Using Sequence Motifs for Enhanced Neural Network Prediction of ...

5 downloads 664 Views 1MB Size Report
atoms is smaller than the threshold, a characteristic sequence (logo) motif, is found. The mo- tifs change as the sequence separation increases: for small.
ISMB-99, 95-105

Using Sequence Motifs for Enhanced Neural Network Prediction of Protein Distance Constraints 

Jan Gorodkin1 2 , Ole Lund1 , Claus A. Andersen1 , and Søren Brunak1 1 Center

for Biological Sequence Analysis, Department of Biotechnology, The Technical University of Denmark, Building 208, DK-2800 Lyngby, Denmark [email protected], [email protected], [email protected], [email protected] Phone: +45 45 25 24 77, Fax: +45 45 93 15 85 2 Department

of Genetics and Ecology, The Institute of Biological Sciences University of Aarhus, Building 540, Ny Munkegade, DK-8000 Aarhus C, Denmark

Abstract Correlations between sequence separation (in residues) and distance (in Angstrom) of any pair of amino acids in polypeptide chains are investigated. For each sequence separation we define a distance threshold. For pairs of amino acids where the distance between Cα atoms is smaller than the threshold, a characteristic sequence (logo) motif, is found. The motifs change as the sequence separation increases: for small separations they consist of one peak located in between the two residues, then additional peaks at these residues appear, and finally the center peak smears out for very large separations. We also find correlations between the residues in the center of the motif. This and other statistical analyses are used to design neural networks with enhanced performance compared to earlier work. Importantly, the statistical analysis explains why neural networks perform better than simple statistical data-driven approaches such as pair probability density functions. The statistical results also explain characteristics of the network performance for increasing sequence separation. The improvement of the new network design is significant in the sequence separation range 10–30 residues. Finally, we find that the performance curve for increasing sequence separation is directly correlated to the corresponding information content. A WWW server, distanceP, is available at http://www.cbs.dtu.dk/services/distanceP/.

Keywords: Distance prediction; sequence motifs; distance constraints; neural network; protein structure.

Introduction Much work have over the years been put into approaches which either analyze or predict features of the threedimensional structure using distributions of distances, correlated mutations, and more lately neural networks, or combinations of these e.g. (Tanaka & Scheraga 1976; Miyazawa & Jernigan 1985; Bohr et al. 1990; Sippl 1990; Maiorov & Crippen 1992; Göbel et al. 1994; Mirny & Shaknovich 1996; Thomas, Casari, & Sander 1996; Lund et al. 1997; Olmea & Valencia 1997; Skolnick, Kolinski, & Ortiz 1997; 

Present address: Structural Bioinformatics Advanced Technologies A/S,  Agern Alle 3, DK-2970 Hørsholm, Denmark Copyright c 1999, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.

Fariselli & Casadio 1999). The ability to adopt structure from sequences depends on constructing an appropriate cost function for the native structure. In search of such a function we here concentrate on finding a method to predict distance constraints that correlate well with the observed distances in proteins. As the neural network approach is the only approach so far which includes sequence context for the considered pair of amino acids, these are expected not only to perform better, but also to capture more features relating distance constraints and sequence composition. The analysis include investigation of the distances between amino acid as well as sequence motifs and correlations for separated residues. We construct a prediction scheme which significantly improve on an earlier approach (Lund et al. 1997). For each particular sequence separation, the corresponding distance threshold is computed as the average of all physical distances in a large data set between any two amino acids separated by that amount of residues (Lund et al. 1997). Here, we include an analysis of the distance distributions relative to these thresholds and use them to explain qualitative behavior of the neural network prediction scheme, thus extending earlier studies (Reese et al. 1996). For the prediction scheme used here it is essential to relate the distributions to their means. Analysis of the network weight composition reveal intriguing properties of the distance constraints: the sequence motifs can be decomposed into sub-motifs associated with each of the hidden units in the neural network. Further, as the sequence separation increases there is a clear correspondence in the change of the mean value, distance distributions, and the sequence motifs describing the distance constraints of the separated amino acids, respectively. The predicted distance constraints may be used as inputs to threading, ab initio, or loop modeling algorithms.

Materials and Method Data extraction The data set was extracted from the Brookhaven Protein Data Bank (Bernstein et al. 1977), release 82 containing 5762 proteins. In brief entries were excluded if: (1) the sec-

ondary structure of the proteins could not be assigned by the program  DSSP (Kabsch & Sander 1983), as the DSSP assignment is used to quantify the secondary structure identity in the pairwise alignments, (2) the proteins had any physical chain breaks (defined as neighboring amino acids in the sequence having Cα -distances exceeding 4  0Å) or entries where the DSSP program detected chain breaks or incomplete backbones, (3) they had a resolution value greater than 2  5Å, since proteins with worse resolution, are less reliable as templates for homology modeling of the Cα trace (unpublished results). Individual chains of entries were discarded if (1) they had a length of less than 30 amino acids, (2) they had less than 50% secondary structure assignment as defined by the program DSSP, (3) they had more than 85% non-amino acids (nucleotides) in the sequence, and (4) they had more than 10% of non-standard amino acids (B, X, Z) in the sequence. A representative set with low pairwise sequence similarity was selected by running algorithm #1 of Hobohm et al. (1992) implemented in the program RedHom (Lund et al. 1997). In brief the sequences were sorted according to resolution (all NMR structures were assigned resolution 100). The sequences with the same resolution were sorted so that higher priority was given to longer proteins. The sequences were aligned utilizing the local alignment program, ssearch (Myers & Miller 1988; Pearson 1990) using the pam120 amino acid substitution matrix (Dayhoff & Orcutt 1978), with gap penalties  12,  4. As a cutoff for sequence similarity we applied the threshold T  290  L, T is the percentage of identity in the alignment and L the length of the alignment. Starting from the top of the list each sequence was used as a probe to exclude all sequence similar proteins further down the list. By visual inspection seven proteins were removed from the list, since their structure is either ’sustained’ by DNA or predominantly buried in the membrane. The resulting 744 protein chains are composed of the residues where the Cα atom position is specified in the PDB entry. Ten cross-validation sets were selected such that they all contain approximately the same number of residues, and all have the same length distribution of the chains. All the data are made publicly available through the world wide web page http://www.cbs.dtu.dk/services/distanceP/.

Information Content / Relative entropy measure Here we use the relative entropy to measure the information content (Kullback & Leibler 1951) of aligned regions between sequence separated residues. The information content is obtained by summing for the respective position in the alignment, I  ∑Li 1 Ii , where Ii is the information content of position i in the alignment. The information content at each position will sometimes be displayed as a sequence logo (Schneider & Stephens 1990). The position-dependent information content is given by qik (1) Ii  ∑ qik log2 pk k

where k refers to the symbols of the alphabet considered (here amino acids). The observed fraction of symbol k at position i is qik , and pk is the background probability of finding symbol k by chance in the sequences. pk will sometimes be replaced by a position-dependent background probability, that is the probability of observing letter k at some position in the alignment in another data set one wishes to compare to. Symbols in logos turned 180 degrees indicate that qik pk .

Neural networks As in the previous work, we apply two-layer feed-forward neural networks, trained by standard back-propagation, see e.g. (Brunak, Engelbrecht, & Knudsen 1991; Bishop 1996; Baldi & Brunak 1998), to predict whether two residues are below or above a given distance threshold in space. The sequence input is sparsely encoded. In (Lund et al. 1997) the inputs were processed as two windows centered around each of the separated amino acids. However, here we extend that scheme by allowing the windows to grow towards each other, and even merge to a single large window covering the complete sequence between the separated amino acids. Even though such a scheme increases the computational requirements, it allow us to search for optimal covering between the separated amino acids. As there can be a large difference in the number of positive (contact) and negative (no contact) sequence windows, for a given separation, we apply the balanced learning approach (Rost & Sander 1993). Training is done by a 10 set cross-validation approach (Bishop 1996), and the result is reported as the average performance over the partitions. The performance on each partition is evaluated by the Mathews correlation coefficient (Mathews 1975) C

 

P N  Pf N f  t t   Nt  N f  Nt  Pf  Pt  N f  Pt  Pf 



(2)

where Pt is the number of true positives (contact, predicted contact), Nt the number of true negatives (no contact, no contact predicted), Pf the number of false positives (no contact, contact predicted), and N f is the number of false negatives (contact, no contact predicted). The analysis of the patterns stored in the weights of the networks is done through the saliency of the weights, that is the cost of removing a single weight while keeping the remaining ones. Due to the sparse encoding each weight connected to a hidden unit corresponds exactly to a particular amino acid at a given position in the sequence windows used as inputs. We can then obtain a ranking of symbols on each position in the input field. To compute the saliencies we use the approximation for two-layer one-output networks (Gorodkin et al. 1997), who showed that the saliencies for the weights between input and hidden layer can be written as skji  s ji  w2jiW j2 K

(3) where w ji is the weight between input i and hidden unit j, and W j the weight between hidden unit j and the output. K is a constant. The kth symbol is implicitly given due to the

Sequence separation 2

Sequence separation 11

250

2000

Sequence separation 16 120

200

100

1000

100 50

-6

60

0

-4 -2 0 2 Physical distance (Angstrom)

20 -20 -15 -10 -5 0 5 10 15 20 Physical distance (Angstrom)

Sequence separation 3

1000

0

25

-20

Sequence separation 12

-10 0 10 20 30 Physical distance (Angstrom)

40

Sequence separation 20 90

200

80

800

70 Number

600 400

Number

150 Number

80

40

500

0

150

Number

Number

Number

1500

100

60 50 40 30

200

50

0

0

20 10

-6

-4 -2 0 2 4 Physical distance (Angstrom)

-20

Sequence separation 4

800

0

-10 0 10 20 Physical distance (Angstrom) Sequence separation 13

70

200

700

Number

Number

Number

300

Sequence separation 60

100

40 30 20

200

50 10

100 0 -8

50

50

150

400

-10 0 10 20 30 40 Physical distance (Angstrom)

60

600 500

-20

-6

-4 -2 0 2 4 6 Physical distance (Angstrom)

0

-20

-10 0 10 20 Physical distance (Angstrom)

30

0 -40

-20 0 20 40 60 80 Physical distance (Angstrom)

Figure 1: Length distributions of residue segments for corresponding sequence separations, relative the respective mean values. Sequence separations 2, 3, 4, 11, 12, 13, 16, 20, and 60 are shown. These show the representative shapes of the distance distributions. The vertical line through zero indicates the displacement with respect to the mean distance.

sparse encoding. If the signs of w ji and W j are opposite, we display the corresponding symbol upside down in the weight-saliency logo.

Results We conduct statistical analysis of the data and the distance constraints between amino acids, and subsequent use the results to design and explain the behavior of a neural network prediction scheme with enhanced performance.

Statistical analysis For each sequence separation (in residues), we derive the mean of all the distances between pairs of Cα atoms. We use these means as distance constraint thresholds, and in a prediction scheme we wish to predict whether the distance between any pair of amino acids is above or below the threshold corresponding to a particular separation in residues. To

analyze which pairs are above and below the threshold, it is relevant to compare: (1) the distribution of distances between amino acid pairs below and above the threshold, and (2) the sequence composition of segments where the pairs of amino acids are below and above the threshold. First we investigate the length distribution of the distances as function of the sequence separation. A complete investigation of physical distances for increasing sequence separation is given by (Reese et al. 1996). In particular it was found that the α-helices caused a distinct peak up to sequence separation 20, whereas β-strands are seen up to separations 5 only. However, when we perform a simple translation of these distributions relative to their respective means, the same plots provide essentially new qualitative information, which is anticipated to be strongly correlated to the performance of a predictor (above/below the threshold). In particular we focus on the distributions shown in Figure 1,

but we also use the remaining distributions to make some important  observations. When inspecting the distance distributions relative to their mean, two main observations are made. First, the distance distribution for sequence separation 3, is the one distance distribution where the data is most bimodal. Thus sequence separation 3 provides the most distinct partition of the data points. Hence, in agreement with the results in (Lund et al. 1997) we anticipate that the best prediction of distance constraints can be obtained for sequence separation 3. Furthermore, we observe that the α-helix peak shifts relative to the mean when going from sequence separation 11 to 13. The length of the helices becomes longer than the mean distances. This shift interestingly involve the peak to be placed at the mean value itself for sequence separation 12. Due to this phenomenon, we anticipate, that, for an optimized predictor, it can be slightly harder to predict distance constraints for separation 12 than for separations 11 and 13. The peak present at the mean value for sequence separation 12 does indeed reflect the length of helices as demonstrated clearly in Figure 2. Rather than using the simple rule that each residue in a helix increases the physical distance with 1.5 Angstrom (Branden & Tooze 1999), we computed the actual physical lengths for each size helix to obtain a more accurate picture. The physical length of the α-helices was calculated by finding the helical axis and measuring the translation per Cα -atom along this axis. The helical axis was determined by the mass center of four consecutive Cα atoms. Helices of length four are therefore not included, since only one center of mass was present. We see that helices at 12 residues coincide with sequence separation 12. Again we use this as an indication that at sequence separation 12 it may be slightly harder to predict distance constraints than at separation 13. Separation and helix length 35

Physical distance (Angstrom)

 Average sequence separation  Average helix length

30 25 20 15 10 5 0

0 

5 

10

15

20

25

30

Separation/Length (Residues)

Figure 2: Mean distances for increasing sequence separation and computed average physical helix lengths for increasing number of residues.

As the sequence separation increases the distance distribution approaches a universal shape, presumably independent of structural elements, which are governed by more local distance constraints. The traits from the local elements do as mentioned by (Reese et al. 1996) (who merge large separations) vanish for separations 20–25. (Here we considered separations up to 100 residues.) In Figure 1 we see that the transition from bimodal distributions to unimodal distributions centered around their means, indicates that prediction of distance constraints must become harder with increasing sequence separation. In particular when the universal shape has been reached (sequence separation larger than 30), we should not expect a change in the prediction ability as the sequence separation increases. The universal distribution has its mode value approximately at  3  5 Angstrom, indicating that the most probable physical distance for large sequence separations corresponds to the distance between two Cα atoms. Notice that the universality only appears when the distribution is displaced by its mean distance. This is interesting since the mean of the physical distances grows as the sequence separation increases. From a predictability perspective, it therefore makes good sense to use the mean value as a threshold to decide the distance constraint for an arbitrary sequence separation. A useful prediction scheme must rely on the information available in the sequences. To investigate if there exists a detectable signal between the sequence separated residues, for each sequence separation, we constructed sequence logos (Schneider & Stephens 1990) as follows: The sequence segments for which the physical distance between the separated amino acids was above the threshold were used to generate a position-dependent background distribution of amino acids. The segments with corresponding physical distance below the threshold were all aligned and displayed in sequence logos using the computed background distribution of amino acids. The pure information content curves are shown for increasing sequence in Figure 3. The corresponding sequence logos are displayed in Figure 4. We used a “margin” of 4 residues to the left and right of the logos, e.g., for sequence separation 2, the physical distance is measured between position 5 and 7 in the logo. The change in the sequence patterns is consistent with the change in the distribution of physical distances. Up to separations 6–7, the distribution of distances (Figure 1) contains two peaks, with the β-strand peak vanishing completely for separation 7–8 residues. For the same separations, the sequence motif changes from containing one to three peaks. For larger sequence separations, the motif consists of three characteristic peaks, the two located at positions exactly corresponding to the separated amino acids. The third peak appear in the center. This peak tells us that for physical distances within the threshold, it indeed matters whether the residues in the center can bend or turn the chain. We see that exactly such amino acids are located in the center peak: glycine, apartic acid, glutamic acid, and lysine, all these are medium size residues (except glycine), being hydrophilic, thereby having affinity for the solvent, but not being on the outer most surface of the protein (see e.g., (Creighton 1993)

Separation: 13 "

! Separation: 2 0.12 0.06

0.10

Bits

Bits

0.08 0.06 0.04

0.04 0.02

0.02



1 



2



4

3



5



6



7



8

10 

9



4

2

6





10 

8

12





14

Position

Separation: 7 !

Separation: 20 "

0.06

0.03

0.04

0.02



16

Position

0.02

18



20 

22

0.01



0.00



4

2



6



10 

8



12

14



0.00

16



4 6 8

2

















10  12 14 16 18 20 22 24 26 28

Position

Position

! Separation: 9

" Separation: 30

0.06

0.02

0.04

Bits

Bits



0.00 11

Bits

Bits

0.00

0.01

0.02



0.00



4

2



6

10 

8



12





14



16

10 

5



20 

15



25

Position

Position

" Separation: 12

" Separation: 60

0.06



30 

35

0.02

0.04

Bits

Bits



0.00

18

0.01

0.02 0.00



2



4



6



8

 10



12



14



16



18

 20

Position

0.00



5













 15 20 25 30 35 40 45 50 55 60 65 10 Position

Figure 3: Sequence information profiles for increasing sequence separation.

for amino acid properties). As the sequence separation increases from about 20 to 30 the center peak smears out, in agreement with the transition of the distance distribution that shifts to the universal distribution in this range of sequence separation. The sequence motif likewise becomes “universal”. Only the peaks located at the separated residues are left. The composition of single peak motifs resemble to a large

degree the composition of the center for motifs having three peaks. However, the slightly increased amount of leucine moves from the center of the one peak motifs to the separated amino acid positions in the three peak motifs. The reverse happens for glycine. Interestingly, as the sequence separation increases valine (and isoleucine) become present at the outermost peaks. In agreement with the helix length shift from below to above the threshold at separations 12

Y T M F

WWW

S

C

W

H Q F H M W N

R P W I C Y W M

K T I

A T

D

R P

M Q Q P F

C

G

F I C C M W Y

W C M

bits

T N S

KA

D

Q P K N Q E

I

YY

G

C H H C Y C W P

C C

W

W

C C

T P Q S N

G

TV S

F T N F K S V

P H H C H W H N P W Y K N H Y F P H C T F S N F P D K P P T P S G

N

P

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

W

S

P Q E N

C H C

1 2 3 4 5 6 7 8 9 10 11

H

T

E

K P S Q D N

W WWWW R M Q H K E M YV T P

M

H

D Q M Q

H P H Q

R

T

R

L A

I

I

I

Ž

 

I

L A

bits

R G

G

S

G S

V

KAAA PE N D

D

S

D E DT TS

I

K E E K

L V

R

H Y F F

W C C –  ¢ W

F

Q

P

F

I

H Q

Q Y M F Q W P Y M WWWWWW R

Q

£

²

Â

V Á

Æ

À

¿

¾

F ½

Y C W

ª

Q

H

W H M W

H H Y H Y H Y

P

P N

L

N D

H H H

K K E E E K

N N P Q P R

W C

R R P N F F

Y

Q

W W M M M Q Q W

©

Y



H H M H M M C C C ºW ¼» ÅÄ

±

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

G

N S D P G R T D P T

K S

N K

H H H

C C

WW C C C W

C

W

D Q PS D

R K N

Y M Y M P M P Q P R H E M M M F F R N H K

«

¤

W H M M W M H H Q M —

S S

M C

M

M



H

Š

x

u

¸

Q R P H ž

F C Y Y C C WW

C C C C C C

n

m

FT

¬

¥

Ÿ

y

h

V

Y MY I F I I N N Y N

¦

™

˜

i

g

­

§

š

N

z

j

T

F

V

I

I

R R R

YY F FQ F R RQQQ R

S

C W M f

Ã

¹

W W Q Q Q W Y

I

D

E

TTT N I

P



{

o

k

]

\ [

AA A L G L G

Y

F

R Q H K N H P N YQ Q H H K I Y R I R V Y E R F P K T K N L P FY P E E S N FQ R S P D Q I v

p

D

d

T

I

F

V

A

E K A N D D L L

WW

N P R

P

Y Y H

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A

D E

V L

G

E

| | bits

G

T I

F

µ

²

Š

´

G ®

£

G G G G G

­

V

Y Y R Y I K Y K P Q K K F F R A F F F R I E P P P E P P E K E I R R R R K I S D S L I E I D V S A I A L L I L A D K A L L AA L E L

T

w

¢

E

€

S T j



v

z

V

’

‘

—

~

q

R

`

¬

œ

«

ˆ

¡



y

p

^

h

}



u

g

†

K

•

™

›

P P

N L A A K

R

NO P

X

\

ƒ„



Œ

”

˜

I

š

I

Q C C C C C C C C V I L G L G E S T N N N N N K S T N D N I E K D S I K S V Q

I

W

[

d

S T V T T U

n

s

{

‚

‹

“

°

¥

ž

¤

¦

¯

C W

W M W W H M Q Y F F R P W W M M M M M Q Y R P

C C C C C C C C W W W W W W

E E

K K E

C

W W H

H

W W

K

R

R N N Q N P

Q

R H

H M

LJ

Å

Ä

V G Ã

T S L T G I I Â

Á

V

À

È

¸

·

S

¾¿

Ç Æ

C I M T H N M M H H C C W W

P R P Y F

Q

W W M M W M Q Q

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

C C

V S

C H M W C

¨ §

H I F Y N F Y M N Y C W M M H

W W W W M H F N I Q Y Y Y P N W H M H M M H M M M Q H M H Y Q P P F M M H E D Q F F Q F R N M P R A A W W M W P H H H H M M H H Q Q Q M Y F Y N F E D F M Q Q Q R P E K K H I H Q M Q Q F Y Y F Y Y F Y F Y N H N E K D Q F F F Q Y P R P R P P P R R R R K E P P A Y Q F P P P K Q Y I I Q V Q A Q K

»

º

M Y

³

±

©

Ÿ



D T T T T T

…

Q

e

–

Ž

t

f

K N

 

|

o

]

Y

V

R R P

N

P E N N

‡

GH

C C C C

¹

H F F

ª

i

_

M

I

N R K Q K N

Q

K K D

a

Z

L G G L

L L

Y

G G

‰

P R Q




7

0

r

m

G

?

8

1

R Y R H

H M

D

G

G

9

Y

F

'

E N P

+

)

(

"

¼

W Q Y F R P W Q Q F Y K F Y R

AA

I

*

$

#

F F F F F M R C Y Y Y Y C

T I

bits

Q

V FS SA TAT L ZY

e

d

`

L r

P K I P R R E E K

VV q

z

x

{

|

„…

‚ 

ƒ

—

†

Š

Ž



–”“

œš™

¡

¢

G V G G G S S K N K V •

‘

›

˜

ž

Ÿ

£

©

¥







A

R Y

¨§ ¦

ǻ

¬

®

­

± ¯°

²

³

º ·

¹

¿

¾

¼½

Ã

ÄÅ Æ

È

Ê

Ç

ÍÎÌË

Ð

Ñ

Ò

Ó

Õ

Ô

Û ÙÚÖ×Ø

à

á



D S

Y H H W

F T P T

Q

M C W C W C

C C

C C C

H

C C C C

C

H

Y

C

C W C

W

ì

ã

Ý ßÞ Ü

â

æ å

í

é

ëê



ò

ù ÷

ú

õ

ö

ø

ü

þ

ÿý











A L AG



+

YNL









C H

5

V 8

VS T 4

7

I

|

L A G

FT V YH I MYN FS WH I Y (





ðñîï

ó

ô

C C C C C C C C C C

C C C C

C W C W C C

I T

'

3

2

T

0 1

;

Y C F C M W H *

6

/

:9

F N H M Y M

C C C W W W H M

.

R Q

Q

R

F P P R R E Q

H

R

M M M M C M M W M Q

W

E E P R

H W

P R

H

H P Q

Q G M

Q P K T N H E D S H

R P R

R P Q P N Q Q Q Y

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69

Y

H H

"

!









ç

ä

Ï

Á

T T T F F P P É P F Q P F F K K D S D S D S N K S T H D S N Y S N N N N N V Y V Y N G D I S T D N K N N V G V V V W G G W W W W W N N Y N Y N N Y M M M S S Y Y S Y Y Y Y N K D I N N D Y D N K D E K D I D N Y W WW

K

Q R E

Y

C

C C W C

M Q P F R Q

Y

H Y H W H H W

W

Y

C W

W

Y

K Y Y F Q Q F F Q H

H H

H H

Y

C C C

W

T S Y N

V T T T T T T w } ~  € ‰T Œ   ’   ¤ ¶µ´ »¸ À I T F F F P F F F R R R Q Q Q S S S Q G H G G H H D G V H H H H H H M N Y N K I S S K I S K M M M D N K N Y N Y K K W N W W I

C C C Q

Q H H H H

C W



T N 

V 

è

‡ ˆ

T T T F G G G S T K T L G V S V S S Y K Y K N V A S N N K N N N N ‹

AG D G S

û

S

D V E F L V L G M F E I R P R L A V P F I L Q Q V I E K K K I P Q Q y

A L

t u

E I D I I L A L D L I I D D E A L A A D A L E E

v

E T I

n

A A I F G TA G A H F G G G G G AA E Y AG Y M G G A D T D V C I V E D N S T G N D S S I S D W D S T D S T D S T E Y K G E K T E N H M G V G D S G V V D D S D S E K D E S T D S D S N K N N G D S G G D S S M C V VVVV N I E K I G G G Y H S S C S H M W Y Y Y I V I I Y N Y Y Y Â

o

m

A

D

p

I

j

g



V

a

A

L

b

G

G

s

k

[

$

#





l

D

c

\

V





L

A

SAA T G G

N

I

^

]

L

_

C W

H H H H F F F F F F F P P F P F P P R P P P R E E V P D V D L A T W M M H M H M H M H M M H H H Q Q Q Q Y Q Q Q Y F Q M Q M M M M M M Y F F F M M M I F M M M M R R R W R W P P R W W P E R W W W P W W I P E I E K I R R P E V D E D I T T T L L A L A W M M M M H M M M H M M M M H H M H M Q Q Q Q W F R W W W W W W F Y Y R R R R E D K I R R E I R P P T K I P P P P P P D V V D L V L A V

L

G



h

S

X

M

@

L

R

K N N QPE KKE E

YH M H

A

AT G I T I F V

i

L G I G V

N D

C

B

I

KTK E E D

N O

I

;

L L K L

P

FY

I

:

Q Q Q Q Q Q F Q Q V F Y T T F F F F F Q Q Q L A R N E I E R E K I R R K I P E P Q P Q F Q R F F Q Q Q Y I P I I K I I K I E K E K K I L A L A L A L A P P L F N R P F R I I P R I P R R P F R K K E F R K E R R R R E K I E T K T E T I E T L L L P P P A P P P W Q L A T K E T R R Y T V F V L I E L A L A A A E E E K L L G L L A G A I R K P V V V V V A A Q E T V E K T E K T V A L Q I L L L A A R A L L L V V L E N

S

D




VST ST I T I

0.0

A

A L G

A L

P

I

D R R Q K P E N

G

?

Q R P Q Q K

J

GG

G

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

-

D D PN D L K A RELL A NDA

5

G

1

D

673

V

8 9

C M W 2 4

D

A

L Y

C C R W W V M F I Y

V

%

L



&

K P E KD

,

WW

C

T

R

NDS K D

I

F

F M C M F P C H R P M I Y Y M I K N H H T FT



S

K

QRSE

I

I

)

A L A G L V

V

S T

R

T

S

QN

H P



C

Y F

C M C

H





I T

U

T

G

R

V

S PD

N



Q Y W R C

N

E Q K

C

V

V

S

LI

K

K G I

Q P T K N P S 0

/

E

A

V

&

KS P E E D

L

'

G D

I

D S

ï

î

-

T E

C

Q C

.

F R

TA

âá





D



R





K

KG PD

G

R

ã

Q H W H M M C C C Y H M

V

)

YR (



N F A R L F M Y G R P V Y

V

E

!

I





Q

W

SPE S D D

KG

RE

YP FY F R I G F I V T

I

ú

ù

L A A L

K

E



K E



P Q D

ñ

ð

A L

GN

F

V

L

Q

G

E D S

VL

I

ò

*

R A N QRR û

ó

A N

L

ë

W

W Ø

å

ä

N CY

Y C

Q W R V Q H C M M W Y I C

V

ì

Q M

L G F Y A H

í

T V

æ

+

D T T K E S G N D K G E

R

LTVT

ç

S

è

V

G D

é

A S L E

| | 0.02

L A

L A R I





G

G D S E T ê

V

Y



T D F S D I E Q S Y R M N H H F Ù

K E T

I

õ

ô

"

G

L

Õ

T L

Ö

G

ü

S P K ö

A

×

D E NK

P RPS

DN E

Ý

V





ý

÷

FL

R

L 

ø

A

Û

E

I



I

F

#



Þ

L A



þ

V

A

$

V



ÿ

L A

V

A



DK ESD S EG K GKNSA

ß

L

L A



I

Ú

0.04



à

I

R

P S D S D N S D

A

Ü

G A

GD AEG

V

Ò



V

S S FT TT I

D

RT I N KD T R



Ñ

,

2

I

Y

C C C H W H H

V

Separation: 60

L

Ó

PK

T

%

TT I

0.0

A

!

E K E E D A D



G G G

B

V A L A V

Q

C H M

L

A LG V G V S

V L



L ½



;

:

A

M





V

Ô

|

I T

V

L

-

L

C

C Y C WW W

&

L A N AA

î

G L G L V V

K



L E

I





R YY

S

I

Ü

þ

D R

I

FPP

WW C C C C C H H C C H H H H ú

L

E

I

Y H H H Y F YY P FY H

H

Ð

C W H M ÊÉÈÇ ÏÎ

ÿ

ò

T I G

A

I

F F R P P FY P P K YF P N ó

Q H M M M M C M F

ß

é

P

C I F Q H M R C W

T

M

M M M C C Q W WW

V

3

V

T

M

à

M Y



P S

I

Ñ







N

Ò

Í

û

H P

A







ê

A G K

ST

F N N P

P

E Q F YQ M Y H

ü

D N K

ô

LT I G

N G S



õ

Q YYQ F Q M Q

V

D SA

Ó

L

L

RD

ë

ä

Ý

Õ

Ô

V

E







KS E

Ö



A

å

Ø

L G EA N

ö

á

Ù

V VV LA L L LA A AA

G

G DG

×





ND

KR

D T N KS E D G

ï

KG A

G

V

KKSG P T P T TD I TVVS ç

PS



V L

TS DN D DT I S

â

Û

Ú

E D S

V

KE

EA K D N I V R S V R R N N E R I F I L QQ Q R R F A R R F A I V L QQQY F R QQ F A L

A

TV ST

V V S

TD V

ð

I



RR

è

A L

ESG

ã

E I E K I



G R

T



ý

D LDSN S N E

V

A



÷

D



ø

G A



KV I K

EAA KL L G

G

æ

G

LE LV AEAED K

G

ù

ì

A

L

L

í

Ï

®

N QPR S

E

Î

|

q

l

e

^

H C M Y C C H M W C N C C W C W

O

V

F

I

¨

¡

›

H Q



ADEA

Þ

Ë

a

`

_

F

¯

œ

•



0.04

L A

GKDG ALSKL

L A

C H M W Ê Ì

U V

P

W

G

S N EG

G

ñ

G

Ð

|

·

T

V

L

‘

E N G G SG V T P VV K D

Separation: 30

A

Ë

Y

H

Y H Y H M C F H M

Y Y C @?

„

T PR KSA

D

A

I

H C C W C H H ;

P }

r

K R P D G S V G N AT S F K F H Y F I F Y N M

B

=




0.0

I

L A

V

“

w

KN PP PKD

DT

S G E A N T

b

I

…

ETN

c

Z

I

W

‹

€

t

V

X

Q

I

L I

Œ

Y

K

J

R G

L

T T S T I I

GA

Ì

L A

L T LA V I FTAV

E

C

N

M

V

VV V

P

G

L G G G L

Y M H

C M W WW C

K D E T

6

'

C C W

†

H P P P R R R Q K

9

8







D R V G

:

1



°

”

‡

N

T K S E T K E E D D

2

#

YQ M M YQ M F M Y M H Q M H M

M



C

³

L A

ˆ

GDSE SESE SD D EKTTEG GS KTKKS T N

L

I

´

DDDG

‚

L

F Q F R I F YQ F R





M M

WW

W

7

Q

Q

Q Y

M Y

C

A

3

$

ƒ

K E

G

T

úùø



G

VVL V A LVV A

GV

VV

F P PY T K P P Y Y F S N N F F S P R P D T F F N N D S P K N E T T PPP N S D G I N I I KSG T I TTD D T I I VS

üû

F

F Y

R Q

I

4

I

R

K DG E

D TDS SS TD VT G

G

I



E

I





µ

V

‰

GGG

V

K R R I L R V QNQ R K

RQQ

R R Q F

E

A

L

I

I

A

S

DA S T

KV

N





L A

L

0.02

L

A

V

I

.

(

AAA

ý

5

)

%

LLLA

S

V SV TT G

óô



S T

G

ðï

L

/

*

N D K V RRR E N 



G

õ

SN

+

K

E EKA K I EVL I EA



G

G

0

D &

G

L

bits

L H Q A P

R

Y

M F A C L C Y M

K

P R N H T V D S E K Q

S

K E D

V N G

bits

F Y M F

V Y R E

N

N

I

P K

V T

bits

V

Q

R

YF I

W C

A

L

H

I

0.0

HYR FH I YFYF

ÿþ

R V H F M C

É

I



 

D L ES

L

L

Í

F H M M Y H

÷ö

-







âá à èéæç å íì î òñ ßÞ ãä ëê

|

I

0.02

A

,





Separation: 12

0.04

V

L





Separation: 9

0.06

F S

AEE

L LDDL AAA KKE KLVV

A

ALES LED K A

V

0.0

L



!

Q

|

C F M C Y W H H

I

0.04

"





C H M H M C W W

0.02

S K

GGA L G

M F F C Q

0.04

D

| |



AA

E

0.06

M

A

V

R

G P

C C C C C

H N C Y

L

YF

Separation: 20

A V L E

0.0

E

0.02

S E

I

N

W

I

P K

0.04

|

É Ð ÇÆÈ Î Ï ÕÔÓ Å ÍËÌ ÑÒ ÝÜÙÖÚÛ× Ø

R

N P R Q

|

0.06

Q

D S

C

WW C

¹

D E

K Q T

L G D

F M Y

M C

F H Y M

I

R F C M M Y H C W H W

P

A

L A

Y H

L I G F V R H Y

0.0

Separation: 7

Y

V R

T S

M W

V

¼½» ¾

I T

K

W

Q

E

Y M M Q M Y H

QQ

N Q

I

T S

Y P

W

F

F

V

N G D

K I V I G T V

VT

F

Y

I

L A

¿

L

T

PG N R

N D

V

Q W M

Q

M M W W

V

V

L G

S

K

ÂÁ IÃ Ê À

E

I

G G

T V D D G V

N T D N S T D I V S G S

M F M

F F Y

F

PKG G A ND S E

E

RQP

R

Q

K

ON

JI

Y

RT

>=

Q M F

H M

T

0

Q

FL A

L

I

/

I

R Q

I R

I

R

RRVL

V

E

|

K

E I R Q

V

)(

S

I

I

R I

#

VUT

KA

L E R

R

V

V

&% '

GF

Q

¬ Œ‹ ’‘ š ¢ ± l f  Š v µ u c ¡ ‰  ˆ « ~ } … ª „ ™ ´²³ ¸·º¶ X\[ZYW ^_] ab` ed jki rtsq {zyxw| ƒ‚€ †‡ Ž ˜

0.02

G N D S G T RA E

321

P Q R

Ä

A

,+*

A E L D R Q

A E

L D

LK

K K

A E L N E I

”“ ›

ENNS K D G K PPPE A D N NS E P

L A

LTVV

.-

R

E

D N A L

hg

A

V

K S E T

A L

M

H

LA

L

D G

V G

K TSG S

4

N

S  E ¦ — E G K ¥S D œ K £ ­® D –• T¤ E KG S

m

I

DG E S T

$

?

A o

V n

G

65

° D ž D § G ¯

A

@ D

; 9 :8

Ÿ ¨

L p

L

A LK

7

LE D D E KR S L A

0.04

T

G

B A < E

G   G © L

|

L C

A

0.06

G

|

A D

A

0.12 0.1 0.08 0.06 0.04 0.02 0.0

AA

bits

Separation: 13

LL

Separation: 2

Figure 4: Sequence logos for increasing sequence separation. A margin of 4 residues on both ends of the logo is used. Symbols displayed upside down indicate that their amount is lower than the background amount. This figure is available in full color at http://www.cbs.dtu.dk/services/distanceP/.

and 13, the center peak shifts from slight over to slight underrepresentation of alanine: helices are no longer dominant for distances below the threshold. The smearing out of the center peak for large sequence separations does not indicate lack of bending/turn properties, but reflects that such amino acids can be placed in a large region in between the two separated amino acids. What we see for small separations, is that the signal in addition must be located in a very specific position relative to the separated amino acids.

>

>

>

>

>

>

=

=

dent of the sequence separation. We even note the drop in performance for sequence separation 12, as predicted from the statistical analysis above. It is also characteristic that, as the distribution of physical distances approaches the universal shape, and as the center peak of the sequence logos vanishes, the performance drops and becomes constant with approximately 0.15 in correlation coefficient. The change in prediction performance takes place at sequence separations for which changes in the distribution of physical distance and sequence motifs change. Due to the not vanishing signal in the logos for large separations, it is possible to predict distance constraints significantly better than random, and better than with a no-context approach. The conclusion is not surprising: the best performing network is that which uses as much context as possible. However, for large sequence separations more than 30–35 residues, the same performance is obtained by just using a small amount of sequence context around the separated amino acids, as in (Lund et al. 1997). We found that the performance between the worst and best cross-validation set is about 0.1 for separations up to 12 residues, then about 0.07 for separations up to 30 residues, and then 0.1–0.2 for separations larger than 30. Hence, at sequence separation 31 the performance start to fluctuate indicating that the sequence motif becomes less defined throughout the sequences in the respective crossvalidation sets. Hence we can use the networks as an indicator for when a sequence motif is well defined. As an independent test, we predicted on nine CASP3 (http://predictioncenter.llnl.gov/casp3/) targets (and submitted the predictions). None of these targets (T0053 T0067 T0071 T0077 T0081 T0082 T0083 T0084 T0085) have sequence similarity to any sequence with known structure. The previous method (Lund et al. 1997) had on the average 64.5% correct predictions and an average correlation coefficient of 0.224. The method presented here has on average 70.3% correct predictions and an average correlation coefficient of 0.249. The performance of the original method corresponds to the performance previously published (Lund et al. 1997). However, the average measure is a less convenient way to report the performance, as the good performance on small separations are biased due to the many large separation networks that have essentially the same performance. Therefore we have also constructed performance curves similar to the one in Figure 5 (not shown). The main features of the prediction curve was present. In Figure 8 we show a prediction example of the distance constraints for T0067. The server (http://www.cbs.dtu.dk/services/distanceP/) provides predictions up a sequence separation of 100. Notice that predictions up to a sequence separation of 30 clearly capture the main part of the distance constraints. We also investigated the qualitative relation between the network performance and information content in the sequence logos. Interestingly, and in agreement with the observations made earlier, we found that the two curves have the same qualitative behavior as the sequence separation increase (Figure 7). Both curves peak at separation 3, both curves drop at separation 12, and both curves reaches the plateau for sequence separation 30. We note that the relative entropy curve increases slightly as the separation increases.

Mean performances

(a) 0.75

r=0 r=2 r=4 r=6 r=8 r=10 r=12 r=14

Fraction correct

0.70

A

0.65 0.60 0.55 0.50 0.45

?

?

0

20 40 60 80 Sequence Separation (residues)

100

@

Mean performances

(b) 0.50

r=0 r=2 r=4 r=6 r=8 r=10 r=12 r=14

Correlation coefficient

0.40

0.30

0.20

0.10

0.00 B

B

0

B

B

B

20 40 60 80 Sequence separation (residues)

B

100

C

Figure 5: The performance as function of increasing sequence context when predicting distance constraints. The sequence context is the number of additional residues r 0 2 4 6 8 10 12 14 from the amino acid and toward the other amino acid. The performance is showed in percentage (a), and in correlation coefficient (b). This figure is available in full color at http://www.cbs.dtu.dk/services/distanceP/. D

E

E

E

E

E

E

E

This is due to a well known artifact for decreasing sampling size, and entropy measures. The correlation between the curves also indicates that the most information used by the predictor stem from the motifs in the sequence logos.

CASP3 target T0067 180

160

140

120

100

80

60

40

20

F

180

Single window curves I

0.50

160 Plain

Correlation coefficient

0.40

140

Profile

120 0.30

100 80

0.20

60 0.10

40 0.00

20

0

20 40 60 80 Sequence Separation (residues)

G

100 G

H

Figure 6: The performance curve using a single windows only. We also show the performance when including homologue in the respective tests. This complete a profile. Relative entropy and NN performance

Arbitrary scale

NN-performance Relative entropy

J

20

40

60

80

]0.4 ; 0.5]

]0.7 ; 0.8]

]0.2 ; 0.3]

]0.5 ; 0.6]

]0.8 ; 0.9]

]0.3 ; 0.4]

]0.6 ; 0.7]

]0.9 ; 1.0]

Figure 8: Prediction of distance constraints for the CASP3 target T0067. The upper triangle is the distance constraints of published coordinates, where dark points indicate distances above the thresholds (mean values) and light points distances below the threshold. The lower triangle shows the actual neural network predictions. The scale indicates the predicted probability for sequence separations having distances below the thresholds. Thus light points represent prediction for distances below the thresholds and dark points prediction for distances above the threshold. The numbers along the edges of the plot show the position in the sequence. This figure is available in full color at http://www.cbs.dtu.dk/services/distanceP/.

L

0

] - ; 0.2]

100 J

Sequence separation (residues) K

Figure 7: Information content and network performance for increasing sequence separation.

Finally, we conclude by an analysis of the neural network weight composition with the aim of revealing significant motifs used in the learning process. In general, we found the same motifs appearing for the each of the 10 cross-validation sets for a given sequence separation. As described in the Methods section, we compute saliency logos of the network parameters, that is we display the amino acids for which the corresponding weights, if removed, cause the largest increase in network error. We made saliency logos for each of the 5 hidden neurons. For the short sequence separations the network strongly punish having a proline in the center of the

input field, that is a proline in between the separated amino acids. This makes sense as proline is not located within secondary structure elements, and the helix lengths for small separations are smaller than the mean distances. (The network has learnt that proline is not present in helices.) In contrast, the presence of aspartic acid can have a positive or negative effect depending on the position in the input field. For larger sequence separations, some of the neurons display a three peak motif resembling what was found in the sequence logos. The center of such logos can be very glycine rich, but also show a strong punishment for valine or tryptophan. Sometimes two neurons can share the motif of what is present for one neuron in another cross-validation set. Other details can be found as well. Here we just outlined the main features: the networks not only look for favorable amino acids, they also detects those that are not favorable at all. In Figure 9 we show an example of the motifs for 2 of the 5 hidden neurons for a network with sequence separation 13.

N N ST

D H D H G

N

V

H T

D S

F P KC

N N

D

G

V S

F

Y W

T V

T

W

R

H

D

R

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

T

L LI

N

WWV

Ö

G Ñ

F

E CSCN

P T

R

L I

¿

C

V

F

Ð

Ú

I

è

ç

Þ

¾

I æ

å

Ó

Ë

F

M Y

Î

I

V

H

I

á

I M E V H C H Y A K Í

A

à

ã

ß

Ò

C ÊÉ

K D

R

P

F

K

K S

M

E

D

V

Y

T

I

DAX P S HH W STXKX G E S D T

Y

R

T

S

Q

P

E

KW

R

H

E E

N V X D X A E Y S Q V I G Q H H F D M D M T Q

M

X V

A Q T X H

P X A H X Y H E L

T »

S

D

³

Q P

M

Y L

H

W A S Q

A

R

R

P

P

Y

V

T Ý

Ø H

M L

G

S

Y

Y

A R T

Y T

F

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

E

Q

Ã

Â

X

K T

W

W P

¯

¨

X A X

­ ® ° X A S D N T

G

¼

E L K S

È

½

G K A P

C N ´

I

ATY

©

¤

£

K

M

K C S Q C W

”

C

T

L

A

I

M



M

E

Ÿ

R

M ž

I

Ž

D



ä

Ì

µ

Ä

ª

 

YA Y

M

R X H A M

S

Q

V

Q R A

I

Q V M

â

Ù

S

L

Ï

D

¥

›

F Q

Å

H S Y Q

•



ˆ

V



T D T X X R Q T N X F A A

¡

I

GG

GG

I YI WI I )

G

/

I V MV

M FM ML F L L F I VMFML MYL F

DD S

XX

L

D P S





G S 

7

Q





&

+

W E

I

C Q

L

I

I

C

VCF F C V D

K

A

6

1

%

C

G

K Q K E P N N P KQ H H R P P



D N

P

Y C

V

F I M U

Q E W F E WW W E R E Y T K L Q S F E K F N H H L E H E H I R R

M

F S N N AW 0

;

J

O

I

C

I

T

L A S N Q T N N V C K K K D B

>

M Q E

N

H

Y A E W

P Q Q

X Y X X T H A D T A P X Y

X X H

E M

T

R

S

H

X

E

H

Q X

M

A

C

R

E

L

S

L

C

F Y

I

Q

K

X

G E X

S W K N K

Q K E

K V R M E

Y L A

X

S G R T

P R X I X X W

A H

H F H

Q

Q

A P A K D T K R

Y

A

T

T N

N

M

M

P

T

N

N S

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

K

Y M A

A A R C F A

= T F L H V M S D T Y W H A R T H L Y Y 54 9 :8 @ ?< M SR

H

R

T

T

S

*

$



C T

P

K



Q C T R

T

V

S

Q

V

L

S

E 



A N C

F H

R S P G E D Q



 T R R L C A G X X X D X      #

T N

A

I V

K



Q

I

H

C

W

T

V WC

F F A

G

F

P

ó

W

W D

G E S E Q E E Q S D KP ö

ù A M õ ú ÿ

H

W

F

Y

D

W

û

÷

ð

R

P

P SSP D PK SPK D ,



Y

ü

Y

I

ô

T ê

G

Y

ý

I L

ñ

W F

Y L

H ìëé ïî

C

ø

L

Y M

M

G W D M C R Y N H

R

-

W

C

ò

I

F V

.

'

V

G



S A

C V





K R G G D



L

D N

3

R

!

P

D

þ



D G D Q

C

2

I

G Q P X

C

V

G G

I L W VV MVL W I YVV (

N

P

D N G N N

X

"



This work was supported by The Danish National Research Foundation. OL was supported by The Danish 1991 Pharmacy Foundation.

R N C X K G K L H F E K M H Q R R H Y A P M Q RX R

’

‘

W

WW

Ô

W

M

Æ

·

±

NS

«

EN S E K

²

¦

™

I

E

N

NFAL YWPP G RL E M R N T H F R G N GGE I MY F GP L I P D

D

¢

œ

š

Û

Õ

Q G DD

Ç

À

¬

CVC QVL Y C

S GLESP N X

¸

I

VV V F

DG

NK

Acknowledgments

D L

D K DKNNP GP KX

W Y FV Y FWV

C

–

I

Ü

N P

N G

W

V

×

X NPR N RP

G

IF

G

Á

¹

§

—

P G

Saliency

K G H

G X H P

N

D C

Y Y

Y

N I

Y W W X

W V

E

Saliency

G

M

R P S F K V Y N X W D F D V V K W K N N Y

H

G G

X H

H T E

N

M

C I F KG N W E W L W E LWC ES R TW FWV CQTQ F C AYV Q H C C K

í

0

H

W

L

F L M

‰

1

G

Q F H

F

C

T

º

“

Š

2

R

Q

F

H

C

N G N C

| |

C I

˜

‹

3

L A

„ † M E E M

K V X X

R

Y

KG D



Œ

4



Q K

R

F R F W K K V Y V F V Y L

y F n X W

D

F

C

0

K

Y W

K

I

| | |

W

G o

l

‡

D

C

S DTS D N H

Saliency

D

p

K Q I

R

E

I

I

…

X N P S D N

M z

f

W

X X F

‚

W R F A V A R A V

u

Y E

M L

`

X

X

V

I

L

E QX C T XS G C

I

\

M

Y

X

R



{

K

Y E Q S N P X H

H

Q

ƒ

D

CT SS D

]

E

K R

Q

S

R

T

A

V

S R

Q

~

N

C

V

E

I

E T S A

T I Y V A L M I F I Y F E D Y S

Q S W E

€

CTSS

L F Q M I L W F I R T

|

q

h

I R I A

K I M E v

E M g

U

I

w

L

a

^

r

i

A

Z

I

LL

x

PG

PPCXX CH H C HP G

1

P

O

T L

C

M

F

L

}

Q

s

m

b

D

T

T

E

V

A M

t

L

j

c

S

0

TD

d

M KQQL AL QLRRM MA

A A M WM QV Q KE E M N

CCC

A

e

Q R

A WL X

A A

K

_

[

YG Y

Q k

W

P

GGGG

1

PPPPP PP P

|

2

predicted separation 3 as having optimal performance. We found that the information content of the logos has the same qualitative behavior as the network performance for increasing sequence separation. Finally, the weight composition of the network was analyzed and we found that the sequence motif appearing was in agreement with the sequence logos. The perspectives are many: the network performance may be improved further by using three windows for sequence separations between 20 and 30 residues. The performance of the network can be used as inputs to a new collection of networks which can clean up certain types of false predictions. Combining the method with a secondary structure prediction should give a significant performance improvement on the short sequence separations. The relation between information content and network performance might be explained quantitatively through extensive algebraic considerations.

Figure 9: Three saliency-weight logos from the trained neural network at sequence separations 9 and 13. For each position of the input field the saliency of the respective amino acids is displayed. Letters turned upside down contribute negatively to the weighted sum of the network input. The top logo (sequence separation 9) illustrates strong punishment for proline and glycine upstreams. On the middle logo (sequence separation 13), N is turned upside down on the peaks near the edges. On the bottom logo the I’s close to the edges contribute positively, and the N in the center peak also contribute positively. The I’s in the center peak contributes negatively (they are upside down). This figure is available in full color at http://www.cbs.dtu.dk/services/distanceP/.

Final remarks We studied prediction of distance constraints in proteins. We found that sequence motifs were related to the distribution of distances. Using the motifs we showed how to construct an optimal neural network predictor which improve an earlier approach. The behavior of the predictor could be completely explained from a statistical analysis of the data, in particular a drop in performance for sequence separation 12 residues was predicted from the statistical analysis. We also correctly

References Baldi, P., and Brunak, S. 1998. Bioinformatics — The machine learning approach. Cambridge Mass.: MIT Press. Bernstein, F. C.; Koetzle, T. G.; Williams, G. J. B.; Meyer, E. F.; Brice, M. D.; Rogers, J. R.; Kennard, O.; Shimanouchi, T.; and Tasumi, M. 1977. The protein data bank: A computer based archival file for macromolecular structures. J. Mol. Biol. 122:535–542. (http://www.pdb.bnl.gov/). Bishop, C. M. 1996. Neural Networks for Pattern Recognition. Oxford: Oxford University Press. Bohr, H.; Bohr, J.; Brunak, S.; Cotterill, R. M. C.; Fredholm, H.; Lautrup, B.; and Petersen, S. B. 1990. A novel approach to prediction of the 3-dimensional structures of protein backbones by neural networks. FEBS Lett. 261:43– 46. Branden, C., and Tooze, J. 1999. Introduction to Protein Structure. New York: Garland Publishing Inc., 2nd edition. Brunak, S.; Engelbrecht, J.; and Knudsen, S. 1991. Prediction of human mRNA donor and acceptor sites from the DNA sequence. J. Mol. Biol. 220:49–65. Creighton, T. E. 1993. Proteins. New York: W. H. Freeman and Company, 2nd edition. Dayhoff, M. O. ans Schwartz, R. M., and Orcutt, B. C. 1978. A model of evolutionary change in proteins. Atlas of Protein Sequence and Structure 5, Suppl. 3:345–352. Fariselli, P., and Casadio, R. 1999. A neural network based predictor of residue contacts in proteins. Prot. Eng. 12:15– 21. Göbel, U.; Sander, C.; Schneider, R.; and Valencia, A. 1994. Correlated mutations and residue contacts in proteins. Proteins 18:309–317. Gorodkin, J.; Hansen, L. K.; Lautrup, B.; and Solla, S. A. 1997. Universal distribution of saliencies for pruning in

layered neural networks. Int. J. Neural Systems 8:489–498. (http://www.wspc.com.sg/journals/ijns/85_6/gorod.pdf). Hobohm, U.; Scharf, M.; Schneider, R.; and Sander, C. 1992. Selection of a representative set of structures from the Brookhaven Protein Data Bank. Prot. Sci. 1:409–417. Kabsch, W., and Sander, C. 1983. Dictionary of protein secondary structure: Pattern recognition and hydrogenbonded and geometrical features. Biopolymers 22:2577– 2637. Kullback, S., and Leibler, R. A. 1951. On information and sufficiency. Ann. Math. Stat. 22:79–86. Lund, O.; Frimand, K.; Gorodkin, J.; Bohr, H.; Bohr, J.; Hansen, J.; and Brunak, S. 1997. Protein distance constraints predicted by neural networks and probability density functions. Prot. Eng. 10:1241–1248. (http://www.cbs.dtu.dk/services/CPHmodels). Maiorov, V. N., and Crippen, G. M. 1992. Contact potential that recognizes the correct folding of globular proteins. J. Mol. Biol. 227:876–888. Mathews, B. W. 1975. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochem. Biophys. Acta 405:442–451. Mirny, L. A., and Shaknovich, E. I. 1996. How to derive a protein folding potential? A new approach to an old problem. J. Mol. Biol. 264:1164–1179. Miyazawa, S., and Jernigan, R. L. 1985. Estimation of effective interresidue contact energies from protein crystal structures: cuasi-chemical approximation. Macromolecules 18:534–552. Myers, E. W., and Miller, W. 1988. Optimal alignments in linear space. CABIOS 4:11–7. Olmea, O., and Valencia, A. 1997. Improving contact predictions by the combination of correlated mutations and other sources of sequence information. Folding & Design 2:S25–S32. Pearson, W. R. 1990. Rapid and sensitive sequence comparison with FASTP and FASTA. Meth. Enzymol. 183:63– 98. Reese, M. G.; Lund, O.; Bohr, J.; Bohr, H.; Hansen, J. E.; and Brunak, S. 1996. Distance distributions in proteins: A six parameter representation. Prot. Eng. 9:733–740. Rost, B., and Sander, C. 1993. Prediction of protein secondary structure at better than 70% accuracy. J. Mol. Biol. 232:584–599. Schneider, T. D., and Stephens, R. M. 1990. Sequence logos: a new way to display concensus sequences. Nucl. Acids Res. 18:6097–6100. Sippl, M. J. 1990. Calculation of conformational ensembles from potentials of mean force. An approach to the knowledge-based prediction of local structures in globular proteins. J. Mol. Biol. 213:859–83. Skolnick, J.; Kolinski, A.; and Ortiz, A. R. 1997. MONSSTER: A method for folding globular proteins with a small number-comment of distance restraints. J. Mol. Biol. 265:217–241. W

Tanaka, S., and Scheraga, H. A. 1976. Medium- and long range interaction parameters between amino acids for predicting three-dimensional structures of proteins. Macromolecules 9:945–950. Thomas, D. J.; Casari, C.; and Sander, C. 1996. The prediction of protein contacts from multiple sequence alignments. Prot. Eng. 9:941–948.