They illustrate this problem quite well using the example of the frog puzzle (though I’m probably biased since I worked that puzzle many, many times growing up) – since all of the frogs are repeated, you can’t know which one goes where.  Likewise, if you have several 500 bp reads that all happen to be within a 1000 bp repeat, you have no way of knowing which read goes with which copy.

The original method, BAC-by-BAC sequencing, simplified the computation by reducing the number of repeats, but complicated the sequencing project.  The current method, mate-pair reads sequencing, attempts to ensure that each read has some kind of unique identifying information associated with it.  To do this, they use inserts of a given length (longer than the read and most repeats), and sequence both ends of the insert.  This makes it almost certain that at least one of the ends will contain a unique, non-repeated portion of the DNA.  Armed with these fragments, they describe the method for obtaining the DNA sequence.

Section 8.10 on protein sequencing and identification leans more toward being a biological overview of the subject and its challenges.  It alludes to some computational problems, but does not delve into any.  Several applications of protein sequencing and identification are given, as well as an intro to mass spectrometry.

Section 8.11 introduces the peptide sequencing problem.  The idea behind sequencing peptides seems to be to break the peptide into two parts (fragment it) and record the masses and ion types: in the fragmenting process peptides can lose molecules such as ammonia or water, and these partial peptides with pieces missing are called ion types.  If is a set of numbers representing the masses of possible chemical groups removed during fragmentation, then is called the set of ion types.

To sequence the peptides, two spectra are generated: given a known peptide, the theoretical spectrum is the set of masses obtained by subtracting all ion types from the masses of all partial peptides.  Conversely, not knowing the peptide, the experimental spectrum is a set of numbers obtained by mass spectrometry.  The goal of the peptide sequencing problem, then, is to find a peptide whose theoretical spectrum is the best match to some experimental spectrum.

The book mentions that a linear time algorithm exists to solve this problem. It then compares SBH to using DNA arrays as mentioned in the previous section. A main difference is that in SBH, the target DNA sequence is unknown while with DNA arrays, the target is either known or “almost” known. Of course, there are problems with the real life application of the algorithm, since it may be difficult to distinguish between perfect matches and highly stable mismatches.

In section 8.7, the book discusses SBH as a Hamiltonian path problem. The idea is to construct a directed graph by introducing a vertex for every l-mer in the spectrum and then connecting two vertices if they overlap. Two l-mers p and q overlap if the first l – 1 letters of p coincide with the last l – 1 letters of q. However, with complicated examples, there may be more than one Hamiltonian path, each corresponding with a different possible reconstruction.

In section 8.8, the book discusses SBH as an Eulerian path problem. This technique is more fruitful than the Hamiltonian path problem, since it can be solved using a linear time algorithm. This time, a graph is constructed whose edges, not vertices, correspond to l-mers from the spectrum. Then a path must be found that visits every edge exactly once. The book thoroughly describes how to produce such a graph on page 273. Finally, after proving that a connected graph is Eulerian if and only if each of its vertices is balanced, it describes the algorithm that is used to find an Eulerian cycle.

Section 8.3 – DNA sequencing

This section talks about fragment assmebly, a method of DNA sequencing in which one can determine an entire genome by cutting it up into thousands of little DNA fragments and then reassembling them based on overlapping patterns. One extremely successful method of DNA sequencing, and probably the most renowned, was developed by Fred Sanger. By leaving out one of the four bases (A, G, C, or T) when making copies of the unknown DNA sequence, and of course running this experiment once for each base, Sanger discovered that he could in fact copy different length fragments of the DNA. He then organized these fragments by length, and read off the sequence accordingly. The way it sort of works is: each location of a particular base (which ever is left out of the experiment) is highlighted by the fragments that are created. For example, the sequence ACGTAAGCTA is cut into the fragments ACG and ACGTAAGC whenever T is left out of the experiment, highlighting that the base following G (in the first fragment) and C (in the second) is a T. If we took the same sequence, ACGTAAGCTA, and left out G, we would get back the fragments AC and ACGTAA. Once we’ve run the experiment four times, we look at all of our ”ladder” of fragments and can reconstruct the original sequence based off of the overlapping patterns seen. Though the process seems simple enough, it does become a bit complicated because billions of fragments must be measured in order to read the ladder.

8.4 – Shortest Superstring Problem

The sequence that contains all of the information from the readings is know as the superstring. There can be more than one superstring for any given set of data, but the one most useful to us is the shortest, in which no information is duplicated. For example, given the set of strings {000, 001, 010, 011, 100, 101, 110, 111}, we can create a superstring by simply linking each fragment one after the other to create the sequence 000 001 010 011 100 101 110 111. However, by taking into account any overlapping info from our original set of eight strings, we can see that the string 0001110100 also contains all information from the orginal set and it is presented in a much more simplified manner. This shortest superstring turns out to be a reasonable first guess at the unknown genomic DNA sequence. 

8.5 – DNA Arrays as an Alternative Sequencing Technique

An alternative sequencing technique, Sequencing by Hybridization, was developed in an attempt to overcome the time-consuming nature of the Sanger method. This method involves constructing a mini DNA array, or chip, that contains thousands of short DNA fragments, called probes. A flourescently labeled strand of DNA is then applied to the array. Those probes which are complimentary to a substring of the added target strand will hybridize, or weakly bond to the target strand at that location. For example, the probe ACCGTGGA would hybridize to the target CCCTGGCACCTA where it is bolded, because ACCGTGGA is complimentary to TGGCACCT. This method of sequencing didn’t really takeoff until1991 with Fodor’s light-directed polymer synthesis, because many were hesitant that creating a bunch of probes and assembling arrays in order to sequence the DNA might  not be any more efficient than Sanger’s method. Yet with Fodor’s technique, building an array with  probes of length  requires only  reactions, rather than the expected .

Section 8.1 is just an introduction to what graphs are, which in our case should be more of a recap. As expected, the book did explain basic terms such as vertices, edges, degree of vertex, indegree, outdegree,and connected and disconnected graphs. I do not think there is a need to define any of them here, but if anyone wants me to, just make a comment, and I will be sure to reply.  The book also included very popular problems involving graphs such as the bridge problem, the Hamiltonian cycle problem, and others similar to them. I am guessing most of us have heard and practiced these problems at least once in our lives.

Section 8.2 gave a nice description on how Seymour Benzer used graphs and graph theory, along with his experiment on bacteria to show that genes are linear.

In Section 7.5 is some history, which I’ll leave to you to read.

Secion 7.1 revisits the Sorting problem from chapter 2, when we looked at an algorithm requiring quadratic time.  With the new divide-and-conquer approach, we will be able to find an algorithm requiring  time.  It looks like there are two keys to this divide-and-conquer sorting algorithm: first, if we’re given two sorted lists of integers it is relatively easy () to combine the two into one sorted list; second, a single-element list is, by definition, already sorted. Therefore it makes sense to construct a recursive algorithm which breaks down a list of integers into single-element lists, then take these sorted lists and repeatedly combine them to form the desired sorted list.

The algorithms themselves are fairly straightforward, and I can follow the first explanation of the  running time, but given the recurrence relation on page 230 I don’t think I could have come up with  on my own.  The second explanation is more intuitive, but the semantics in the last paragraph of section 7.1 seemed inconsistent.  He uses the term “recursion tree” to refer to the entire graph (which is not a mathematical tree, based on my understanding) in Figure 7.1, but goes on to state that it has 3 levels, which would indicate that either the Divide or the Conquer portion is the actual recursion tree.   Also, it looks like he uses “top level” (line 4) to refer to the level that is, strictly speaking, the second level from the bottom.  For those more familiar with computer science, is this typical?

In 7.2, the goal is to reduce the space complexity of the sequence alignment algorithm.  It is important to note that the algorithm presented will only compute the score of an alignment, not the alignment itself.  The implication seems to be that if we need the actual alignment, the space complexity remains .

Previously, in order to find the longest path in an edit graph, we needed the entire backtracking matrix, which again requires  space.  In order to do this, we need to come up with a way to find the longest path in the edit graph without using a backtracking matrix.  The new technique depends on the fact that we can find the middle vertex of the longest path without actually knowing the longest path – all we need to do is find the length of the longest path from the source to each vertex in the middle column (prefix(i), where i indexes each vertex in the middle column), and the length of the longest path from each vertex in the middle column to the sink (suffix(i)).  Then, since the length of the longest path from source to sink passing through vertex i in the middle column is length(i) = prefix(i) + suffix(i), maximizing length(i) over all i gives the length of the longest path from source to sink and, more importantly, determines the middle vertex in this path.

It took me a while to understand why this reduces the space requirement – i.e., why no backtracking matrix is required – but I think the key is that in finding this middle vertex, all we need are the lengths of two longest paths.  Since we don’t need the actual paths, we don’t need a backtracking matrix.

It’s easy to see that continuing this process – repeatedly finding middle vertices – will eventually give us each vertex in the longest path, hence the path itself.  Since this process must terminate, and each stage requires time equal to the rectangles to the left and right of the current middle column, the time required to compute the longest path is a partial sum of the geometric series with common ration 1/2 and first term equal to the area of the edit graph, nm, hence is less than 2nm.  So this algorithm requires  time.  We’re also told that it requires  space, since to compute the alignment scores in a column we only use the scores from the preceding column, so we only need 2n values.

Section 7.3 revisits the Longest Common Subsequence problem and develops an algorithm with running time, using block alignment and the Four-Russians technique.  As a side note, I looked up an article on the Four-Russians technique to find out why it’s called that – it turns out that the technique comes from a paper by four authors, but only one is Russian…

Block alignment of two DNA sequences – in our case, both are assumed to have length n – is simply dividing the grid formed by the sequences into subgrids of size t x t, where t divides n.  A block path is one which enters and leaves each square at the corners.  The block alignment problem is to find the longest block path through the graph.  To do this, we must find the alignment score for each of the squares, which means solving alignment problems, then piecing together the alignment scores to find the longest path.  Correct me if I’m wrong, but I think this ends up requiring time. However, we’re looking for an algorithm with subquadratic time.

This is where the Four-Russians technique comes in, and in order to apply it we must have .  The key lies in constructing minialignments and storing each alignment score in a lookup table – if we assume  (which, since it’s a scalar multiple I think that for running time this counts as ), this lookup table will have only n entries.  The running time calculations make use of :

Computing the score for each t x t minialignment takes time, so with n entries it takes running time.  Apparently, though, the overall running time is dominated by the accesses (could someone explain why there’s that number of accesses?), and each access takes time, so the overall running time is .

They model a putative exon with a weighted interval. Its weight is supposed to reflect the actual biological likelihood that the interval is an exon. They display the problem using a graph like the one seen on page 202. Again, the dynamic programming technique is utilized, and the longest path ending at vertex  is determined. This is repeated for every vertex from 1 to 2n with being the solution to the problem.

Of course, a common theme is that this algorithm has problems with its biological application. For instance, the endpoints of the putative exons are not well defined. Also, the optimal chain may not actually be a valid alignment.

In section 6.14, they discuss the spliced alignment approach. In it, a related protein within one genome is used to reconstruct the exon-intron structure of a gene in another genome. They use the example of “It was brilliant thrilling morning and the slimy, hellish, lithe doves gyrated and gambled nimbly in the waves” as the genome sequence and Lewis Carroll’s line “twas brillig, and the slithy toves did gyre and gimble in the wabe” as the target sequence.

They then discuss an algorithm that computes the similarity score between the i-prefix of the spliced alignment graph and the j-prefix of the target sequence. The score is computed until the end of the sequence, at which point the maximum over all possible blocks is the solution. Again this has problems, one of which is the mosaic effect. If the number of short blocks is high, then chains of these blocks can replace actual exons in spliced alignments. This occurs because it is more probable to create a long string from many short strings than from the same number of longer strings. To avoid these problems, the book suggests that candidate exons should be subjected to some filtering procedure.

In section 6.15, they present an interesting history on dynamic programming algorithms.

6.11 – Gene Prediction

During transcription, DNA is used as a  template to create a corresponding strand of mRNA. Like DNA, mRNA is simply a string of nucleotides (the only difference being that Uracil is used instead of Thymine). In this mRNA strand, each segment of 3 nucleotides is referred to as a triplet or codon. In 1961, Brenner and Crick found that each of these codons, or 3-nucleotide segments, codes for 1 amino acid. These amino acids then basically string together to form a protein. Yanofsky went on to prove that the gene and its protein product are colinear (meaning that the first codon codes the first amino acid, the second codon codes for the second amino acid, and so on).

And so it was up until Sharp and Roberts’ discovery of split genes in 1977, that most everyone in the biology world thought that proteins were merely encoded by a long string of triplets, each one directly after the other. Sharp and Roberts were studying a particular protein, found in adenovirus, known as hexon. They discovered that this protein was built from 4 separate fragments of the adenovirus genome. These 4 continuous fragments are known as exons, and contain all of the gene’s to-be-expressed information. The segments between these exons are called introns, and contain all of the information not expressed, or the “junk”. The book compares these split genes to a magazine article that begins on the cover page and then continues on page 13 and then picks up again on, say, page 40.

Now, this intron-exon model does not exist in prokaryotic organisms which makes gene prediction for prokaryotes considerably easier  than predicting gene location in eukaryotes. To complicate things even further for gene prediction in eukaryotes: 1) the complexity of the organism is not a direct indication of the complexity of its genome (as the book mentioned,  the salamander has genome 10 times larger than the human, indicating that the salamander’s genome contains a great deal more of the “junk DNA”) and    2) the jumps between different parts of split genes differ from species to species (i.e. the information found on one exon in the human genome may be found split across two exons in the related gene of a mouse).

As for methods of going about predicting gene location, we are introduced to two main approaches. First is the statistical approach, which looks for features that appear frequently in genes and infrequently elsewhere. We will take a closer look at this approach shortly. Second (and perhaps the more promising of the two), we have the similarity-based approach, which is based on the idea that a newly sequenced gene has a decent chance of being related to an already known gene. The similarity-based approach is covered in greater depth in section 6.13.

Section 6.12 – Statistical Approaches to Gene Prediction

In this section, we look at attempting to locate a gene by detecting small statistical variations between the exons and introns in our mRNA strand. One method, and seemingly the simplest, is to look at Open Reading Frames (ORFs). An ORF is any segment of the mRNA strand that begins with the start codon, ATG, and ends with one of the three “stop” codons (TAA, TAG, or TGA). ORFs can overlap due to the three different “reading frames” (starting at positions 1, 2, and 3) and the additional three “reading frames” on the reverse strand. Typically, “junk DNA” contains a considerably greater amount of the stop codons than does the coding-region of DNA. Thus, any ORF that is reasonably long enough (meaning there were not a considerable amount of stop codons present) could indicate a potential gene. Unfortunately, searching for lengthy ORFs may cause any short genes or genes with short exons to go unnoticed.

Biases in codon usage is one of the statistical features used in many of the statistical gene prediction algorithms. With 64 possible triplet combinations and only 20 standard amino acids, it is obvious that more than one triplet or codon will code for a particular amino acid. As such, merely knowing the sequence of amino acids tells us very little about the sequence of nucleotides composing our mRNA. This is where using the statistical biases comes into play. The codon usage array (page 199) shows which of those triplets is most likely to occur. Note that the codon usage arrays differ for coding and noncoding-DNA, as well as from species to species. As I understand it, if we are given an ORF containing the amino acid Arg, and by consulting our usage array we see that CGC codes for Arg 38% of the time and AGG codes for Arg only 3% of the time, then it is reasonable to assume that our mRNA strand more likely contains the triplet CGC. Sliding down our genomic sequence, we can often relate peaks in likelihood to genes. Unfortunately, however, this and other methods, such as Borodovsky’s in-frame hexamer count, though beneficial for gene prediction in prokaryotes, are often greatly complicated by the eukaryote’s exon-intron model. Often exons in eukaryotes are too short to produce any reliable peaks in the likelihood plot. As a result, many researchers have turned to studying splicing signals at exon-intron junctions using Hidden Markov Models, such as GENSCAN developed in 1997 by Burge and Karlin. This too, though, is not always completely accurate.

Section 6.9 touches on affine gap penalties as an example of a nicer approach to penalizing gaps in alignments. According to the book, it is a linearly weighted score for large gaps. It is easy and common sense to make adjustments on edit graphs (discussed in previous sections) for alignments with affine gap penalties by adding vertical and horizontal edges the length of the gap score from any vertex to another south or east of it. Also, increasing the number of edges in the edit graph does increase the running time for the alignment algorithm but not by a whole lot. Lastly, the book does explain how to reduce alignments with affine gap penalties to the Manhattan Tourist problem.

Section 6.10 introduces the idea of multiple alignment as opposed to pairwise alignment in cases where pairwise alignment is not always the best way to find biologically related sequences, especially when sequences have weak similarities between them.  To be honest, it was not necessary to include that much to define multiple alignment, I would have understood it as well if they said it was similar to pairwise alignment, but with more  than two sequences, and no column should contain only spaces. The rest of this section shows different methods to finding out the string with the most common characters in each column of the alignment, their flaws, and any improvements.  A programming algorithm involving k-dimensional edit graph would solve it, but the running time increases by a lot as k gets bigger. Computing optimal pairwise alignments between every pair of strings and then combine them together into a multiple alignment would not really work because some pairwise alignments are incompatible.

In fact, all the approaches discussed in this section have one issue or the other, but at least they work. This just shows that it is difficult to attain perfection when it comes to algorithms. However, there is always room for improvement, and as time goes on, someone always comes up with a new idea that makes everything better.

The overarching goal of these 3 sections is to find a way to measure how ‘similar’ two strings of either nucleotides or of amino acids is, continuing the ideas from the previous sections. The real mathematical difficulty here is to capture the notion of similarity with a mathematical definition which is both robust and efficiently computable.

In sections 6.4 and 6.5 we considered adding ‘gaps’  to the Hamming distance to get a better notion of the distance between two strings. In 6.6, we consider a generalization of this method. Let δ be a (k+1) x (k+1) matrix, called the scoring matrix, where k is the number of letters in the alphabet we are using (whether they be nucleotides or amino acids). We’ll abuse notation a bit and write δ(x,y) for the entries of δ, where x and y are either letters or the gap character ‘-’.

The idea is that we’ll parse through the strings as we did before, either diagonally or treating the character as a gap, and at each step we add δ(x,y) to a running total, where x and y are characters extracted from the respective strings, or the gap. For a particular way to parse the strings, we get a number, which I’ll call the similarity of the two strings, and then we minimize that number over all possible paths. Algorithmically, there’s a nice way to do this, which I won’t discuss since it’s rather prominently placed in the chapter.

While this certainly is better than any of the previous methods, and is probably in a form that is usable as is, it isn’t terribly satisfying to me. At the very least, using it to predict the probability of a particular mutation occurring over a long period of time is certainly not very accurate, so there’s a question as to whether this represents the actual “biological similarity” of the sequences (if such a thing actually exists). To explain better, I’d need to draw an analogy to statistical mechanics, which I expect won’t be familiar to most people here. If there’s any interest in this analogy (which I think is pretty interesting, but extremely tangential) then I can post it in the comments. In the analogy, the similarity we have come up with corresponds to the limit that the ‘temperature’ (analogous to mutation rate) goes to 0. On the other hand, my idea is really of only theoretical interest, since it doesn’t lend itself to an algorithmic solution.

A special case of this solution is the  case where δ has diagonal elements 1 in all but the ‘-’ row and column, -μ off-diagonal elements in these rows and columns, and -σ in the last row and column, for real μ, σ (typically nonnegative). This doesn’t seem to actually speed up the algorithm significantly, but it certainly reduces the amount of memory required. It’s useful for DNA sequence comparisons, but less so for protein sequences because these can often change quite a lot and still be qualitatively very similar.

An observation I made when reading is that, if the diagonal elements (other δ(-,-), which is unimportant) are all equal and positive, and if δ is symmetric, and all the off-diagonal elements are less than the diagonal elements, then given the set of strings of fixed length n, we can do an affine transformation on the similarity function to make it into a metric space. If we allow the affine function do also be a function of the lengths of the two strings (affine in each of these parameters as well) then we can make the set of all strings into a metric space. The assumptions all seem somewhat reasonable, but I’m no expert in biology. I don’t know if this is useful, but metric spaces are nice enough things that it’s plausible that it has some application.

Section 6.7 deals with the problem of comparing protein sequences. They first explain an algorithm that one might think up to compute good values of δ. The big problem with this “algorithm” is that it needs δ to compute δ. So if we don’t already know what δ should be, the algorithm won’t actually give us δ. We might expect that this means it’s useless, but in fact in many cases we can start with a guess, and if it’s good enough the algorithm will make it better. We can think of the algorithm as a continuous function , and study the dynamical system generated by repeated application of . Our desired δ is a fixed point of . If it is attracting on some “large” subset of then the algorithm applied iteratively will converge to δ if we start with a member of this “large” subset. There are several interesting analytical issues here (most notably proving that the fixed point is attracting), but this is digressing a lot. I did try to prove this for about 5 minutes, but wasn’t successful.

Section 6.7 also gives an alternate way to get a good δ, which is essentially based on statistical analysis of protein sequences. It should be fairly straightforward to understand, and I don’t have much to say about it.

Section 6.8 discusses the problem of finding substrings of two given strings which are very similar. In many cases, we can’t expect the strings to be similar overall, but certain parts should be very similar. This is called the local alignment problem (as opposed to the global problem). The problem has the same complexity as the global problem. I honestly didn’t find that too surprising, but apparently the authors did. It only requires a small modification of the method from section 6.6, so I’ll leave out the in-depth description since everyone will read it in the book.

In all honesty, this solution isn’t terribly mathematically elegant to me, because I don’t see any reason to expect that the similarities of pairs of strings of different lengths are comparable the way we’ve defined it. If anyone sees a good argument as to why they should be, that’d be interesting.

These algorithms are probably robust enough to work for biological data, but I expect more specialized algorithms will outperform them most of the time.