Multiple sequence alignment (MSA) is a prerequisite for virtually all comparative
sequence analyses, including phylogeny reconstruction, functional motif or domain characterization,
sequence-based structural alignment, inference of positive selection, and profile-based homology
searches. All such analyses take the MSA input for granted, regardless of uncertainties in the
alignment. Since errors in upstream stages tend to cascade downstream, alignment errors are an important
concern in molecular data analysis.
The GUIDANCE web server is a powerful and user-friendly tool for assigning confidence scores to each
residue, column, and sequence in an alignment and for projecting these scores onto the MSA [1]. The server identifies columns and sequences that are unreliably aligned and
enables their automatic removal from the MSA, in preparation for downstream analyses.
Three algorithms for quantifying MSA uncertainty are implemented in the server. The GUIDANCE score is
based on the robustness of the MSA to guide-tree uncertainty and relies on the bootstrap approach [2]. The Heads-or-Tails (HoT) score measures alignment uncertainty due to
co-optimal solutions [3].
GUIDANCE3/GUIDANCE2 is an integrative methodology that accounts for: (1) uncertainty in the process of indel
formation;
(2) uncertainty in the assumed guide tree; (3) co-optimal solutions in the pairwise
alignments used as building blocks in progressive alignment algorithms (as HoT).
GUIDANCE is meant to be used for weighting, filtering, or masking unreliably aligned
positions in sequence alignments before subsequent analysis. Analyses
where GUIDANCE filtering could be useful include phylogeny reconstruction, reconstruction of the history
of specific insertion and deletion events, inference of recombination events, etc.
GUIDANCE2 and GUIDANCE3 also provide a set of alternative alignments that can be used when adopting a
statistical point of view, i.e., performing statistical analyses that rely on many possible alignments
that are supported by the data.
The set of alternative MSAs can also be concatenated to improve phylogenetic accuracy by averaging
over alignment uncertainty. Additionally, selecting the best predicted MSA using the GUIDANCE3 deep
learning model (see Methodology) has been shown to improve phylogeny estimation
compared to using a single default alignment.
GUIDANCE scores cannot tell you which alignment is better. For example, align the same
sequences using either PRANK or MAFFT and assign GUIDANCE scores to both. If the PRANK alignment has an
average score of 0.8 while the MAFFT alignment has a score of 1, this does not mean that the MAFFT alignment is more
accurate. GUIDANCE measures the robustness of the alignment, so a perfect score means that MAFFT will
consistently align the sequences in the same way, regardless of the perturbations introduced by GUIDANCE.
Still, this alignment may be consistently wrong. Therefore, GUIDANCE scores cannot be used to choose between alternative
alignments. They can only be used to evaluate a given alignment produced by a specific alignment program and
identify columns for which the aligner is less confident relative to other columns in the same alignment.
GUIDANCE3 addresses this limitation through its deep learning–based MSA selection, which can identify
the most accurate alignment among the generated alternatives (see Methodology).
GUIDANCE is also not appropriate for evaluating an alignment produced by an approach other than those
supported in GUIDANCE (MAFFT, PRANK, and Clustal Omega). For example, you should not
run GUIDANCE on an alignment produced by T-COFFEE. Likewise, do not upload to GUIDANCE an alignment that you
corrected manually, even if it was originally produced by one of the supported aligners. Similarly,
alignments generated using special features (e.g., MAFFT alignment that uses RNA structure information) cannot
be evaluated by GUIDANCE. In general, we recommend always uploading unaligned sequences and avoiding
the option to upload aligned sequences.
GUIDANCE scores reflect the robustness of an alignment to perturbations.
For this goal, a standard MSA is first generated, hereby termed "base
MSA". The user may choose between MAFFT (the FFT-NS-1 variant) [5],
PRANK [6], and Clustal Omega [4]. The main idea
behind the GUIDANCE3 / GUIDANCE2 and GUIDANCE methodologies is to construct a set of MSAs. GUIDANCE uses bootstrap
trees as guide-trees to the alignment algorithm, and compare them to the base MSA in order to estimate
its confidence level (Figure 1). Similarly, GUIDANCE2 and GUIDANCE3 use bootstrap trees, vary the gap penalty score of
the alignment program scoring scheme, and employ HoT methodology (see details below).
Comparing the base alignment to the set of alternative alignments results in scores between 0-1 for each
residue, residue-pair, column and sequence of the MSA.
An in-depth description of the algorithm behind GUIDANCE can be found
in ref. [2].
FIGURE 1
A schematic flowchart of the GUIDANCE algorithm. A base MSA is produced by any progressive alignment
method. Bootstrap neighbor joining (NJ) trees are reconstructed and given as guide trees to the
progressive alignment program, producing a set of MSAs. GUIDANCE scores are then calculated by
comparing each MSA to the base MSA, and are color coded on each residue in the alignment.
HoT (Heads-or-Tails) scores measure the alignment uncertainty by
generating a set of co-optimal MSAs and comparing them to the standard
alignment. Co-optimal MSAs are a set of alignments that are given the
same maximal score by the alignment algorithm. The co-optimal MSAs set
is constructed by reversing the sequences at each of the
pairwise-profiles-alignment steps of the progressive alignment algorithm
[3]. The comparison results in scores between 0-1 for each residue, residue-pair,
column and sequence of the MSA.
While the GUIDANCE2 and HoT scores assess the reliability of a given alignment, they cannot determine
which of several alternative alignments is the most accurate. Most widely used MSA methods optimize the
sum-of-pairs (SoP) score, yet the alignment with the optimal SoP score often does not correspond to the
most accurate alignment when compared to a reference.
GUIDANCE3 addresses this limitation by incorporating a deep learning model that integrates a collection
of MSA features to identify the best alignment from the set of alternatives generated during the
bootstrap procedure. The model takes as input the full set of alternative MSAs produced from the same
unaligned sequences and predicts their relative ranking with respect to alignment accuracy. Across
simulated and empirical datasets, this approach more accurately identifies the top-ranking MSA than the
SoP score and several widely used alignment programs. Using simulations, selecting MSAs based on this
model has also been shown to lead to more accurate phylogenetic reconstructions.
The deep learning model is available both through the GUIDANCE3 web server and as a standalone
Command Line Interface (CLI) tool (guidance3-predict), which accepts a folder of
alternative MSAs and returns the predicted best alignment together with a ranked list of all
alternatives.
Running time depends on the dataset size (number and length of
sequences) and (for GUIDANCE/GUIDANCE2/GUIDANCE3 scores) on the number of bootstrap
repeats. The major component of the running time is the multiple
alignment program used, thus MAFFT runs will be fastest and PRANK runs
slowest. In Figure 2, a comparison between the running time of GUIDANCE2 and other MSA reliability
methods when using MAFFT as the alignment method.
Note that GUIDANCE2 and GUIDANCE were run
with the default 100 bootstrap repeats, but this number can be reduced
to shorten the running time. HoT running time depends on the number of
branches in the guide tree, which increases linearly with the number of
sequences. Zorro, TCS, ALiScore, TrimAl and NOISY were run on a pre-calculated MSA.
Please note that the GUIDANCE3 supports parallel computing and early convergence stopping,
thus a significant reduction in running times is possible. In addition, GUIDANCE3 uses IQ-TREE for guide-tree inference
(replacing SEMPHY used in GUIDANCE2), which natively supports multi-threading and further improves running time.
FIGURE 2: Time performance as a function of the sequence length.
Sets of 40 simulated protein sequences with different lengths were aligned
using MAFFT and analyzed by alignment reliability methods.
GUIDANCE directs you to a web page called "GUIDANCE Job Status
Page". This web page is automatically updated every 30 seconds, showing
messages regarding the different stages of the server activity. When the
calculation finishes, several links appear. For simplicity, we only
describe the output of the GUIDANCE method. Similar output is produced
by the HoT method, also implemented in this server.
Among other benchmarks used to evaluate the performance of GUIDANCE2 we used a benchmark of 541 simulated protein sequences. Sequences were simulated using INDELible. In order to have realistic parameters for the simulations, we first selected 541 MSAs from the OrthoMaM database, for which CDS are available for all 40 mammals included in the database. This parameter setup resulted in MSAs similar to OrthoMaM alignments (visual comparison of alignment length, number and length of indels). The control files for INDELible and the resulted simulated MSAs used as benchmark can be downloaded here.