1 of 25

Lecture 8

2 of 25

Importing the data into Galaxy

Click the new-history icon at the top of the history panel.

Click on the   icon (History options) on the top of the history panel

Select the option Create New from the menu

Renaming a history

Click on   (Edit) next to the history name (which by default is “Unnamed history”)

Type the new name

Click on Save

3 of 25

Import the sample FASTQ files to your history

https://zenodo.org/record/800651/files/F3D0_R1.fastq

https://zenodo.org/record/800651/files/F3D0_R2.fastq

https://zenodo.org/record/800651/files/F3D141_R1.fastq

https://zenodo.org/record/800651/files/F3D141_R2.fastq

https://zenodo.org/record/800651/files/F3D142_R1.fastq

https://zenodo.org/record/800651/files/F3D142_R2.fastq

https://zenodo.org/record/800651/files/F3D143_R1.fastq

https://zenodo.org/record/800651/files/F3D143_R2.fastq

https://zenodo.org/record/800651/files/F3D144_R1.fastq

https://zenodo.org/record/800651/files/F3D144_R2.fastq

https://zenodo.org/record/800651/files/F3D145_R1.fastq

https://zenodo.org/record/800651/files/F3D145_R2.fastq

https://zenodo.org/record/800651/files/F3D146_R1.fastq

https://zenodo.org/record/800651/files/F3D146_R2.fastq

https://zenodo.org/record/800651/files/F3D147_R1.fastq

https://zenodo.org/record/800651/files/F3D147_R2.fastq

https://zenodo.org/record/800651/files/F3D148_R1.fastq

https://zenodo.org/record/800651/files/F3D148_R2.fastq

https://zenodo.org/record/800651/files/F3D149_R1.fastq

https://zenodo.org/record/800651/files/F3D149_R2.fastq

https://zenodo.org/record/800651/files/F3D150_R1.fastq

https://zenodo.org/record/800651/files/F3D150_R2.fastq

4 of 25

Import Reference DataImport the following reference datasets

https://zenodo.org/record/800651/files/HMP_MOCK.v35.fasta

https://zenodo.org/record/800651/files/silva.v4.fasta

https://zenodo.org/record/800651/files/trainset9_032012.pds.fasta

https://zenodo.org/record/800651/files/trainset9_032012.pds.tax

https://zenodo.org/record/800651/files/mouse.dpw.metadata

5 of 25

Organizing our data into a paired collection

Click on the checkmark icon param-check at top of your history.

Select all the FASTQ files

Tip: type fastq in the search bar at the top of your history to filter only the FASTQ files;

Click on All selected

Select Build List of Dataset Pairs from the dropdown menu

Click on Auto-pair to create the suggested pairs.

Or click on “Pair these datasets” manually for every pair that looks correct.

6 of 25

Name your collection at the bottom right of the screen

You can pick whatever name makes sense to you

Click the Create Collection button.

A new dataset collection item will now appear in your history

Quality Control

Make.contigs with the following parameters

“Way to provide files”: Multiple pairs - Combo mode

“Fastq pairs”: the collection you just created

Leave all other parameters to the default settings

7 of 25

Data Cleaning

As the next step, we want to improve the quality of our data.

Summary.seqs with the following parameters

“fasta”: the trim.contigs.fasta file created by Make.contigs tool

“Output logfile?”: yes

Start End NBases Ambigs Polymer NumSeqs

Minimum: 1 249 249 0 3 1

2.5%-tile: 1 252 252 0 3 624

25%-tile: 1 252 252 0 4 6235

Median: 1 252 252 0 4 12470

75%-tile: 1 253 253 0 5 18705

97.5%-tile: 1 253 253 6 6 24316

Maximum: 1 502 502 247 243 24939

Mean: 1 252.74 252.74 0.72128 4.46445

# of Seqs: 24939

The Start and End columns tell us that the majority of reads aligned between positions 1 and 252, which is what we expect to find given the reference file we used. However, some reads align to very different positions, which could indicate insertions or deletions at the terminal ends of the alignments or other complicating factors

8 of 25

We do this data cleaning using the Screen.seqs tool, which removes

sequences with ambiguous bases (maxambig) and

contigs longer than a given threshold (maxlength).

Screen.seqs with the following parameters

“fasta”: the trim.contigs.fasta file created by Make.contigs tool

“group”: the group file created in the Make.contigs tool step

“maxlength”: 275

“maxambig”: 0

9 of 25

Optimize files for computation

Microbiome samples typically contain a large numbers of the same organism, and therefore we expect to find many identical sequences in our data. In order to speed up computation, we first determine the unique reads, and then record how many times each of these different reads was observed in the original dataset. We do this by using the Unique.seqs tool.

Unique.seqs with the following parameters

“fasta”: the good.fasta output from Screen.seqs tool

“output format”: Name File

10 of 25

This step has greatly reduced the size of our sequence file; not only will this

speed up further computational steps, it will also greatly reduce the amount

of disk space (and your Galaxy quota) needed to store all the intermediate files generated during this analysis. This Unique.seqs tool created two files, one is a FASTA file containing only the unique sequences, and the second is a so-called names file. This names file consists of two columns, the first contains the sequence names for each of the unique sequences, and the second column contains all other sequence names that are identical to the representative sequence in the first column.

To recap, we now have the following files:

  • a FASTA file containing every distinct sequence in our dataset (the representative sequences)
  • a names file containing the list of duplicate sequences
  • a group file containing information about the samples each read originated from

To further reduce file sizes and streamline analysis, we can use the Count.seqs tool to combine the group file and the names file into a single count table.

11 of 25

Count.seqs with the following parameters

“name”: the names output from Unique.seqs tool

“Use a Group file”: yes

“group”: the group file we created using the Screen.seqs

Count.seqs tool summarizes the number of times each unique sequence was observed across each of the samples

From now on, we will only work with the set of unique sequences, but it’s important to remember that these represent a larger number of total sequences, which we keep track of in the count table.

In the following we will use the unique sequences together with the count table as input to tools instead of the complete set of sequences. If this is done for the Summary.seqs tool tool it will report both the number of unique representative sequences as well as the total sequences they represent.

12 of 25

Sequence Alignment

Alignments are a powerful way to compare related DNA or protein sequences.

They can be used to capture various facts about the sequences aligned, such as

common evolutionary descent or common structural function. We take the general view that the alignment of letters from two or multiple sequences represents the hypothesis that they are descended from a common ancestral sequence.

Sequence comparison is a crucial aspect of bioinformatics analysis that involves comparing newly determined biological sequences with previously known sequences stored in databases.

Sequence alignment is considered the most essential step in comparing biological sequences. Sequence alignment arranges two or more nucleotide or amino acid sequences to identify regions of similarity between the sequences. These regions of similarity are helpful in understanding the functional, structural, and evolutionary relationships between the sequences.

13 of 25

Two commonly used sequence alignment algorithms are global alignment and local alignment.

Global alignment: Global alignment is a method of comparing two sequences, which aligns the entire length of the sequences by maximizing the overall similarity. This method is used when comparing sequences that are of the same length. 

Local alignment: In local alignment, instead of attempting to align the entire length of the sequences, only the regions with the highest density of matches are aligned. This is useful for identifying short conserved regions in protein or nucleotide sequences.

14 of 25

Pairwise Alignment

  • Pairwise sequence alignment is the type of sequence alignment that involves aligning two sequences to identify the optimal pairing of the sequences.
  • It is based on a scoring system that assigns positive scores to matching characters and negative scores to mismatching characters or gaps. 
  • The main objective of pairwise sequence alignment is to obtain the highest possible score, which indicates the degree of similarity between the two sequences.

Multiple Sequence Alignment

  • Multiple Sequence Alignment involves aligning multiple (three or more) biological sequences to achieve optimal sequence matching. 
  • Multiple sequence alignments are used to identify conserved sequence regions and to construct phylogenetic trees, which help us understand the functional and evolutionary relationships between different species or groups of organisms.

15 of 25

Align.seqs with the following parameters

“fasta”: the fasta output from Unique.seqs tool

“reference”: silva.v4.fasta reference file from your history

There are different alignment characters in this output:

.: terminal gap character (before the first or after the last base in our query sequence)

-: gap character within the query sequence

Summary.seqs with the following parameters:

“fasta”: the align output from Align.seqs tool

“count”: count_table output from Count.seqs tool

“Output logfile?”: yes

16 of 25

More Data Cleaning

1. Remove any reads not overlapping the region V4 region (position 1 to 252) using Screen.seqs tool.

2. Remove any overhang on either end of the V4 region to ensure our sequences overlap only the V4 region, using Filter.seqs tool.

3. Clean our alignment file by removing any columns that have a gap character (-, or . for terminal gaps) at that position in every sequence (also using Filter.seqs tool).

4. Remove redundancy in the aligned sequences that might have been introduced by filtering columns by running Unique.seqs once more.

5. Group near-identical sequences together with Pre.cluster tool. Sequences that only differ by one or two bases at this point are likely to represent sequencing errors rather than true biological variation, so we will cluster such sequences together.

17 of 25

Screen.seqs with the following parameters

“fasta”: the aligned fasta file from Align.seqs tool

“start”: 1

“end”: 252

“maxhomop”: 8

“count”: the count table file from Count.seqs

Filter.seqs with the following parameters

“fasta”: good.fasta output from the latest Screen.seqs tool

“vertical”: yes

“trump”: .

“Output logfile”: yes

18 of 25

Unique.seqs with the following parameters

“fasta”: the filtered fasta output from Filter.seqs tool

“name file or count table”: the count table from the last Screen.seqs

19 of 25

Import the sample FASTQ files to your history

https://zenodo.org/record/815875/files/SRR531818_pampa.fasta

https://zenodo.org/record/815875/files/SRR651839_anguil.fasta

https://zenodo.org/record/800651/files/silva.v4.fasta

https://zenodo.org/record/815875/files/trainset16_022016.pds.fasta

https://zenodo.org/record/815875/files/trainset16_022016.pds.tax

20 of 25

Merge.files tool with the following parameters

    • “Merge” to fasta files
    • “Inputs” to the two sample fasta files

Make.group tool with the following parameters

    • “Method to create group file” to Manually specify fasta files and group names
    • “Additional”: Add two elements to this repeat
      • First element
        • “fasta - Fasta to group” to SRR531818_pampa file
        • “group - Group name” to pampa
      • Second element (click on “Insert Additional”)
        • “fasta - Fasta to group” to SRR651839_anguil file
        • “group - Group name” to anguil

21 of 25

Unique.seqs tool with the following parameters

“fasta” to the merged fasta file

“output format” to 

Count.seqs tool with the following parameters

“name” to the name file from Unique.seqs

“Use a group file” to yes

“group” to the group file from Make.group

22 of 25

Summary.seqs tool with the following parameters

“fasta” parameter to the fasta from Unique.seqs

“count” to count table from Count.seqs

“output logfile?” to yes

Screen.seqs tool with the following parameters

“fasta” to the fasta file from Unique.seqs

“minlength” parameter to 225

“maxlength” parameter to 275

“maxambig” parameter to 0

“maxhomop” parameter to 8

• “count” to the count file from Count.seqs

23 of 25

Using 1 processors.

Start End NBases Ambigs Polymer NumSeqs

Minimum: 1 80 80 0 3 1

2.5%-tile: 1 104 104 0 3 501

25%-tile: 1 242 242 0 4 5001

Median: 1 245 245 0 4 10001

75%-tile: 1 245 245 0 4 15001

97.5%-tile: 1 247 247 0 6 19501

Maximum: 1 275 275 2 31 20000

Mean: Align.seqs 1 237.519 237.519 0.00495 4.24965

# of unique seqs: 19502

total # of seqs: 20000

24 of 25

Align.seqs tool with the following parameters

“fasta” to the good.fasta output from Screen.seqs

“Select Reference Template from” to Your history

“reference” to the silva.v4.fasta reference file

“flip” to Yes

Summary.seqs tool with the following parameters

“fasta” parameter to the aligned output from Align.seqs

“count” parameter to count_table output from Screen.seqs

25 of 25

Screen.seqs tool with the following parameters

  • “fasta” to the aligned fasta file
  • “start” to 3080
  • “end” to 13424
  • “count” to the group file created by the previous run of Screen.seqs