Kapur Lab - Penn State · USDA · Bioinformatic Diagnostic Tools (bdtools)

vSNP3

Whole-genome SNP analysis for bacterial and viral isolates: how sequencing reads become a high-resolution genotype, and how the SNP database is built and maintained.

Links

01

Resolution/what WGS measures

vSNP3 compares every base in the genome.

vSNP3 aligns reads to a reference genome and counts every difference, reporting the number of SNPs between any two isolates. Methods that read only a few loci, such as spoligotyping and MLST, assign isolates to a single type; comparing the whole genome separates isolates within that type and measures how far apart they are.

Traditional typing spoligotype / MLST — a few loci TYPE SB0121 8 isolates, 1 type. All 8 share one type; no differences are resolved. same isolates, whole genome Whole-genome SNP typing every base, against one reference 112 TB-101TB-104TB-107 0–4 SNPs apart group 1 96 TB-102TB-109 2 SNPs apart group 2 180 240 310 TB-103TB-106TB-108 180–310 SNPs from either group
The same eight isolates under both methods. Numbers on the branches are SNPs (illustrative): distances show which isolates are closely related, which are not, and by how much.
vSNP3 · Kapur Lab · Penn State · USDA
02

Overview/the whole process

The whole process on one page.

Reads in, aligned to a reference, one VCF per sample collecting in a database that only grows — then split into groups, each read as a tree and a SNP table.

01 FASTQ in R1 R2 one pair per sample 02 Step 1 · align to a reference reference genome SNP no coverage 03 one VCF each .vcf SNPs + coverage gaps 04 VCF database it only grows 01–03 run once per sample  ·  04 onward can be re-run over any selection 05 Defining SNPs MTBC0:1263022 MTBC0:2096350 MTBC0:643174 a few absolute positions split the set 06 A tree and a SNP table per group Lineage4_group1 · n=18 A C G T A C G C G Lineage4_group2 · n=12 A G T A C T A Lineage2_group1 · n=9 A C G G T A T A C G each group its own SNP profile, and the tree that follows from it
vSNP3 · Kapur Lab · Penn State · USDA
03

The database/reference types

The database is organized by reference type.

A reference type is one folder per reference genome, holding everything needed to compare samples against it — together with every VCF already processed there.

Mycobacterium_AF2122/ ├── *.fasta the reference genome ├── *.gbk annotation → a SNP gets a gene ├── *_define_filter.xlsx defining SNPs + per-group filters ├── *_metadata.xlsx file stem → display label └── *_remove_from_analysis.xlsx samples/regions to drop
  • Add an organism by adding a folder. The reference name is the folder name.
  • The VCF set only grows as samples are processed against that reference.
Why references cannot be mixed ref A 1,484,567 ref B 1,102,318 no merge The same biological site carries a different number in each reference.
  • The reference defines the positions. Two samples called against the same reference can be compared position by position. Against a different reference the same site carries a different number, so Step 2 will not mix them.

Each reference type is its own database — separate Step 1, Step 2 and comparisons

  • Mycobacterium_H37step 1 sample1, sample2 … step 2 comparisons
  • Mycobacterium_AF2122step 1 sample1, sample2 … step 2 comparisons
  • mtbc0_v1.1step 1 sample1, sample2 … step 2 comparisons
  • Brucella_abortus1step 1 sample1, sample2 … step 2 comparisons
  • …one folder per reference, added as needed
vSNP3 · Kapur Lab · Penn State · USDA
04

Step 1/per sample

Step 1 aligns one sample’s reads to a reference and calls its variants.

Reads go in; every difference from the reference is called and every uncovered region is recorded. This is the slow stage, and it runs once per sample.

FASTQ R1 + R2 align Reference genome MTBC0 SNP zero coverage call variants + record gaps <sample>_zc.vcf SNPs + zero-coverage map <sample>.bam the alignment, kept stats depth, %Q20, %zero coverage
  • Zero coverage is recorded. A - later means no data at that position, not a match.
  • The BAM is kept, so the reads behind any call can be reviewed later.
  • Adding samples is incremental. Alignment is done once.

In bdtools: pick the reference, Grab ready-to-run samples, then Runbdtools · vSNP GUI v0.4.80 · Step 1.

A few lines of a _zc.vcf

#CHROM  POS      REF  ALT  QUAL     INFO                                    FORMAT     sample
MTBC0   6406     C    T    3514.88  AC=2;AF=1;DP=119;MQ=60;TYPE=snp         GT:DP:AD   1/1:119:0,119
MTBC0   8285     C    T    2588.83  AC=2;AF=1;DP=91;MQ=60;TYPE=snp          GT:DP:AD   1/1:91:2,88
MTBC0   13228    GCCCCCCGCT  GCCCCCGCT  1056.36  AC=2;DP=39;TYPE=del        GT:DP:AD   1/1:39:0,38
MTBC0   17868    N    .    .        no reads covered this position          GT         ./.

The SNP, its depth and quality — and a row for every position with no coverage at all.

vSNP3 · Kapur Lab · Penn State · USDA
05

Quality control/between Step 1 and Step 2

Review the Step 1 statistics before running Step 2.

Step 1 flags each sample PASS or REVIEWbdtools · vSNP GUI v0.4.80 · Step 1 Results with the reason. Failing samples are excluded before a comparison runs.

MetricHealthyReview when
Average depth of coverage≥ 40×< 20×
Reads mapping to the reference> 90%< 80%
% reference with zero coverage< 5%> 10%
R1/R2 % passing Q20> 50%very low
Quality SNP countorganism-typicala spike

Low mapping usually means contaminating reads — host DNA or another organism. Parse the sample down to your target taxon with Kraken from inside Step 1, then re-run it.

Step 1 QC · 38% mapped contaminating reads Kraken · parse to taxon ⌷ Import → Step 1 DRR184968-Mycobacterium- tuberculosis-complex Re-run: 1 sample QC · 96% mapped
Kraken runs from inside Step 1; the parsed sample carries its taxon in the name, and only that cleaned sample is re-aligned.
Kraken · Krona chart of one sample
vSNP3 · Kapur Lab · Penn State · USDA
06

Large sample sets/defining SNPs

Defining SNPs sort samples into groups before the comparison is run.

Defining SNPs assign each sample to a group, so every table you actually read is smaller and internally comparable. A whole-complex table is thousands of positions across dozens of samples, and the differences between lineages drown out the few SNPs that separate recent cases.

One table for everything MTBC0:1263022 MTBC0:2096350 MTBC0:643174 60 samples × 3,100 positions — too large to read. Between-lineage differences dominate. AMBER = THE DEFINING POSITIONS sort Lineage4_group1 n=18 Lineage4_group2 n=12 Lineage2_group1 n=9 Each group: its own SNP table, tree, and FASTA. Groups nest Lineage4 Lineage4.1 Lineage4.1.2 A subgroup is just another defining position inside its parent. #MTBC0:2096350 a ‘#’ parks a provisional group — it stays in the file and reports nothing until asked.
In <reference>_define_filter.xlsx: row 1 is a chrom:position, row 2 names the group, and the rows beneath list positions to filter out within it.

Adding a group is adding a name and a position to that file — the next run reports it as its own group. A real run of this MTBC set produced 173 groups, each with its own table, tree, and SNP distances.

vSNP3 · Kapur Lab · Penn State · USDA
07

Step 2/comparison

Each Step 1 result joins the database; Step 2 compares any selection from it.

Every finished sample is copied into the project’s VCF database, which only ever grows. Step 2 takes whichever samples you tick, splits them into groups, and builds a table and tree for each one. No reads are re-aligned.

From Step 1 S-0213_zc S-0214_zc today’s samples add VCF database · grows only 214 samples processed to date select Step 2 defining SNPs split the set into subgroups Lineage4_group1 n=18 table + tree Lineage4_group2 n=12 table + tree Lineage2_group1 n=9 table + tree Several small comparisons instead of one huge one — much faster, and each table is readable.
The database holds every sample processed against this reference; each was aligned once, whenever it was first run.
  • You tick what goes in: this project’s samples plus optional shared panels. See the builderbdtools · vSNP GUI v0.4.80 · Step 2 build
  • One table and tree per subgroup come out of the same run. See the outputsbdtools · vSNP GUI v0.4.80 · Step 2 Results
vSNP3 · Kapur Lab · Penn State · USDA
08

Reading the output/table and tree together

The SNP table and the tree show the same data two ways.

Same samples, same SNPs, presented two ways: the tablebdtools · vSNP GUI v0.4.80 · cascade table lists them position by position, the tree draws the same differences as distance. A block of shared SNPs in the table is the branch that groups those samples in the tree. In the built-in tree viewerbdtools · vSNP GUI v0.4.80 · tree viewer, clicking a clade opens the table for just those samples.

Cascading SNP table Tree p1 p2 p3 p4 p5 p6 p7 p8 reference G A C T A G C T S1 T T T · · · · · S2 T T T · · · · · S3 T T T C C · · · S4 · · · · · A · - S5 · · · · · A G · S6 · · · · · · · T 3 shared SNPs · = same as reference    - = no coverage S1 S2 S3 S4 S5 S6 = this branch S4 and S5: 1 SNP apart S6: separate 1 SNP
The three SNPs that S1, S2 and S3 share (highlighted, left) are the branch that groups them (highlighted, right). Rows are aligned across both views. Branch length is proportional to the number of SNPs, and the scale bar shows one SNP.
vSNP3 · Kapur Lab · Penn State · USDA
09

Validation/the v in vSNP

vSNP stands for validate SNP: any call can be checked in the read alignment.

In bdtools a cell in the cascading table is a linkbdtools · vSNP GUI v0.4.80 · cascade table: clicking it opens the alignment at that position. When two isolates differ by only a few SNPs, each position affects the result, so they are reviewed in the read alignmentbdtools · IGV · MTBC0:1,263,022 rather than taken from the table alone.

SNP table pos → A C G T A C G T A C T T A C G - CLICK A CELL a ‘-’ is no coverage, not a match IGV Confirmed — use it 46× depth · 44/46 alt · both strands REAL SNP good depth, both strands, clean flanks Rejected — inside a repeat repeat 31× depth · 3/31 alt · one strand FALSE SNP low fraction, one strand, repetitive region Rejected — at an indel edge indel 38× depth · 5/38 alt · beside a gap FALSE SNP alt bases only on read ends that stop at the indel
Cells that look the same in the table. The read alignment tells them apart, which is why Step 1 keeps the BAM.
vSNP3 · Kapur Lab · Penn State · USDA
10

Summary/six properties

Six properties of the vSNP3 workflow.

What the preceding slides add up to.

01

Zero coverage is recorded

A gap is reported as missing data, never as agreement with the reference.

02

Step 1 and Step 2 are separate

Align once per sample; compare as often as you like.

03

One reference per dataset

Positions stay comparable. References are never mixed.

04

Grouping is yours to maintain

Defining SNPs are a curated file, not a clustering threshold.

05

Sample differences are visually validated

A table cell opens the read alignment at that position.

06

Every run is repeatable

Inputs, parameters and versions are stored with every run.

In bdtools every step above is a button, with the same layout as the other tools in the suite. The command line is optional.

vSNP3 · Kapur Lab · Penn State · USDA
11

Running it/five steps

Running vSNP3 in bdtools: five steps.

All of it in the browser. Allow about an hour, most of it unattended while Step 1 aligns.

  1. 01Open the dashboard and launch vSNP3bdtools · dashboard. Check the paths in Settings.
  2. 02Create a project and add inputsbdtools · vSNP GUI v0.4.80 · Projects & Inputs — server files, drag-and-drop, or SRA accessions.
  3. 03Step 1: choose the reference, then Runbdtools · vSNP GUI v0.4.80 · Step 1. This is the longest stage.
  4. 04Review Step 1 Resultsbdtools · vSNP GUI v0.4.80 · Step 1 Results. Exclude anything flagged REVIEW.
  5. 05Step 2: tick what to compare, then Runbdtools · vSNP GUI v0.4.80 · Step 2 Results. Read the table and tree together.

Practice data. docs/TRAINING.md Module 4 runs the whole workflow on nine public TB accessions; Module 5 scales it to a labelled MTBC panel. docs/TRAINING.html is the same guide as one standalone file you can email or hand out.

Reference type Step 1 align QC review Defining SNPs Step 2 compare Table + tree Validate SNPs
vSNP3 · Kapur Lab · Penn State · USDA
12