Page tree
Skip to end of metadata
Go to start of metadata


(warning) External links on this page can only be accessed from outside the RE (warning)

This aggregate dataset contains information on a subset of participants who have since been withdrawn from research. Their use in any new analyses is not permitted. Thus, it is extremely important to remove these samples from your analyses an ensure that you are only using samples included in the latest data release.

The list of samples for the consented participants can be found in the 'cancer_analysis' table in the labkey, for the latest data release.

For the main programme version 13 data release, the list of consented samples are detailed in the file main_programme_v13_somatic_samples.txt, located in the folder /gel_data_resources/main_programme/aggregation/aggregated_somatic_strelka/somAgg/v0.2/docs/

To filter the aggregate to these samples, all bcftools commands should include the flag -S /gel_data_resources/main_programme/aggregation/aggregated_somatic_strelka/somAgg/v0.2/docs/main_programme_v13_somatic_samples.txt

Submit a ticket to the Genomics England Service desk if you are unsure of how to filter the dataset for any other use.

Brief Overview

The somatic aggregate multi-sample VCF (somAgg) comprises somatic genomic data from 16,341 tumour samples. It is an aggregation of single nucleotide variants and small indels (<=50bp) from single-sample somatic vcf files that have been successfully sequenced and interpreted. The input data can be found in the cancer_analysis table in Labkey, for the Main Programme data release V12 of the 100,000 Genomes Project, under column somatic_small_variants_annotation_vcf. 

Description

We have aggregated 16,341 somatic vcf files from the 100,000 Genomes Project which we made available as a multi-sample VCF dataset (somAgg). somAgg comprises over 573 million annotated single nucleotide variants and small indels (<=50bp) from quality controlled tumour whole genomes. For a breakdown of variants per chunk see here.

All samples in the dataset have a matched germline, both deep whole-genome sequenced with an average coverage of 100x and 30x, respectively. All samples were sequenced with 150bp paired-end reads in a single lane of an Illumina HiSeq X instrument and uniformly processed on the Illumina North Star Version 4 Whole Genome Sequencing Workflow (NSV4, version 2.6.53.23); which comprises the iSAAC Aligner (version 03.16.02.19) and small variant calling using tumour-normal subtraction performed by Strelka2 (version 2.4.7). Samples were aligned to the Homo Sapiens NCBI GRCh38 assembly with decoys. 

The dataset was constructed from the aggregation of single-sample annotated somatic vcf files using bcftools (version: 2019.02.26). Variant normalisation and decomposition was implemented by vt (version 0.57721). Annotation was performed by Cellbase against the ENSEMBL (version 90/GRCh38), COSMIC (version v86/GRCh38) and ClinVar (October 2018 release) databases.

The site QC annotation of the somAgg has been obtained from the single sample annotated VCFs. No additional site QC has been conducted and all samples in the cancer_analysis table have been included (i.e. no sample QC/filtering was conducted). 

The multi-sample VCF is split into 1,371 roughly equal 'chunks' across the genome for faster processing. Each chunk contains the full set of samples and is in the VCF.gz file format with accompanying tabix index files (.tbi). Chromosomes 1-22, X, Y, and M are included. 

The usage of GT

In the somatic aggregated files there are only two possible GT values:

  • 0/1 indicating that sample is a carrier of the variant
  • 0/0 indicating that sample does not carry the variant

All variants are in their bi-allelic forms (instead of potential multi-allelic) and samples that have multi-allelic sites are indicated by the FORMAT tag: SAMPLE_MULTIALLELIC (See Genotype-level Metrics for further details on SAMPLE_MULTIALLELIC). 

Definitions:

  • Multi-allelic: where a single variant contains three or more observed alleles, counting the reference as one, therefore allowing for two or more variant alleles (heterozygous genotype example: 1/2)
  • Bi-allelic: where a variant contains two observed alleles, counting the reference as one, and therefore allowing for one variant allele (heterozygous genotypes are always: 0/1)

Extended Details

Each step of the pipeline to generate somAgg is documented in the sections below: 

Code Book

A code book of popular queries to help you use somAgg is found here:  somAgg Code Book

somAgg Manifest & Location

Manifest

The somAgg dataset comprises: 

  • A multi-sample VCF file for each chunk containing the genotypes and per variant quality metrics and filter flags

Location

All somAgg (v0.2) outputs can be found in the following folder within the Genomics England Research Environment: 

/gel_data_resources/main_programme/aggregation/aggregated_somatic_strelka/somAgg/v0.2/

This folder is accessible from the Desktop Environment and from the HPC as shown below:

Desktop AccessHPC Access

Overview of Quality Control Flags

Variants in the multi-sample VCF files are flagged against this set of basic site quality metrics. Note that hard variant filtering has not been applied to the dataset (no variants have been removed). 

For more information on the Strelka flags, please refer to their manual on GitHub.

Sample QC

All 16,341 samples included in somAgg have successfully passed our internal sequencing and interpretation pipeline. These samples are listed in the LabKey table cancer_analysis. Some quality control statistics for these samples are provided below.

Sample AttributeDescription
Tumour Cross-Contaminationless than 5%
Germline Cross-Contaminationless than 3%
Median Fragment Sizegreater than 279bp
Excess of Chimeric Readsmean of 0.3%
Percentage of Somatic Mapped Readsmean of 93.4%
Percentage AT Dropoutmean of 3.1%

Single sample Genomics England Filters

On the single sample vcf level (somAgg input), Genomics England has defined extra FILTERs that are described here. In the single vcf file, a variant is only flagged with PASS after having passed all Strelka and the filters listed below.

BCNoise10Indel

Applied to indels only. It aims to flag calls with too many filtered basecalls. More specifically, a variant is flagged if the average fraction of filtered basecalls within 50 bases of the indel exceeds 0.1, i.e. FDP50/DP50 > 0.1.

PONnoise50SNV

Applied to SNVs only. It aims to flags variants in a region of mapping/sequencing error. More specifically, a variant is flagged if SomaticFisherPhred is below 50, indicating somatic SNV is systematic mapping/sequencing error. Different from other filters, this filter is only applied to variants that pass all Strelka filters.

SimpleRepeat

Applied to both SNVs and indels. It aims to flag variants that overlap a repetitive regions, since these are prone to error.  More specifically, a variant is flagged if overlapping simple repeats as defined by Tandem Repeats Finder: http://hgdownload.cse.ucsc.edu/goldenPath/hg38/database/simpleRepeat.txt.gz

CommonGnomADVariant

Applied to both SNVs and indels. It aims to flag variants commonly found in GL, assuming these are not cancer relevant. More specifically, a variant is flagged if its population germline allele frequency is above 1% in gnomAD dataset.

CommonGermlineVariant

Applied to both SNVs and indels. It aims to flag variants commonly found in GL, assuming these are not cancer relevant. More specifically, a variant is flagged if its population germline allele frequency is above 1% in a Genomics England sub-cohort.

The cohort that was used to generate this Germline allele frequencies can be found on:

CGV
vi /gel_data_resources/interpretation_support_data/cancer/CommonGermlineVariant/agg_samples.non_genetic.tsv

Note however, that a few samples have been analysed with previous versions of this cohort, and hence some inconsistency has been carried over to the somAgg. 

RecurrentSomaticVariant

Applied to both SNVs and indels. It aims to flag variants commonly found in the somatic samples. More specifically, a variant is flagged  as recurrent somatic variant if its frequency is above 5% in a Genomics England cohort. This cohort is made of 910 FF-PCRfree, 128 FF-nano and 232 FFPE samples. AF are calculated individually and if AF > 0.05 in any of these three cohorts variants were flagged. This flag resulted from a study that showed that there was an increased number of small variants in FFPE. Also, AF is defined assuming diploid and variant frequency (VF) = 2 * AF.

The file with the resulting AF that were used for annotation can be found here:

CGV
vi /gel_data_resources/interpretation_support_data/cancer/RecurrantSomaticVariant/cancer_mainProgram_2017.merged.AF.sorted.vcf.gz

Variant- and genotype- level flags (FILTER)

The FILTER field has not been populated in this version of the aggregate. Hence, all variants have FILTER "." in the respective field of the aggregate VCF. All filter flags of the individual annotated VCF files have been moved to the INFO or FORMAT fields in the aggregate. Variant-level flags have been moved to the INFO field of the aggregate. Genotype-level flags have been kept in the FORMAT field of the aggregate. Note that no variants have been filtered out on the basis of these filters in this version of the aggregate.

Filter flags are marked in purple on the Variant- and Genotype- level metrics and flags below.

Variant-level Metrics (INFO)

Per variant quality metrics are kept in the INFO field of the multi-sample VCF files. The INFO tags with descriptions are shown in the table below. Note that the source column in the table indicates if the TAG is generated by the variant caller (Strelka), has been added as part of Genomics England sequencing and interpretation pipeline (internal) or as part of post-processing/annotation specifically for the aggregate (BRS).

INFO TAGSNV/indelSourceDescription
RepetitiveRegion*SNVStrelkafilter flag: variants that overlap LINE repeat region*
CommonGermlineVariantbothInternalfilter flag: variants with a population germline allele frequency above 1% in a Genomics England sub-cohort
RecurrentSomaticVariantbothInternalfilter flag: recurrent somatic variants with frequency above 5% in a Genomics England cohort
SimpleRepeatbothInternalfilter flag: variants overlapping simple repeats as defined by Tandem Repeats Finder: http://hgdownload.cse.ucsc.edu/goldenPath/hg38/database/simpleRepeat.txt.gz
CommonGnomADVariantbothInternalfilter flag: variants with a population germline allele frequency above 1% in gnomAD dataset
ICindelStrelkaNumber of times RU repeats in the indel allele. (Indel Counts of RU)
IHPindelStrelkaLargest reference interrupted homopolymer length intersecting with the indel
PNOISESNVStrelkaFraction of panel containing non-reference noise at this site
PNOISE2SNVStrelkaFraction of panel containing more than one non-reference noise obs at this site
RCindelStrelkaNumber of times RU repeats in the reference allele. (Reference Counts of RU)
RUindelStrelkaSmallest Repeating sequence Unit in inserted or deleted sequence
AF1000G-Strelka

The allele frequency from all populations of  the 1000 genomes projected

AA-StrelkaThe inferred allele ancestral (if determined) to the chimpanzee/human lineage
GMAF-StrelkaGlobal minor allele frequency (GMAF); technically, the frequency of the second most frequent allele
Format: GlobalMinorAllele|AlleleFreqGlobalMinor
cosmic-StrelkaThe numeric identifier for the variant in the Catalogue of Somatic Mutations in Cancer (COSMIC) database
Format: GenotypeIndex|Significance
clinvar-StrelkaClinical significance
Format: GenotypeIndex|Significance
EVS-Strelka

Allele frequency, coverage and sample count taken from the Exome Variant Server (EVS)
Format: AlleleFreqEVS|EVSCoverage|EVSSamples

RefMinor-Strelka

Denotes positions where the reference base is a minor allele and is annotated as though it were a variant

phyloP-StrelkaPhyloP conservation score. Denotes how conserved the reference sequence is between species throughout evolution
CSQT-Strelka

Consequence type as predicted by the Illumina Annotation Engine (IAE).
Format: GenotypeIndex|HGNC|Transcript ID|Consequence

CSQR-StrelkaPredicted regulatory consequence type.
Format: GenotypeIndex|RegulatoryID|Consequence
CT-InternalConsequence type as predicted by CellBase
AF_GNOMAD-InternalAllele frequency from all populations of gnomAD genome data set
AF_GEL_GL-InternalAllele frequency from the Genomics England germline cohort
AN_GEL_GL-InternalTotal number of alleles in called genotypes from Genomics England germline cohort
AC_GEL_GL-InternalAllele count in genotypes from Genomics England germline cohort
AF_GEL_SOM_FFpcrfree-InternalAlternate Allele Frequency in the Genomics England FFpcrfree cohort
AF_GEL_SOM_FFnano-InternalAlternate Allele Frequency in the Genomics England FFnano cohort
AF_GEL_SOM_FFPE-InternalAlternate Allele Frequency in the Genomics England FFPE cohort
HomopolimerIndel-StrelkaIndels intersecting with reference homopolymers of at least 8 nucleotides
SegmentalDuplication-StrelkaVariants intersecting with Segmental Duplications: http://hgdownload.cse.ucsc.edu/goldenPath/hg38/database/genomicSuperDups.txt.gz

* Repetitive regions have been introduced when some samples of the 100,000 Genomes Project had already been sequenced and analysed so it is not consistently applied throughout the cohort.

Genotype-level Metrics (FORMAT)

Genotype-level metrics are kept in the FORMAT field of the multi-sample VCF files. The FORMAT tags with descriptions are shown in the table below. Note that the source column below indicates if the TAG is generated by default by the variant caller (Strelka), has been added as part of Genomics England sequencing and interpretation pipeline (internal) or or as part of post-processing/annotation specifically for the aggregate (BRS). The SNV/indel column indicates whether the respective FORMAT field has been populated for SNPs, indels or both.

FORMAT TAGSNV/indelSourceDescription
PASSbothInternalfilter flagAll internal and Strelka filters passed. Note that all samples had Repetitive Regions Variant-level flag checked for.
LowQuality-Strelkafilter flag: Locus has low support for variant allele, ALT=.
BCNoiseIndelindelStrelkafilter flag: Average fraction of filtered basecalls within 50 bases of the indel exceeds 0.3
HighDepthindelStrelkafilter flag: Locus depth is greater than 3x the mean chromosome depth in the normal sample
LowQscoreSNVStrelkafilter flag: The empirically fitted VQSR score is less than 2.75
QSI_refindelStrelkafilter flag: Normal (germline) sample is not homozygous ref or sindel Q-score < 30, ie calls with NT!=ref or QSI_NT < 30
BCNoise10IndelindelInternalfilter flag: flags if average fraction of filtered basecalls within 50 bases of the indel exceeds 0.1, FDP50/DP50 > 0.1
PONnoise50SNVSNVInternalfilter flag: flags if SomaticFisherPhred is below 50, indicating somatic SNV is systematic mapping/sequencing error (applies only to SNVs that pass Strelka filters)
AUSNVStrelkaNumber of 'A' alleles Used in tiers* 1,2
CUSNVStrelkaNumber of 'C' alleles Used in tiers* 1,2
DPbothStrelkaRead depth for tier1 (used+filtered)
DP2indelStrelkaRead depth for tier2
DP50indelStrelkaAverage tier1 read depth within 50 bases
FDPSNVStrelkaNumber of basecalls filtered from original read depth for tier1
FDP50indelStrelkaAverage tier1 number of basecalls filtered from original read depth within 50 bases
GUSNVStrelkaNumber of 'G' alleles Used in tiers* 1,2
SDPSNVStrelkaNumber of reads with deletions spanning this site at tier1
SUBDPSNVStrelkaNumber of reads below tier1 mapping quality threshold aligned across this site
SUBDP50indelStrelkaAverage number of reads below tier1 mapping quality threshold aligned across sites within 50 bases
TARindelStrelkaReads strongly supporting alternate allele for tiers 1,2. Note that, according to this, alternate allele means the reference allele in addition to any other conflicting/overlapping candidate indel alleles.
TIRindelStrelkaReads strongly supporting indel allele for tiers 1,2
TORindelStrelkaOther reads (weak support or insufficient indel breakpoint overlap) for tiers 1,2
TUSNVStrelkaNumber of 'T' alleles Used in tiers* 1,2
GTbothBRSGenotype, 0/1 for all called variants, i.e. any variant that has been called, regardless of variant allele frequency or filter flag, has GT=0/1. When a variant has **not** been called in a given sample, GT=0/0 
ALTMAPSNVStrelkaTumor alternate allele read position MAP
ALTPOSSNVStrelkaTumor alternate allele read position median
cDPSNVStrelkaCombined depth across samples
MQSNVStrelkaRMS Mapping Quality
MQ0SNVStrelkaNumber of MAPQ == 0 reads covering this record
NTbothStrelkaGenotype of the normal in all data tiers, as used to classify somatic variants. One of {ref,het,hom,conflict}.
OVERLAPindelStrelkaSomatic indel possibly overlaps a second indel
QSIindelStrelkaQuality score for any somatic variant, ie. for the ALT haplotype to be present at a significantly different frequency in the tumor and normal
QSI_NTindelStrelkaQuality score reflecting the joint probability of a somatic variant and NT
QSSSNVStrelkaQuality score for any somatic snv, ie. for the ALT allele to be present at a significantly different frequency in the tumor and normal
QSS_NTSNVStrelkaQuality score reflecting the joint probability of a somatic variant and NT
ReadPosRankSumSNVStrelkaZ-score from Wilcoxon rank sum test of Alt Vs. Ref read-position in the tumor
SGTbothStrelkaMost likely somatic genotype excluding normal noise states
SNVSBSNVStrelkaSomatic SNV site strand bias
TQSIindelStrelkaData tier used to compute QSI
TQSI_NTindelStrelkaData tier used to compute QSI_NT
TQSSSNVStrelkaData tier used to compute QSS
TQSS_NTSNVStrelka

Data tier used to compute QSS_NT

VQSRSNVStrelkaRecalibrated quality score expressing the phred scaled probability of the somatic call being a FP observation
SAMPLE_MULTIALLELICSNVInternalOriginal chr:pos:ref:alt encoding for SNVs
SAMPLE_VARIANTindelInternalOriginal chr:pos:ref:alt encoding for indels
VAFbothStrelkaVariant allele frequency,
SNV:  VAF = dALT / (dALT + dREF), where dALT and dREF are read depth for tier 1 for ALT and REF respectively, i.e. the tier 1 counts for AU, CU, GU, or TU.** 
indel: VAF = TIR / (TIR + TAR), considering only tier 1 counts.
SomaticFisherPhredSNVInternalPhred score of Fisher's test of somatic allele ratio vs PoN allele ratio (applies only to SNVs that pass Strelka filters)

* Strelka tier is not the GEL tier: the algorithm divide calls into two tiers according to level of confidence. From the paper: The first tier (tier1) is a set of input data filtration and model parameter settings with relatively stringent values, whereas the second tier (tier2) uses more permissive settings. All calls are initially made using tier1 settings, after which the variant is called again using tier2. Strelka reports the minimum of the two somatic call qualities: Q=min(Qtier1, Qtier2)

** Note that the way VAF is calculated for SNVs, it does not take multi-allelic into account. The reason for that is to remove potential noise. However, multi-allelic sites may have VAFs whose sum is larger than 1. In the most extreme case, you will have REF completely replaced by the two (or more) possible ALT and each ALT will have VAF = 1. 

Help & Support

Help with somAgg

Please reach out via the Genomics England Service Desk for any issues related to the somAgg aggregation or companion datasets, including "somAgg" in the title / description of your inquiry.


  • No labels