Variant Call Format

Last updated
Variant Call Format
Filename extension
.vcf
Developed by 1000 Genomes Project
Latest release
4.3
January 13, 2021;2 years ago (2021-01-13)
Type of format Bioinformatics
Extended from Tab-separated values
Extended togVCF
Open format?Yes
Website samtools.github.io/hts-specs/VCFv4.3.pdf

The Variant Call Format (VCF) is a standard text file format used in bioinformatics for storing gene sequence variations. The format was developed in 2010 for the 1000 Genomes Project and has since been used by other large-scale genotyping and DNA sequencing projects. [1] [2] VCF is a common output format for variant calling programs due to its relative simplicity and scalability. [3] [4] Many tools have been developed for editing and manipulating VCF files, including VCFtools, which was released in conjunction with the VCF format in 2011, and BCFtools, which was included as part of SAMtools until being split into an independent package in 2014. [1] [5]

Contents

The standard is currently in version 4.3, [6] [7] although the 1000 Genomes Project has developed its own specification for structural variations such as duplications, which are not easily accommodated into the existing schema. [8]

Additional file formats have been developed based on VCF, including genomic VCF (gVCF). gVCF is an extended format which includes additional information about "blocks" that match the reference and their qualities. [9] [10]

Example

##fileformat=VCFv4.3 ##fileDate=20090805 ##source=myImputationProgramV3.1 ##reference=file:///seq/references/1000GenomesPilot-NCBI36.fasta ##contig=<ID=20,length=62435964,assembly=B36,md5=f126cdf8a6e0c7f379d618ff66beb2da,species="Homo sapiens",taxonomy=x> ##phasing=partial ##INFO=<ID=NS,Number=1,Type=Integer,Description="Number of Samples With Data"> ##INFO=<ID=DP,Number=1,Type=Integer,Description="Total Depth"> ##INFO=<ID=AF,Number=A,Type=Float,Description="Allele Frequency"> ##INFO=<ID=AA,Number=1,Type=String,Description="Ancestral Allele"> ##INFO=<ID=DB,Number=0,Type=Flag,Description="dbSNP membership, build 129"> ##INFO=<ID=H2,Number=0,Type=Flag,Description="HapMap2 membership"> ##FILTER=<ID=q10,Description="Quality below 10"> ##FILTER=<ID=s50,Description="Less than 50% of samples have data"> ##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype"> ##FORMAT=<ID=GQ,Number=1,Type=Integer,Description="Genotype Quality"> ##FORMAT=<ID=DP,Number=1,Type=Integer,Description="Read Depth"> ##FORMAT=<ID=HQ,Number=2,Type=Integer,Description="Haplotype Quality"> #CHROM POS      ID         REF   ALT    QUAL  FILTER   INFO                             FORMAT       NA00001         NA00002          NA00003 20     14370    rs6054257  G     A      29    PASS    NS=3;DP=14;AF=0.5;DB;H2           GT:GQ:DP:HQ  0|0:48:1:51,51  1|0:48:8:51,51   1/1:43:5:.,. 20     17330    .          T     A      3     q10     NS=3;DP=11;AF=0.017               GT:GQ:DP:HQ  0|0:49:3:58,50  0|1:3:5:65,3     0/0:41:3 20     1110696  rs6040355  A     G,T    67    PASS    NS=2;DP=10;AF=0.333,0.667;AA=T;DB GT:GQ:DP:HQ  1|2:21:6:23,27  2|1:2:0:18,2     2/2:35:4 20     1230237  .          T     .      47    PASS    NS=3;DP=13;AA=T                   GT:GQ:DP:HQ  0|0:54:7:56,60  0|0:48:4:51,51   0/0:61:2 20     1234567  microsat1  GTC   G,GTCT 50    PASS    NS=3;DP=9;AA=G                    GT:GQ:DP     0/1:35:4        0/2:17:2         1/1:40:3
Demonstrating the difference between the binary BCF and VCF formats. Binary BCF versus VCF format.png
Demonstrating the difference between the binary BCF and VCF formats.

The VCF header

The header begins the file and provides metadata describing the body of the file. Header lines are denoted as starting with #. Special keywords in the header are denoted with ##. Recommended keywords include fileformat, fileDate and reference.

The header contains keywords that optionally semantically and syntactically describe the fields used in the body of the file, notably INFO, FILTER, and FORMAT (see below).

The columns of a VCF

The body of VCF follows the header, and is tab separated into 8 mandatory columns and an unlimited number of optional columns that may be used to record other information about the sample(s). When additional columns are used, the first optional column is used to describe the format of the data in the columns that follow.

NameBrief description (see the specification for details).
1CHROMThe name of the sequence (typically a chromosome) on which the variation is being called. This sequence is usually known as 'the reference sequence', i.e. the sequence against which the given sample varies.
2POSThe 1-based position of the variation on the given sequence.
3IDThe identifier of the variation, e.g. a dbSNP rs identifier, or if unknown a ".". Multiple identifiers should be separated by semi-colons without white-space.
4REFThe reference base (or bases in the case of an indel) at the given position on the given reference sequence.
5ALTThe list of alternative alleles at this position.
6QUALA quality score associated with the inference of the given alleles.
7FILTERA flag indicating which of a given set of filters the variation has failed or PASS if all the filters were passed successfully.
8INFO   An extensible list of key-value pairs (fields) describing the variation. See below for some common fields. Multiple fields are separated by semicolons with optional values in the format: <key>=<data>[,data].
9FORMATAn (optional) extensible list of fields for describing the samples. See below for some common fields.
+SAMPLEsFor each (optional) sample described in the file, values are given for the fields listed in FORMAT

Common INFO fields

Arbitrary keys are permitted, although the following sub-fields are reserved (albeit optional): [6]

NameBrief description
AAancestral allele
ACallele count in genotypes, for each ALT allele, in the same order as listed
AFallele frequency for each ALT allele in the same order as listed (use this when estimated from primary data, not called genotypes)
ANtotal number of alleles in called genotypes
BQRMS base quality at this position
CIGARcigar string describing how to align an alternate allele to the reference allele
DBdbSNP membership
DPcombined depth across samples, e.g. DP=154
ENDend position of the variant described in this record (for use with symbolic alleles)
H2membership in hapmap2
H3membership in hapmap3
MQRMS mapping quality, e.g. MQ=52
MQ0Number of MAPQ == 0 reads covering this record
NSNumber of samples with data
SBstrand bias at this position
SOMATICindicates that the record is a somatic mutation, for cancer genomics
VALIDATEDvalidated by follow-up experiment
1000Gmembership in 1000 Genomes

Any other info fields are defined in the .vcf header.

Common FORMAT fields

NameBrief description
ADRead depth for each allele
ADFRead depth for each allele on the forward strand
ADRRead depth for each allele on the reverse strand
DPRead depth
ECExpected alternate allele counts
FTFilter indicating if this genotype was “called”
GLGenotype likelihoods
GPGenotype posterior probabilities
GQConditional genotype quality
GTGenotype
HQHaplotype quality
MQRMS mapping quality
PLPhred-scaled genotype likelihoods rounded to the closest integer
PQPhasing quality
PSPhase set

Any other format fields are defined in the .vcf header.

See also

Related Research Articles

<span class="mw-page-title-main">PDF</span> Portable Document Format, a digital file format

Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems. Based on the PostScript language, each PDF file encapsulates a complete description of a fixed-layout flat document, including the text, fonts, vector graphics, raster images and other information needed to display it. PDF has its roots in "The Camelot Project" initiated by Adobe co-founder John Warnock in 1991. PDF was standardized as ISO 32000 in 2008. The last edition as ISO 32000-2:2020 was published in December 2020.

Waveform Audio File Format is an audio file format standard, developed by IBM and Microsoft, for storing an audio bitstream on personal computers. It is the main format used on Microsoft Windows systems for uncompressed audio. The usual bitstream encoding is the linear pulse-code modulation (LPCM) format.

Resource Interchange File Format (RIFF) is a generic file container format for storing data in tagged chunks. It is primarily used for audio and video, though it can be used for arbitrary data.

ZIP is an archive file format that supports lossless data compression. A ZIP file may contain one or more files or directories that may have been compressed. The ZIP file format permits a number of compression algorithms, though DEFLATE is the most common. This format was originally created in 1989 and was first implemented in PKWARE, Inc.'s PKZIP utility, as a replacement for the previous ARC compression format by Thom Henderson. The ZIP format was then quickly supported by many software utilities other than PKZIP. Microsoft has included built-in ZIP support in versions of Microsoft Windows since 1998 via the "Plus! 98" addon for Windows 98. Native support was added as of the year 2000 in Windows ME. Apple has included built-in ZIP support in Mac OS X 10.3 and later. Most free operating systems have built in support for ZIP in similar manners to Windows and Mac OS X.

In bioinformatics and biochemistry, the FASTA format is a text-based format for representing either nucleotide sequences or amino acid (protein) sequences, in which nucleotides or amino acids are represented using single-letter codes.

The archiver, also known simply as ar, is a Unix utility that maintains groups of files as a single archive file. Today, ar is generally used only to create and update static library files that the link editor or linker uses and for generating .deb packages for the Debian family; it can be used to create archives for any purpose, but has been largely replaced by tar for purposes other than static libraries. An implementation of ar is included as one of the GNU Binutils.

<span class="mw-page-title-main">Comma-separated values</span> File format used to store data

Comma-separated values (CSV) is a text file format that uses commas to separate values. A CSV file stores tabular data in plain text, where each line of the file typically represents one data record. Each record consists of the same number of fields, and these are separated by commas in the CSV file. If the field delimiter itself may appear within a field, fields can be surrounded with quotation marks.

<span class="mw-page-title-main">Shapefile</span> Geospatial vector data format

The shapefile format is a geospatial vector data format for geographic information system (GIS) software. It is developed and regulated by Esri as a mostly open specification for data interoperability among Esri and other GIS software products. The shapefile format can spatially describe vector features: points, lines, and polygons, representing, for example, water wells, rivers, and lakes. Each item usually has attributes that describe it, such as name or temperature.

Real-Time Messaging Protocol (RTMP) is a communication protocol for streaming audio, video, and data over the Internet. Originally developed as a proprietary protocol by Macromedia for streaming between Flash Player and the Flash Communication Server, Adobe has released an incomplete version of the specification of the protocol for public use.

FASTQ format is a text-based format for storing both a biological sequence and its corresponding quality scores. Both the sequence letter and quality score are each encoded with a single ASCII character for brevity.

Pileup format is a text-based format for summarizing the base calls of aligned reads to a reference sequence. This format facilitates visual display of SNP/indel calling and alignment. It was first used by Tony Cox and Zemin Ning at the Wellcome Trust Sanger Institute, and became widely known through its implementation within the SAMtools software suite.

SAMtools is a set of utilities for interacting with and post-processing short DNA sequence read alignments in the SAM, BAM and CRAM formats, written by Heng Li. These files are generated as output by short read aligners like BWA. Both simple and advanced tools are provided, supporting complex tasks like variant calling and alignment viewing as well as sorting, indexing, data extraction and format conversion. SAM files can be very large, so compression is used to save space. SAM files are human-readable text files, and BAM files are simply their binary equivalent, whilst CRAM files are a restructured column-oriented binary container format. BAM files are typically compressed and more efficient for software to work with than SAM. SAMtools makes it possible to work directly with a compressed BAM file, without having to uncompress the whole file. Additionally, since the format for a SAM/BAM file is somewhat complex - containing reads, references, alignments, quality information, and user-specified annotations - SAMtools reduces the effort needed to use SAM/BAM files by hiding low-level details.

Ensembl Genomes is a scientific project to provide genome-scale data from non-vertebrate species.

GeneTalk is a web-based platform, tool, and database for filtering, reduction and prioritization of human sequence variants from next-generation sequencing (NGS) data. GeneTalk allows editing annotation about sequence variants and build up a crowd sourced database with clinically relevant information for diagnostics of genetic disorders. GeneTalk allows searching for information about specific sequence variants and connects to experts on variants that are potentially disease-relevant.

Sequence Alignment Map (SAM) is a text-based format originally for storing biological sequences aligned to a reference sequence developed by Heng Li and Bob Handsaker et al. It was developed when the 1000 Genomes Project wanted to move away from the MAQ mapper format and decided to design a new format. The overall TAB-delimited flavour of the format came from an earlier format inspired by BLAT’s PSL. The name of SAM came from Gabor Marth from University of Utah, who originally had a format under the same name but with a different syntax more similar to a BLAST output. It is widely used for storing data, such as nucleotide sequences, generated by next generation sequencing technologies, and the standard has been broadened to include unmapped sequences. The format supports short and long reads (up to 128 Mbp) produced by different sequencing platforms and is used to hold mapped data within the Genome Analysis Toolkit (GATK) and across the Broad Institute, the Wellcome Sanger Institute, and throughout the 1000 Genomes Project.

<span class="mw-page-title-main">Binary Alignment Map</span>

Binary Alignment Map (BAM) is the comprehensive raw data of genome sequencing; it consists of the lossless, compressed binary representation of the Sequence Alignment Map-files.

Compressed Reference-oriented Alignment Map (CRAM) is a compressed columnar file format for storing biological sequences aligned to a reference sequence, initially devised by Markus Hsi-Yang Fritz et al.

ANNOVAR is a bioinformatics software tool for the interpretation and prioritization of single nucleotide variants (SNVs), insertions, deletions, and copy number variants (CNVs) of a given genome.

The BED format is a text file format used to store genomic regions as coordinates and associated annotations. The data are presented in the form of columns separated by spaces or tabs. This format was developed during the Human Genome Project and then adopted by other sequencing projects. As a result of this increasingly wide use, this format had already become a de facto standard in bioinformatics before a formal specification was written.

Tabix is a bioinformatics software utility for indexing large genomic data files. Tabix is free software under the MIT license.

References

  1. 1 2 Danecek, Petr; Auton, Adam; Abecasis, Goncalo; Albers, Cornelis A.; Banks, Eric; DePristo, Mark A.; Handsaker, Robert E.; Lunter, Gerton; Marth, Gabor T.; Sherry, Stephen T.; McVean, Gilean; Durbin, Richard (2011-08-01). "The variant call format and VCFtools". Bioinformatics. 27 (15): 2156–2158. doi:10.1093/bioinformatics/btr330. ISSN   1367-4803. PMC   3137218 . PMID   21653522.
  2. Ossola, Alexandra (20 March 2015). "The Race to Build a Search Engine for Your DNA". IEEE Spectrum . Retrieved 22 March 2015.
  3. "Understanding VCF format | Human genetic variation". EMBL-EBI . Archived from the original on 2023-04-20. Retrieved 2023-11-10.
  4. Garrison, Erik; Kronenberg, Zev N.; Dawson, Eric T.; Pedersen, Brent S.; Prins, Pjotr (2022-05-31). "A spectrum of free software tools for processing the VCF variant call format: vcflib, bio-vcf, cyvcf2, hts-nim and slivar". PLOS Computational Biology. 18 (5): e1009123. Bibcode:2022PLSCB..18E9123G. doi: 10.1371/journal.pcbi.1009123 . ISSN   1553-734X. PMC   9286226 . PMID   35639788.
  5. Danecek, Petr; Bonfield, James K; Liddle, Jennifer; Marshall, John; Ohan, Valeriu; Pollard, Martin O; Whitwham, Andrew; Keane, Thomas; McCarthy, Shane A; Davies, Robert M; Li, Heng (2021-01-29). "Twelve years of SAMtools and BCFtools". GigaScience. 10 (2). doi:10.1093/gigascience/giab008. ISSN   2047-217X. PMC   7931819 . PMID   33590861.
  6. 1 2 "VCF Specification" (PDF). Retrieved 20 Oct 2016.
  7. "Specifications of SAM/BAM and related high-throughput sequencing file formats". GitHub . Retrieved 24 June 2014.
  8. "Encoding Structural Variants in VCF (Variant Call Format) version 4.0 | 1000 Genomes" . Retrieved 20 October 2016.
  9. "GVCF - Genomic Variant Call Format". GATK. Broad Institute.
  10. "gVCF Files". Illumina, Inc. Retrieved 2023-11-10.
  11. "HTS format specifications". samtools.github.io. Retrieved 2022-02-22.
  12. "Bio-IT World". www.bio-itworld.com. Retrieved 2018-10-26.