Remove Empty Variants From VCF

Tyler Marrs — Mon, 26 Sep 2016 03:48:50 GMT

I ran into an issue where I tried to use known SNPs from dbSNP that consisted of some empty alleles. If you are using GATK's base quality score recalibrator (BQSR), then it will complain about the VCF not being valid. There are a handful of Perl one-liners that are posted within forums, but they did not appear to work. Here is my solution.

The script below takes a VCF as input (gzipped or not) and writes a new VCF. It simply checks for empty REF and ALT fields. When that criteria is met, these variant lines are skipped while writing out the new VCF.

#! /usr/bin/env python

# This script removes empty alleles from a VCF. Some data sources such as
# dbSNP provide improperly formatted VCFs. This causes headache when trying
# to use strict tools such as the GATK.

# It works by checking for empty string in the REF and ALT columns. When
# one of these lines are crossed it skips writing it to the new VCF output.

import argparse
import gzip

def parse_args():
    """
    Parses the command line arguments.
    """
    parser = argparse.ArgumentParser(description='Strip empty alleles from VCF')
    parser.add_argument('input', type=str, help='Input VCF - gzip accepted')
    parser.add_argument('output', type=str, help='Output VCF')

    return parser.parse_args()

def vcf_open(path):
    """
    Obtain file handle if gzipped or normal text file.
    """
    if path.endswith('.gz'):
        return gzip.open(path, 'rt')

    return open(path, 'r')

def read_and_write_vcfs(input, output):
    """
    Reads in VCF and write out corrected VCF.
    """
    with vcf_open(input) as f:
        with open(output, 'w') as w:
            for line in f:
                if not line.startswith('#'):
                    records = line.strip().split("\t")
                    ref = records[3]
                    alt = records[4]

                    if is_empty(ref) or is_empty(alt):
                        print("skipping.... " + line)
                        continue

                w.write(line)

def main():
    args = parse_args()
    read_and_write_vcfs(args.input, args.output)

if __name__ == '__main__':
    main()

Convert 23 and Me Raw Data to VCF

Tyler Marrs — Mon, 04 Jul 2016 18:25:55 GMT

Have you ever wanted to perform analysis on your 23 and Me data? Well the first step to do this consists of converting the raw output that they provide into a widely used format; in this case VCF. The VCF format enables you to use several tools to annotate each SNP with various meta information. You can use tools such as snpEff or even impute your data using the Sanger imputation server. I created an online web service that converts your raw 23 and Me data into the VCF format. It automatically detects the appropriate reference genome for you. There is also no issues with missing SNPs during the conversion process as the same reference genomes that 23 and Me use are used.

To get started, visit this page to convert your data: 23converter.tylermarrs.com.

Tyler Marrs (Posts about vcf)

Remove Empty Variants From VCF

Convert 23 and Me Raw Data to VCF