GFF3 formats are tabular files with 9 fields per line, separated by tabs.First 8 fields share almost same data structure. But the 9th field varies a lot depending on the features. The last field(9th field) varies a lot from one gene prediction algorithm to another.Presently I am trying to build my own gff module that will parse one gff file and return one structured data structure( python dictionary structure). For that, I need a good understanding of the 9th field of the gff format.So can any one please help me by providing various gff files.
How 9th column of a gff file varies from one gene prediction algorithm to another?
file formats;python;gff3;sequence annotation
The first place to start is the GFF3 specification. This is the official word on what is and is not allowed in a GFF3 file. For example, users can define arbitrary attribute keys, so long as they do not begin with an uppercase letter (these are reserved for official use).But your question doesn't seem to be about what is allowed, but what is commonly used. I have a question for you: which gene predictors are YOU using? Or are you using gene annotations produced by others?Here are a few examples.NCBI RefSeqIf you're using GFF3 from NCBI, chances are it looks something like this.NC_007070.3     Gnomon  gene    329235  331223  .       +       .       ID=gene14;Dbxref=BEEBASE:GB42168,GeneID:551678;Name=LOC551678;gbkey=Gene;gene=LOC551678;gene_biotype=protein_codingNC_007070.3     Gnomon  mRNA    329235  331223  .       +       .       ID=rna27;Parent=gene14;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;Name=XM_624067.4;gbkey=mRNA;gene=LOC551678;model_evidence=Supporting evidence includes similarity to: 66 ESTs%2C 24 Proteins%2C and 99%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 113 samples with support for all annotated introns;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4NC_007070.3     Gnomon  exon    329235  329459  .       +       .       ID=id117;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4NC_007070.3     Gnomon  exon    329850  330082  .       +       .       ID=id118;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4NC_007070.3     Gnomon  exon    330166  330301  .       +       .       ID=id119;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4NC_007070.3     Gnomon  exon    330376  331223  .       +       .       ID=id120;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4NC_007070.3     Gnomon  CDS     329333  329459  .       +       0       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1NC_007070.3     Gnomon  CDS     329850  330082  .       +       2       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1NC_007070.3     Gnomon  CDS     330166  330301  .       +       0       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1NC_007070.3     Gnomon  CDS     330376  330416  .       +       2       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1MAKERThe MAKER annotation workflow (paper, software) is a pretty commonly used gene annotation tool, and produces GFF3 output like this.scaffold_12     maker   gene    652527  655343  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959;Name=maker-scaffold_12-augustus-gene-0.959scaffold_12     maker   mRNA    652527  655343  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1;Parent=maker-scaffold_12-augustus-gene-0.959;Name=maker-scaffold_12-augustus-gene-0.959-mRNA-1;_AED=0.24;_eAED=0.18;_QI=0|0|0|0.66|0.5|0.33|3|0|218scaffold_12     maker   exon    652527  652817  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:exon:1203;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1scaffold_12     maker   exon    654877  655170  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:exon:1204;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1scaffold_12     maker   exon    655272  655343  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:exon:1205;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1scaffold_12     maker   CDS     652527  652817  .       +       0       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:cds;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1scaffold_12     maker   CDS     654877  655170  .       +       0       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:cds;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1scaffold_12     maker   CDS     655272  655343  .       +       0       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:cds;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1scaffold_12     maker   gene    941547  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851;Name=snap-scaffold_12-processed-gene-0.851scaffold_12     maker   mRNA    941547  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1;Parent=snap-scaffold_12-processed-gene-0.851;Name=snap-scaffold_12-processed-gene-0.851-mRNA-1;_AED=0.22;_eAED=0.22;_QI=0|0|0.25|0.25|1|1|4|661|95scaffold_12     maker   exon    941547  941631  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1206;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   exon    942343  942367  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1207;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   exon    942780  942920  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1208;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   exon    943200  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1209;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   CDS     941547  941631  .       +       0       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   CDS     942343  942367  .       +       2       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   CDS     942780  942920  .       +       1       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   CDS     943200  943236  .       +       1       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1scaffold_12     maker   three_prime_UTR 943237  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:three_prime_utr;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1GeneMark, SNAP, AugustusIf we look at the output of several common ab initio gene prediction tools, none of them actually produces GFF3 by default. Here is some output from GeneMark (paper, software).543   1   +  Initial         799051     799097         47          1 2 - -543   2   +  Terminal        799236     799266         31          3 3 - -544   3   -  Terminal        802357     802514        158          3 2 - -544   2   -  Internal        802607     802685         79          1 1 - -544   1   -  Initial         802829     802843         15          3 1 - -Here is some output from SNAP (paper, software).Einit   8230    8239    +       9.329   0       1       0       scaffold_12-snap.4Exon    8848    8869    +       2.484   2       2       2       scaffold_12-snap.4Exon    10121   10208   +       15.302  1       0       2       scaffold_12-snap.4Exon    11361   11420   +       5.969   0       0       2       scaffold_12-snap.4Exon    11471   11535   +       0.921   0       2       1       scaffold_12-snap.4Eterm   12169   12187   +       18.163  1       0       1       scaffold_12-snap.4Einit   14569   14668   +       -6.918  0       1       0       scaffold_12-snap.5Exon    15029   15203   +       -1.023  2       2       0       scaffold_12-snap.5Exon    16171   16307   +       -9.230  1       1       1       scaffold_12-snap.5Eterm   16667   16698   +       9.829   2       0       0       scaffold_12-snap.5Einit   17809   17898   -       -5.390  0       0       0       scaffold_12-snap.6Exon    17299   17350   -       6.978   0       1       2       scaffold_12-snap.6Eterm   17029   17129   -       -2.857  2       0       0       scaffold_12-snap.6Both of these formats appear to be tab-delimited, but that's where the similarity with GFF3 ends.The output of Augustus (paper, software) looks like GTF (a variant of the GFF format) once you lose all the lines beginning with a # symbol, but Augustus also has a GFF3 output mode.# Predicted genes for sequence number 1 on both strands# start gene g1scaffold_12     AUGUSTUS        gene    2841    3400    1       +       .       g1scaffold_12     AUGUSTUS        transcript      2841    3400    1       +       .       g1.t1scaffold_12     AUGUSTUS        start_codon     2841    2843    .       +       0       transcript_id g1.t1; gene_id g1;scaffold_12     AUGUSTUS        intron  3027    3097    1       +       .       transcript_id g1.t1; gene_id g1;scaffold_12     AUGUSTUS        CDS     2841    3026    1       +       0       transcript_id g1.t1; gene_id g1;scaffold_12     AUGUSTUS        CDS     3098    3400    1       +       0       transcript_id g1.t1; gene_id g1;scaffold_12     AUGUSTUS        stop_codon      3398    3400    .       +       0       transcript_id g1.t1; gene_id g1;# protein sequence = [MAIKNAEHDLRVIVDAIEGLGLKVAPHKTEAMAFPASALCGRRGAAPPKIRLGGSSILVGSRSRWYISGHSENSSKSP# RTEGKETTPLQQRDPLDAPLWVSGVVAHCCGGPEGQEGCPGLAAQGSDQGVLRIRDGLLCGYDGCGDHRPRPSDSSAGGGLCRP]# end gene g1#### start gene g2scaffold_12     AUGUSTUS        gene    4712    15229   0.21    +       .       g2scaffold_12     AUGUSTUS        transcript      4712    15229   0.21    +       .       g2.t1scaffold_12     AUGUSTUS        start_codon     4712    4714    .       +       0       transcript_id g2.t1; gene_id g2;scaffold_12     AUGUSTUS        intron  4858    5591    0.74    +       .       transcript_id g2.t1; gene_id g2;scaffold_12     AUGUSTUS        intron  5686    15028   0.23    +       .       transcript_id g2.t1; gene_id g2;scaffold_12     AUGUSTUS        CDS     4712    4857    0.72    +       0       transcript_id g2.t1; gene_id g2;scaffold_12     AUGUSTUS        CDS     5592    5685    0.42    +       1       transcript_id g2.t1; gene_id g2;scaffold_12     AUGUSTUS        CDS     15029   15229   0.43    +       0       transcript_id g2.t1; gene_id g2;scaffold_12     AUGUSTUS        stop_codon      15227   15229   .       +       0       transcript_id g2.t1; gene_id g2;# protein sequence = [MGRNSHRSCCVVNCKITSAKSDCKFYKFPTAKWKINQRKMWVAAVKRQKYIKDEISHAETQTEITEVTGATKVNYANK# KYICLLFVRTYVLRMLVDVALSNLRFSLFGIRKSLEIFGQSEKADQTRWRLPSCEMEWIESRKGKMRE]# end gene g2###Parsing attributesSo...how should YOU handle attributes in GFF3's 9th column? That depends a lot on what you want to do with the data.The most important attributes are ID and Parent, which are used to define relationships between features and subfeatures. (These relationships implicitly define a directed acyclic graph of features, although most GFF3 parsers don't directly support traversal of this graph.)But not only do you have to handle the pre-defined attributes discussed in the GFF3 specification, you also have to be able to handle any number of arbitrary attributes whose keys you may not know beforehand.The simplest way to handle this would be to parse the attribute column into a dictionary of key/value pairs. Once it's in this form, it's trivial to see what attributes are there and how to access them.For example, if we have a feature with the following attributes...ID=mRNA42;Parent=gene19;integrity=0.95;foo=bar...we would want it in a dictionary like so.attributes = {    'ID': 'mrna42',    'Parent': 'gene19',    'integrity': '0.95',    'foo': 'bar',}Python code to parse that might look something like this.attributes = dict()for keyvaluepair in attributestring.split(';'):    for key, value in keyvaluepair.split('='):        attributes[key] = valueOne thing to consider is that it's possible for an attribute to have multiple values (separated by commas). It's not commonly used, but it is valid and something that should be handled or at least checked for.