top down mass spectrometry 
 describes an analytical process for 
 the identification and characterization of whole proteins .
the canonical  top down  experiment consists of a 
 precursor scan to obtain the intact mass of the proteoform(s ) under study and a tandem mass spectrum ( ms / ms ) 
 obtained using ion fragmentation techniques such as ecd , etd , hcd , cid , or uvpd .
the defining characteristic of a top down experiment 
 is that the precursor ion is an intact proteoform , not the typical small peptides ( less than 3 kda ) produced 
 from intentional enzymatic digestion prior to mass spectrometry ( ms ) . 

thus , the mass of the precursor ion should represent a native proteoform 
 present in the sample , with its fragment ion masses providing extensive 
 characterization and verification of the primary structure .
as larger 
 proteins are targeted , experiments tend toward acquisition logic involving 
 spectral averaging for both the precursor and fragment ions to improve 
 the data quality .
these combined data are then analyzed to infer the 
 neutral masses of all intact and fragment ion species observed . for 
 proteins analyzed by electrospray ionization , this analysis to infer 
 mass ( aim ) uses either isotopic spacings for direct charge state assignment and/or deconvolution of protein charge states .
historically , top down mass spectrometry has targeted the 
 in - depth 
 characterization of a small number of proteoforms ; however , the past 5 years has seen a gradual transition to 
while the 
 number of proteins able to be identified has risen into the thousands , 
 the extent of characterization of each individual proteoform varies 
 and currently there is no scoring framework that captures this aspect 
 of top down proteomics . in certain cases , a protein ( arising from 
 a specific gene ) may be identified confidently with no inference problem 
 whatsoever , yet may be only partially characterized as shown in figure 1 . in figure 1b , two  equivalent  

proteoforms , one with a post - translational modification ( ptm ) in the 
 first position and the other with the ptm in the second , will each 
 have the same number of matching fragment ions .
scoring systems based 
 only on matching ions will report equal scores for these two proteoforms . 

this example illustrates how a proteoform can be clearly identified 
 ( i.e. , the evidence supporting the identification of either of the 
 two positional isomers is strong ) , yet not fully characterized .
this 
 problem of ptm site localization is encountered in bottom up , and 
 has been dealt with in various ways . in targeted top down 
 ms generating just a few spectra , manual reanalysis of data , or curation 
 of the primary literature , can be used to select one protein form 
 over another highly related one . in such cases , expert decisions are 
 made to distinguish proteoforms with similar primary structures taking 
 into account cleavage propensity of pairs of amino acids , complementary 
 ion pairs , sequence tags , and mass errors of precursor and fragment 
 ions .
incomplete fragmentation may or may not lead to partial characterization 
 of a protein molecule . in panel
( a ) , the matched fragment ions uniquely 
 determine the location of the post - translational modifications ( ptms ) . 
 in panel ( b ) ,
incomplete fragmentation in the middle of the protein 
 backbone does not yield a definitive ptm localization . in this particular 
 example
, the ptm could be located at either of two amino acids , resulting 
 in an identified , but partially characterized proteoform .
the c - score 
 framework was developed to handle such cases , routinely encountered 
 in top down proteomics .
n - terminal fragment ions are colored red , 
 while c - terminal fragment ions are shown in blue .
the numbers below 
 the fragment ions indicate how many ptms are reported on by each ion . with the shift to high - throughput , 
 fully automated data collection , 

we now seek a framework for scoring of protein identification and 
 proteoform characterization that builds in domain knowledge to achieve 
 the quality of manual analysis without requiring it .
all of the mathematical 
 symbols used below are aggregated into table 1 . fundamentally , the problem of identifying which proteoform is most 
 consistent with an experiment combines both aspects of protein identification 
 ( which gene ) and characterization ( which proteoform ) . in practice , 

the problem is one of testing a series of hypotheses about which proteoform 
 was present in the mass spectrometer , and then picking the 
each candidate proteoform ( not simply its 
 protein sequence ) constitutes a hypothesis , but since candidate proteoforms 
 does not represent all possible hypotheses to be tested , a scoring 
 system must allow for the possibility that the best scoring proteoform 
 is near the correct answer , but not exactly correct .
further , it is 
 frequently the case that the observed data can not conclusively differentiate 
 between two or more related proteoforms ( as in figure 1b ) , or the case where multiple proteoforms were actually fragmented 
 together .
typically , the way a proteoform score is used experimentally 
 is not too dissimilar the use of scores in annotating novel nucleotide 
 sequences with blast . for a given search ,
a minimum cutoff threshold 
 is picked by the operator ( a priori ) , and for each query the answer 
 accepted as correct must be both the highest scoring result and greater 
 than the cutoff threshold .
the problem of inferring 
 which proteoform , from the articulated 
  prior  list of proteoforms in a database , was observed 
 within the mass spectrometer is well - suited to a bayesian approach 
 ( figure 2 ) . in this case
, bayes law can be 
 rearticulated as follows:1where ( 1 ) pr(proteoformi|datams / ms ) is read as the probability of the ith proteoform given the ms / ms data , and is known as the posterior probability of proteoform i given 
 the observed data ; ( 2 ) pr(proteoformi ) 
 is known as the prior probability of proteoform i ; ( 3 ) pr(datams / ms|proteoformi ) is read as the probability of the data given proteoform i , and is known as the likelihood of the 
 data given proteoform i ; ( 4 ) pr(datams / ms ) is known as the probability of the data . by convention , 
 this can be taken as the sum of all prior probabilities multiplied 
 by their likelihoods across all hypotheses .
notice that this term scales the posterior probability to be the 
 fraction of the total of the numerators over all proteoforms interrogated . 

thus , a multiple testing correction across the database search is 
 integral in this approach .
this differs from controlling for the overall 
 false discovery rate of an experiment which can , for example , be handled 
 posthoc with a search against a scrambled sequence database in a manner 
 analogous to that used in earlier work .
problem 
 of assigning which proteoform was present in the mass spectrometer 
 during automated data collection can be envisioned as sorting a list 
 of candidate proteoforms based on the observed ms data and a mathematical 
 model of the process by which the observations were collected .
bayes 
 law provides a useful foundation for building these models . in practice , 
 it is not required that the list of candidate proteoforms be explicitly 
 articulated prior to the analysis .
it is sufficient that all candidate 
 forms can be calculated for an explicitly stated set .
for example , 
 listing base protein sequences and the ptms to be considered on these 
 sequences defines a list of proteoforms , even if the list is never 
 explicitly written . to be precise
, we will 
 define the following variables to restate 
 bayes law from eq 1 .
let mo = observed precursor mass , mi = mass of the ith of n observed fragment ions , so { mi}i  =  1n is the set of all n observed fragment ions , and q = the qth of k candidate 
 proteoforms in the database .
the posterior probability of hypothesis 
 q , as per eq 1 , can be restated aswhere pr(q ) is the  prior  

since the probability of 
 the data is , by convention , known , two 
 values are needed to calculate the 
 posterior probability for each hypothesis : the prior probability and 
 the likelihood for the data . in many applications ,
the prior probabilities 
 can be taken to be  all hypotheses are equally probable  

( i.e. , uniform prior probabilities ) . if there are  k  competing hypotheses , that is , 
k  proteoforms in the candidate list , then each proteoform 
 has a prior probability of 1/k , but this does not 
 need to be the case .
it is possible that one would want to assign 
 proteoforms of proteins that contain experimentally demonstrated ptms 
 ( or transcripts informed by rna - seq ) higher prior probabilities than 
 proteoforms that contain chemically possible , but otherwise rarely 
 observed modifications .
for example , if there are two proteoforms 
 that differ only in the location of a ptm such as in figure 1b , and one of the ptms was known to occur , while 
 the other had never been reported , and the set of observed fragment 
 ions failed to differentiate the two forms , then the known form should 
 be considered the most probable form to have been observed .
this degree 
 of differentiation can be achieved with such a scoring system by first 
 setting and then improving the prior probabilities with continued 
 experimentation .
prior probabilities are , by definition , based on 
 information known prior to data collection and are in practice always 
 somewhat arbitrary in their determination . in sharp contrast
, the 
 likelihood of the data given the proteoform is calculated under an 
 explicit mathematical model of the processes used to generate top 
 down ms data ; calculation of this likelihood requires generative models .
generate  the probability of the 
 observed data , given the proteoform in question .
therefore , they take 
 as input the proteoform , and as much knowledge of the measurement 
 process as can be encoded in the model - creation process . for example , 
 a generative model could include the propensity of individual pairs 
 of amino acids to dissociate during fragmentation ( e.g. , x - p cleavage 
 in ecd and etd is not possible , yet dp 
 bonds are preferentially cleaved in threshold dissociation ) , or the model could include a function for 
 the difference between the observed and theoretical intact mass .
the 
 more knowledge of the measurement process the generative model includes , 
 the better , but since the task is to rank order the list of candidate 
 proteoforms ( figure 2 ) , some details can be 
 safely excluded from the generative model if they do not shift the 
 proteoform rankings .
for example , during 
 manual data interrogation of an error - tolerant , top down search result , 
 many researchers prefer the observed intact mass to match the theoretical , 
 but allow for the possibility that the masses may not match because 
 the best proteoform in the database is not the correct one .
any scoring 
 system that does not include the closeness of the observed ms1 to 
 the theoretical ms1 is ignoring a valuable observation .
next , this 
 bayesian framework explicitly states the process in the form of two 
 generative models ; one for the precursor mass , and the other for the 
 fragment ion masses .
since the models are clearly articulated , they 
 can be defended , rejected , or modified based on community discourse . 
 in practice , this means that the process model can be modified and 
 updated to reflect new understanding of the measurement process , or 
 to reflect changes to the process used .
for example , an automated 
 data analysis system can use the same scoring mechanism , but employ 
 different parameter sets that reflect experimental differences .
experiments 
 that use cid or hcd will employ different parameter sets in the generative 
 models used for scoring than the same top down experiment employing 
 ecd / etd or uvpd for protein ion fragmentation .
it should be noted that the calculation of scores in this system 
 differs from bayesian approaches commonly seen in biological sequence 
 analysis . in those applications ,
the 
 generative models usually have unknown parameters that need to be 
 estimated from the sequence data , which is usually taken as being 
 perfectly known .
the application here is more like that of edwards 
 where the process is considered known .
as will be seen , our generative models have no unknown parameters ; 
 instead all values are taken from our knowledge of mass spectrometry , 
 or from prior studies that focused on determining the needed value . 

thus , instead of inferring values from the data collected for the 
 study in question , we have a framework for applying knowledge gained 
 in analytic chemistry to the problem of protein inference .
currently , 
 prosightpc and prosight ptm report an expectation , or 
 e - value , for each proteoform .
this score is calculated by multiplying 
 the probability of getting at least the observed number of matching 
 fragment ion due to chance by the number of proteoforms interrogated . 
 for this calculation
, the probability of getting at least a given 
 number of matching fragment ions is determined using poisson - based 
 model . here
we describe the implementation 
 of this bayesian approach to scoring , and compare one set of generative 
 models to the prosight ptm expectation value used within prosightptm 
 and prosightpc .
we show that our implementation , with its current generative models , 
 provides an improvement over the existing scoring system as measured 
 by area under the curve ( auc ) on the resulting roc chart ( auc of 0.99 versus 0.78 for a complex human 
 example and auc 0.85 versus 0.80 in pseudomonas ) .
the 
 observed data in a canonical ms / ms experiment includes ( a ) the neutral 
 precursor mass , which gives the total molecular weight of the proteoform 
 under study , and ( b ) a set of neutral fragment ion masses , that is , 
 masses of the products resulting from fragmentation of the proteoform
. 
 also required is a database of possible proteoforms , each of which 
 serves as a  hypothesis  that could potentially explain 
 the observed ms / ms data .
the database of possible proteoforms was 
 generated by combinatorial expansion of all potential proteoforms 
 using the known modification information in the uniprot knowledgebase .
since the observed data mo and each fragment ion , mi , are independently conditioned on q , we could take unfortunately , this 
 approach suffers 
 from two limitations .
first the magnitude of i  =  1n  pr(mi|q ) scales with n. larger lists of ms2 fragment ions lower the calculated 
 value of the ms2 likelihood , relative to lists with few fragment ions , 
 which makes it impossible to directly compare scores between separate 
 queries with differing numbers of ms2 fragment ions .
this can be mitigated 
 by weighting by the geometric mean of the number of ms2 fragment ions ; 
 1/n .
second , this approach weighs the ms1 data as 
 simply a single additional matching fragment ion . to avoid this , we 
 introduce two scaling functions , f and g , to map the ranges of the individual generative model to a normalized 
 range ; for the precursor 
 generative model , the function f is simply the identity 
 function . for the fragment ion generative 
 model ,
we define g by its logarithm base , which is 
 simply a linear function on the logarithm base 10 of the fragment 
 probability ; g sets the logarithm base 10 of the 
 minimum possible fragment probability to the logarithm base 10 of 
 the minimum possible precursor probability and likewise for the maxima . 
 if min1 is the minimum precursor probability , max1 is the maximum precursor probability , min2 is the minimum 
 fragment probability , and max2 is the maximum fragment 
 probability , then under an assumption 
 of uniform prior , and given the likelihood functions from above , we 
 have2equation 2 therefore 
 reduces the posterior probability of a hypothesis to the data likelihood 
 computation , with a normalization factor equal to the sum of the likelihoods 
 under all possible hypotheses .
since we have assumed 
 that the given database of proteoforms is an exhaustive set of hypotheses , 
 these generative models must allow for the possibility of observing 
 related proteoforms that are not present in the database .
the c - score system requires two generative 
 models ; one for the precursor mass mo and 
 the other for the set of observed fragment ion masses , { mi}. these models fully prescribe how 
 to compute the likelihood terms on the right - hand side of eq 2 .
note that the particular form of eq 2 implies that all the likelihood ( probability ) terms need 
 only be computed up to a constant factor .
this constant factor cancels 
 between the numerator and denominator of the rhs of eq 2 .
the probability pr(mo|j ) of an arbitrary protein sequence j ( of theoretical mass mj ) producing the observed precursor mass mo can be modeled as a function of m , the difference between the mo and mj .
we create the 
 probability distribution pr(mo|j ) such that masses within m of mj have the highest 
 probability , and this probability reduces as a truncated gaussian 
 function with  = 1 ,  = 30 da , and a minimum value of 
 1  10 .
notice that we only need to specify 
 pr(mo|j ) to a constant factor ( i.e. , we only need to specify a non - negative 
 function f ( mo , j ) proportional to the probability ) , and the 
 normalization factor ( 1/0max(mo)f(mo,j)dmo ) that converts it to a probability 
 density function is assumed implicitly . to specify the probability 
 distribution pr(mi|j ) , we 
 need an ms2 generative model based on our knowledge of the measurement 
 process during tandem mass spectrometry .
we note that each fragmentation 
 event involves cleavage of the intact proteoform at one bond on the protein backbone , resulting in exactly two fragments ( although both may not be observed in the spectrometer ) . 

the fragmentation propensity depends on the pair of adjacent amino 
 acids flanking the cleavage site .
the generative model we choose is 
 based on this observation , and uses the following basic ideas : ( 1 ) each theoretically possible fragment ion mass defines a region 
 of width 2m ( m  m , m + m ) called a permissible region .
an observed mass mi within 
 a permissible region has a probability proportional to the cleavage 
 propensity of gas phase protein ions at that permissible region .
( 2 ) an observed fragment ion mass mi outside of any 
 permissible region has a small , constant probability .
these 
 ideas are captured in the probability distribution function 
 of figure 3 . in principal , for
any given j , the probability distribution pr(mi|j ) is constructed by assigning a height to every point in the 
 range ( 0 , mj ) , where mj is the theoretical precursor 
 mass . in practice
, ms2 mass lists can contain unexpected ions with 
 mass values greater than mj , and so an arbitrarily large mass of 4 million daltons is 
 taken in place of mj . 

( the value of 4 million daltons was selected as it is safely above 
 the mass of the largest protein known , titin at 3.9 mda , and allows 
 for a modified form of titin to be present . in practice , this value 
 is effectively infinity . )
the assigned heights across the entire range 
 are divided by the total area under the curve , thus defining a probability 
 density function .
the heights are assigned as follows : step 
 1 : determine all theoretical fragment ions that j can give rise to , noting their mass and 
 the n- and c - terminal flanking amino acids at each cleavage site .
step 2 : for each theoretical fragment ion , calculate a weight , 
  , proportional to the product of the cleavage frequencies for 
 both flanking amino acids as previously determined for the appropriate 
 fragmentation method .
thus , if  
 and  are the two flanking amino acids , and f and f are their 
 respective cleavage frequencies , we set  = ff. the permissible 
 region associated with a theoretical fragment ion mass , m , is assigned a consistent height equal to the weight  computed 
 for the corresponding theoretical fragment ion .
step 3 : any 
 point in the range ( 0 , mj ) but outside of all permissible regions is assigned 
 a height of noise , which is a constant .
ms2 generative model for fragment ions observed 
 in tandem mass 
 spectrometry ( ms / ms ) . for instruments capable of ms / ms with accurate 
 mass
, thin ( e.g. , low part - per - million wide ) permissible regions occur 
 with a search - defined width around each theoretical fragment ion mass . 

the 
 heights of these regions are scaled by the propensity of the cleavage 
 events required to form the theoretical ion .
different fragmentation 
 methods require different weights , for example ecd has different fragmentation 
 propensities than cid . a very low probability , the weight 
 for chemical or electronic noise , is assigned to any observed 
 mass that does not match one of the theoretical masses .
we noted above that the probability density function 
 is obtained 
 by dividing the above - mentioned  height  function by 
 the sum of the area under the curve , which is different for different 
 j .
this total area , denoted by tj , is computed as follows : 
 the kth permissible region has a height equal to 
 k and a width equal to 2m .
regions outside of permissible regions have a height of 
 noise and a total width of p  
 2(len  1)m , where len is the length of the 
 protein sequence .
the resulting probability density function is3 the expressions on the right - hand side specify a probability density while the expression on the left - hand side is a 
 probability mass , which , strictly speaking , should 
 be equated to the probability density over a very small mass interval , 
 . however , such a correction can be safely ignored here because 
 the probability mass pr(mi|q ) is used only in the context 
 of eq 2 , with equal powers of  in numerator 
 and denominator . using eqs 2 and 3
, 
 it is now possible to calculate a posterior probability pr(q|mo,{mi } ) for every sequence q in the database .
this posterior probability 
 is proportional to the likelihood pr(mo,{mi}|q ) since we assume uniform priors ( eq 2 ) .
therefore , our search for the maximum a posteriori hypothesis 
 q is equivalent to a maximum likelihood 
 estimation ( mle ) search , that is , we report the q that maximizes pr(mo,{mi}|q ) .
we report the 
 best hypothesis q along with its 
  phred - like  characterization 
 score , which can be written 
 as c = 10log10(1  pr(q|mo,{mi } ) ) .
this final c - score transformation 
 scales the posterior probability of q to the familiar range used in many other bioinformatic applications . 

c - scores span the standard phred - like score range of 0 to > 500 . 

practical 
 ranges of the c - score are evaluated with specific examples and reported 
 in the main text below .
therefore , a c - score of 40 is sufficient to 
 judge a proteoform as extensively or fully characterized , while proteoforms 
 with c - scores between 3 and 40 are identified , but only partially 
 characterized .
a c - score below 3 indicates insufficient evidence for 
 either identification or characterization .
note also that since the 
 c - score represents a nonlinear transformation of the posterior probability , 
 which is itself normalized by pr(datams / ms ) , there is a 
 functional relationship between the highest score in a search , and 
 the second highest score ( supporting information figure 1 ) .
the 
 typical usage of a 
 score such as the c - score is in high throughput data processing .
an 
 operator picks a threshold level of the score and then asserts that 
 any query scoring above the threshold identifies the target data . 

this is done routinely in annotating dna sequences with tools such 
 as blast . to test the utility of a new scoring model , a set of data 
 where
the correct answer can be considered known is needed ; this forms 
 the  ground truth  of the analysis .
when the new scoring 
 system gives the known true answer both as the highest score , and 
 the scoring above the operator - defined threshold , the system is said 
 to have delivered a  true positive  .
likewise , when 
 the best proteoform scoring above threshold is not the known correct 
 answer , it is scored as a false positive , and so on for both true 
 and false negatives .
sensitivity is the proportion 
 of positives which are actually classified as such for a given threshold . 
 likewise , specificity is the proportion of negatives 
 that are correctly identified as such . for
any given arbitrary 
 threshold of a score , a specificity and sensitivity can be calculated 
 from a known data set . by iterating over a number of thresholds , and 
 plotting the resulting sensitivity against its corresponding specificity 
 ( or by convention , against 1-specificity ) a  receiver operating 
 characteristic  curve
this curve is known to 
 be indicative of the utility of the classifier . by repeating this 
 iterative analysis of with multiple scoring systems , the relative 
 utility of the systems
so , to compare the 
 new c - score with the existing prosight e - value , test input data sets 
 with known correct answers are needed .
these data sets are searched 
 against the appropriate proteoform database and , for each proteoform 
 returned , the e - value and c - score is calculated .
next , an arbitrary 
 list of threshold values is generated for each score , and for each 
 threshold value in this list , a list of identified proteoforms is 
 generated .
the identified proteoform is taken as the form that is 
 both above the threshold score , and to be the highest scoring proteoform . 

it is therefore possible for a search not to return a proteoform , 
 if no proteoform scores above the cutoff threshold . when searching 
 a test data set with known correct answers , it is possible to classify 
 the search results as either a true positive , true negative , false 
 positive , or false negative . from these classifications ,
these specificity 
 and sensitivity couplets then become points on the roc chart curve 
 for a given score .
the human data files used in this analysis 
 were acquired in the course other studies published previously .
briefly , mitochondrial membrane proteins were isolated from hela 
 s3 cells and separated using a gelfree 8100 fractionation system ( expedeon ) 
 as described previously . for the analysis at 
 hand , 295 top down experiments from a nanolc - velos orbitrap elite 
 ms analysis of a gelfree fraction containing 1520 
 kda proteins
were selected for intensive manual interrogation to provide 
 a set of highly curated  known positives  to test parameter 
 sets within the generative models used in development of the c - score 
 framework .
pseudomonas aeruginosa were 
 grown on rich media to mid - log phase , were isolated by centrifugation , 
 lysed , separated with gelfree , and analyzed with mass spectrometry 
 using the methods referenced above . for the secondary data set ,
a descriptive analysis of intact 
 and tandem mass spectral data was performed using rawmeat 2.1 ( http://vastsci.com/rawmeat/ , vast scientific ) . using these 
 data ,
related ms2 scans were merged and individual prosight upload 
 format ( puf ) files were created for each 
 data set . a combination of qualbrowser and prosightpc 3.0 ( both thermofisher , 
 san jose , ca ) was used , and all potential precursors predicted to 
 have been in the ms1 isolation window were added as potential proteoforms 
 for the experiments . both ms1 and ms2 data were deisotoped using the 
 xtract algorithm embedded within prosightpc 3.0 , generating 623 single 
 experiment puf files .
( containing 460  000 
 forms ) and  homo_sapiens_2012_02_top_down_complex  ( containing 
 just over 10 million forms ) which are prosight warehouses ( .pwf files ) 
 for human proteins built against uniprot release 2012_02 , available 
 for download ( ftp://prosightftp:gsx1gon@prosightpc.northwestern.edu/2012_02/eukaryotes/homo%20sapiens/ ) . experiments with single 
proteoforms were 
 selected and verified by at least two group members trained in top 
 down proteomics data analysis , to be considered as true answers , and 
 while subjective each met the following two criteria : the most abundant 
 fragment ions must be accounted for , and the intact mass difference 
 between theoretical and experimental must be small ( < 10 ppm ) or 
 must be explainable ( 1 da errors from deisotoping , a previously 
 unknown or unannotated post - translational modification , oxidation , 
 etc . ) .
this will typically involve the assignment of > 50% of fragment 
 ions appearing at a signal - to - noise of 10:1 or higher .
however , we 
 note that both experimental conditions ( e.g. , overfragmentation , generating 
 internal ions ) and the selection of data processing parameters will 
 affect the quality of data and thus the fraction of ions matched for 
 each experiment .
if evidence was not sufficient to uniquely 
 identify one proteoform for an experiment ( arising either from poor 
 data quality or from multiple proteins being fragmented simultaneously ) , 
 the experiment was excluded from further study here . using these heuristics , 
 295 puf files where only one proteoform was present were selected 
 for use here .
these files are described in supporting 
 information table 1 and provided in supporting 
 information data set 1 . to facilitate a comparison of c - scores 
 and e - values , a custom c #
this application takes , as input , a collection of puf files , the list 
 of manually validated correct answers , and the c - score version number 
 ( with a specific parameter set ) to use .
the c - score version allows 
 various iterations of generative models to be tested against the e - value 
 and each other .
the output from the application is an excel spreadsheet 
 containing a correct score and actual score for both the e - value and 
 the c - score . during execution , this application runs prosightpc absolute mass and biomarker searches on each 
 puf file to compile actual search scores .
the correct scores are then 
 calculated using the list of correct answers ( supporting information table 1 ) .
the c - score will be available 
 for testing within the proteoform characterization tool , hosted by the consortium for top down proteomics 
 ( http://www.topdownproteomics.org ) . to 
 compare the e - value to the c - score
, we 
 chose as our first input a set of 295 puf files ( described above ) 
 and a c - score version of 1.0 .
the list of correct proteoforms for 
 each of the 295 targets was provided as a csv file .
the custom console 
 application was then used to generate an excel spreadsheet that was 
 further analyzed .
two roc curves ( one for each score ) were plotted 
 on the same chart by checking , for each target , whether the top scoring 
 proteoform was the correct proteoform ( figure 4 ) .
subsequently , additional roc curves were generated , where a target 
 was counted as incorrect if the correct proteoform merely tied for 
 the best score , but other proteoforms also shared the same score value . 

we also calculated the area under each curve as a quantitative measure 
 of difference .
two additional roc curves where generated as before 
 but the correct answer would only be considered a true positive if 
 it uniquely had the best score . thus , for these curves , if the correct 
 proteoform failed to out - score all other proteoforms in the database 
 it was not considered a true positive .
this process was repeated for 
 136 proteoforms from pseudomonas aeruginosa ( data are provided in supporting information data set 2 ) .
( a ) receiver operating characteristic curves comparing 
 c - scores 
 and e - values on the 295 experiments in the manually curated test data 
 set .
the area under the curve for the blue c - score is 0.99 compared 
 to 0.78 for the orange e - value .
notice the large difference in sensitivity 
 of the e - value in the low fpr region , between when ties for the best 
 score are considered as acceptable for identification or not .
although 
 present in c - scores , the problem is much less pronounced , and at a 
 much higher sensitivity .
( b ) histogram of the 295 c - scores obtained 
 from searching the human database ( forward ) compared to the c - scores 
 obtained from searching the same experimental data against a scrambled 
 decoy database .
of the decoy hits , only 7% had a c - score above 40 , 
 likewise 42% of the forward c - scores are above this value .
the c - score 
 was substantially better than the e - value at identifying 
 and characterizing the correct proteoform from the data set of 295 
 human test cases , as well as the 136 pseudomonas test 
 cases .
figure 4 shows a receiver operating 
 characteristic ( roc ) curve for both scores on the human data , and 
 the area under the curve for the c - score was 0.99 compared to only 
 0.78 for the e - value . for the pseudomonas test cases
, 
 the c - score also outperformed the e - value with aucs of 0.85 to 0.80 , 
 respectively .
clearly , the c - score ( with the v1.0 parameter 
 set ) dominates the 
 e - value for all levels of specificity and sensitivity .
since the e - value 
 is simply a nonlinear transformation of the number of matching fragment 
 ions , it seems reasonable to assert that this improvement comes from 
 the new score s ability to include additional factors such 
 as known fragmentation propensities and ms1 mass differences both 
 of which are known to be relevant in characterizing proteoforms .
these 
 and other such factors are not considered by the current e - value , 
 nor is the score easily extended to consider such factors . when 
 the data are sufficient to completely characterize a proteoform , 
 the c - score is often well above 40 , indicating hyperconfidence in 
 the characterization power of the underlying data ( figure 5a ) .
however , a major limitation of the e - value occurs 
 when many proteoforms share the same ( seemingly confident ) score . 

this can be seen in figure 4 as the vertical 
 distance between the two e - value lines in the specificity range of 
 0.81.0 ( fpr 0.00.2 ) .
figure 5b and c shows two example cases where there is equal evidence for 
 two distinct proteoforms , one with an n - terminal acetylation and the 
 other with the acetylation localized to k7 . in this example
, these 
 two proteoforms share a confident e - value of 7  10 ; however , their c - scores are tied , yet at a much less confident 
 value of 3 . using the data from these two manually annotated sample 
 sets , and with the understanding of the c - score model , we assert three 
 operating ranges for the c - score : c - score > 40  proteoform 
 is both identified and fully characterized ; 3  c - score  
 40  proteoform is identified , but only partially characterized ; 
 c - score < 3  proteoform is neither identified nor characterized .
( a ) a 
 result with a very high c - score of 525 , indicating a fully 
 characterized proteoform of protein cytochrome b - c1 , o14957 .
( b , c ) two proteoforms with equivalent e - values 
 and c - scores for 10 kda heat shock protein , p61604 .
these data show 
 the complementarity between the e - value ( scores protein identification 
 well ) and the c - score ( reflects confidence in characterization of 
 related proteoforms on a phred - like score from 0 to > 500 ) . to achieve separation between 
 the two proteoforms in figure 5b and c , future 
 iterations of the c - score
will allow 
 for differential probabilities for n - terminal acetylation and internal 
 acetylation .
n - terminally acetylated , so one may posit that , in the absence of evidence to the contrary , 
 n - terminal acetylation is more likely than internal acetylation . the 
 c - score model can incorporate this bias and many similar issues that 
 provide biochemical and biological context . to achieve even more specificity 
 and sensitivity in the c - score framework , one may use a proteomic 
 database , such as gpmdb , to use peptide - level information linked to 
 the specific gene product to inform these differential probabilities 
 in top down proteomics experiments . in 
 the case of uniprot entry p61604 , gpmdb reports that n - terminal 
 acetylation was observed in 69/69 studies , making the proteoform reported 
 in figure 5b a much more likely than the proteoform 
 reported in figure 5c .
the characterization 
 of high - throughput lc  ms / ms experiments , 
 where hundreds of ms / ms targets are automatically generated for each 
 lc run , forms the foundation of high throughput top down proteomics . 

our probabilistic formulation leverages the knowledge of fragmentation 
 propensities and may be further extended to incorporate other types 
 of prior information to improve the scoring ( vide supra ) . to characterize 
 unknown ptms ( e.g. , a previously unknown methylation )
, we can extend 
 our c - score model to give a higher probability to common ptm mass 
 shifts ( i.e. , 42.0105 da for acetylation , 79.966 da for phosphorylation , 
 etc . ) , thereby boosting the data likelihood
. this will be particularly 
 useful in complex eukaryotic proteomes where the inclusion of all 
 theoretically possible proteoforms in the database ( even at search 
 run time ) is computationally prohibitive .
this framework provides 
 a great deal of flexibility for having 
 an appropriate scoring systems for a given experimental procedure . 

our current assumption of uniform priors on candidate proteins makes 
 our procedure a maximum likelihood ( ml ) inference , but the framework 
 can be readily extended to allow for nonuniform priors without additional 
 overhead .
nonuniform priors may be useful when researchers have a 
 priori belief that some protein forms ( e.g. , particular combinations 
 of ptms or splice variant ) are more likely to be present in the sample 
 ( e.g. , from rna - seq data ) .
further , the score remains robust when 
 used with instruments with decreasing mass accuracy .
supporting information figure 2 shows the roc chart resulting 
 from rerunning the analysis on the human test data set with increasing 
 mass tolerances . as tolerance increases , the discriminative power 
 of the score decreases as more fragment ions will match by chance .
in sum , we have shown a promising scoring system for protein identification 
 and proteoform scoring to better capture the information content in 
 high - resolution top down proteomics .
the c - score model lays forth 
 a path to increase the sophistication of protein identification and 
 characterization platforms to extract maximum value from ms - based 
 proteomics in automated fashion .
the conceptual and demonstrable advances 
 outlined above provide a deterministic process to advance the utility 
 of proteomics for nonexperts .
non - bioinformaticists / non - proteomicists  
 to advance protein - based science based on proper description of the 
 fully articulated primary structure for whole proteoforms .
when coupled 
 with high value experimentation , we posit that the faithful and more 
 efficient conversion of data streams into knowledge will significantly 
 advance major breakthroughs in mechanistic biology and on the front 
 lines of disease research including improved discovery and validation 
 of protein - based biomarkers .
 the 
 automated processing of data generated by top down proteomics 
 would benefit from improved scoring for protein identification and 
 characterization of highly related protein forms ( proteoforms ) . here 
 
 we propose the  
 c - score  ( short for characterization 
 score ) , a bayesian approach to the proteoform identification and characterization 
 problem , implemented within a framework to allow the infusion of expert 
 knowledge into generative models that take advantage of known properties 
 of proteins and top down analytical systems ( e.g. , fragmentation propensities , 
  off - by-1 da  discontinuous errors , and intelligent 
 weighting for site - specific modifications ) . 
 the performance of the 
 scoring system based on the initial generative models was compared 
 to the current probability - based scoring system used within both prosightpc 
 and prosightptm on a manually curated set of 295 human proteoforms . 
 
 the current implementation of the c - score framework generated a marked 
 improvement over the existing scoring system as measured by the area 
 under the curve on the resulting roc chart ( auc of 0.99 versus 0.78 ) . 