Gonadal steroids, in particular 5 alpha-dihydrotestosterone (DHT) and 17 beta-estradiol (E2), have been shown to feed back on the hypothalamic-pituitary-gonadal (HPG) axis of the ranid frog. However, questions still remain on how DHT and E2 impact two of the less-studied components of the ranid HPG axis, the hypothalamus and the gonad, and if the feedback effects are consistently negative. Thus, the goal of the study was to examine the effects of DHT and E2 upon the HPG axis of the gonadally-intact, sexually mature male leopard frogs, Rana pipiens.R. pipiens were implanted with silastic capsules containing either cholesterol (Ch, a control), DHT, or E2 for 10 or 30 days. At each time point, steroid-induced changes in hypothalamic GnRH and pituitary LH concentrations, circulating luteinizing hormone (LH), and testicular histology were examined.Frogs implanted with DHT or E2 for 10 days did not show significant alterations in the HPG axis. In contrast, frogs implanted with hormones for 30 days had significantly lower circulating LH (for both DHT and E2), decreased pituitary LH concentration (for E2 only), and disrupted spermatogenesis (for both DHT and E2). The disruption of spermatogenesis was qualitatively similar between DHT and E2, although the effects of E2 were consistently more potent. In both DHT and E2-treated animals, a marked loss of all pre-meiotic germ cells was observed, although the loss of secondary spermatogonia appeared to be the primary cause of disrupted spermatogenesis. Unexpectedly, the presence of post-meiotic germ cells was either unaffected or enhanced by DHT or E2 treatment.Overall, these results showed that both DHT and E2 inhibited circulating LH and disrupted spermatogenesis progressively in a time-dependent manner, with the longer duration of treatment producing the more pronounced effects. Further, the feedback effects exerted by both steroid hormones upon the HPG axis were largely negative, although the possibility exists for a stimulatory effect upon the post-meiotic germ cells. Rana catesbeiana, in which gonadectomy elevated circulating gonadotropins, and estrogen and androgen replacement suppressed this elevation [2) and 5α-dihydrotestosterone (DHT), could directly target the pituitary to modulate the release of luteinizing hormone (LH) and follicle-stimulating hormone (FSH) in these frogs [It is well established in mammals that gonadal steroid hormones are potent negative feedback regulators of the hypothalamic-pituitary-gonadal (HPG) axis. In ranid frogs, the first evidence supporting this notion came from a study on the bullfrog, levation . It was se frogs .2 as feedback regulators of gonadotropin secretion in ranid frogs, there is still some confusion regarding the exact nature of these feedback effects. For example, in female and juvenile R. catesbeiana, DHT suppressed the post-gonadectomy rise in circulating gonadotropins [R. pipiens), DHT had no effect on the post-castration rise in gonadotropin in males [2 were less variable, some conflicting data also exist. In both R. catesbeiana and R. pipiens, E2 consistently inhibited LH and FSH secretion both in vivo and in vitro, demonstrating a direct and powerful negative feedback effect of E2 at the level of the pituitary [2 treatment significantly stimulated the proliferation of primary spermatogonia (I SPG) in the green frogs, R. esculenta [2 in the positive feedback of the HPG axis.Despite the established role of DHT and Eotropins , yet it otropins ,5, suggeituitary -3. Howevsculenta ,7, sugge2 and DHT impact the HPG axis in gonadally-intact male R. pipiens. We will do so by measuring parameters that reflect the function of the HPG axis, including hypothalamic GnRH, pituitary and circulating LH, and spermatogenic activity. Moreover, steroid treatments were administered over two periods, 10 days and 30 days, to examine if the nature of steroidal feedback effects remains consistent over time. These results should allow us to determine if DHT and E2 act consistently as negative feedback regulators at different levels of the HPG axis.Results from the previous studies revealed the complex nature in which estrogen and androgen feed back on the reproductive axis, and suggest that the nature of the feedback effects might vary depending on the species, sex, reproductive stage of the frogs used, and the duration of steroid hormones administered. Further, there are still significant gaps in our knowledge regarding how these two steroid hormones feed back on the other two components of the HPG axis, the hypothalamus and the gonad. Therefore, the goal of the present study is to understand how ERana pipiens, were obtained from Carolina Biologicals (Burlington, NC) from April to July of 2004. Since the life histories of these animals were not entirely clear, all experiments were conducted within the three-month period to minimize the possible confounding effects of seasonality. As a control, we performed histological analysis on testes of representative frogs from every batch. When testicular histology was compared, little differences were observed among batches of frogs arriving at different times (data not shown). Frogs were kept under a 12L:12D photoperiod and fed live crickets every other day. All frogs were allowed to acclimate to the laboratory environment for at least one week before surgical implant.All experimental procedures were conducted in compliance with the animal protocol approved by the Institutional Animal Care and Use Committee at the University of Colorado. Mature male northern leopard frogs, 2 were prepared as previously described [2 (Sigma, St Louis, MO) and sealed on both ends with silicon glue. Ch implant was widely used as a negative control for the experimental treatment of cholesterol-based compounds such as steroid hormones [R. pipiens. Filled capsules were equilibrated in 0.6% saline for at least 48 hours before implant. For implant, frogs were anesthetized by immersion in 0.03% benzocaine. A small incision was made at the base of the left leg and a silastic capsule inserted subcutaneously. The incision was then closed with silk thread. Frogs were monitored for recovery from anesthesia and then returned to their respective holding tanks.One-cm silastic capsules containing crystalline cholesterol (Ch: control), DHT, or Eescribed , with sohormones -11. In aTen or 30 days after implant, frogs were weighed and sacrificed by quick decapitation using a guillotine. Trunk blood was collected into heparinized tubes, centrifuged, and plasma stored at -70°C until the measurement of LH and steroid hormones by radioimmunoassays (RIAs). Testes were removed, their masses recorded, and immersion-fixed in Bouin's fixative overnight. Hypothalami were excised from the brain by four cuts: a coronal cut 1 mm rostral to the optic chiasm, a coronal cut on the caudal border of the optic tectum, and two sagittal cuts along the lateral margins of the median eminence. Hypothalami were flash-frozen on dry ice and stored at -70°C until extraction and the measurement of GnRH by RIA. Pituitary glands were removed, sonicated in 500 μl phosphate-buffered saline (PBS), and stored at -70°C until the measurement of LH by RIA. All carcasses were later inspected for the presence of the silastic capsule in the legs. Animals whose capsules were lost were excluded from data analysis.R. catesbeiana) [R. pipiens [Plasma and pituitary LH levels were measured by a LH RIA developed for the bullfrog (sbeiana)  and vali pipiens . The iod125I]GnRH added to representative homogenates prior to extraction, was 86%. GnRH RIA was carried out with an antiserum specific for the mammalian form of GnRH (R1245, provided by Dr. Terry Nett at the Colorado State University) using a protocol described in detail elsewhere [Hypothalamic GnRH was extracted with 1 N HCl from the frozen tissues as previously described . The reclsewhere ,14,15. T2 and DHT RIAs were performed using the RIA kits from Diagnostic Systems Laboratories (Webster, TX). These RIA kits have been validated previously for the measurement E2 and DHT in R. pipiens [2 RIA and 4 pg/ml for the DHT RIA. The intra- and inter-assay coefficients of variation were 5.3% and 4.9%, respectively, for the E2 RIA, and 3.1% and 8.4%, respectively, for the DHT RIA. Both RIAs are highly specific and cross-react minimally with other steroid hormones.E pipiens . The limAfter fixation, testes were dehydrated through ascending concentrations of ethanol, defatted in Histoclear, and embedded in paraffin. Thirteen-μm sections were cut on a rotary microtome, mounted on poly-L-lysine-coated slides, and stained with hematoxylin and eosin.Testes were processed for ICC of PCNA, a cell cycle S phase marker, to identify I SPG and secondary spermatogonia (II SPG) undergoing cell proliferation . Testicuet al. [Five to eight testes  per treatment group were assessed for the following histological parameters: seminiferous tubule diameter, the number of I SPG per tubule, and the number of cysts within each tubule containing II SPG, primary spermatocytes (I SPC), or secondary spermatocytes (II SPC). These germ cells were defined according to Rastogi et al. . To scor10-transformed data followed by the Tukey's post-hoc test. Differences were considered significant when P < 0.05.Differences among groups were analyzed by the one-way analysis of variance (ANOVA) on log2 levels were 540 ± 110.5  and 1966 ± 13.2 pg/ml , and circulating DHT levels were 0.9 ± 0.3  and 37.5 ± 4 ng/ml . For animals implanted for 30 days, circulating E2 levels were 488 ± 370  and 2255 ± 650 pg/ml , and circulating DHT levels were 2.1 ± 1.4  and 24.6 ± 2.7 ng/ml .Steroid hormone RIAs were performed to monitor circulating steroid hormone levels in Ch- and hormone-implanted animals. For animals implanted for 10 days, circulating E2 groups implanted for 10 or 30 days , the diameter of the seminiferous tubules, and the presence of four germ cell types (I SPG, II SPG, I SPC, II SPC) in the testes. Overall, no differences were observed in any of the six parameters among treatment groups in the 10-day-implanted animals Fig. . HoweverGSI Fig. , whereasGSI Fig. . NeitherGSI Fig. . RepreseGSI Fig.  showed mGSI Fig. . In contes Figs. . Whereases Figs. , DHT- anPC Figs. . The loses Figs. , a resules Figs. . Interesen Figs. .2  of GnRH because this is the predominant hypophysiotropic form of GnRH in the diencephalon of ranid frogs [R. pipiens is at present unclear and awaits further investigation.DHT and Eid frogs  and the id frogs . Howeverid frogs , and itsid frogs . The abi2 treatment for the longer duration (30 days) significantly reduced pituitary LH concentration, although a substantial amount of LH still remained in the pituitary glands of these E2-treated animals. Thus, the potent suppression of circulating LH in animals treated with E2 for 30 days was most likely attributable to the low secretory activity of the pituitary gonadotropes rather than the depletion of LH stores. Somewhat surprising was the inability of E2 treatment for 10 days to suppress circulating LH. The pituitaries of ranid frogs were shown to be extremely sensitive to the inhibitory actions of E2. In R. pipiens, in vitro exposure of the pituitary to E2 concentrations as low as 100 pg/ml for only 48 hours significantly suppressed both basal and GnRH-stimulated gonadotropin secretion [2 implants could elevate circulating E2 to about 2000 pg/ml, one might expect a substantial decline in circulating LH of these animals after only 10 days of implant, but this was not the case. It is possible that additional in vivo mechanisms exist in these frogs to buffer against short-term estrogenic inhibition. Some of these might include changes in the clearance rate of circulating LH [2 by binding to E2 and decreasing the availability of the bioactive hormone.Only Eating LH  and the ating LH . The forR. pipiens. It is at present unclear if reduced circulating LH is a direct consequence of suppressed secretory activity of the gonadotropes or reduced output from the GnRH system. Based on the previous report that DHT had no direct inhibitory effect on pituitary gonadotropin secretion [Another unexpected observation was that DHT treatment for 30 days also suppressed circulating LH. Our results differ from a previous study demonstrating the inability of DHT to suppress post-gonadectomy rise in LH  and showecretion , it seem2 was observed at the level of the testes. In animals implanted for 30 days with these two steroids, a marked loss of several germ cell types (I SPG, II SPG, and I SPC) was observed. Interestingly, although E2 exerted a greater disruptive effect on spermatogenesis, similar trends of reduction with virtually no qualitative difference were also observed in DHT-treated testes, suggesting similar outcome might be attained with longer DHT exposure. That DHT- and E2-induced disruption of spermatogenesis differed only in the degree of severity suggests the disruption occurred primarily via a common pathway, possibly through the inhibition of gonadotropins. In this study, we could not measure circulating FSH because we lack a homologous FSH RIA. However, previous studies have reported that FSH secretion in R. pipiens was under the negative control of E2 [One of the most pronounced negative feedback effects of DHT and E2-treated animals. Although the significant reduction of I SPG in E2-treated testes could partially account for the loss of germ cell types that arose from I SPG, this cannot be the sole cause. For example, substantial PCNA-positive I SPG still remained in the testes of frogs implanted with E2 for 30 days, yet virtually no II SPG remained in the testes of these animals. Similarly, in 30-day-DHT-treated testes, there was no significant decline in I SPG, but II SPG were markedly reduced. These observations indicate a disproportionate loss of II SPG that may have resulted from their failure to survive. These data were consistent with a previous report in the newt that the mitotic penultimate SPG failed to survive when circulating FSH was suppressed [Although the roles of FSH and LH in anuran spermatogenesis are not entirely clear, data from other amphibians suggest FSH is essential for supporting the proliferation and survival of spermatogonia ,27. Impoppressed . Taken t2- or DHT-implanted frogs were mature spermatozoa, which occupied the largest bulk of the tubular lumen. It is possible that low circulating FSH had little influence on the germ cells once they entered meiotic division. Under low circulating LH, spermiation was inhibited, and mature spermatozoa continued to accumulate in the tubule. Another possibility is that E2 and DHT, while suppressing the presence of pre-meiotic germ cells, actually stimulated the entrance of existing I SPC into meiosis and promoted the survival of post-meiotic germ cells. This possibility was partially supported by the previous observation that testes of frogs treated with DHT for 20 days had fewer II SPG, but significantly more I SPC in the midst of meiotic division [2 facilitated the progression of spermatogenesis to the extent that the intermediate germ cell types could no longer be adequately replenished, leaving tubules filled with post-meiotic cells (primarily spermatozoa) and very little else. Regardless, the trend towards reduced GSI in steroid hormone-treated animals, along with the reduced presence of pre-meiotic germ cells, indicate an overall negative effect of these hormones upon the testes. The abundance of spermatozoa in DHT- and E2-treated animals nevertheless raised an interesting possibility for the existence of a positive steroidal effect on spermatogenesis.An interesting observation is that although pre-meiotic germ cells (I SPG, II SPG, and I SPC) were adversely affected by steroid hormone implants at 30 days, meiotic or post-meiotic germ cells (II SPC, spermatids and spermatozoa) appeared unaffected or stimulated. In fact, the most conspicuous germ cells in the seminiferous tubules of 30-day Edivision . Along t2, in addition to affecting spermatogenesis by lowering circulating gonadotropins, have also been shown to act directly upon the amphibian testes. Both androgen and estrogen binding sites were found in the amphibian testes [2 acted directly upon the amphibian testes to suppress androgen secretion [2 directly stimulated SPG I proliferation in R. esculenta [R. pipiens. Whether or not this discrepancy was due to the differences in experimental paradigms or species used is at present unclear.Worth mentioning is the possibility that DHT and En testes -33. A nuecretion -36, stimecretion ,38, and ecretion . Similarecretion . Of intesculenta ,37. Unde2 for 20 days [2 reduced the presence of II SPG and I SPC, whereas DHT reduced only the presence of the former. However, the study represented only a snapshot in time, so no information was available regarding the progression of events that led to the altered formation of germ cells. Moreover, it was unclear if the treatment with these two steroid hormones for shorter or longer periods could impact the testes differently. Our current results showed that both steroids inhibited circulating gonadotropin and disrupted spermatogenesis progressively in a time-dependent manner, with the longer duration of treatment producing the more pronounced effects. Further, the changes in the testes were qualitatively similar between DHT and E2 treatments, suggesting declining gonadotropin levels might be the common underlying cause for the disrupted spermatogenesis. These results reflect the highly sensitive nature of the anuran reproductive axis to estrogenic and androgenic modulation. We showed that the continuous exposure of mature frogs to high physiological levels of steroid hormones for a relatively short period could profoundly alter their pituitary and testicular function. In particular, the potency of estrogen hormones raises concerns regarding the potential reproductive disruption that can occur when mature frogs are exposed to short-term and low-level environmental estrogen mimics.We previously showed that spermatogenesis in the ranid frogs was altered in mature male frogs implanted with DHT and E 20 days . SpecifiPST designed the experiments, analyzed the data, and prepared the manuscript. AEK and JTJ prepared the silastic capsules, implanted the animals, removed the tissues, and performed all the hormone measurements. KBW implanted some animals, prepared all histological samples, counted the germ cells, and performed the PCNA ICC. All authors read and approved the final manuscript.


Conserved protein sequence motifs are short stretches of amino acid sequence patterns that potentially encode the function of proteins. Several sequence pattern searching algorithms and programs exist foridentifying candidate protein motifs at the whole genome level. However, amuch needed and importanttask is to determine the functions of the newly identified protein motifs. The Gene Ontology (GO) project is an endeavor to annotate the function of genes or protein sequences with terms from a dynamic, controlled vocabulary and these annotations serve well as a knowledge base.This paperpresents methods to mine the GO knowledge base and use the association between the GO terms assigned to a sequence and the motifs matched by the same sequence as evidence for predicting the functions of novel protein motifs automatically. The task of assigning GO terms to protein motifsis viewed as both a binary classification and information retrieval problem, where PROSITE motifs are used as samples for mode training and functional prediction. The mutual information of a motif and aGO term association isfound to be a very useful feature. We take advantageof the known motifs to train a logistic regression classifier, which allows us to combine mutual information with other frequency-based features and obtain a probability of correctassociation. The trained logistic regression model has intuitively meaningful and logically plausible parameter values, and performs very well empirically according to our evaluation criteria.In this research, different methods for automatic annotation of protein motifs have been investigated. Empirical result demonstrated that the methods have a great potential for detecting and augmenting information about thefunctions of newly discovered candidate protein motifs. With the completion of many genome sequencing projects and advances in the methods of automatic discovery of sequence patterns  project  is a conThe basic approach for determining the function of a motif is to study all the sequences that contain the motif (pattern). Intuitively, if all the functional aspects of the sequences matching a motif are known, we should be able to learn which function is most likely encoded by the motif, based on the assumption that every protein function is encoded by an underlying motif. This means that we would need a knowledge base of protein sequences, in which the functions of a sequence are annotated as detailed as possible. In addition, we would also need prediction methods that can work on a given set of protein sequences and their functional descriptions to reliably attribute one of the functions to the motif that matches these sequences. To determine the function of any novel motif, we would first search the protein knowledge base to retrieve all the functional descriptions of the proteins containing the motif, and then use such prediction methods to decide which function is encoded by the motif. In this research, we use the Gene Ontology database as our protein knowledge base and explore statistical methods that can learn to automatically assign biological functions (in the form of GO terms) to a protein motif.vice versa. However, the database usually returns more than one GO terms that may or may not describe the function of the motif in the query. Thus, we need methods to disambiguate which GO term describe the function of the motif (assign a GO term to a motif) and determine how confident we are as the assignment is concerned. We use statistical approaches to learn from known examples and cast disambiguation task into a classification problem. Furthermore, the probability output by the classifier can be used to represent its confidence for the assignment.Our approach is based on the observation that the Gene Ontology database contains protein sequences and the GO terms associated with the sequences. In addition, the database also contains information of known protein motifs, e.g. the PROSITE patterns that match the sequences. Thus, the protein sequences in the database provide a sample of potential associations of GO term with motifs, among which some are correct  and some are not. This provides us an opportunity to perform supervised learning to identify discriminative features and use these features to predict whether a new association is correct or not. Current Gene Ontology database is implemented with relational database system, which allows one to perform queries like "retrieve all GO terms associated with the sequences that matches a given motif" and p-value. They found that, in the database they worked with, most sequences only had one functional GO term. Therefore, they could assign the GO term of a sequence to the motif that matched with highest score with fairly good accuracy. However, due to restrictive assumption that each sequence has only one GO term, their approach can not address the potential problem that a sequence matching a motif has multiple associated GO terms, which is common case now, and how to resolve such ambiguity.Recently, Schug et al  publisheWe use the May 2002 release of the Gene Ontology sequence database (available online ), which Using the information stored in the Gene Ontology and PROSITE, we manually judged a set of 1,602 cases of distinct PROSITE-GO associations to determine whether the association is correct or not. The PROSITE-GO association set has been judged in two different ways. One way is to label an association as correct if and only if the definition of the GO term and the PROSITE motif match perfectly according to the annotator. Gene Ontology has the structure of a directed acyclic graph (DAG) to reflect the relations among the terms. Most terms (nodes in the graph) have parent, sibling and child terms to reflect the relation of "belonging to" or "subfamily". The second way of judging GO-PROSITE association is to label an association as correct if the GO term and the PROSITE motif are either exact match or the definitions of GO term and PROSITE motif are within one level difference in the tree, i.e., the definition of GO term and the PROSITE motif have either a parent-child relation or a sibling relation according to the GO structure. Thus we have two sets of labeled PROSITE-GO associations, the perfect match set and the relaxed match set (with neighbors). Both sets are further randomly divided into training (1128 distinct associations) and test (474 distinct associations) sets. Since the test sample size is fairly large, the variance of the prediction accuracy can be expected to be small. Thus we have not considered any alternative split of training and test sets.T, and none of the sequences that do not match the motif is assigned the term T, then it is very likely that the motif pattern is encoding the function described by term T. Of course, this is only an ideal situation; in reality, we may see that most of, but not all of the proteins matching a motif pattern would be assigned the same pattern, and also some proteins that do not match the motif may also have the same term. Thus, we want to have a quantitative measure of such correlation between GO terms and motif patterns.Intuitively, we may think of the GO terms assigned to a protein as one description of the function of a protein in one language (human understandable) while the motifs contained in the protein sequence as another description of the same function in a different language . We would like to discover the "translation rules" between these two languages. Looking at a large number of annotated sequences, we hope to find which terms tend to co-occur with a given motif pattern. Imagine that, if the sequences that match a motif are all assigned a term X and Y [p(X = x, Y = y) with the expected joint distribution under the hypothesis that X and Y are independent, which is given by p(X = x)p(Y = y). A larger mutual information indicates a stronger association between X and Y, and I = 0 if and only if X and Y are independent.A commonly used association measure is mutual information (M.I.), which measures the correlation between two discrete random variables X and Y . It basiT to a sequence and the matching of a sequence with a motif M as two binary random variables. The involved probabilities can then be empirically estimated based on the number of sequences matching motif M (NM), the number of sequences assigned term T (NT), the number of sequences both matching M and assigned T (NT-M), and the total number of sequences in the database. Table For our purpose, we regard the assignment of a term c to define a simple decision rule: assign term T to motif M, if and only if I ≥ c. For a given cutoff c, the precision of term assignment is defined as the ratio of the number of correct assignments to that of the total assignments according to the cutoff c. In Figure We set out to test whether we can use mutual information as a criterion to assign a GO term to a PROSITE motif. One approach is to use a mutual information cutoff value most likely to be appropriate for the motif. To address this problem, we can use a different cutoff strategy, and adopt a decision rule that assigns a GO term to a motif based on the ranking of mutual information, which is a common technique used in information retrieval text categorization [M in the annotated data set, all observed motif-term associations containing M are retrieved and ranked according to mutual information, then the term that has highest mutual information is assigned to M. Alternatively, if we use this approach to facilitate human annotation, we can relax the rule to include GO terms that have lower ranks, thus allowing multiple potential GO terms to be assigned to a motif, assuming that a human annotator would be able to further decide which is correct. In this method, the key in making a decision is to select a cutoff rank that covers as many correct associations as possible (high sensitivity) while also retrieves as fewer incorrect associations as possible (high specificity). The optimal cutoff can be determined by the desired utility function.However, a drawback of such an approach is that, given a motif, sometimes, many observed motif-term associations can have mutual information above the cutoff value, making it difficult to decide which pair is correct. While in other cases, the mutual information of the observed motif-term pairs may all be below the cutoff value, but we still would like to predict what terms are rization . More spFigure While the mutual information measure appears to give reasonable results, there are three motivations for exploring more sophisticated methods. First, the mutual information value is only meaningful when we compare two candidate terms for a given motif pattern; it is hard to interpret the absolute value. While a user can empirically tune the cutoff based on some utility preferences, it would be highly desirable to attach some kind of confidence value or probability of correctness to all the potential candidate motif-term associations. Second, there may be other features that can also help predict the function (term) for a motif. We hope that the additional features may help a classifier to further separate correct motif-term assignment from wrong ones. Third, there exist many motifs with known functions (e.g., those in the PROSITE database), and it is desirable to take advantage of such information to help predict the functions of unknown motifs. This means that we need methods that can learn from such information. In this section, we show that the use of logistic regression can help achieve all three goals. Specifically, we use logistic regression to combine the mutual information with other features, and produce a probability of correct assignment. The motifs with known functions serve as training examples that are needed for estimating the parameters of the regression function.T) and PROSITE motif (M) co-occur (NT-M). (2) The number of sequences in which T occurs (NT). (3) The number of sequences in which M occurs (NM). (4) The number of distinct GO terms (G) seen associated with M (NG|M). (5) The number of distinct PROSITE patterns (P) seen associated with T (NP|T). In addition, we also consider, as a feature, the similarity of the sequences that support a motif-term pair. Intuitively, if a motif is conserved among a set of diverse sequences, it is more likely that the motif is used as a building block in proteins with different functions. Thus, the average pair-wise sequence similarity of the sequence set can potentially be used as a heuristic feature in the logistic regression classifier. Given a set of sequences, we use a BLAST search engine to perform pair-wise sequence comparisons. We devised a metric AvgS to measure the averaged pair-wise sequence similarity per 100 amino acids (see methods) and use it as an input feature for classifier.We now discuss the features to be used in logistic regression, in addition to the mutual information discussed in the previous section. The goal is to identify a set of features that is helpful to determine whether association of any pair of a GO term and a motif is correct or not, without requiring specific information regarding the function of GO term and motif. For a distinct motif-term pair, we collect following frequency-based features: (1) The number of sequences in which the GO term (Y, so that Y = 1 means correct assignment and 0 means incorrect. We represent a motif-term pair by a vector of features X = (X1,..., Xk), where k is the number of features. The seven features/variables used in our experiments are NT-M, NT, NM, NG|M, NP|T, AvgS, and M.I.. Suppose we have observed n motif-term pairs, then we have n samples of (yi, xi), i = 1, 2, ..., n, where, yi is the correctness label and xi is the feature vector for the corresponding motif-term pair. Our goal is to train a classifier which, when given a motif-term pair and feature vector X, would output a label Y with value 1 or 0. Alternatively, we can also consider building a classifier which outputs a probability that Y = 1 instead of a deterministic label. Thus, our task is now precisely a typical supervised learning problem; many supervised learning techniques can potentially be applied. Here, we choose to use logistic regression as our classification model because it has a sound statistical foundation, gives us a probability of correct assignment, and can combine our features naturally without any further transformation.To cast the prediction problem as a binary classification problem, we augment our data set of motif-term pairs with a class label variable G, given by 2l(D|β f)/l(D|β f-), where, l(D|β f) and l(D|β f-) are the log-likelihood of the model with feature f and the model without feature f, respectively. Since we add or drop one feature at a time, G follows χ2 distribution with degree of freedom of 1 [p-value of 0.1 as a significant threshold. Figure In order to build a model only with the truly discriminative features, it is a common practice to perform feature selection for logistic regression. We use a combined forward and backward feature selection algorithm. Starting from the intercept, we sequentially add features into the model and test if the log-likelihood increases significantly; we keep the current feature if it does. After the forward selection, we sequentially drop features from the model, to see if dropping a feature would significantly reduce the log-likelihood of the model; if it does, we exclude the feature from the model, otherwise continue. When testing the significance, we use the likelihood ratio statistic dom of 1 . We use p(Yi = 1|Xi) for each test case. Table After fitting the model using the training set, we tested the model on the test set, i.e., we used the model to compute an output Y = 1, thus, a small change in the number of cases introduces a large change in percentage. For example, when the threshold is set at 0.9, only three cases are covered by the rule and two of them are correct, thus percent correct drop to 66%.As the results from the logistic regression are the conditional probability that an association of a GO term with a given motif is correct, we need to decide the cut off threshold for making decision. We calculate the sensitivity and specificity for a different threshold from 0.1 to 0.9 with a step of 0.1 and plotted the ROC curves as shown in Figure C and D. We see that using mutual information alone performs almost as well as logistic regression with additional features. However, the area under the curve (0.816) is smaller than that of logistic regression (0.875), indicating that logistic regression does take advantage of other features and has more discriminative power than mutual information alone.To see whether the additional features are useful, we also performed ROC analysis using different mutual information cutoff threshold on the perfect match test set. The result is shown in Figure β1, β2 and β3 for the three features NT-M, NT and NM, which are also involved in the calculation of mutual information, have a very interesting interpretation – they indicate that the roles of these three variables in the logistic regression model actually are to compromise the effect of mutual information! Indeed, according to the formula of the mutual information, a strong correlation corresponds to a high NT-M, low NT, and low NM, but the coefficients shown in Table NT) with respect to any pattern matched by the sequence to which the term is assigned. But, intuitively, one occurrence is very weak evidence, and at least should be regarded as weaker than when we have a term occurring 10 times in total and co-occurring 9 times with the same motif. The key issue here is that mutual information only reflects the correlation between variables, but does not take into account the strength of evidence, therefore, tends to over-favor the situation where there is a perfect correlation but very little evidence. However, the number of sequences in which the co-occurrence happens, which is called the "support" for the association, is also very important.The coefficients NG|M and NP|T, are also meaningful. Their negative signs indicate that the more terms a motif co-occurs with or the more motifs a term co-occurs with, the less likely a particular association is correct. This also makes sense intuitively, since all those co-occurring terms can be regarded as "competing" for a candidate description of the motif's function, so the more terms a motif is associated with, the competition is stronger, and thus the chance that any particular term is a correct description of function should be smaller. Thus, the logistic regression model not only performs well in terms of prediction accuracy but also gives meaningful and logically plausible coefficient values.The coefficients for the other two parameters, In this paper, we explore the use of the Gene Ontology knowledge base to predict the functions of protein motifs. We find that the mutual information can be used as an important feature to capture the association between a motif and a GO term. Evaluation indicates that, even used alone, the mutual information could be useful for ranking terms for any given motif. We further use logistic regression to combine mutual information with several other statistical features and to learn a probabilistic classifier from a set of motifs with known functions. Our evaluation shows that, with the addition of new features and with the extra information provided by the motifs with known functions, logistic regression can perform better than using the mutual information alone. This is encouraging, as it shows that we can potentially learn from the motifs with known functions to better predict the functions of unknown motifs. This means that our prediction algorithm can be expected to further improve, as we accumulate more and more known motifs.new motifs e.g., those discovered using TEIRESIAS, SPLASH or other programs. This would further enable us to perform data mining from the Gene Ontology database in several ways. For example, we can hypothesize the functions of a large number of novel motifs probabilistically, then we will be able to answer a query, such as "finding the five patterns that are most likely associated with the GO term tyrosine kinase". This is potentially very useful because it is not uncommon that substantial knowledge about the functions and sub-cellular location of a given protein is available even though a structural explanation for the functions remains obscure. On the other hand, we believe that our methods will facilitate identifying potentially biological meaningful patterns among the millions of patterns returned by pattern searching programs. A sequence pattern that associates with certain GO term with high M.I. or probability is more like to be a meaningful pattern that that with low scores. Furthermore, our methods can also be used in automatic annotation of novel protein sequences as suggested in Schug et al and Rigoutsos et al [Although we have so far only tested our methods on the known motifs, which is necessary for the purpose of evaluation, the method is most useful for predicting the functions of new and unknown motifs. For the future work, we can build a motif function prediction system and apply our algorithm to many candidate os et al ,13,22. OHaving stated the potential uses of our approaches, we also realize that there exist some limitations for our methods. For example, in order to predict the function of a newly identified sequence pattern correctly, we would require functional annotations of the sequences of GO database be complete and accurate, which may not always be the case. In this paper, we mainly used the motifs with known function to evaluate the capability of the methods developed in this research. Our result shows that the methods work well with known sequences patterns. Currently, the annotation of motif function with GO term is carried out manually at the European Bioinformatics Institute (the GOA project). Such approach is warranted because human annotation is more accurate than automatic ones. However, as the amount of information regarding protein functions accumulates and a large number of new potential motifs are discovered, it will be very labor intensive to annotate the potential association of protein function and protein patterns. By then, the methods studied in this research will potentially prove to be useful to discover the underlying protein motifs that are responsible for the newly annotated function. For example, the methods can be used as prescreening to narrow down to the most possible associations of protein function and motifs, thus facilitate human annotation.In summary, we have developed methods that disambiguate the associations between of Gene Ontology terms and protein motifs. These methods can be used to mine the knowledge contained in the Gene Ontology database to predict the function of novel motifs, discover the basis of a molecular function at primary sequence level and automatically annotated the function of novel proteins.Mutual information is defined as followsp(X = x, Y = y), p(X = x) and p(Y = y) can be empirically estimated from the data by counting occurrence/co-occurrence followed by normalization.In which the probabilities The sensitivity and specificity of the rules are calculated asTP (True Positive) is the number of associations labeled as correct among the retrieved motif-term pairs meeting the ranking cutoff criteria, FN  is the number of associations labeled as correct but not retrieved, TN (True Negative) is the number of associations labeled as incorrect and not retrieved, and FP  is the number of associations labeled incorrect but are retrieved.where AvgS) of a sequence set is as followsCalculation of the average pair-wise sequence similarity per 100 amino acids  is a delta function which equals 1 if i = j and 0 otherwise.Where Sp(Y = 1|X) and X1, ..., Xk:The logistic regression model is a conditional model that assumes the following linear relationship between β = (β0, β1, ..., βk) is the parameter vector. We can fit the logistic regression model (i.e., estimate the parameters) using the Maximum Likelihood method – essentially setting the parameters to values at which the likelihood of the observed data is maximized . In our experiments, we use iteratively reweighted least squares (IRLS) algorithm [where, lgorithm  to fit t


Bax in NT-3 knockout mice rescues these neurons and allows for examination of their axon growth in the absence of NT-3 signaling. TrkC-positive peripheral and central axons from dorsal root ganglia follow proper trajectories and arrive in close proximity to their targets but fail to innervate them. Peripherally, muscle spindles are absent and TrkC-positive axons do not enter their target muscles. Centrally, proprioceptive axons branch in ectopic regions of the spinal cord, even crossing the midline. In vitro assays reveal chemoattractant effects of NT-3 on dorsal root ganglion axons. Our results show that survival factor NT-3 acts as a short-distance axon guidance molecule for muscle sensory afferents as they approach their proper targets.Neurotrophin-3 (NT-3) is required for proprioceptive neuron survival. Deletion of the proapoptotic gene  In vivo and in vitro experiments suggest a role for NT-3 in guiding axons from the spinal cord to their muscle targets Neurotrophin-3 (NT-3) is a key requirement for the development of proprioceptive inputs to motor neurons . Mice deNT-3 heterozygote and Bax knockout (KO) mice to obtain mice with double KO of both NT-3 and Bax genes, and examined proprioceptive axonal projections. All NT-3 and double KOs died within 48 h after birth (NT-3 KOs at embryonic day (E) 15. Two subsets of DRG cells expressing either TrkA or TrkC were detected in double KOs, similar to wild-type (WT) or Bax KO animals. Surprisingly, at postnatal day (P) 0, a few cells expressed TrkC even in NT-3 KO animals in the Bax+/+ genetic background, and some cells co-expressed TrkA and TrkC, regardless of the genotype   C.Bax KO embryos. There was no detectable TrkC expression in NT-3 KO spinal cord, indicating complete absence of proprioceptive fibers. In double KO spinal cord, TrkC-positive fibers entered the dorsal spinal cord and descended medially in a manner similar to that seen in WT or Bax KO cases. However, it was not possible to follow TrkC immunolabeled fibers all the way to their terminal zones in any of the cases. Next we examined the central projections of DRG axons with the lipophilic tracer DiI at P0. In WT and Bax KO pups, proprioceptive afferents entered the dorsal spinal cord and followed a medial course towards the ventral horn. They then turned laterally towards motor neurons in the lateral motor column, where they branched and terminated . Control cultures were set up using bovine serum albumin (BSA)– or phosphate buffer saline (PBS)–soaked beads (n = 12). DRG axons began extending towards the localized NT-3 source by the end of the first day and consistently displayed a strong chemoattraction by 3 d in vitro, whereas they did not show such preference for BSA-loaded control beads (n = 16). NT-3 may act through either TrkC, or the p75NTR. We repeated the co-culture experiments with DRG explants derived from NTRp75 KO mice (n = 18). Axons of these ganglia also showed strong chemoattraction towards the NT-3 beads (n = 6) to deplete soluble NT-3 from the collagen gels (n = 4 each) (n = 15). Control cultures were set using PBS-loaded beads (n = 6). DiI labeling through the DRGs revealed numerous fibers entering the spinal cord at ectopic regions and growing towards the NT-3 beads . We co-cultured collagen-embedded E13 WT DRG explants with NT-3-soaked sepharose beads  F. There -3 beads A, surrou-3 beads C–6E. In -3 beads B. No axo-3 beads F. Axons -3 beads G. NT-3 inestin promoter in the central nervous system, the course of the proprioceptive afferents are altered and directed towards the regions with high levels of ectopic NT-3 expression in the spinal cord  may represent cutaneous innervation . PresentBax/NT-3 double KO cases, and often incomplete DiI labeling gives the impression that there are no axons reaching the ventral spinal cord. We have also seen cases similar to theirs (n = 4), but at higher magnification these axons did not have terminal boutons and were not completely labeled. However, with complete fills (n = 5), it was possible to trace these axons into the ventral midline and across to the contralateral side, and visualize terminal boutons at their tips  to NT-3 heterozygote males on a 129 Sv background to generate double heterozygote animals. Progeny was genotyped with PCR, and animals heterozygous for both genes were bred to obtain the double KOs. Primers used for the Bax locus were R661, GTT GAC CAG AGT GGC GTA GG; R662, CCG CTT CCA TTG CTC AGC GG; and R663, GAG CTG ATC AGA ACC ATC ATG. Primers used for the NT-3 locus were NT3A, CGT GGT GAG GTT CTA TTG GCT AC; NT3B, CAG AGC ACC CTG CCC AAA GCA GAG; NT3R, CCT TGA CAA TAC TGA ATG CC; and NEOF, GGG AAC TTC CTG ACT AGG. WT, Bax KO, and NT-3 KO littermates were used as controls. A total of 11 Bax/NT-3 double KO mice were analyzed, of these nine were P0 pups. The p75 colony on a 129 S1 background was received from Jackson Laboratory. We used tail DNA to genotype animals using the primers IMR0013, CTT GGG TGG AGA GGC TAT TC; IMR0014, AGG TGA GAT GAC AGG AGA TC (generic neo primers); IMR0710, TGT TAC GTT CTC TGA CGT GGT GAG; and IMR0711, TCA GCC CAG GGT GTG CAC TC (p75 locus). For embryonic experiments, day of plug positivity was considered E0. All of the protocols used in this study were approved by the Louisiana State University Health Sciences Center Institutional Animal Care and Use Committee and conformed to the National Institutes of Health guidelines for use of experimental animals.We crossed Frozen spinal cord sections (10 μm thick) were blocked and incubated in a cocktail of rabbit anti-TrkA and goat anti-TrkC antibodies , and the gastrocnemius muscle was isolated and sectioned into 40-μm-thick slices on a vibratome.Gastrocnemius muscle was dissected out in P0 pups and sectioned longitudinally, or, in some cases, the whole leg at the level of gastrocnemius muscle with tibial nerve was sectioned in cross section. Sections were incubated with monoclonal S46 antibody  reactive to the spindle-specific slow-tonic myosin heavy chain isoform and developed by Vector mouse on mouse kit as described above, followed by rabbit anti-NF-M antibody (Chemicon) and CY3 conjugated goat anti-rabbit antibody (Chemicon) in a sequential double-labeling protocol. In another method, DiI crystals were placed in DRGs of E17 embryos from different genotypes  were inserted in DRGs. Labeled spinal cords were incubated at 37 °C for 8 d and sectioned on a vibrotome at 100 μm. Sections were observed under the fluorescent microscope and photographed, and images were transferred to Adobe Photoshop, inverted, and adjusted for brightness and contrast. Photoconverted in 0.15% DAB  in 0.1M Tris buffer (pH 8.2).2, and the pH was adjusted to 7.5. Individual ganglion explants were placed in 24-well plates and covered with freshly prepared collagen. Sepharose beads with an average diameter of 150 μm were used. Beads were washed twice with PBS, air dried, and loaded with 10, 20, 50, or 100 ng/μl NT-3 (Collaborative Research, Chemicon) at 4 °C overnight with constant shaking. For negative control, beads were loaded with BSA (10–100 ng/μl) or PBS. Either a single neurotrophin-loaded bead or a single BSA (or PBS)–loaded bead was implanted about 200–1,200 μm away from the ganglion explant. Collagen-embedded cultures were then placed at 33 °C for 15 min for the matrix to harden. Serum-free culture medium was then added to each well. In cultures with WT DRG and control beads, 10% serum was added to ensure viability of the explants. TrkC-Fc  was added (20 μg/ml) into the culture medium.DRG explants were derived from Swiss Webster, Bax null, and p75 null mouse embryos (E13). For collagen matrix assays, DRGs were dissected out under a stereomicroscope using tungsten needles. Collagen matrix was prepared with 430 μl of collagen (3 mg/ml dissolved in 0.1M acetic acid, Sigma), 50 μl of 10X DMEM medium, and 2.5 μl of 0.8 M NaCan = 15) or PBS-loaded (n = 6) sepharose beads were prepared as described above, and placed in the midline at mid-spinal cord level. Inserts were placed in six-well plates with serum-free medium at the bottom of the wells, and kept at 33 °C for 3 d in the presence of 5% CO2. Cultures were then fixed with 4% paraformaldehyde in PBS, and small crystals of DiI were placed into the DRGs. Samples were incubated in a 37-°C incubator, allowing the dye to diffuse, and photographed under a Nikon (Tokyo, Japan) inverted epifluoresence microscope.Spinal cords with attached DRGs were dissected out from Swiss Webster mouse embryos at E13, and sectioned into 300-μm-thick slices. Tissue slices were placed on Millicell Tissue Culture Inserts (Millipore, Billrica, Massachusetts, United States). NT-3-loaded  WT embryo showing proprioceptive axons contacting motor neuron dendrites in the ventral horn, forming synapses.(B) Bax/NT-3 double null spinal cord. Although labeled fibers enter the ventral spinal cord, they extend towards the midline instead of the ventral horn and never contact the motor neuron dendrites.(C) High-power image of the inset in (A). Arrows point to the proprioceptive fibers contacting motor neurons (asterisk).(D) High-power image of the inset in (B). Notice that there are no sensory axons contacting labeled motor neurons (asterisk).Scale bar: 100 μm (A and B), 50 μm (C and D).(24 MB TIF).Click here for additional data file.Figure S2Although numerous muscle and nerve fibers were labeled, no muscle spindles could be identified because the characteristic morphology of sensory nerve ending wrapped around muscle bag fiber had not begun to develop in any of the genotypes yet. Scale bar: 25 μm.(13 MB TIF).Click here for additional data file.


There have been significant improvements in the design and management of humanitarian aid responses in the last decade. In particular, a significant body of knowledge has been accumulated about public health interventions in emergencies, following calls for developing the evidence base of humanitarian health interventions. Several factors have prompted this, such as the increased volume of humanitarian assistance with subsequent higher levels of scrutiny on aid spending, and greater pressure for improving humanitarian aid quality and performance. However, documentation of the ability of humanitarian interventions to alleviate suffering and curb mortality remains limited. This paper argues that epidemiological studies can potentially be a useful tool for measuring the impact of health interventions in humanitarian crises. Survey methods or surveillance systems are mainly used for early warning or needs assessment and their potential for assessing the impact of aid programmes is underutilised. In the last decade, humanitarian aid agencies have put a great deal of effort into improving the design and management of humanitarian responses in conflict situations: humanitarian agencies have agreed on international standards, numerous technical guidelines have been produced, training programmes are offered on different aspects of humanitarian programming, and aid programmes are evaluated more systematically. Initiatives as distinct as the Code of Conduct, Sphere, Active Learning Network for Accountability and Performance in Humanitarian Action, the Humanitarian Accountability Partnership International, the Quality Project from the French group Urgence, réhabilitation, développement and People in Aid all have in common a concern for the quality, performance and impact of humanitarian assistance . In partDespite the technical progress in the last decade, knowledge of the impact of humanitarian interventions in alleviating suffering and ultimately reducing mortality in the health and other sectors remains limited. There is a real question whether all the developments in accountability and implementation actually improve the overall performance of humanitarian assistance ,8. WhereThis paper examines the current practice of humanitarian agencies for measuring the impact of health interventions. It is based on a review of the literature as well as on the authors' own field experience. The question explored is whether field epidemiology provides a useful set of tools and methods to determine more accurately the impact of health interventions.The issue of impact is particularly high on the current humanitarian agenda. The increasing interest in impact analysis arises from a number of interlinked developments: the rapid increase in the overall volume of humanitarian assistance in the last decade- from 2 billion in 1990 to $5.5 billion by 2000 [There is no definition of impact in humanitarian assistance. The most commonly used definition of impact in international aid is the one provided by the OECD/DAC which defines impact very widely as ' [t]he positive and negative, primary and secondary, long-term effects produced by a development intervention, directly or indirectly, intended or unintended' . This paA central question in an impact assessment is 'what would have happened in the absence of the aid programme?' In theory, there are two approaches to answering this question :1. To compare the impact with a control group that did not receive aid – a "with/without" comparison.2. To do a "before/after" comparison for the beneficiaries of an intervention.These two approaches pose different sets of issues. The "with/without" comparison with the use of control groups creates ethical problems: it is difficult to deliberately exclude a group from access to potentially life-saving relief. It may happen that some particular groups do not receive relief, due to problems with access or lack of resources, but, as Hallam warns, comparisons between people who received assistance and those who did not need to be used very carefully . As a reThe "before/after" comparison generates different problems. In order to be valid, the comparison implies that no other factor influences the impact, so that it can be fully attributed to the intervention. In reality, a multiplicity of other factors have an influence on the impact, such as the presence of other aid programmes, local coping mechanisms, or changes in the social and economic environment. For example, it is very difficult to attribute a reduction of conflict-related excess mortality to humanitarian aid only: variations in the baseline mortality  can be significant. There is an increasing recognition of the importance of these other factors, and that humanitarian aid, however vital, is only one element of the picture -18.There are three main considerations that are necessary for measuring the impact of health programmes: the strength of the evidence suggesting causation between the intervention and the change in health status, the validity of a baseline of comparison, and the validity of the indicator employed. These conditions also apply to other types of humanitarian assistance.Over time, a great deal of debate has arisen over what epidemiological evidence constitutes proof that some exposure or input produced an effect versus what evidence simply implied an association. This distinction can be more than academic, as was seen over three decades of debate regarding the effects of smoking on health. Bradford-Hill put forward criteria for attempting to ascribe causation between an exposure and a health outcome . These cBradford-Hill said that all of the following conditions can contribute to the argument that an exposure induces a health consequence:1. The greater the strength of the association, the more likely that it is causative2. There is a dose-response relationship between the exposure and the health outcome3. Exposure consistently induces the health consequence in different settings at different times4. The exposure occurs before the health outcome5. There is a biologically plausible explanation for the exposure resulting in the health outcome6. There are not more plausible explanations for the health outcome7. Experimental results add particular weight to the evidenceFor virtually all cause and effect health relationships, some of these criteria will not apply. For a programme to be shown to have an impact, criteria 4, 5 and 6 should always be met. Of particular concern to programme evaluation is the issue of biological plausibility and the amount of service provided. Programmes need to be evaluated with particular regard to the likelihood that the level of inputs provided could plausibly result in the outcome reported. That is, the number of clinic visits, or the amount of food provided per child etc. need to be sufficient to induce the health effects observed. Criteria of causation are important for interventions and settings where project impacts are usually not, or cannot be, measured.These criteria of causation can be applied to populations and programmes as readily as Bradford-Hill applied them to specific disease agents. For interventions with a vast literature documenting the attributable benefits (e.g. measles vaccination or Vitamin A supplements), the need to show "proof" that the intervention produced a health benefit may be small, but for many other emergency interventions  there may be little or no evidence that such programmes produce any health benefits, making the importance of documenting any benefits great. Most humanitarian programmatic efforts fall somewhere in between, employing types of programmes that have produced documented benefits in some settings, but have failed in others, and may or may not be producing benefits in the setting at hand. Table Attempts to analyse the impact of humanitarian interventions are often handicapped by a lack of baseline data and a lack of knowledge about regular seasonal variations in key indicators of impact. For instance, baseline mortality rates are often not known. Countrywide figures are either unreliable, out of date, or not appropriate as they do not capture the regions where the conflict is occurring. A related problem that is continuously faced by humanitarian agencies is the lack of reliable population statistics .When there are no baseline data, established norms can be used instead. For example, when people are arriving in a new location or are returning home, it is often impossible to determine the baseline before their arrival. In those cases, norms can be applied as an assumed baseline or as a threshold above or below which the indicator should not fall. Programmes lacking baseline data which keep mortality "low" or keep water and food provision high may be successful in terms of meeting their objectives but still not be able to quantify the impact of the intervention.The identification and use of relevant indicators is a crucial part of determining the impact of an intervention. Although the terminology varies, the literature generally distinguishes between performance (or process) indicators and impact (or outcome) indicators.• Performance (or process) indicators concerns both the outputs of a programme (number of latrines built, number of training conducted, the quantities of food delivered) and the process of implementation (coverage of a programme, equity of distribution, targeting)• Impact (or outcome) indicators are measures of the actual achievements intended by a programme. Mortality and malnutrition rates are the most commonly used impact indicators for humanitarian programmes, although interventions that aim to support livelihoods as well as save lives might require a broader set of indicatorsFor example, in a measles immunisation campaign, the immunisation coverage within a certain age group is a performance indicator, and the incidence rate of measles cases within that group is an impact indicator .The Standardised Monitoring and Assessment of Relief and Transition (SMART) inter-agency initiative is an attempt to systematise the collection of impact indicators. SMART has recently stipulated that two measures  are the most basic essential indicators for assessing the severity of population stress and for monitoring the overall effort of the humanitarian community. Those involved in the SMART initiative are currently developing standardised survey methodologies with associated reporting formats and software to address some of the challenges to effective evaluation. Part of the purpose of SMART is to enable global comparisons about the extent of humanitarian need in order to enable resources to be focused where they are most needed. A better understanding of needs based on nutrition and mortality might also enable better analysis of impact. Of course, malnutrition and mortality rates will not necessarily enable impact to be attributed to particular projects or agencies and may only be able to demonstrate the extent to which a relief system as a whole is meeting the needs of a population .There is a tendency for humanitarian agencies to collect performance rather than impact indicators. This is due to various reasons, such as donor requirements that tend to favour the collection of performance indicators or the belief that impact indicators  are sometimes difficult to collect. Arguably, it is easier for humanitarian agencies to monitor their own activities than to go out and monitor or assess the effect these activities have on the population they are assisting. Performance indicators may in some cases provide sufficient evidence about the likely impact of interventions and hence be used as a proxy for impact. However, when the validity of the proxy measure is unclear, there are risks in using performance indicators as a measure of success. For example, there is strong evidence that immunising children against measles has a direct effect on reducing mortality from measles; therefore, immunisation coverage can be used as a proxy for impact on mortality ,25. OtheIn the humanitarian sector, assessment of impact has most often been seen as a sub-set of evaluation. Impact is one of the Organisation for Economic Co-operation and Development / Development Assistance Committee (OECD/DAC) evaluation criteria ,27. HoweWhereas the question of impact is unarguably important in evaluation practice, a detailed analysis of impact requires a different form of investigation. This may be done either through ongoing monitoring of project implementation, or as a separate research exercise . The tools developed in field epidemiology provide a set of potentially useful methods for measuring the impact of interventions. Existing approaches such as survey methods or surveillance systems, although seldom used by humanitarian agencies, can provide significant insight about the impact of humanitarian aid. So far, these tools remain mainly used in early warning systems or needs assessment . The twoMost surveys are an attempt to actively go out and survey a representative sample of the population, although they may take different forms. WHO and others have produced manuals to guide health workers to conduct specific kinds of surveys, with nutritional anthropometry and Expanded Program on Childhood Immunizations (EPI) coverage methodologies being among the most succinctly described. Aid workers often do not have sufficient skills to take a valid sample and analyse the results of a survey. This is why many initiatives to improve the quality of relief programmes have emphasised the importance of training relief workers in survey methodologies. Some organisational headquarters and some groups such as Epicentre have specialists who can be deployed to assist with the conducting of surveys.Uncertainty about population figures creates particular difficulties in constructing sampling frames for use in surveys. Census data may be many years old while the crisis may have had a dramatic impact on demographics and population numbers due to migration and high mortality. Although cluster surveys are a compromise measure, in many situations  it may prove difficult to gain access to the 30 clusters proscribed. Nomadic groups may also prove difficult to sample ,28,29. DSurveillance is the systematic collection of information over time for decision making. Surveillance systems are often part of general monitoring systems and have been used for analysing impact in both health and nutrition programmes.Aid agencies sometimes evaluate health programs by establishing a surveillance system at the beginning of a funding cycle and contrasting the rate of health events at the beginning and the end. This is valid if either: a) all of the events of interest are captured by the surveillance network, or b) the data from within the system are representative of the health conditions of the entire population and remain consistently so over the course of the project. Neither of these conditions is commonly met for clinic-based surveillance systems in rural and urban areas, although both of these conditions are often met in well-defined settings like refugee camps. If the majority of a population does not have access to formal health care then a clinic or hospital-based surveillance system will be able to tell very little about the health conditions of the broader population. Of course, not all surveillance systems are linked to utilisation of formal health services. Sentinel site surveillance systems for nutrition monitoring, for instance, involve the monitoring of purposively selected communities in order to detect changes in context, programme and outcome variables. Surveillance systems can be less costly than surveys and may reveal more in-depth information on the causes of malnutrition.A recurrent problem with the use of epidemiological methods such as surveys and monitoring is the lack of appropriate skills for conducting good quality assessments. Reviewers from the Centers for Disease Control and Prevention (CDC) evaluated the monitoring of projects and the measurement of nutritional status and mortality in Somalia from the period 1991–93 . They deThe measurement of anthropometry is relatively standardised compared to many other health outcomes such as mortality and mental health status. For example, a mortality survey in Kabare Health Zone in the Eastern Democratic Republic of Congo (DRC) in 1999 was conducted simultaneously with an EPI coverage survey and only included households with a child under 5 years of age. The resulting estimate of crude mortality (1.9 per 1000 per month) was far lower than in a later repeat survey (finding 2.7 per 1000 per month) and, in fact, the initial survey missed most of the excess mortality . For somThus, without improved staff skills and capacity and a significant change in attitude among donors, it is likely that humanitarian agencies will continue to rely heavily on process indicators and not be expected to prove that programmes influenced the health of the targeted beneficiaries.All final reports of health-related programmes funded by the US Department of State, Bureau of Population, Migration, and Refugees (BPRM) and submitted in 2003 were reviewed for the Humanitarian Policy Group study. Proposals that contained objectives of health-related activities (e.g. shelter provision, food transport) but that did not specifically say they would influence health status were excluded. The remaining 15 final reports were evaluated against the following five criteria:• Was there a health-related objective?• Was the baseline rate measured or a comparison group identified?• Was the health-related outcome measured and reported?• Was the societal level of the evaluation appropriate given the intervention?• Were there any major issues supporting or raising concerns about the reported outcome data?The societal level of a health project and evaluation was categorised as being either on the patient level, the household level or the community level. The expectation was that programmes that intervened on a specific level should be evaluated on that level. For instance, a curative health programme might have benefits at the individual level but it may not be possible to evaluate its impact at a wider level.Six of the 15 reports did not attempt to measure or report any health-related rates or status. Proposals corresponding to five of these six reports only contained process indicators as the initial objectives, and thus the lack of documented health benefits was assured before the projects began. An additional three of the 15 reports contained health data-based objectives but did not present any health-status data, instead reporting process indicators such as the numbers of clinics supported, consultations given, or tons of food distributed.Only four of 15 final reports could demonstrate a health benefit, and three others were likely to have produced a population-based benefit although this was not documented. These four were the only projects to measure baseline rates. Nine of the 15 did not have objectives and measures that matched to societal level. The results of this analysis confirm the general conclusion reached at the July 2002 SMART Monitoring and Evaluation Workshop, that while NGO's and agencies often want to monitor health outcomes, they usually monitor process indicators. Problems with process indicators seen in the BPRM review include:• The cited activity may be related to the health outcome, but the significance of this effort depends on the activities being done well and in sufficient numbers • The health-related objective is only distantly related to the health outcome (e.g. a programme in Uganda wanted to induce "food self-sufficiency" but reported tons of food distributed)• In some cases, the link between the process indicator and the outcome was simply implausible Interestingly, a mental health programme in Guinea, with perhaps the most difficult-to-measure outcomes, had the most rigorous documentation, which included pre-intervention and post-intervention patient evaluations and the use of non-patient controls. Representatives for the other three programmes which documented impacts felt that very little of the project budget (perhaps <2%) was spent on documenting the impacts.Over 20 NGOs were providing general health services in the eastern DRC in 2000 and 2001 with funding from either Office of U.S. Foreign Disaster Assistance (OFDA) or European Commission's Humanitarian Aid Office. According to OFDA, only two of those agencies could show health benefits associated with their programmes . This seThere is an increasing interest in impact analysis at higher levels and a 'system-wide approach to performance' . SeveralA number of issues must be considered with wider levels of impact assessment. First, there is no reason to think that the constraints encountered when measuring the impact of particular interventions are erased when looking at a wider level. A particular difficulty is that of aggregation. The wider the level is, the more aggregation impact data require. Clearly, a donor or an aid agency looking at the overall effectiveness of its aid over a number of years needs far more aggregation than the evaluation of the impact of a single project conducted by a single agency. Finally, wider levels of impact assessment also generate new problems such as that of responsibility. Who is responsible for the collective impact of a number of individual humanitarian projects? Projects may have a positive impact taken individually, but the overall humanitarian effort may be insufficient compared with the level of needs. Who will account for the overall success or failure (if that is in fact possible to measure) of the humanitarian enterprise? This is a typical question that came out of the system-wide Rwanda evaluation. There is also a need for consensus in the relief community about the fundamental objective of health programmes.Despite existing efforts to improve the quality, accountability and performance of humanitarian aid in the health sector and more broadly, this paper has shown that there is limited knowledge about the health impact of humanitarian aid. The epidemiological tools potentially useful for analysing the impact of aid programmes are seldom used. As a result, humanitarian efforts rest on a limited evidence base. This is in large part due to lack of epidemiological skills found within NGOs working on the ground. Addressing this skills deficit will be essential if the rigour of routine assessments is to be improved.In the current practice, the health impact of programmes is too often assumed rather than demonstrated. This is largely due to the use of performance or process indicators as proxy for impact, without the necessary evidence that the intervention is robustly linked with a health outcome. There needs to be a consensus regarding which types of intervention (measles immunisation, assuring that people have enough food and water) are linked to good health and the levels of service that are sufficient, in order to document that aid money is well spent. Further research on the links between particular interventions and health outcomes is required to build up this evidence base.Efforts to document project impact should be woven into monitoring and surveillance activities, not only to reduce costs, but as a tool to improve program quality. The absence of systematic monitoring and surveillance in the humanitarian sector is a serious obstacle to assessing the impact of humanitarian aid. All too often an assessment of the impact is considered as a separate activity that takes place at the end of a project. The question of impact must be included throughout the project cycle, from the formulation of objectives to the final evaluation.For health impacts to be more widely documented there needs to be adequately trained, experienced, and motivated staff present at the design and evaluation phases of projects. Part of the solution is increased funding for training and retaining staff who can act as a resource, but there must be also an increased collaboration between donors and relief workers that develops a culture rewarding the documentation of programme failures as well as successes as learning opportunities. While initiatives such as SMART provide a potentially useful platform for analysing the global impact of humanitarian aid, there is a risk that the efforts will focus exclusively on technical discussions regardless of the wider political dimension of humanitarian aid. Some agencies also fear that these mechanisms will reinforce the donor control over humanitarian agencies, instead of solely aiming to increase the quality and performance of humanitarian aid. Nonetheless, increasing accountability in all sectors of international aid and increasing expectations for the wellbeing of the world's downtrodden will eventually demand consistent and widespread documentation of humanitarian benefits. Existing epidemiological techniques can adequately do so, if only they were employed. The challenge will be to make this documentation occur through positive self-improvement motives rather than as a reactive response to criticism.LR, a field epidemiologist, wrote the theoretical parts of the paper and conducted the review of reports of 15 health-related programmes; CAH, a researcher and humanitarian worker, wrote the sections that relate to humanitarian assistance and the practice of impact assessment. This paper is based on a research project into the impact of humanitarian aid carried out by the Humanitarian Policy Group at the Overseas Development Institute.The author(s) declare that they have no competing interests.


Much of the microarray data published at Stanford is based on mouse and human arrays produced under controlled and monitored conditions at the Brown and Botstein laboratories and at the Stanford Functional Genomics Facility (SFGF). Nevertheless, as large datasets based on the Stanford Human array began to accumulate, a small but significant number of discrepancies were detected that required a serious attempt to track down the original source of error. Due to a controlled process environment, sufficient data was available to accurately track the entire process leading to up to the final expression data. In this paper, we describe our statistical methods to detect the inconsistencies in microarray data that arise from process errors, and discuss our technique to locate and fix these errors.To date, the Brown and Botstein laboratories and the Stanford Functional Genomics Facility have together produced 40,000 large-scale (10–50,000 feature) cDNA microarrays. By applying the heuristic described here, we have been able to check most of these arrays for misidentified features, and have been able to confidently apply fixes to the data where needed. Out of the 265 million features checked in our database, problems were detected and corrected on 1.3 million of them.Process errors in any genome scale high throughput production regime can lead to subsequent errors in data analysis. We show the value of tracking multi-step high throughput operations by using this knowledge to detect and correct misidentified data on gene expression microarrays. Expression microarrays, with the capability to measure the mRNA expression level of tens of thousands of genes simultaneously, have found broad application in both clinical and basic research -7. With This example is from the analysis of experiments from an ovarian tumor dataset that firh and plate i and plate n and plate o for all four comparisons of the first PCR round to the second.With MuFu, a check using a 384-well plate partition of the data shows a discrepancy but no obvious plate rotation or misidentification. We then repartition and repeat our tests at the 96-well plate level. The results are shown in Table h and i are swapped, and n and o are swapped, most likely in the plates from the first PCR round. The ambiguity is broken by sequencing a small sampling of wells, which confirms that the misidentifications are in the print plates from the first round of PCR, and not the second. In Figure In all four comparisons, the two distributions resolve themselves well, as can be seen in Figure p, identified in both the rank and rotated-rank matrices across print batches is a clear candidate for a plate rotation. The data from the distance matrix comparisons is shown in Table We compare four arrays A1, A2, A3 and A4 from a particular print production batch A to four arrays B, C, D and E from four other distinct print production batches. The total number of 384 well microtiter plates in print batch A is 113. The results of the plate rotation test are shown in Table q has a poor self-match comparison. Indeed, when the distance matrix is examined, we see the plate matches instead plate r across the four array comparisons. We check the three criteria for plate misidentification and summarize these data in Table In this example we compare four arrays A1, A2, A3 and A4 from a particular print batch A to four other arrays B, C, D and E from four other distinct print batches. As seen in Table Out of the 265 million features checked in our database using MuFu, problems were detected and corrected on 1.3 million of them. That we were able to find and correct both previously known and unknown problems gives us confidence in the algorithm. That so few problems existed overall (0.5%) reassures us as to the robustness of our process in general.Microarray data, by its sheer volume, presents interesting laboratory information management challenges. The data arrive at the investigators desk after a significant number of steps. We have found that the better you track production, quality control, and experimental steps, the better chance you have of uncovering the reasons behind discrepancies that may appear in the data. Often, statistical analyses look only at the data presented as final expression values or ratios, without taking into account relevant quality control data. In our effort to understand our errors and the source of large systematic discrepancies in our data we have found the MuFu algorithm a useful test of certain classes of process errors, and as a check for general problems with specific process steps or microtiter plates. We use MuFu to find, verify and fix problems that can be attributed to an error in plate processing. We also flag plates for which we can find no specific problem but we see yield inconsistent results. These may be plates that, at some point in the process, had a problem (cross contamination, a PCR problem) that was not detected while the process step was being carried out. The fact that we can test the data, using our standard quality control hybridizations, for these types of quality issues is reassuring and has helped us gain confidence in our data.Obviously, there are many other classes of error that creep into microarray data. Aside from the gross process errors that are amenable to detection, as we have described here, there is also a large class of more subtle systematic errors that can contribute to the overall systematic error on the expression ratio measurement. Isolating the source of these individual errors is sometimes quite difficult. Properties of the microarray feature such as spot size, shape and uniformity can contribute, but the majority of systematic errors are introduced at the time the experiment is performed. Slide post processing, RNA quality, labelling, hybridization and washing all lend the possibility for introducing systematic errors. Improvements in protocols and hybridization apparatus have helped reduce these errors and should continue to do so in the future. As these sources of error are identified and eliminated, expression microarrays will continue to provide progressively more sensitive measures of gene expression..Process errors in any genome scale high throughput production regime can lead to subsequent errors in data analysis. We have shown the value of tracking multi-step high throughput operations by using this knowledge to detect and correct misidentified data on gene expression microarrays. We generalized our procedure using a simple heuristic, which found and fixed several problems with the proper assignment of gene identifier with physical microarray feature. We found thirteen print runs (9K arrays) that had four plates mistracked, six print runs with single 384 plate rotations, and one instance of a plate rotation at the 96 well plate level. One skipped plate was detected, as well as one plate printed twice. Out of the 265 million features checked in our database using MuFu, problems were detected and corrected on 1.3 million of them. That we were able to find and correct both previously known and unknown problems gives us confidence in the algorithm. That so few problems existed overall (0.5%) reassures us as to the robustness of our process in general. A list of corrected arrays can be found at We follow the simple heuristic outlined here. The flowchart for the program is shown in Figure Aij be the intensity measurements of array A, subset i and gene index j. The measurements are usually of the channel (Cy3 or Cy5) used as a common reference. For example, if we partition the data by the 384-well plate each feature once occupied at some point in its process history, A is the array id, i is the plate id, and j is the well index between 1–384. Let Ai be the vector (Ai1,...,Ai384). In our vernacular this is the 384-well plate expression vector for plate i. Let Bij andBi be the similar definitions for array B. We also reverse the data vectors from array A and label it Ai' . In our notation, Ai' = (Ai384,...,Ai1) is the expression vector for a plate rotated by 180°.1. Begin by partitioning the gene expression data on an array into subsets according to plate. Other partitions can also be made but the plate level partition is the most useful for our purposes. Let Dmn, 1 ≤ m, n ≤ N} in which each element Dmn = 1 - corr(Am, Bn). For the 384-well plate example, Am is the 384-well expression vector for plate m on array A, Bn is the 384-well expression vector for plate n on array B, and Dmn is the distance between the two vectors in this 384 dimensional space. We also generate the corresponding rotated-distance matrix {D'mn, 1 ≤ m, n ≤ N} for the plate rotation case, in which each element D'mn = 1 - corr(A'm, Bn). We tried several correlation functions including Euclidean and Pearson but found the Pearson to be best suited to this task.2. Generate a distance matrix {Rmn, 1 ≤ m, n ≤ N} by converting the distances to ranks such that the row m in the rank matrix is the order statistic of the row m of the distance matrix. We also generate the corresponding rotated-rank matrix {R'mn} for {D'mn}. Ideally, for the case where we are comparing identical subsets from two different arrays we expect the diagonal elements of the rank matrix to all be equal to one, which means that each subset of genes from the first array matches its corresponding subset in the second array the best. In general, due to the statistical variation in array data, the diagonal elements are not all equal to one, even if there are no misidentification errors. The examples show that this does not hinder us from making a clear distinction between real problems in the data and statistical fluctuations.3. Generate a rank matrix {A1, A2, A3, A4 in the example) from the print batch in question to a control sample of arrays (B, C, D, E in the example) selected from several distinct print batches. By comparing the rank R, and rotated-rank R', matrix assignments for comparisons across array batch boundaries we can quickly flag possible rotated plates. A visualization of this test is shown in Table A) against all of the control batches (in this case batches B, C, D and E) in the non-rotated case, we conclude that the corresponding flagged plate or partition from batch A may be misidentified. Note that in the limit that the partitions of the array are all uniform in expression ratios there is a 5% probability of a spurious flag. For this reason it is better to use high quality, highly variegated control arrays for such tests. Next, to better resolve ambiguous cases and to check our rank matrix determination we use the distance rather than rank matrix. If, for example, a plate x is to be considered a rotated plate, the following three criteria must be met.A plate rotation may have an affect on a single microarray batch if it occurs during array printing or may persist across many print batches if it happens during a 96-well (PCR) process step. In any case, the mismatch will persist across many array comparisons. To check for plate rotations in a print batch, we compare an array sample (D'xx <Dxx. The rotated-distance must be less than the non-rotated distance.1. D'xx is close to the mean of the distribution, {Dmm, 1 ≤ m ≤ N}, and Dxx is close to the mean of the distribution {D'mm, 1 ≤ m ≤ N}.2. D'xx is an outlier of the distribution {D'mm, 1 ≤ m ≤ N}, and Dxx is an outlier of the distribution {Dmm, 1 ≤ m ≤ N}.3. D'mm, 1 ≤ m ≤ N} and {Dmm, 1 ≤ m ≤ N} separate well. In Figure The second and third criteria above are valid if the distributions of {x for a match with plate y with the following criteria:If a plate has consistently poor self-match rankings and a plate rotation has been excluded as a possible source of error, this next step tests for a plate misidentification. From the rank matrix we can identify the best alternative match. We test a suspect plate Dxy <Dxx. The mismatch distance is shorter than the match distance.1. Dxy is close to the mean of the distribution {Dmm, 1 ≤ m ≤ N}, and Dxx is close to the mean of the distribution {Dmn, 1 ≤ m ≠ n ≤ N}.2. Dxx is an outlier of the distribution {Dmm, 1 ≤ m ≤ N}, and Dxy is an outlier of the distribution {Dmn, 1 ≤ m ≠ n ≤ N}.3. Dmm, 1 ≤ m ≤ N} and {Dmn, 1 ≤ m ≠ n ≤ N} are well resolved.Again, the second and third criteria above are valid if the distributions of {IT, MF developed the heuristic and carried out analysis and implementation.MS brought this problem to our attention and provided test data.MD provided advice and checked our work with an alternative technique.All authors read and approved the final manuscript.