In a previous paper, we introduced MUSCLE, a new program for creating multiple alignments of protein sequences, giving a brief summary of the algorithm and showing MUSCLE to achieve the highest scores reported to date on four alignment accuracy benchmarks. Here we present a more complete discussion of the algorithm, describing several previously unpublished techniques that improve biological accuracy and / or computational complexity. We introduce a new option, MUSCLE-fast, designed for high-throughput applications. We also describe a new protocol for evaluating objective functions that align two profiles.We compare the speed and accuracy of MUSCLE with CLUSTALW, Progressive POA and the MAFFT script FFTNS1, the fastest previously published program known to the author. Accuracy is measured using four benchmarks: BAliBASE, PREFAB, SABmark and SMART. We test three variants that offer highest accuracy (MUSCLE with default settings), highest speed (MUSCLE-fast), and a carefully chosen compromise between the two (MUSCLE-prog). We find MUSCLE-fast to be the fastest algorithm on all test sets, achieving average alignment accuracy similar to CLUSTALW in times that are typically two to three orders of magnitude less. MUSCLE-fast is able to align 1,000 sequences of average length 282 in 21 seconds on a current desktop computer..MUSCLE offers a range of options that provide improved speed and / or alignment accuracy compared with currently available programs. MUSCLE is freely available at  Multiple alignments of protein sequences are important in many applications, including phylogenetic tree estimation, secondary structure prediction and critical residue identification. Many multiple sequence alignment (MSA) algorithms have been proposed; for a recent review, see . Two attLN) in the sequence length L and number of sequences N  / [min  - k + 1 ].     (1)τ is a k-mer, LX, LY are the sequence lengths, and nX(τ) and nY(τ) are the number of times τ occurs in X and Y respectively. This definition can be motivated by considering an alignment of X to Y and defining the similarity to be the fraction of k-mers that are conserved between the two sequences. The denominator of F is the maximum number of k-mers that could be aligned. Note that if a given k-mer occurs more often in one sequence than the other, the excess cannot be conserved, hence the minimum in the numerator. The definition of F is an approximation in which it is assumed that (after correcting for excesses) common k-mers are always alignable to each other. MUSCLE also implements a binary approximation FBinary, so-called because it reduces the k-mer count to a present / absent bit:Here FBinary = Στ δXY(τ) / [min  - k + 1 ].     (2)δXY(τ) is 1 if τ is present in both sequences, 0 otherwise. As multiple instances of a given k-mer in one sequence are relatively rare, this is often a good approximation to F. The binary approximation enables a significant speed improvement as the size of the count vector for a given sequence can be reduced by an order of magnitude. This allows the count vector for every sequence to be retained in memory, and pairs of vectors to be compared efficiently using bit-wise instructions. When using an integer count, there may be insufficient memory to store all count vectors, making it necessary to re-compute counts several times for a given sequence.Here, mutation distance, i.e. the number of mutations that occurred on the historical path between the sequences. The historical path through the phylogenetic tree extends from one sequence to the other via their most recent common ancestor. The mutation distance is trivially additive. The fractional identity D is often used as a similarity measure; for closely related sequences 1 - D is a good approximation to a mutation distance . As sequences diverge, there is an increasing probability of multiple mutations at a single site. To correct for this, we use the following distance estimate .     (6)Following MAFFT, we also implemented a weighted mixture of minimum and average linkage:dMixPC = (1 - s) dMinPC + s dAvgPC,     (7)s is a parameter set to 0.1 by default. Clustering produces a pseudo-root (the last node created). We implemented two other methods for determining a root: minimizing the average branch weight .     (15)Dk is the last position in X that is aligned to a letter in Y. Extract the special case of a gap of length 1:Here, xy = max { max(k<x-1) [Mky + bXk+1], Mx-1y + bXx}.     (16)DHence,xy = max { Dx-1y, Mx-1y + bXx }.     (17)DSimilarly,xy = max { Ixy-1, Mxy-1 + bYy }.     (18)Ix and the inner loop over increasing y. For fixed x, define vectors Mcurry = Mxy, Mprevy = Mx-1y, Dcurrx = Dxy, Dprevx = Dx-1y; for fixed x, y define scalars Icurr = Ixy, Iprev = Ixy-1. Now we can re-write (14), (17) and (18) to obtain the following recursion relations:Let the outer loop iterate over increasing curry = Sxy + max { Mprevy-1, Dprev y-1 + tXx-1, Iprevy-1 + tYy-1 }     (19)Mcurry = max { Dprevy, Mprevy + bXx }     (20)Dcurr = max { Iprev, Mprevy + bYy }.     (21)ILX × LY matrix is needed for the trace-back that produces the final alignment.An i Σj f xi f yj Sij = Σi f xi Wyi, where Wyi = Σj f yj Sij. The vector Wyi is used LX times, and it therefore pays to compute it once and cache it. Observe that a typical profile column contains << 20 different amino acids. We sort the frequencies in decreasing order; the summation Σi f xi Wyi is terminated if a frequency f xi = 0 is encountered. This typically reduces the time spent in the summation, especially when sequences are closely related. As with Wyi, the sort order is computed once and cached. Observe that the roles of the two profiles are not symmetrical. It is most efficient to choose X, for which frequency sort orders are computed, to be the profile with the lowest amino acid diversity when averaged over columns. With this choice, the summation terminates earlier on average then if the other profile is identified as X. Note that out of N - 1 iterations of progressive alignment, a minimum of  and maximum of N - 1 profile-profile alignments will include at least one profile containing one sequence only, and in the refinement phase exactly N of the 2N - 1 edges in the tree terminate on a leaf. At least half of all profile-profile alignments created in the MUSCLE algorithm therefore include a profile of one sequence only. Special cases where one or both profiles is a single sequence can be handled in separate subroutines, saving overhead due to unneeded loops that are guaranteed to execute once only. This optimization is especially useful for the LE function as it enables the logarithm to be incorporated into the W vector.The inner-most dynamic programming loop, which computes the profile function, deserves careful optimization. We will consider the case of PSP; similar optimizations are possible for LE. PSP = Σk-mers in common between two sequences, then attempt to extend the match. In the case of diagonal identification we found compressed alphabets to significantly out-perform the standard amino acid alphabet  be the amino acid of sequence s in column x, and S be the substitution score of amino acids i and j. It is convenient to impose an (arbitrary) ordering on the sequences and amino acid types. Then,Notice that computation of the SP score dominates the time complexity of refinement and of MUSCLE overall, introducing O.     (23)SPδ = 1 if s [x] = i, 0 otherwise, and ni [x] = Σs δ. We say ni [x] is the count of amino acid type i in column x. We can now transform the sum over pairs of sequences into a sum over pairs of amino acid types:Define a = Σx Σi ni Σj nj>i S + 1/2 Σx Σi (ni2 - ni) S.     (24)SPFrequencies are computed as:f xi = ni [x]/ N.     (25)Using frequencies,N2LP) but (25) and (26) are O(NLP). For N >> 20, this is a substantial improvement. Let SPg be the contribution of gap penalties to SP, so SP = SPa + SPg. It is natural to seek an O(NLP) expression for SPg analogous to (26), but to the best of our knowledge no solution is known. Note that in MUSCLE refinement, the absolute value of the SP score is not needed; rather, it suffices to determine the difference in the SP scores before and after re-aligning a pair of profiles. Let SP be the contribution to the SP score from a pair of sequences s and t, so SP = Σs Σt>s SP, and denote the two profiles by X and Y. Then we can decompose SP into intra-and inter-profile terms as follows:For simplicity, we have neglected sequence weighting; it is straightforward to show that (26) applies unchanged if weighting is used. Note that (23) is O + Σs∈Y Σt∈Y:t>s SP + Σs∈X Σt∈Y SP     (27)SP = ΣNote that the intra-profile terms are unchanged in any alignment that preserves the columns of the profile intact, which is true by definition in profile-profile alignment. This follows by noting that any indels added to align the profiles are guaranteed to be external gaps with respect to any pair of sequences in the same profile. It therefore suffices to compute the change in the inter-profile term:XY = Σs∈X Σt∈Y SP.     (28)SPπ- be the alignment path before re-alignment and π+ the path after re-alignment. The change in alignment can be specified as the set of edges in π- or π+, but not both; i.e., by considering a path to be a set of edges and taking the set symmetric difference Δπ = (π- ∪ π+) - (π- ∩ π+). The path π+ after re-alignment is available from the dynamic programming traceback. The path π- before re-alignment can be efficiently computed in O(LP) time. Note that in order to construct the profile of a subset of sequences extracted from a multiple alignment, those columns that contain only indels in that subset must be deleted. The set of such columns in both profiles is therefore available as a side effect of profile construction, and this set immediately implies π-. It is a simple O(LP) procedure to compute Δπ from π- and π+. Note that SPa is a sum over columns, and there is a one-to-one correspondence between columns and edges in π. The change in SPa can therefore be computed as a sum over columns in Δπ, with a negative sign for edges from π-, reducing the time complexity from O(NLP) to O(N|Δπ|). We now turn our attention to SPg. We say that a gap G intersects Δπ if and only if any indel in G is in a column in Δπ, and denote by Γ the set of gaps that intersect Δπ. If a gap does not intersect Δπ, i.e. does not have an indel in a changed column, its contribution to SPg is unchanged. It therefore suffices to consider penalties for gaps in Γ, again with negative signs for edges from π-. The construction of Γ is straightforward in O(NLP) time. Finally, a sum over pairs in Γ is needed, reducing the O(N2) component to the smallest possible set of terms.This reduces the average time by a factor of about two. We can further improve on this by noting that in the typical case, there are few or no changes to the alignment. This suggests computing the change in SP score by looking only at differences between the two alignments. Let NLP) time. Define a two-symbol alphabet {X, -} in which X represents any amino acid and - is the indel symbol. There are four dimers in this alphabet: XX, X-, -X and --, which denote by no-gap, gap-open, gap-close and gap-extend respectively. Re-write a multiple alignment in terms of these dimers, adopting the convention that dimer ab composed of symbol a in column x-1 and symbol b in column x is written in column x. Now consider the contribution to SPg of an aligned pair of dimers, written as ab↔cd. Clearly XX↔X- adds a gap-open penalty; XX↔-X adds a gap-close , and compute the change in SP by considering only those columns in Δπ. We find use of the dimer approximation to marginally reduce benchmark scores. By default, MUSCLE therefore uses the exact SP score for N ≤ 100 and the dimer approximation for N > 100, where the higher time complexity of the exact score becomes more noticeable.We next describe an approximation to SP that can be computed in O against the number of false pairs with score ≤ S (y axis); we call the resulting graph a discrimination plot. Ideally, all true pairs would score higher than all false pairs, in which case the profile function would be a perfect discriminator and would always produce perfect alignments. A function that perfectly discriminates will appear as a Γ-shaped plot; a function that has no ability to discriminate will appear as a diagonal plot along the line x = y. If a function F has a discrimination plot that is always above another function G  > DG(x) ∀ x, where DF is the discriminator plot for F as a function of x), then F has a superior ability to discriminate true from false pairs compared with G. If the plots intersect, the situation is ambiguous and neither function is clearly superior. We used sets of structural alignments from [m(F) where m(x) is a monotonically increasing function). However, a monotonic transformation may change the alignments produced by a profile function, so we can regard high discrimination as a necessary but not sufficient condition for a good profile function. One can turn this into a virtue by noting that the discrimination plot allows the relative probability of true versus false to be determined from a score. It is therefore possible to numerically determine a log-odds function from the discrimination plot, which can be evaluated by table look-up. Using discrimination plots for PP2, we found the optimal transformation for LE to be close to linear, in contrast to other functions we tried, including PSP (results not shown). This observation further encouraged us to explore the performance of LE in an MSA algorithm. Testing on multiple alignment benchmarks we find LE to give superior results on BAliBASE, but statistically indistinguishable results on other databases (results not shown). MUSCLE therefore uses LE as the default choice as it sometimes gives better results but has not been observed to give lower average accuracy on any of our tests. It is also useful to introduce a method with a distinctively different scoring scheme as an alternative that may give better results on some input data and may provide unique features for incorporation into jury or consensus systems. One drawback of LE is its relatively slow performance due to the need to compute a logarithm for each cell of the dynamic programming matrix.We have previously attempted a systematic comparison of profile functions . The metnts from  (PP) andnts from  (PP2). Pnts from  with ≤ 3nts from , LAMA [4nts from . Using PLP = O(L + N), the e-string construction for the root alignment, and a fixed number of refinement iterations.The complexity of MUSCLE is summarized in Table FBinary is used as a distance measure (Equation 2), the PSP profile function is used, and diagonal finding is enabled.MUSCLE offers a variety of options that offer different trade-offs between speed and accuracy. In the following, we report speed and accuracy results for three sets of options: (1) the full MUSCLE algorithm including Stages 1, 2 and 3 with default options; (2) Stages 1 and 2 only, using default options (MUSCLE-prog); and (3) Stage 1 only using the fastest possible options (MUSCLE-fast), which are as follows: Q, the number of residue pairs correctly aligned divided by the length of the reference alignment. For more discussion of the reference data, assessment methodology and a comparison of MUSCLE with T-Coffee and NWNSI, the most accurate MAFFT script, see [In Tables ipt, see .LP is O(L). Results are shown in Figure To compare speeds for a larger number of sequences, we created a test set by using PSI-BLAST to search the NCBI non-redundant protein sequence database for hits to dienoyl-coa isomerase (1dci in the Protein Data Bank ), selectMUSCLE demonstrates improvements in accuracy and reductions in computational complexity by exploiting a range of existing and new algorithmic techniques. While the design–typically for practical multiple sequence alignment tools–arguably lacks elegance and theoretical coherence, useful improvements were achieved through a number of factors. Most important of these were selection of heuristics, close attention to details of the implementation, and careful evaluation of the impact of different elements of the algorithm on speed and accuracy. MUSCLE enables high-throughput applications to achieve average accuracy comparable to the most accurate tools previously available, which we expect to be increasingly important in view of the continuing rapid growth in sequence data..MUSCLE is a command-line program written in a conservative subset of C++. At the time of writing, MUSCLE has been successfully ported to 32-bit Windows, 32-bit Intel architecture Linux, Solaris, Macintosh OSX and the 64-bit HP Alpha Tru64 platform. MUSCLE is donated to the public domain. Source code and executable files are freely available at


The mortality rate among patients with infective endocarditis, especially associated with the presence of complications or coexisting conditions such as renal failure and the use of combined medical and surgical therapy remains still high. Prolonged parenteral administration of a bactericidal antimicrobial agent or combination of agents is usually recommended, however, the optimal therapy for infective endocarditis associated with renal injury is not adequately defined.Patient was a 24-years old man who presented to our hospital with fever, fatigue, and rapidly progressive glomerulonephritis. He had a history of ventricular septum defect (VSD). A renal biopsy specimen revealed crescentic glomerulonephritis and echocardiogram revealed VSD with vegetation on the tricuspid valve. Specimens of blood demonstrated Propionibacterium Acnes. The intensive antibiotic therapy with penicillin G was started without clinical improvement of renal function or resolution of fever over the next 7 days. After the short-term treatment of low dose of corticosteroid combined with continuous antibiotics, high fever and renal insufficiency were dramatically improved.Although renal function in our case worsened despite therapy with antibiotics, a short-term and low dose of corticosteroid therapy with antibiotics was able to recover renal function and the patient finally underwent tricuspid valve-plasty and VSD closure. We suggest that the patients with rapidly progressive glomerulonephritis associated with infective endocarditis might be treated with a short-term and low dose of corticosteroid successfully. Infective endocarditis has been classified as acute or subacute-chronic based on the clinical presentation and often presents extracardiac findings such as fever, anorexia, weight loss, malaise, and night sweats . The proA 24-year old man was admitted to our hospital because of macrohematuria and general malaise, along with insidious deterioration of renal function. The patient had been diagnosed as having ventricular septum defect (VSD) without complications and had been well six months before admission, when the patient presented a temperature of 38.0°C and a productive cough. One month before admission, the patient was admitted elsewhere because of fever, general malaise, and macrohematuria. The temperature was 38.6°C, the pulse was 120 beats per minute, and respirations were 24 times per minute. The blood pressure was 130/58 mmHg. On physical examination, the patient appeared acutely ill and pansystolic murmur of Levine III/IV was noted at the 4th left sternal border (LSB). The urine was positive for hematuria (+++) and protein (+++); the sediment contained 150–160 RBC/hpf and 10–15 granular casts/hpf. Laboratory test revealed 2.04 mg/dl of serum creatinine, 33.3 mg/dl of BUN, and 9.0 g/dl of hemoglobin. Echocardiogram demonstrated VSD and vegetation on the tricuspid valve associated with regurgitation. The patient was transferred to our hospital for evaluation of renal dysfunction.On the first hospital day, the temperature was 39.4°C and the physical examination demonstrated pansystolic murmur at the 4th LSB again, but no lymphoadenopathy and no localizing signs for a focus of infection. Laboratory findings were: anemia , presence of d-dimer 10.3 mg/ml), BUN 41 mg/dl, serum creatinine 2.49 mg/dl, proteinuria 0.96 g/24 h, 24 h Ccr 30.0 ml/min, hypocomplementemia , positive for inflammatory sign , and positive for cryoglobulinemia  or resolution of fever over the next 7 days. The initial antibiotics were replaced with ampicillin and imipenem in addition to low dose of corticosteroid treatment initiated with intravenous methylprednisolone 0.5-g per day for 3 consecutive days followed by oral prednisolone 30-mg, 20-mg, and 10-mg per day each for 3 days, respectively. On day 16, laboratory test demonstrated normalization of inflammatory signs , serum complements, and circulating immune complex and negative for blood culture and cryoglobulin. After the short-term treatment of low dose of corticosteroid combined with continuous antibiotics, high fever and renal insufficiency were dramatically improved  declare that they have no competing interests.Koya D and Shibuya K cared the patients, took the picture, and wrote the paper.Haneda M and Kikkawa R discussed the case.The pre-publication history for this paper can be accessed here:


The Allen Brain Atlas of gene expression in the mouse brain is poised to serve as an outstanding resource to neuroscience Armed with billions of cells, elaborate circuitry, and a seemingly animate anatomy, capable of growing as it learns, the brain is a marvelously enigmatic organ. Much to the chagrin of those that study it, the brain remains perhaps too mysterious.Although genetic information exploded out of the Human Genome Project, it has been of little consequence to neuroscience—a discipline still grappling with the boundaries and names for distinct brain regions. According to United States National Institute of Mental Health Director Thomas Insel, over 99% of the neuroscience literature focuses on only 1% of the estimated 15,000–16,000 genes expressed in the brain. David Van Essen, a neurobiologist at Washington University in St. Louis, Missouri, likens the current genetic map of the brain to a 17th century map of Earth. A voyage around the Earth had already proven it was round, but landmass resolution was still vague at best.Magellan's benefactors, though, never bankrolled a technical advance quite like the Allen Brain Atlas. Neuroscience's unlikely sugar daddy, Microsoft cofounder and the world's fifth wealthiest man, Paul Allen, created the $100 million dollar Allen Brain Institute in Seattle, Washington, two years ago. The first explicit goal of the institute was to create an open-access, visual, searchable online map of genes expressed in the brain, as well as of brain circuitry and cell location. Roughly one petabyte of data—equal to the memory necessary to hold the information held in about 50 Libraries of Congress—will be produced as a result.In mid-December, the first 2,000 genes were uploaded. By 2006, the Allen team plans to have as many as 24,000 genes online. While the ultimate goal is to map the human brain, the atlas ushers in a new era of neurogenetics—an attempt to make connections between anatomical, genetic, and behavioral observations.The initial effort will focus on the standard, inbred lab mouse strain known affectionately as C57BL6. Like it or not, mice are remarkably similar to humans—sharing 99% of our genes. Humans, at this time, provide too many hurdles—not the least of which is a lack of willing brain donors that are the same age. Since the C57BL6 strain is inbred, the mice are also much more uniform than humans—a key to constructing the most accurate representative map possible of one species' adult brain.dyspyndin, a gene linked to schizophrenia. Insel is banking on the atlas to locate genes linked to conditions including bipolar disorder, schizophrenia, and autism. Once identified, such critical genes can be examined in detail and used in studies aimed at disease cures or drug screens. Presumably, the atlas will be a boon for drug discovery and development by providing information on drug targets present within brain cells.With a map of mouse genes in hand, scientists will be able to develop informed hypotheses about genes that may affect human brain function and dysfunction. “People will be able to look first in the mouse atlas, then more selectively focus on human cases,” says Gregor Eichele, director of the Max Planck Institute for Experimental Endocrinology in Hannover, Germany. Indeed, researchers may do well to focus their efforts on those specific cells in which homologous genes are expressed in the mouse. Insel suggests an initial mental health application: find Eichele points out that the Allen Brain Atlas will eliminate time wasted testing fruitless hypotheses. “If you think gene X may be involved in [hippocampal-dependent] memory, but it's not expressed in the hippocampus, you shouldn't bother following that line of research,” he says. Insel agrees that where genes are expressed in the brain will be most telling. “In the brain, more than any other organ, function follows form,” he says.Cellular resolution of expression patterns will prove necessary to uncover as yet unknown relationships between circuitry, cell type, and gene expression in the brain, says Arthur Toga, a neuroscientist at the University of California, Los Angeles, and Allen Brain Atlas advisor. Ed Lein, a neuroscientist at the Allen Brain Institute, thinks that mapping at the cellular scale will also redefine anatomy. Traditionally, neuroanatomists have delineated brain regions pretty much by eye, identifying clusters of cells and patterns of connections that look the same. “We're starting to redefine boundaries of regions by cell type,” says Lein, defining cell type by gene expression pattern.The difficulty, according to Allen Brain Institute scientific advisor David Anderson, a neuroscientist at the California Institute of Technology in Pasadena, California, is in understanding how a gene mutation affects behavior. “It is impossible without a knowledge of the circuits in which a gene is expressed,” he says. Anderson himself studies innate behaviors such as fear responses. Specifically, he's most interested in understanding the function of different regions of the almond-shaped amygdala. Once he finds genes expressed in neurons within regions of the amgydala implicated in fear, he plans to determine neuron function by creating transgenic animals in which specific neuron activity has been silenced.Thus far, Anderson's laboratory has endured the slog of using microarray techniques to identify genes with expression patterns linked to fear behavior, followed by in situ hybridization, which traditionally involves twenty-odd complex, error-prone steps, to find just some of the genes expressed in different parts of the amygdala. “The Allen Brain Atlas will identify a whole set of genes that we would have had to spend several years to find,” he says. Indeed, locating where even one gene of interest is expressed in the brain eats up valuable research time, especially when there are so many potentially interesting genes. In most cells, 10,000 genes can be expressed.Eichele improved the slow error-prone process into an automated, fast, “high throughput” method that caught the attention of the Allen team because it was capable of meeting the needs of such an ambitious project. In situ hybridization uses labeled probes for specific messenger RNA sequences, allowing scientists to test individual brain tissue samples for gene expression.Automated in situ hybridization data will be generated for the entire mouse transcriptome—the full complement of activated genes in a particular tissue at a particular time—on a genome-wide scale. The mouse is a young adult at 56 days old, free from the confounding factors of development. After it is sacrificed, the mouse brain is immediately frozen, then sliced very thinly—to get forty sections from each millimeter of thickness—so that the probes for hybridization can expose gene expression in individual cells.The in situ data will be matched in a three-dimensional framework to the reference atlas developed by Allen's team. The resulting images will be turned into a virtual microscope, allowing users to focus down on genes expressed in regions of interest.While the Allen Brain Atlas is somewhat like other genomics projects in scale, it is unique. “No one's gone into a 3-D structure like a tissue and examined it in a systematic way,” says Allan Jones, senior director of Allen Brain Atlas Operations. In that way, he adds, it's a much richer dataset than the Human Genome Project. “As we're ramping up—fully by spring of next year—we'll be generating about 1,000 microscope slides a day with four mouse brain sections on each slide,” says Jones. Each day, those sections are scanned and stitched together electronically into 300-megabyte batches. “Scaling up a lab process is currently the biggest challenge,” says Jones. “In effect, we're turning an art form into something that gives high-quality data day in and day out,” says Jones. “If you are off slightly when cutting a 2-D brain slice, it becomes very difficult to map back into a 3-D context.”While it is a logical starting point, a spatial understanding of gene expression is just one way to mine the brain. Other avenues currently being pursued explore individual variability, development, and comparisons between species See . The sheThe Allen database will provide a spatial map of neurogenetic data, specifying where the 99% of shared mammalian genes are expressed in the brain. Other online databases are striving to provide alternative axes of information that will detail individual variability, species comparisons, and changes during development.Going deep on the genetic variability axis of understanding is Web QTL, a collection of images from 800 brains of 35 different strains of mice. “We're interested in genetic sources of variation,” says Rob Williams, a neurobiologist at the University of Tennessee in Memphis. “We study many strains of mice and map the upstream modulators that control expression differences.”Van Essen's online atlas strives to map the structural and functional areas of the cerebral cortex, believed to be the seat of thought, learning, emotion, sensation, and movement, for humans, macaques, rats, and mice. In constructing the SuMS (for “surface management system”) database, they've put most of their effort into comparing datasets between species.Finally, GenSAT (Gene Expression Nervous System Atlas) follows gene expression as it changes through the development of an organism. Using a method that manipulates “bacterial artificial chromosomes” to insert, change, or delete parts of large gene sequences, one transgenic mouse is created for each gene. When reporter genes are added to the bacterial artificial chromosomes, cells with selective gene activity glow. This advance takes a step towards relating gene expression patterns to connectivity between brain regions.“It's clear that they want to do a first-rate job of working with all this information, but I'm not sure they entirely appreciate how hard it's going to be to manage the staggering amount of information they're getting,” says Van Essen. As manager of his own comparative anatomy database, he understands the magnitude of the task. He points out that properly digitizing the data in electronic form and registering one particular slice to a standardized reference with meaningful coordinates is not trivial.“In an atlas, there will be considerable variability from one individual to the next—even in inbred mouse strains,” says Van Essen. Even if it's only 20% variability, it is still going to pose challenges for managing experimental data. Van Essen acknowledges that this isn't just an impediment for Allen's team, but for neuroscience as a whole. “These tools, in general, don't emerge from a vacuum. They emerge best when rich, challenging datasets are staring people in the face,” he says.An even larger problem is communicating the data effectively. Figuring out how to navigate the tremendous morass of data will be a bioinformatic stumbling block. Lein acknowledges that figuring out how to best annotate the data is one of the bigger challenges for the future—particularly in cases where gene expression doesn't match the agreed-upon boundaries of anatomical regions.For now, users will be able to mine the data by gene name only. The initial release consists of an image viewer to view the in situ hybridization data for one or several genes at a time, along with a reference atlas to determine the structures in which genes are expressed. Future releases will allow the user to conduct more sophisticated searches, such as by anatomical structures.“Not only is there this 3-D structure, but there are lots of studies where people are trying to understand what drives the turning on and off of genes,” says Jones. “At the end, if the atlas has a big impact, it will be in providing the precise coordinates for those people to tease apart what specific DNA elements drive expression within regions or structures.”And while the Allen Brain Atlas will provide a fine level of detail, there are limitations. Much like the Human Genome Project, the information will be the starting point and not the end point of understanding brain function. “It won't change strategy for doing experiments,” says Nobel laureate and Columbia University neuroscientist Eric Kandel. “The atlas will be a catalyst rather than a direction setter.”Overall, neuroscience is entering a new era. Insel notes that recent work has proven the brain to be extraordinarily dynamic, birthing neurons throughout a lifespan. Brain functions seem more modular than global. And there is no real separation between the mind and brain. “Mental disorders are brain disorders,” he says.Over the next 5–10 years, neurogenomics will fuel a golden age of discovery in neuroscience. In fact, scientists may even reach an overarching goal—understanding how the wild card of environment impacts brain function. “We want predictive genetics able to accommodate environmental differences,” says Rob Williams, neurobiologist at the University of Tennessee in Memphis.For Kandel, one thing is certain. “Most of the mysteries of the brain lie ahead of us.”


As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge.th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options.We analysed the results from the Extended Matching Questions Examination taken by 4Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified.Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards. It is acknowledged from medical student learning behaviour that assessment often drives learning . TherefoAt the University of Leeds the undergraduate medical course includes an integrated medical and surgical specialities program . Acknowledging that no single assessment format can adequately assess all the learning objectives within the course blueprint, a combination of assessments  are currently used to assess the student's competence. Although a combined score is used to assess the competence of the students, test scores within the individual assessments reflect a richer profile of the individual, allowing an understanding of strengths and weaknesses that can result in improvement in the individual and the educational programme. Analysis of the quality of individual assessments is essential to this process. The focus of this paper is the use of item response theory to examine the validity of the typical use of a single score obtained from the summative EMQ examination, to characterise each student and their individual differences, in short the investigation of the relative unidimensionality of the EMQ examinationtheme (e.g. leg or cancer pain), the lead-in for the questions that gives the students instructions on what to do (e.g. "for each patient select the most likely diagnosis"); the questions in the form of vignettes giving the pertinent information based on which the student is to select the correct answer; and finally the potential answers   Figure . The resThe use of EMQs has been advocated to overcome some of the criticisms levelled at the use of multiple-choice questions to test factual and applied knowledge. There are advantages to using EMQs :• The format of themes aid the organisation of the examination, and the use of blueprinting is a natural aid to the process of writing EMQs• As questions are written in themes or general topic it allows the teacher to write many questions for that theme and then share these questions out randomly to create more than one examination paper• Good questions provide a structure designed to assess application of knowledge rather than purely recall of isolated facts• The approach to writing these questions is systematic, which is very important when several people are contributing questions to one exam• The extended list of options allows the inclusion of all relevant options, and reduces the opportunity for students to 'guess' the correct answer as in MCQs• EMQs were found to be more discriminating than two and five option versions of the same questions resulting in a greater spread of scores, and reliability was higher as a consequence of this ,6.The importance of ensuring validity and reliability of the EMQ procedure is crucial. Current evidence for this is limited but does support good reliability and validity ,6,7. ThiTraditionally, methods of analysis based on classical test theory have been used to evaluate such tests. The focus of the analysis is on the total test score; frequency of correct responses (to indicate question difficulty); frequency of responses (to examine distractors); reliability of the test and item-total correlation  -11. AlthDevelopments of the Classical Test Theory can be found in modern test theory and, in particular, the Rasch model  of Item not to find a model that better accounts for the data, but to understand statistical misfit as substantive anomalies that need to be understood, and by being understood, to lead to the construction of more valid and reliable tests. This is the approach taken in this study. That is, analysis of data based on existing items will be considered closely both statistically and substantively with a view to guiding better question construction. Thus the aim of this paper is to explore the use of Rasch analysis to determine the validity of the EMQ examination currently taken by 4th year medical undergraduates.The use of the Rasch model entails a different perspective, or paradigm, from IRT approaches in general . Where dth year medical students were used as the source data. The examination is designed to test factual and applied knowledge taught in the Medical and Surgical Specialties course and is taken by the students at the end of the 16 weeks course. The course is run three times per year and rotates with two other courses . All questions were devised by the lead educational supervisor within each specialty. Training in EMQ writing was provided to the medical specialty supervisors. The examination consisted of 98 EMQs distributed across eight specialties and 27 themes, each with two to four EMQs. Each themed group of EMQs had eight to 15 possible response options  was used to determine the impact of the term in which the examination was taken on student performance.difference between the student's ability [θ] and the difficulty of the question [β] (i.e. the ability required to answer the question correctly), and only a function of that differenceThe Rasch model is a probabilistic unidimensional model which asserts that (1) the easier the question the more likely the student will respond correctly to it, and (2) the more able the student, the more likely he/she will pass the question compared to a less able student. The model assumes that the probability that a student will correctly answer a question is a logistic function of the θ and β. Even though each response to each question must depend upon the students' ability and the questions' difficulty, in the data analysis, it is possible to condition out or eliminate the student's abilities  in order to estimate the relative question difficulties [From this, the expected pattern of responses to questions can be determined given the estimated iculties ,16. Thusiculties . The furiculties ,19. In ticulties ,21If the above assumptions hold true then the relationship between the performance of students on an individual question and the underlying trait  can be described by an S shaped curve (item response function). Thus the probability of answering the question correctly consistently increases as the location on the trait (knowledge) increases Figure . The steWhen the observed response pattern does not deviate significantly from the expected response pattern then the questions constitute a true measurement or Rasch scale . Taken wIn this analysis, responses to the EMQ are analysed as dichotomous options, that is, one correct answer and all of the other options are analysed together as one incorrect response. To determine how well each question fits the model, and so contributes to a single trait, a set of 'fit' statistics are used which test how far the observed data match those expected by the model. The trait refers to the required knowledge base that the student must acquire within the medical and surgical specialties course. The Item – Trait Interaction Statistic , reflects the degree of invariance across the trait. A significant chi-square value indicates that the relative location of the question difficulty is not constant across the trait. In addition, question fit statistics are examined as residuals . An estimate of the internal consistency reliability of the examination is based on the Person Separation Index where the estimates on the logit scale for each person are used to calculate reliability.Misfit of a question indicates a lack of the expected probabilistic relationship between the question and other questions in the examination. This may indicate that the question does not contribute to the trait under consideration. In the current study students are divided into three ability groups  denoting each Class interval with approximately 65 students in each. Furthermore, significance levels of fit to the model are adjusted to take account of multiple testing (e.g. for 24 items the level would be 0.002 and for 98 the level would be 0.0005) .As well as invariance across the trait, questions should display the same relative difficulty, irrespective of which externally defined group is being assessed. Thus, the probability of correctly answering a question should be the same between groups given the same ability level of the student. For example, given the same ability, the students should not be more likely to answer a question correctly simply because they sat the exam in the third term instead of the first or second term. This type of analysis is called Differential Item Functioning (DIF) . The basDIF is identified by two way analysis of variance (ANOVA) of the residuals with the term in which the examination was taken by the student as one factor and the class interval as the other . Two typAlthough EMQ are analysed as though they have dichotomous response categories (correct or incorrect), it is possible to examine how the separate incorrect options within an individual EMQ are contributing to the student's response. This procedure is very similar to the technique of Graphical Item Analysis(GIA) , though In view of our limited sample size (and particularly the ratio of students to items) we elected in the first instance to examine in detail the psychometric properties of the musculoskeletal component of the EMQ examination, acknowledging the limitations associated with the accuracy of the person estimate based upon 24 items (29). Subsequent analysis of the whole examination is reported to demonstrate the potential benefits of using Rasch analysis, but again acknowledging the limited conclusions that can be drawn on student ability and question difficulty estimates for the whole examination as a result of looking at 98 items with 193 students.Data were collected from 193 students . Total scores ranged from 36 to 78 out of a maximum mark of 98 . Initially, analysis of data was undertaken from the combined specialties of rheumatology and orthopaedic questions, which consisted of 24 EMQs'.2 = 79.73, p = 0.003). This suggests that the musculoskeletal questions mapped on to a single dimension of applied knowledge in this case and within the power of the test of fit. This was further supported by a principal components analysis of the residuals identifying a first residual factor accounting for just 8% of the variation. However, the Person Separation Index was low, 0.50, indicating a low reliability. This, however, can be ascribed to the intrinsic homogeneity of the students who are selected under rigorous criteria and who are all studying for the same exam.To estimate individual question difficulty using the Rasch model, all the incorrect response options were treated together as one incorrect option. The fit of the 24 questions to the Rasch model was acceptable, both in terms of individual item fit Table  and overlocation . Three out of the 98 EMQs showed significant misfit to the model at the individual level; one Infectious Diseases question, one Oncology question and one Rehabilitation Medicine question. Once these misfitting questions were omitted from the analysis, the overall fit of the remaining 95 EMQs to the Rasch model improved, and showed no significant Item – Trait Interaction. The questions from each of the component specialties within the course had reasonable spread of difficulty across the logit scale  and thus supported the notion of a valid unidimensional scale. Exploratory analysis of the whole examination showed that only three out of the 98 EMQs displayed significant misfit to the measurement model. These EMQs should be reviewed for ambiguities in question format and relationship to the course blueprint. DIF was identified in five of the 98 questions (these were different to the items displaying misfit) suggesting that the pattern of student responses to these EMQs were dictated by the term in which the exam was undertaken. The reason for this apparent variation needs to be understood before such questions are included into the bank of EMQs used for future examinations. In the present analysis only 'term' was considered for DIF, but DIF by gender also needs to be considered as gender can influence the response students make to the distractors .the incorrect response is selected more frequently than correct response, suggesting that the question appears extremely hard in that the majority of students are consistently selecting the same wrong option. This may relate to the stem being ambiguous, poor delivery of the teaching with students having experienced difficulty in understanding the topic such that the chosen response seems plausible to everyone.Rasch analysis revealed the variety of response patterns made by the students. In some questions the incorrect response is selected more frequently than the correct response but in only some ability groups (distractor), this suggests that the distractor option creates a strongly discriminating question. In this case less able students are more likely to select a distractor response while the more able students are likely to select the correct option.In those questions where most students select the correct answer, regardless of their overall ability level, the question provides little information to distinguish the ability of the students. Kehoe [Where s. Kehoe  has suggThe data presented in this paper on the analysis of response pattern within individual questions is purely descriptive. Although visual inspection of the response patterns is informative, Wang  is curreThe use of the extended matching format with up to 14 response options reduces the impact of guessing on the student's overall mark compared with standard multiple choice questions. However, it could also be argued that setting questions within themes puts the test at risk of a breach of the local independence assumption, in that responses to questions on the same theme may be locally dependent. The PCA of the residuals reject this and support the assumption of local independence.Rasch analysis also allows for the development of a bank of questions that have been calibrated with one another in terms of their difficulty. This allows different examinations to be constructed  while retaining the same level of overall difficulty. This will reduce the likelihood of students in consecutive years taking harder or easier exams when the standard they have to attain is unchanged.Classical test theories , widely used to investigate the quality of student assessment, make few assumptions about the characteristics of the questions such as whether they form a unidimensional construct. Therefore this approach can be used in a variety of measurement situations. A comparison between the classical approach and the Rasch approach with regard to discrimination is given in Figure In contrast, the Rasch measurement model checks two important assumptions: (a) the probability of answering one question correctly does not increase the probability of answering another question correctly within the examination  and (b) all questions in the examination map on to one construct . With respect to a) above, questions that incorporate sections whose answers influence the response to other sections within the same question cannot be analysed using this approach and, to b), unidimensionality is a requirement for the summation of any set of items (33).In addition, arguments have been made to use the two parameter and three parameter logistic models (the latter which adds a guessing parameter)  as theseFinally, a consistent problem with criterion related tests is the apparent low reliability, as expressed by a low person separation index. This is to be expected, as traditional tests of reliability are not appropriate for criterion-referenced tests  where thCase and Swanson ,5 have sThe author(s) declare that they have no competing interests.All authors have contributed to the analysis and writing of this paper and its revisions, and all have read approved the final version.The pre-publication history for this paper can be accessed here:


Priority setting, also known as rationing or resource allocation, occurs at all levels of every health care system. Daniels and Sabin have proposed a framework for priority setting in health care institutions called 'accountability for reasonableness', which links priority setting to theories of democratic deliberation. Fairness is a key goal of priority setting. According to 'accountability for reasonableness', health care institutions engaged in priority setting have a claim to fairness if they satisfy four conditions of relevance, publicity, appeals/revision, and enforcement. This is the first study which has surveyed the views of hospital decision makers throughout an entire health system about the fairness of priority setting in their institutions. The purpose of this study is to elicit hospital decision-makers' self-report of the fairness of priority setting in their hospitals using an explicit conceptual framework, 'accountability for reasonableness'.160 Ontario hospital Chief Executive Officers, or their designates, were asked to complete a survey questionnaire concerning priority setting in their publicly funded institutions. Eight-six Ontario hospitals completed this survey, for a response rate of 54%. Six close-ended rating scale questions , and 3 open-ended questions  of priority setting in your hospital?) were used.Overall, 60.7% of respondents indicated their hospitals' priority setting was fair. With respect to the 'accountability for reasonableness' conditions, respondents indicated their hospitals performed best for the relevance (75.0%) condition, followed by appeals/revision (56.6%), publicity (56.0%), and enforcement (39.5%).For the first time hospital Chief Executive Officers within an entire health system were surveyed about the fairness of priority setting practices in their institutions using the conceptual framework 'accountability for reasonableness'. Although many hospital CEOs felt that their priority setting was fair, ample room for improvement was noted, especially for the enforcement condition. Priority setting, also known as rationing or resource allocation, occurs at all levels of every health care system, including governments, funded provincial/territorial agencies, pharmaceutical benefit-management organizations, hospitals and clinical programs . CountriDaniels and Sabin have proposed a framework for priority setting in health care institutions called 'accountability for reasonableness' -7, which'Accountability for reasonableness' may be an effective framework for describing the components of priority setting, evaluating the fairness of priority setting in hospitals and identifying 'good' practices and opportunities for improvement. It can help hospitals improve their priority setting practices and can be an effective driver of health care reforms -10.'Accountability for reasonableness' has been used to evaluate priority setting in health systems . HoweverThe purpose of this study is to elicit hospital decision-makers' self-report of the fairness of priority setting in their hospitals using an explicit conceptual framework, 'accountability for reasonableness'.We conducted a survey of Chief Executive Officers, or their designates, of Ontario hospitals. The survey questionnaire covered 102 items, including hospital profile information . In this paper we focus on the results of 9 questions concerning priority setting, fairness, and the four conditions of 'accountability for reasonableness' . There are 160 hospitals in Ontario and all were invited to participate in this study.160 Ontario hospital Chief Executive Officers, or their designates, were asked to participate. 86 Ontario hospitals completed this survey, for a response rate of 54%. The average bed count of the responding hospitals was 250.4, with a range of 18 to 1265 beds. The average operating budget of the hospital sample was $75.8 million, ranging from $3.3 million to $733 million. The sampled hospitals represented a blend of teaching, small, community and specialized service facilities across the province.The survey was pre-tested and mailed to Chief Executive Officers of all Ontario hospitals in January 2001. Data from returned surveys were entered into an electronic database for further analysis. Of the 9 questions analyzed in this study, 6 asked for a rating on a 5-point scale ranging from 'Not Well' or '1' to 'Very Well' or '5', and 3 were open-ended.Close-ended ratings were analyzed statistically , with raOverall, 60.7% of respondents indicated their hospitals' priority setting was fair, while 79% stated their hospitals met their priority setting goals. With respect to the 'accountability for reasonableness' conditions, respondents indicated their hospitals performed best for the relevance (75.0%) condition, followed by appeals/revision (56.6%), publicity (56.0%), and enforcement 39.5%). (Refer to Table 9.5%. (Re = 3.94) condition significantly higher than the publicity ( = 3.57), appeals ( = 3.52), and enforcement ( = 3.25) conditions (p < .05) (paired-samples t test). While publicity and appeals conditions were not significantly different, respondents rated these conditions significantly higher than the enforcement condition (p < .05). The distribution of ratings suggested that there was ample opportunity for improvement, with most room for improvement in meeting the enforcement condition.Respondents rated the relevance . Fairness ratings were also positively correlated with each of the 4 'accountability for reasonableness' conditions (p < .01), as were meeting priority setting goals (p < .05).The bivariate correlation between each of the accountability variables of relevance, publicity, appeals/revision and enforcement was strongly positive (p < .01). The Pearson correlation coefficients among the 4 conditions ranged from 0.41 to 0.57. The internal consistency (Cronbach's α) of all of the 'accountability for reasonableness' conditions was very good (α = .78).Bed size  and operating budget  were negatively correlated with the rating of fairness in priority setting.In this section we describe participants' responses to the open-ended questions organized according to themes. We have included verbatim responses to help illustrate the themes.Decision makers emphasized four priority setting goals which were complex, interdependent, and required balancing. Some respondents said that (1) meeting needs existed in relation to delivering (2) quality services, or was constrained by resource availability. Examples of these goals include "quality care within available resources" and "access to high quality service in areas of greatest need". While (3) meeting budgets was identified as a goal by some decision makers, it did not exist in isolation from other goals such as meeting strategic directions or needs. One decision maker said his goal was "ensuring a balanced situation at year end while meeting the direction set out in our strategic plan". Achieving (4) organizational goals was expressed as a limiting, or organizational focusing process, which involved alignment or balancing of multiple goals and values.Decision makers described a deliberative process by which their priority setting goals were operationalized, providing examples of good practices and challenges or barriers for achievement. Overall, decision makers appeared confident that their priority setting goals were the correct ones, in no case did they comment that expressed goals were unrealistic, so contributing to lack of achievement.Decision makers pointed to five factors of importance in meeting their priority setting goals. Review processes (1) were described in which different review processes are brought to bear pointing to areas requiring additional resources. For example, a decision maker said "annually we review and ensure that resources are being allocated to priority programs and services". Leadership (2) was implicated as an important factor in meeting priority-setting goals. For example, a decision maker said "We do not set goals that are pie in the sky, they must be achievable". With respect to (3) stakeholder consultation, some respondents pointed to the need to involve the wider community in decision making. Decision-makers felt that improvements in priority setting were contingent upon (4) access to relevant information. Finally, some decision-makers emphasized the importance of (5) decision making tools, or benchmarking, to improve decision making.Respondents gave information to support their self-evaluation, and provided examples to demonstrate the fairness of priority setting processes from their perspective.In evaluating the fairness of priority setting, decision makers emphasized five themes. Stakeholder consultation raises the level of (1) inclusivity, bringing different points of views and interests to bear in solving common problems. For example, a decision maker said "we have a service providers network in our community that has direct input into resource allocation and program planning". Respondents described review processes (2) involving the deliberation of concerned parties, using relevant data, based on need, cost and other values (e.g. evidence). Reporting systems (3) provide an institutional feedback loop, pointing to areas where limits may be fairly set. With respect to (4) revision/appeals, revising arguments or values based on iterative processes involving various stakeholders is a feature of fairness, and helps to improve buy-in, but may increase organizational tensions. Finally, (5) governance may also be involved in developing decision-making models, and in ensuring the conditions of relevancy, appeals/revision, and publicity are met. For example, a decision maker said "The Board and senior management have developed decision-making models for key program/service changes...our appeals process is informal, but open to access including the Board".To our knowledge, this is the first published survey of hospital decision makers covering an entire health system, using Daniels and Sabin's framework of 'accountability for reasonableness'. It provides data to understand how fair priority setting processes are, what decision makers' priority setting goals are, and how well these goals are met.Several studies, using either survey or qualitative methodologies sometimes in combination with hypothetical "tradeoffs", have examined priority setting from theoretical perspectives aimed at capturing the public or professional's preferences or values in priority setting -24. RelaThis study makes five contributions to knowledge on priority setting. First, there is ample room for improvement in fair practices within Ontario hospitals as described by their Chief Executive Officers. Although many hospital CEOs felt that their priority setting was fair, ample room for improvement was noted, especially for the enforcement condition, followed by publicity, appeals/revision and relevance. While 79% of decision makers felt their hospitals met their priority setting goals, only 60.7% felt their processes were fair. According to decision makers, the perceived 'gap' was greater in meeting fairness than in priority setting goals.Second, it is feasible for decision makers to assess the fairness of priority setting in their institutions on a quantitative basis according to the 'accountability for reasonableness' framework, with greater than half of respondents saying their processes are fair. There is a high degree of association between 'fairness' and the internal components of 'accountability for reasonableness', lending support that fairness can be operationalized through 'accountability for reasonableness'. As well, the internal conditions of 'accountability for reasonableness' are highly related, and positively correlated, suggesting relevance, publicity, appeals/revision, and enforcement are measuring comparable aspects of fairness. Consistent with this, the high Cronbach's α finding is promising for the development of an 'accountability for reasonableness' scale of perceived fairness of priority setting in health care institutions. Measuring decision makers' self-report of their perception in meeting the various conditions of 'accountability for reasonableness', the scale would provide a practical tool for decision makers to assess self-reported views over time, noting progress in meeting conditions, while providing a reference point for needed improvement (i.e. enhance meeting 'publicity' condition). Finally, the finding of a negative relationship between number of hospital beds or budgets and self-report on fairness is consistent with the view that smaller hospitals may be perceived to have fairer processes, with greater involvement of their local communities in decision making and emphasis on transparency and "trust".Third, decision makers' views were surveyed at a health system level to determine what their actual priority setting practices were. Four complex, interrelated priority setting goals were described, suggesting a balance of need, quality, budget and organizational goals in priority determination. In addition, the study points to a close alignment of factors required in the meeting of priority setting goals and in the elements of fairness. In providing decision maker perspectives on characteristics of fairness, this study also adds to previous empirical work suggesting fair priority setting depends on a fair priority setting process . AdditioFour, this study has shown, from a blend of qualitative and quantitative approaches, that it is feasible to operationalize the 'accountability for reasonableness' framework, through the design of survey instruments to facilitate self-evaluation, and identification of good practices. This data can be shared with decisions makers to improve the fairness of their priority setting processes in meeting the four conditions of 'accountability for reasonableness'.Finally, this study expands on the likely relationship between leadership and priority setting in which leadership was found to contribute to perceptions of fairness in two committees engaged in priority setting for new health care technologies. Study results indicate that greatest room for improvement exists in meeting the 'enforcement' condition, with decision-makers describing critical leadership success factors, including: the role of governance in establishing policy and in meeting this condition, the need to set achievable corporate goals, and the significance of a lobbying funding function. Further study is needed to clarify the nature of this leadership contribution.The study is limited, first, in the response rate from hospitals. Only 54% of Ontario's 160 hospital CEO's responded to the survey. It is possible that hospitals responding to this survey were those doing, or perceived doing, better on self-report than others. However, there is no evidence to suggest that this is the case, and the sample did include small, medium and larger teaching hospitals, as well as specialized facilities to mitigate against representative selection bias. Second, social desirability bias was possible in that the views of decision makers expressed in the survey may not have corresponded to what they actually believed, or did. However, unpublished data  based onIn this first survey of Chief Executive Officers within a health region reporting on their assessment of the fairness of priority setting practices in their institutions according to 'accountability for reasonableness', ample room for improvement was observed, especially for the enforcement condition. Additional study is required to understand how application of 'accountability for reasonableness' will vary within and across institutions, and is shaped or influenced by various parameters, including the nature of corporate leadership. These evaluations can help to identify good practice opportunities for improvement that can be shared between local institutions or used within government to drive health care reforms.The author(s) declare they have no competing interests.DR was the primary analyst and principal author of the manuscript. DKM, CK and PAS were involved in study conception, analysis and drafting the manuscript. CK and DKM were involved in data collection. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:


This practice guideline was developed to provide recommendations to clinicians in Ontario on the preferred standard radiotherapy fractionation schedule for the treatment of painful bone metastases.A systematic review and meta-analysis was performed and published elsewhere. The Supportive Care Guidelines Group, a multidisciplinary guideline development panel, formulated clinical recommendations based on their interpretation of the evidence. In addition to evidence from clinical trials, the panel also considered patient convenience and ease of administration of palliative radiotherapy. External review of the draft report by Ontario practitioners was obtained through a mailed survey, and final approval was obtained from the Practice Guidelines Coordinating Committee.Meta-analysis did not detect a significant difference in complete or overall pain relief between single treatment and multifraction palliative radiotherapy for bone metastases. Fifty-nine Ontario practitioners responded to the mailed survey (return rate 62%). Forty-two percent also returned written comments. Eighty-three percent of respondents agreed with the interpretation of the evidence and 75% agreed that the report should be approved as a practice guideline. Minor revisions were made based on feedback from the external reviewers and the Practice Guidelines Coordinating Committee. The Practice Guidelines Coordinating Committee approved the final practice guideline report.For adult patients with single or multiple radiographically confirmed bone metastases of any histology corresponding to painful areas in previously non-irradiated areas without pathologic fractures or spinal cord/cauda equine compression, we conclude that:Where the treatment objective is pain relief, a single 8 Gy treatment, prescribed to the appropriate target volume, is recommended as the standard dose-fractionation schedule for the treatment of symptomatic and uncomplicated bone metastases.• Several factors frequently considered in clinical practice when applying this evidence such as the effect of primary histology, anatomical site of treatment, risk of pathological fracture, soft tissue disease and cord compression, use of antiemetics, and the role of retreatment are discussed as qualifying statements.Our systematic review and meta-analysis provided high quality evidence for the key recommendation in this clinical practice guideline. Qualifying statements addressing factors that should be considered when applying this recommendation in clinical practice facilitate its clinical application. The rigorous development and approval process result in a final document that is strongly endorsed by practitioners as a practice guideline. Radiotherapy is a well-recognized, effective modality in the palliative treatment of painful bone metastases. Bone metastases are a common manifestation of distant relapse from many types of malignant tumours, especially from cancers of the lung, breast, and prostate. With the advent of effective systemic therapies and improvements in supportive care, cancer patients are expected to live longer and may suffer from metastatic disease for a considerable length of time. Many patients with bone metastases suffer from compromised mobility and performance status.optimal" with different treatment goals, one could recommend that a "preferred" dose-fractionation is one that provides pain relief without undue toxicity and is least onerous to the patient.The optimal dose-fractionation schedule for the treatment of bone metastases is unclear. Two surveys of Canadian patterns of practice found that various fractionation schedules are employed by radiation oncologists, ranging from a single large-dose fraction  to a more prolonged course of 30 Gy/10 fractions over 2 weeks . While dDuring the past decade, significant clinical trial efforts have been devoted to comparing single large-dose radiation 8 Gy to 10 Gy) with multifraction regimens (five to ten fractions) . The two Gy with This provincial guideline was initiated to summarize the evidence and to provide recommendations on the preferred standard radiotherapy fractionation schedule for the treatment of painful bone metastases.This practice guideline was developed by Cancer Care Ontario's Practice Guidelines Initiative (PGI), using the methods of the Practice Guidelines Development Cycle . The praEvidence was selected and summarized by four members of the Supportive Care Guidelines Group (SCGG) and methodologists. Members of the SCGG disclosed potential conflict of interest information, reviewed the analysis of the evidence, and prepared draft recommendations. The SCGG includes palliative care physicians, medical and radiation oncologists, psychiatrists, nurses, psychologists, a chaplain, an anesthetist, a surgeon, methodologists, and administrators. After reviewing the evidence and considering issues of patient convenience and resource utilization, the SCGG reached consensus on draft recommendations. The systematic review and meta-analysis, conducted as the initial step in formulating this practice guideline, has been described elsewhere .External review by Ontario practitioners was obtained through a mailed survey consisting of items that address the quality of the draft practice guideline report and recommendations and whether the recommendations should serve as a practice guideline. The efficacy of the practitioner feedback survey process has been previously described . Final aPractitioner feedback was obtained through a mailed survey of 95 radiation oncologists across Ontario. The survey consisted of items evaluating the methods, results, and interpretive summary used to inform the draft recommendation and whether the draft recommendation should be approved as a practice guideline. Written comments were invited. Follow-up reminders were sent at two weeks (post card) and four weeks (complete package mailed again). The SCGG reviewed the results of the survey.Fifty-nine of the 95 surveys were returned (62% return rate). Key results of the practitioner feedback survey are summarized in Table The practice guideline report was circulated to PGCC members for review and approval. Eleven of the fourteen PGCC members completed and returned ballots. Ten PGCC members approved the practice guideline report as written, and one member approved the guideline and provided suggestions for consideration by the SCGG. Suggestions made were to reword the Target Population section and to clarify the second qualifying statement. The SCGG agreed with the suggestions and modified the guideline accordingly.Standard" refers to what is applicable to the majority of patients, with a preference for patient convenience and ease of administration without compromising treatment efficacy or morbidity. Our meta-analysis of all published randomized trials found no difference in pain relief between single fraction and multifraction treatments . Antiemetic agents should be considered as a prophylaxis, given that 30% or more of patients experienced vomiting following single or multifraction treatment in the two studies that specifically collected patient-assessed nausea/vomiting data ,10.For patients where the treatment objective is pain relief, a single 8 Gy treatment, prescribed to the appropriate target volume, is recommended as the standard dose-fractionation schedule for the treatment of symptomatic and uncomplicated bone metastases. This recommendation applies to adult patients with single or multiple radiographically confirmed bone metastases of any histology corresponding to painful areas in previously non-irradiated areas without pathologic fractures or spinal cord/cauda equina compression. It does not apply to the management of malignant primary bone tumour. The following qualifying statements are provided to support the application of the recommendation in clinical practice:Standard" refers to what is applicable to the majority of patients, with a preference for patient convenience and ease of administration and without compromising treatment efficacy or morbidity.• "• The recommendation does not apply to lesions previously irradiated, or lesions causing cord compression or pathologic fractures, because such patients were mostly excluded from clinical trials examining fractionation schedules.• Prophylactic antiemetic agents should be considered when a significant proportion of the gastrointestinal tract is in the irradiated volume.• Patients and referring physicians should be advised that repeat irradiation to the treated area may be possible.• There is insufficient evidence at this time to make a dose-fractionation recommendation for other treatment indications, such as long-term disease control for patients with solitary bone metastasis, prevention/treatment of cord compression, prevention/treatment of pathologic fractures, and treatment of soft tissue masses associated with bony disease.This practice guideline incorporates recommendations based on a systematic review, comprehensive consideration of how the evidence may be applied to clinical practice, feedback from Ontario practitioners, and input from the Practice Guidelines Coordinating Committee prior to final approval. It is strongly endorsed by practitioners for whom it was developed.ASTRO, American Society for Therapeutic Radiology and Oncology; CARO, Canadian Association of Radiation Oncologists; CT, computed tomography; Gy, gray(s); met, metastasis(es); MRI, magnetic resonance imaging; NCIC, National Cancer Institute of Canada; PGCC, Practice Guidelines Coordinating Committee; PGI, Practice Guidelines Initiative; RTOG, Radiation Therapy Oncology Group; SCGG, Supportive Care Guidelines Group.The authors declare that they have no competing interests.JW was the lead author responsible for designing and conducting the systematic review of the literature and the meta-analysis that informed the practice guideline, and for drafting and modifying the practice guideline report. JW was a member of the Supportive Care Guidelines Group and a Radiation Oncologist at the Juravinski Cancer Centre during the development of this guideline. RW reviewed all drafts of the guideline report and made major contributions to performing the meta-analysis that informed the practice guideline, and provided extensive input to the guideline as a radiation oncologist and methodologist. RW is co-Chair of the Supportive Care Guidelines Group. NL and MJ coordinated input from members of the SCGG, conducted literature searches, and drafted and edited the guideline report. MJ conducted duplicate data extraction and meta-analysis. NL updated the literature search, incorporated new data, conducted the practitioner feedback survey, and coordinated approval of the guideline by the Practice Guidelines Coordinating Committee. As members of the Supportive Care Guidelines Group, AB and TW provided substantial feedback on the guideline report at several points during its development, from both a radiation oncology and methodology perspective. Members of the SCGG provided feedback on all draft guideline reports.The pre-publication history for this paper can be accessed here:


Recent analysis of the yeast gene network shows that most genes have few inputs, indicating that enumerative gene reconstruction methods are both useful and computationally feasible. A simple enumerative reconstruction method based on a discrete dynamical system model is used to study how microarray experiments involving modulated global perturbations can be designed to obtain reasonably accurate reconstructions. The method is tested on artificial gene networks with biologically realistic in/out degree characteristics.It was found that a relatively small number of perturbations significantly improve inference accuracy, particularly for low-order inputs of one or two genes. The perturbations themselves should alter the expression level of approximately 50–60% of the genes in the network.Time-series obtained from perturbations are a common form of expression data. This study illustrates how gene networks can be significantly reconstructed from such time-series while requiring only a relatively small number of calibrated perturbations, even for large networks, thus reducing experimental costs. This process is expected to lead to novel classes of drug based on a network approach to cellular dynamics.Recent technological advances have led to an explosive growth in high-throughput genomic and proteomic data such as DNA microarrays. The rapid growth in available data has led in turn to a need for novel quantitive methods for analysis. As a consequence of this need, the reconstruction of gene network architectures from DNA microarray expression data has become a major goal in the field of systems biology. An increased understanding of the network architectures and their respective dynamics will enable novel approaches to disease treatments by allowing us, for example, to identify drug targets Frequently, the gene expression data itself is derived from perturbation experiments such as stress conditions, temperature shifts, and chemical treatments; for example, the widely used yeast cell-cycle datasets of Cho  and SpelO(log(n)) experiments (input/output pairs) might be required for n genes, but that to infer these relationships requires the use of computationally costly enumeration methods.A range of computational and mathematical techniques have been adopted in the effort to find a successful gene network reconstruction technique. Reconstruction methods often have to negotiate a tradeoff between intensive (often intractable) computations, and having to perform a large number of costly experiments. Certain progress can be achieved by making simplifications, such as imposing a limit on the number of inputs to each gene, or making steady state assumptions about the system ,4. Some n ~ 100), and still tractable for large networks (n ~ 1000), though this would require some parallelisation. The global perturbations themselves are simulated by changing the state of each gene at random. A perturbation intensity measure q, defines the probability that each gene will change state (see Methods).In this paper, we propose to explore the issue of how perturbation microarray experiments might be designed, and to suggest how such experiments might be optimised so as to maximize inference capability. Logical gene network models have previously been used to investigate gene network robustness , perturbsensitivity  was measured against P, the number of additional perturbations. Figure P).A discrete dynamical model was used to generate time series data from random networks (see Methods). To measure the effect of adding perturbations on inference ability, inference i which has one or more negative inputs, as well as having default value OFF. Since the discrete dynamics for this gene will be the same as if it had no inputs at all (i.e. zero gene expression for t > 0), the presence of the inputs is impossible to infer. This underestimation effect is clear in table Yi|, see Methods) with the actual solution sizes (i.e. the indegree distribution), and shows that the method is only able to produce roughly half the number of one and two input solution sets that actually exist.It should be noted that the algorithm tends to underestimate the number of inputs a gene may have. This is to be expected in genes for which dynamics cannot be informative: for example, consider a gene P can be explained at least partially, in the following way. Since the time series are discrete, many of the genes may have identical behaviour over time despite having different inputs (i.e. si(t) = sj(t) for two different genes i and j). If we define a "concatenated" time series vector  for gene i, and then map each gene i onto Si, we obtain a many-to-one mapping. As we increase the number of perturbations, we might expect the number of distinct time series also to increase. We define a simple measure to quantify this mapping, M = n'/N where n' is the number of distinct vectors Si, and N is the number of genes. The maximum value of M = 1 indicates that the mapping of genes to time series is one-to-one, whereas lower values indicate degenerate mappings. The manner in which M increases with the number of perturbations is shown in figure The increase in sensitivity with N, and the perturbation intensity parameter q .The experiments described above were repeated to consider variations in two other parameters: the network size P* required to reach a given high accuracy criterion was measured for different values of the network size N. The high accuracy criterion was defined as average sensitivity = 0.95 for one-input solution sets . To find P*, we first find the number of perturbations P+, such that average sensitivity P+ ≥ 0.95, and average sensitivity (P+ - 1) < 0.95. If average sensitivity P+ > 0.95, we use simple linear interpolation to find the  value of P* between P+ and (P+ - 1) for which average sensitivity = 0.95.To consider the first case, the minimum number of perturbations P* are shown in figure log(N) against P* (logarithms used are base 10). A least squares best fit gives P* ≃ 1.75 log(N) + 7.02, which, for N = 1000, gives P* ≃ 12.26. In order to obtain a measure of variance for P*, we would need to calculate P*-equivalent values for many individual networks separately, then consolidate these values to obtain the relevant statistics. However, because it was only feasible to consider medium-sized networks (20 ≤ N ≤ 70), and for any such network we often find only a small number of one-input solution sets, such statistics were found to be unreliable.The resulting values for q. Figure q, and figure q < 0.6. Together with a corresponding minimisation of the standard deviation in this interval (though it still remains fairly high in absolute terms), these results suggest that perturbation intensity should be close to this range to optimise inference accuracy.The second case (varying perturbation intensity) suggests an optimal range for P, and b) perturbation intensity, q.A recent analysis of the yeast genetic network has shown that 93% of genes are regulated by between 1 and 4 genes . This suN. These relatively modest experimental requirements would presumably imply lower experimental costs. The results also suggest that the perturbations should be calibrated , so as to alter the expression levels of approximately half the genes in each experiment. Generating perturbations which alter the expression level of half the genes at random may be difficult to achieve in practice, though experiments can be designed to come as close to this goal as possible. Even in the absence of optimal perturbations, we hope the simulation approach described here will still serve as a useful tool for planning experiments.The inference method itself is most useful for low order inputs, with inference accuracy maximized for predicted single input genes. More accurate methods have been proposed, though these generally require a much larger number of experiments ,15. MethN genes, the state of each gene si  is represented by the binary values 0(OFF) and 1(ON). Additionally, each gene is assigned a default ON/OFF state θi ∈ {0, 1}. The gene interactions are described by an (N × N) matrix C, composed of elements Cij ∈ {-1, 0, +1}, representing the positive(+1), zero(0) or negative(-1) influence of gene j on gene i. State transitions are calculated as follows:For a system of ith gene at the next timestep, si(t + 1), is therefore determined by the balance of positive versus negative inputs which are ON at the previous timestep t. If the balance is positive, then ui(t) > 0 and the next state will be 1(ON). Similarly, if the balance is negative, then ui(t) < 0 and the next state will be 0(OFF). If ui(t) = 0 , then the default value θi determines the next state. This default value needs to be given a priori, and for the purpose of this study will be random.The state of the s(t) and the default vector θ, the problem is to find the necessary model parameters (C) which will reproduce these dynamics. Specifically, a system initialised at s(0) should reproduce the given dynamics s(t) for t > 0. Note that multiple s(t) expression patterns may be defined, which will be denoted as sr(t) for r = 0, .., P, corresponding to time series with different initial states sr(0). Our problem is to find at least one interaction matrix that will reproduce all given dynamics sr(t). The problem of finding an appropriate matrix C may be broken up into N sub-problems, since in this system, each gene i may be solved independently from the others. More precisely, the inputs to gene i , can be found independently of the other genes. This reduces the search space from  down to O(N3N). Each input zi to gene i is represented as an ordered pair , j ∈ {1, .., N}, g ∈ {± 1}, indicating an input from gene j of sign g. A solution y(i) for gene i is a set of K inputs  (with y(i) = φ if K = 0). For K inputs there are  solutions to evaluate. Starting with K = 0 (no inputs), we progress up to a maximum of K = 4, exhaustively evaluating all possible solutions for each K. However, making a parsimony assumption, if solutions are found for some Ks < 4, the method no longer evaluates for K >Ks. Note that the method does not stop as soon as a solution is found, but evaluates all possible solutions for Ks. The failure rate (percentage of genes for which no solution was found for K ≤ 4) never exceeded 3% of the genes in any single network for which reconstruction was attempted.Assuming we are given the state dynamics s0(t) is generated by setting s0(0) = θ. The other time series sr(t), r > 0 are obtained from initial conditions which are perturbations of θ, and correspond to standard experiments such as stress conditions, or chemical treatments. Since, experimental perturbations can usually be modulated in intensity , this was represented using modulated artificial perturbations. Perturbed initial states sr(0) were generated by randomly changing each state s0(0) with probability q. This means that, on average, there will be qN random state differences between each perturbed initial state sr(0), and θ.The control time series y1(i), y2(i), ... are found for gene i, these are consolidated into a solution set, Yi = ∪l{yl(i)}. Note that some information about the solutions has been lost using this approach. For example, a solution set  obtained from a single two-input (K = 2) solution: , may be equal to another solution set  resulting from two single-input (K = 1) solutions:  with  and .Assuming one or more solutions sensitivity and specificity. Here, accuracy was measured using sensitivity, defined as true positives / . The relatively large number of true negatives, makes specificity, defined as true negatives / , an uninformative statistic. Here, true positives are members of the solution set Yi which are also true inputs , and false negatives are those true inputs which are not members of the solution set Yi.However, this consolidation is convenient in that the solution set is easily compared with the known network structures using standard accuracy measures such as N ≤ 70). Inferences on R random networks (each with N genes), will produce approximately RN sensitivity measurements (slightly fewer due to the nonzero failure rate).Accuracy statistics were gathered from inferences performed on a large number of medium-sized random networks  [k is given by:It appears to be the case in gene networks that indegree follows an exponential distribution, whereas outdegree appears to follow a scale-free distribution. More specifically, for the yeast network, the probability distribution for indegree nstants) . Here, anstants)  were crenstants) . The exppk = (1 - eβ-)eβk-,β = 0.45 is a constant. Similarly, the power law distribution  for outpk = Ckτ -eγk-,C, γ, and τ = 1 are constants. Since the algorithm begins by generating in/out-degree pairs for each node, we require equal means for both indegree (<kin >) and outdegree (<kout >). Following [where ollowing , we obtaβ is given, we obtain a value <kin > = 1.76, and fit the free parameter γ = 0.436 to obtain <kout > = <kin > Since the resulting networks are unweighted, non-zero weights  are assigned at random with probability 0.5, as in [Since 5, as in . It shouTM devised and implemented the experiments and drafted the manuscript. RS and AP supervised the project. All authors read and approved the final manuscript.