This paper describes a program that solves elementary mathematical problems, mostly in metric space theory, and presents solutions that are hard to distinguish from solutions that might be written by human mathematicians. U is an open subset of Y, then X, and the second is a proof that if A and B are subsets of X, then The main purpose of this paper is to describe a program that solves elementary mathematical problems, mostly but not exclusively in metric space theory, and that presents the solutions in a form that is hard to distinguish from solutions that human mathematicians might write. The following two proofs are examples of the program’s output.x be an element of U is open, there exists f is continuous, there exists  Let x be an element of f is an injection,  Let The main challenge we have faced in creating this program, discussed more extensively below, is that it does not seem to be possible to reconstruct genuinely human-like writeups from the proofs produced by automated provers from the machine-oriented tradition. In order to be able to produce such writeups we have had to use much more restricted proof methods than those available to modern provers. This in turn makes it more challenging for the prover to solve any individual problem, and indeed the program does not solve any problems that are beyond the reach of existing fully automatic provers. We should also note that we see this prover as the first stage of a long-term project to write a program that solves more complex problems in a ‘fully human’ way. We shall say a little about this wider project later in the paper.For the remainder of this section, we shall discuss the human-style output further. We shall explain why we think it is a goal worth pursuing, and expand on the difficulties just mentioned. In Sect. These days there is a thriving subcommunity of mathematicians who use interactive theorem provers such as Coq, HOL, Isabelle and Mizar. However, it is also noticeable that the great majority of mathematicians do not use these systems and seem unconcerned about the possibility that their arguments are incorrect, even though there is plenty of evidence that incorrectness pervades the published literature.There are several reasons for this, some of which are discussed by Bundy . One is not 100 % confident that a result is correct, and in such situations interactive theorem provers have an extremely valuable part to play: among their recent triumphs are verification of the four-colour theorem  with[natb]], a, b]).flow like real mathematical proofs. Mathematical proofs are like natural language texts—such as the document you are reading—in that sentences link together to form coherent discourses. From a mathematician’s perspective, this is what allows a proof to read as presenting a coherent argument rather than merely a sequence of facts. The principles behind discourse structure have been extensively studied by linguists . We do Non-resolution theorem proving  is, to produce a proof that can be read and checked for correctness by a human mathematician; the standard to be met is “peer review”, just as for journal publication. By contrast, the purpose of Weierstrass is As we shall discuss more extensively in Sect. e [The article just cited does not provide any actual ity of e  does notAnother recent human-oriented system is Analytica , a theorWe should emphasize that the work we describe is not meant to be competitive with any of the provers described here when considered as a prover; as noted above, the additional constraints involved in generating human-readable writeup are important to our project, but they also rule out many sound and effective tactics. Thus, our system is operating at a disadvantage, though it is our hope that ultimately the extra work we have to do now will help us see how to design systems that are more powerful than existing systems at solving the kinds of problems that human mathematicians are good at.input, including MIZAR [There is also an extensive literature on systems that accept natural language-like ng MIZAR , NaProChng MIZAR , ForTheLng MIZAR  and Mathng MIZAR ; we willThe basic architecture of our prover is, as we have mentioned, very similar to the ‘waterfall’ architecture used in the Boyer-Moore provers. The program contains a number of heuristics that transform the goal in a sound manner. These heuristics are ranked by attractiveness. The program operates fully automatically by repeatedly applying the most attractive heuristic that has an effect. The underlying logic is essentially first-order logic with equality, though there are some caveats to be made here  which we will outline as we go through the paper.Notwithstanding the overall waterfall architecture, the details of the operation of the program are actually better described with reference to the goal-oriented tactic-based style of proof used in some interactive LCF-style theorem provers . The corWe should stress that there is a key respect in which our system is not LCF-like. Like the Weierstrass system described in Sect. used to define the integers and thence the rational numbers. A full treatment requires an analysis of the operation of identification, which in turns requires modifications to the underlying logic [While it would certainly be possible to produce a program that generated human-like proofs and certified them correct, this is not an easy exercise for a number of reasons. One of these relates to the different number systems in use in mathematics: most work in theorem proving treats, say, the natural number 3 as being distinct from the rational number 3/1, whereas human mathematicians consistently treat them as the same object. One way of seeing this is to consider a standard definition likeng logic .In the remainder of this section, we highlight some other significant respects in which our approach differs from more typical approaches.targets. If we represent the goal as a sequent B is of the form Rather than having a list or stack of goals, we have a single goal, which may be regarded as a structurally shared version of the collection of goals found in a prover such as HOL. This structural sharing is necessary both for efficiency and in order to model human reasoning processes closely enough to produce human-like write-ups. A human reasoner will generally keep track of a list of background ‘assumptions’ which are known to be true (and likely to be helpful) and a separate list of statements that need to be proved. Under many circumstances the human will reason forwards from the assumptions without knowing in advance which of the statements to be proved will be deduced. If each goal were represented independently, this kind of forwards reasoning would have to be performed more than once. Thus, in our system the goal consists of a list of assumptions and a list of statements to be deduced from those assumptions, which we refer to as We should emphasise that our targets are not the same as the consequents Like many other systems, the program also has a library of statements that it may assume. However, the role that our library plays is very different. With an interactive program whose main aim is proof verification, the library will typically be a huge database of statements that have already been fully checked and can therefore be used safely.By contrast, for us the library is provided by the user and represents a body of results and definitions that a human mathematician would know and feel free either to quote or to use silently when trying to solve the problem at hand. Thus, the question of whether or not a statement is appropriate for our library is one that has to be considered carefully, and the answer varies from problem to problem. For example, if the system is asked to prove the associativity of set intersection, then we do not want it to say, ‘This result is already in the library, so we are done.’ But for a more advanced problem, we want the associativity of set intersection to be available to the program (or perhaps even built in to its reasoning processes) rather than requiring the program to formulate and prove it as a lemma. As ever, this is because we are trying to imitate human mathematical thought and so generate human-like output: one of the things an undergraduate mathematician has to learn is what it is appropriate to assume when proving a result.Thus, to use the program, it is not usually enough simply to input the goal and hope that the program will manage to prove it. Typically, the user has to add appropriate background results to the library first.The library that we have used with the problems we have worked on is small, so we have not had to face the potential problem of the program using a result that is ‘more advanced’ than what it is supposed to be proving. However, when this problem does arise, as it inevitably will, we plan to order our library results using a relation ‘is more advanced than’, so that for each problem we can simply instruct the library not to use results beyond a certain point in the ordering.We have also adopted a policy of being sparing about what we allow in the library. Human mathematicians do not store all the true mathematical statements that they come across in their brains. Rather, they seek out statements or techniques that have a kind of general quality. Exactly what this quality is, we do not claim to understand. It may be that for a more sophisticated program one should not expect to be able to judge the appropriateness of a statement in advance, but should store promising statements and then gradually forget about them if they turn out not to be especially useful.Two examples of results that we have in the library we currently use are the statement that a closed subset of a metric space contains its limit points, and transitivity of d to be less than A statement such as the triangle inequality would be most usefully stored in the library in the following form.Results, i.e. facts that the program can utilise while constructing a proof.Definitional expansions.Term rewrite rules.Constructions.Results and  expansions are the key to reasoning at a high level, which is in turn one of the keys to producing human-like proofs. Term rewrite rules and instructions play a much smaller role in the program. The former are used to change e.g. f(g(x)), which is an instinctive and automatic step for any human mathematician. Constructions generally ‘finish off’ a problem by supplying an object with certain properties that are being looked for. For example, suppose the program needs to find x s.t. At the moment, the library contains four kinds of data:Another unusual aspect of our approach involves paying close attention to the removal or deletion of assumptions. We include tactics for removing assumptions not because they have a significant effect on the output of the program, but because of our commitment to modelling how human mathematicians think. The particular phenomenon we are trying to model is that when humans work out proofs, they often find it obvious that a statement has been ‘used up’ and will have no further role to play in the proof. We have modelled this by incorporating tactics that, under appropriate circumstances, remove assumptions from the list of all assumptions.One reason we are interested in the removal of assumptions is that it forces us to think about a relatively simple special case of a hard and important general problem in theorem proving, namely the problem of deciding which statements are likely to be relevant to the solution of a given problem. It is possible to write programs that search through very large numbers of statements until they find something that magically works, but humans do not do this. So we feel that any efforts we make to understand this aspect of how humans do mathematics will pay dividends when we try to scale up to systems that will find more complex proofs. Another advantage of including tactics that remove assumptions is that it makes it considerably easier to debug cases where the program is stuck, by removing a lot of the ‘noise’ that makes it hard to understand intermediate goals.In general, natural language generation is a complex process. It involves multiple levels of planning, which draw on both domain knowledge and models of the intended audience, and also a phase when the actual text is generated, which draws on syntactic, morphological and lexical information. An overview of the process may be found in . BecauseFortunately, mathematical language has properties that make the task considerably simpler than it is for the English language in general. Foremost among these is the fact that mathematical proofs almost always have a particularly simple rhetorical structure. To some degree this is because the domain of discourse includes only timeless facts, which itself rules out a large proportion of the rhetorical relations found in general text. But the main reason is that there is a strong convention that further constrains the rhetorical structure of proofs. A proof proceeds by the presentation of a sequence of assertions, each of which follows from the premises of the theorem being proved or from previous assertions. This structure is not accidental; it is a direct reflection of the fact that mathematicians process proofs by reading and verifying one sentence at a time, and would not expect the justification of a fact presented in one sentence to be deferred to a later sentence. .This convention gives us an easy way to produce write-ups of our proofs. An obvious strategy is to allow each application of a tactic to generate some number of sentences (possibly zero), and then to concatenate the output from the different tactics to produce the final text. Note that this strategy is viable only because we are absolutely rigorous about requiring our tactics to reflect steps in human reasoning; in effect, the strategy is mimicking a human who is carefully writing down a proof while coming up with it, which is quite straightforward for an experienced mathematician. . As we shall see below, this simple strategy produces surprisingly good results, though with a weakness that needs to be dealt with by a postprocessing phase that turns out to be straightforward.template generation: each tactic is associated with a template, or ‘piece of text with holes’, and the holes are filled in with concrete information about the facts and objects used in the actual application. So, for example, forwards reasoning may be associated with a very simple template ‘since Because we have a fixed list of tactics, implementing the strategy only requires us to specify which sentences (if any) are produced for the applications of each tactic. A very simple way to do this is to use A is open and  Since A is open’ and ‘Note that individual facts are expressed in idiomatic ways, rather being displayed in a way that directly reflects the underlying predicate calculus; thus we have ‘An example of the text produced by this method is as follows:x be an element of A is open and B is open and  Let The main problem with this text is that it suffers a lack of coherence, in the sense defined in : the senA is open and  Since the repetition of the underlined phrase is awkward. Because it is introduced by the sentence immediately preceding the ‘since’ clause, it is awkward to have it spelt out explicitly within that clause. Similarly, consider: Since Here, having two identical ‘since’ clauses in consecutive sentences is again awkward: the repetition of material is unwieldy and unidiomatic.cue phrase. For example, ‘since A is open and A is open, ...’. The cue phrase ‘therefore’ (which assumes that the relevant reason has just been given) is less explicit than the cue phrase ‘since’ (which subordinates an explicitly stated reason), so it avoids spelling out information that is clear from the context. In other cases repetition can be avoided by combining sentences; thus the previous example may be changed intoWe are of the opinion that  correctl Since The initial ‘sentence by sentence’ process described above is followed by a series of transformations that manipulate pairs of consecutive sentences in order to resolve the issues just mentioned. . Applying this series of transformations to the example text above yields:x be an element of A is open, there exists B is open, there exists  Let One particular point worth emphasising is that the write-up process is deterministic: it will always produce the same output text for any given proof. This is for two reasons. First, if any non-determinism had been present we would have had to evaluate many outputs for any given proof, which would have made iterative improvement and fine-tuning of the write-ups considerably slower. Secondly, and more importantly, if the process were nondeterministic, our claim that the program produced human-like output would be suspect, in that we would have been able to run the program several times and ‘cherry pick’ output. Unfortunately, this determinism has an undesirable (but fully anticipated) side-effect. When one compare several proofs produced by the program, the write-ups are much more similar than those a human would produce. For example, most proofs produced by the program end with the phrase ‘we are done’. In the long run, we will undoubtedly need to introduce nondeterministic stylistic variation, allowing the program to vary the text generated for a particular step in just the way human would, despite the difficulties that will cause.Finally, it is worth noting that during the evaluation process described in Sect. functions, each with a distinct name and an (implicit) arity. An example is compose.A collection of predicates, each with a distinct name and an (implicit) arity. An example is less_than. Equality is represented by a distinguished predicate.A collection of mathematical types, each with a distinct name. An example is positive real number. At the moment the collection of types is specialized for use in problems involving metric spaces.A collection of variable is specified by a name , a mathematical type and a ‘variable type’, which indicates whether or not the variable is a normal variable or one of two kinds of metavariable, discussed in Sect. A term is either a variable or a function together with a list of terms of appropriate arity.A atomic formula consists of a predicate together with a list of terms of appropriate arity.An formula consists of one of the following, where An atomic formula.A As discussed in Sect. box is either a nontrivial box or the special box A nontrivial box consists of a list of variables, a list of formulae  and a list of targets.A target consists of either a formula or a list of boxes.A In the absence of variables, the box consisting of variables The formulae used in the program are essentially formulae of first-order logic, augmented with metavariables. More formally, we have:Targets that consist of a list of boxes correspond to a disjunction of the formulae corresponding to the individual boxes.The goal consists of a single box, and tactics are functions that map boxes to boxes. In the rest of this document, we will display the box consisting of variables statement to refer to a formula that appears either as an assumption or a target in some box. Statements and boxes may be tagged with additional information; for example, when a statement has been used together with certain other statements by a tactic, it is tagged to indicate this. Tags never affect the logical interpretation of the tagged material, but are used when the program is evaluating whether tactics are permissible. In particular, tags are used to prevent repeated application of the same tactic to the same statements.We use the term Both the human-like output and debugging code prettify the output in conventional ways, for example by writing existential statement, whereas the statementconjunctive. It is often useful to look more than one level down the parse tree. For example, the program would call the statementuniversal conditional statement. . Similarly, the existential statement above can be further classified as an existential conjunctive statement.When a statement is built out of atomic statements using connectives and quantifiers, the program classifies it according to the operations that appear at the top of its parse tree. For example, the statementx has an inverse” is pre-existential because it expands (in a suitable context) toA is unbounded” is pre-universal because it expands toelementary if it does not introduce a quantifier. For example, the expansion of Finally, many atomic statements can be expanded into statements that are no longer atomic. For example, the statement A is a subset of a metric space X and x is an accumulation point of A ifis substantive: it is an element of X that has the further interesting property of belonging to A.Consider the following definition. If We capture this in our program by representing background conditions through our type system. That is, instead of explicitly saying that Note that this is important for the categorization of statements discussed in the previous subsection. For example, we want to think of a statement such asIt is also important for deciding when a deduction is likely to be relevant. Suppose that we have a universal-conditional statement in the library that has several premises that match our assumptions. To mimic human reasoning, we would usually like this to count as evidence that the library statement is relevant, but not if the assumptions are merely background statements: if a library result requires a positive real number, we will not get excited just because we have a positive real number floating around.The following lines are taken directly from the program’s code: they list, in order of priority, the names of the tactics it can use. . In Sects. We stress here that because our system is fully automatic and intended to model human thought processes, our efforts have been concentrated less on the tactics themselves and more on how the program chooses which tactic to apply. For this program, the general form of the answer is as follows: it just chooses a tactic of the first type it can apply from the list above. Thus, if a tactic of type deleteDone can be performed, it performs it. If not, but a tactic of type deleteDoneDisjunct can be performed, then it performs that. Otherwise, it tries deleteDangling. And so on. In this way our architecture is similar to the ‘waterfall’ architecture used by Boyer and Moore in their provers NQTHM and ACL2. Like them we have tended to give higher priority to ‘lower-risk’ tactics, since this appears to correspond well to the way humans choose what to do; one of the challenges in generating human-like proofs is to assess correctly the risk of the different tactics. We discuss this point more fully in Sect. In the remainder of this section we shall describe the tactics, splitting them into some broad categories. One general preliminary remark is that a large proportion of the tactics contain minor ‘postprocessing’ steps, applying operations such as existential elimination. This is appropriate in precisely the cases where a human would apply such an operation in so automatic a fashion that it would not have any effect on the writeup. We will not mention these postprocessing steps in the tactic descriptions below as to do so would make them much longer. .These tactics remove assumptions  unmatchable if there are no available matches for it.For example, suppose that we have the statements The program is not allowed to substitute the same variable twice into the same assumption. . This can create further circumstances where an assumption is unmatchable. For example, suppose we apply forwards reasoning to the statements Tidying tactics are tactics that do not substantially change the content of a target, but put it into a more convenient form.x, a new assumption P(x) and a new target Q(x). This corresponds to the human tactic of saying (or thinking), ‘Let x be such that P(x); we need to show that Q(x).’If the target takes the form The tactic could be thought of as a composition of two more basic tactics, one an application of universal generalization and the other an application of implication introduction, but again our avoidance of bare conditionals demands that we treat it as unitary.P(x) to help us prove a different target. In that situation, we create a target that consists of a box.If there is more than one target, then this tactic has to be modified, since we cannot use the new assumption P(x) to prove Q(x) but not to prove R. The assumptions in the outermost box (which in the diagram above we do not enclose) can be used to prove both statements.That is, if we have a goal of the formx be such that P(x).”This tactic generates a representation which (if in isolation) would eventually be transformed into text like “Let P as an assumption and the original targets as its targets, and the other is a box with Q as an assumption and the original targets as its targets.If there is an assumption of the form This tactic and its counterpart splitDisjunctiveTarget constitute work in progress; they are important for generating certain human-like proofs, but (because they split the proof into cases) they are not compatible with the incremental, concatenative tactic-by-tactic generation of writeups. We intend to extend the writeup generation mechanism to accommodate these tactics in future versions. Note that neither tactic is used in the problems we evaluate on.P and Q. This tactic does not contribute to the writeup.If there is a target of the form P and Q.If there is a target that consists of a formula of the form This tactic exists for technical reasons: one reason is that the program sometimes likes to attach tags to statements, for example to record whether they have been used (which affects the deletion rules), but it has no facility for attaching tags to parts of statements. Therefore, if we want to use a tag to record information about one disjunct of a disjunctive target, we need to ‘split’ the target first.Also see the note regarding splitDisjunctiveHypothesis above.P is not a conditional statement, then this tactic removes the universal quantifier.If the target is of the form This tactic generates a representation which (if in isolation) would eventually be transformed into text like “Take P(x) and Q(x). The other circumstances are similar.This tactic is formed from a family of related sub-tactics whose common element is that under appropriate circumstances they replace a target with The sub-tactics generate representations that are eventually transformed into text like “Therefore, setting B that has no assumptions and contains no metavariables  to obtain a new assumption Q(x).The most basic form of this tactic is universal modus ponens: that is, a combination of substitution into a universal-conditional assumption followed by an implication elimination that uses the resulting conditional statement. In other words, one uses assumptions of the form x for u, and finally applying forward chaining. In order to produce a human-like writeup, our program does the same. In general, for each type of reasoning tactic that involves a universal conditional assumption, there is a variant that does the same thing to the expansion of a pre-universal conditional assumption.A slight variant of this tactic that is worth highlighting is illustrated by the following simple piece of reasoning: if we know that P(x), Q(x), and R(x).This tactic also handles deductions that involve more than one premise, such as using assumptions of the form This tactic generates a representation which (if in isolation) would eventually be transformed into text like “Since A’, ‘A is closed’ and ‘This is reasoning that is ‘mathematical’ rather than ‘purely logical’. For example, from the statements ‘not to unify the two forms of reasoning, so that library reasoning can be given a lower priority.Logically speaking, one could unify forwards library reasoning with ordinary forwards reasoning by adding the entire  content of the library to our list of assumptions. However, there are one or two aspects of library reasoning that give it a different flavour. The main one is that library results contain no free variables: they are general facts that apply universally. This distinguishes them from assumptions, which are more contingent. A second difference is that forwards library reasoning is normally used to deduce an atomic assumption from other atomic assumptions. A universal conditional statement is involved, but it is in the library and is not a assumption. For these reasons, reasoning that uses library results tends to be used by humans only when other forms of forwards reasoning are not available. Therefore, for the program it is important H is a subgroup of a group G and It is also important to place some kind of restriction on forwards library reasoning to stop the program making obviously irrelevant deductions. For instance, if it knows that A, A is closed and This tactic generates a representation which (if in isolation) would be transformed into text like “Since A is bounded’ might result in the introduction of a variable M and a new hypothesis As its name suggests, this means replacing a pre-existential assumption by its definition expansion. . What the name of the tactic does not reveal is that this expansion is followed immediately by an existential elimination. So for example expansion of the hypothesis ‘A is bounded, it follows that there exists M such that x would automatically be treated as an entity one could refer to: the program is like a human in considering this kind of instance of ‘there exists M such that ...’ as equivalent to ‘let M be such that ...’.  would be transformed into text like “Since This takes a assumption that has an elementary expansion  and replaces it by that expansion. This is sometimes combined with some tidying. For example, if the assumption in question is This tactic generates a representation which (if in isolation) would be transformed into text like “Since Q(x) and an assumption P(x).This is the obvious backwards counterpart of forwards reasoning, using universal modus tollens instead of universal modus ponens. The most basic form is thus that we are given a target Q(x) and an assumption Q(x) by the k targets More generally, if we have a target As with forwards reasoning, there is a simple variant where the role of the universal conditional assumption is played by a pre-universal conditional assumption instead. For example, given a target The contribution of this move to the writeup is complex. An example of the output it can produce is “We know that Q(x), then the target can be replaced by This is backwards reasoning that makes use of a general result in the library. However, it is slightly subtler than forwards library reasoning, because it always uses assumptions as well as a target. The precise rule is that if there are assumptions B is open,” if one wished to prove that A was open. This would be making use of the result that an intersection of two open sets is open.An example of this kind of reasoning would be to say, “It is sufficient to prove that Once again, the restriction we place is for pragmatic reasons: we do not want the program to make highly speculative transformations of the goal that introduce several new targets, since humans are usually reluctant to do this, especially when solving simple problems of the kind we are focusing on. But this is another situation where we would hope to improve the program in a future version, since humans do sometimes introduce multiple targets and then tick them off one by one.The contribution of this move to the writeup is complex. An example of the output it can produce is “Since A is open, i.e. that ...”. In the presence of metavariables, it generates a representation that will eventually be transformed into text like “we would like to show that This replaces a target by an elementary expansion of that target, if it has one. In the absence of metavariables, it generates a representation that will eventually be transformed into text like “we would like to show that This replaces a pre-universal target by its expansion. This tactic will be followed by one of the tidying tactics peelAndSplitUniversalConditionalTarget or peelBareUniversalTarget. It is usually the first tactic that the program applies when faced with a naturally stated problem.This tactic does not generate any write-up text.w into a metavariable. The metavariable needs at some stage to be chosen in such a way that the problem can be solved. If the variable only ever appears in targets, then one simple way in which this can often be done is to identify another variable x with the property that if we substitute x for w, then every target that involves w is equal to a assumption. In that situation, all those targets are replaced by As we are just about to discuss in more detail, we sometimes convert a variable This tactic generates a representation that will (after postprocessing) be transformed into text like “We may take g(f(x)). These rewriting rules are intended to represent operations that are so automatic that a human would not comment on them, and accordingly this tactic does not contribute to the writeup.There are a few term rewriting rules stored as data in the library. An example is that the term metavariables. The rough idea of a metavariable is that one reasons with it as though it had been chosen, deferring the actual choice until later when it becomes clearer what choice will make the argument work. Mathematicians often use this trick: a classic example is the ‘We now come to a class of tactics alluded to earlier: tactics that help us deal with existential targets when it is not immediately clear what to substitute for the existentially quantified variable. A standard technique for this, which is essentially the technique we use, is to form We have found it convenient to introduce two kinds of metavariable, to model two styles of reasoning that are logically similar but psychologically quite different. As ever, this mimicking of human reasoning is necessary to produce a human-like writeup. These are displayed with diamonds or bullets, as described below.To illustrate this, suppose we have a target such as x indicates that x needs to be chosen.More precisely, what the program does to is replace the statement with a box whose variables include the metavariable that is being introduced. In the example above, it would have no assumptions and a single target x.It is important for the program not to interchange quantifiers accidentally. For this reason, we tag the box just created with the variable After ‘unlocking’ the statement, the program applies the peelAndSplitUniversalConditionalTarget tactic inside the box. After that, we have a box that looks like this.x s.t. P(x) whenever Q(x).”This tactic generates a representation that will (after postprocessing) be transformed into text like, “We would like to find This tactic replaces a target of the form This tactic will never be applied to an existential universal conditional target, since that will have been dealt with by unlockExistentialUniversalConditionalTarget. The main reason we have two separate tactics here is that we prefer to bundle the unlocking together with the peelAndSplitUniversalConditionalTarget tactic when that is possible.To see what unlockExistentialTarget allows the program to do, suppose that we have a target of the form x is a witness to the non-emptiness of u so that there is a new target As another example of the use of unlocking, suppose that we wished to prove that u that belongs to u to be in A and in B. Aha, I can take x.’ The program’s unlocking models the silent disappearance of the existential quantifier before the second sentence of the above, which we need to model to produce a human-like writeup.This may seem a little circuitous, but it actually models quite closely how humans think. A human might say, ‘I want to show that x s.t. P(x).”This tactic generates a representation which will (after postprocessing) be transformed into text like “We would like to find This does exactly what it says: it replaces a pre-existential target by its expansion. It generates a representation that will eventually be transformed into text like “We would like to show that ... .”, explicitly presenting the expansion.There are certain tactics that the program will not apply to a ‘diamonded’ metavariable. In particular, it will not do any reasoning with an assumption that involves such a metavariable: for that it needs another kind of metavariable, roughly speaking corresponding to the human operation of ‘pretending that a variable has been chosen’ and then reasoning with it. Logically this is not an important difference, but it is a useful one for us because it reflects a difference in the way human mathematicians think and write. This helps the program to produce more convincing write-ups. When we convert a ‘diamonded’ variable into a full metavariable in this way, we change the diamond to a bullet.We do not need separate tactics for reasoning that involves assumptions with bulleted metavariables: we just allow the reasoning tactics described above to handle such metavariables.not allowed to depend on. This is for two reasons. First, it seems to reflect more accurately how human mathematicians think about such variables, and secondly, it is more economical: there are typically many fewer variables on which a bulleted variable is not allowed to depend than variables on which it is allowed to depend.An important technicality is that if we postpone the choice of a metavariable, we must keep track of which other variables it is allowed to depend on. However, what we actually do is note which variables it is This tactic generates a representation that will (after postprocessing) be transformed into text like “Assume now that ... .”, explicitly stating all assumptions involving the relevant metavariable.If we are told that two objects are equal, then we can eliminate all mention of one object in favour of the other. The precise rules governing when and how mathematicians tend to avail themselves of this opportunity are not obvious. The rules below are best regarded as a temporary solution: they do not always result in realistically human choices, and we intend to replace them by more satisfactory rules when we understand better what humans do.y by x and eliminates the assumption.If there is an assumption of the form This tactic generates a representation that will eventually be transformed into text like “Since v is a variable and t is a term, then this tactic replaces all occurrences of t by v.If there is an assumption of the form This tactic generates a representation that will eventually be transformed into text like “Since As we have already said, the tactics we use above are all either standard in themselves or simple combinations of standard tactics (with the possible exception of our distinction between ‘diamonded’ variables and more standard metavariables). Our main concern is not the set of tactics available to the program, but the way the program chooses which tactic to apply to any given goal. We have attempted to design this so that the program can produce a realistically human style of output in an incremental fashion. That is, each tactic needs to produce a list of human-like English sentences, or more accurately a list of elements of a datatype that correspond to such sentences. The postprocessing described in Sect. How should the program decide which tactic to use in any given situation? Our methodology for answering this question was to work through large numbers of problems ourselves, making a note of which tactics seem appropriate. After a while we were in a position to make a first guess at a suitable method for choosing tactics. We then tried the method out on more problems, adjusting it when it led to inappropriate choices. After several iterations of this, we arrived at the order of priority of the tactics that we set out in the previous section.If our only justification for the order of priority were that it leads to good results for the problems we have tried so far, it would not be very strong: what gives us any confidence that the order of priority will be appropriate for other problems that may be quite different from the ones we have looked at so far? However, there is an informal guiding principle that explains quite a lot  of the order of priority, which is that the program prefers “safe” tactics to “dangerous” tactics. As we mentioned earlier, the same is true of the order of priority chosen by Boyer and Moore in their ‘waterfall’ architecture .Broadly speaking, a tactic is safe if the risk that it will lead to an undesirable result, such as a dead end or a step that is completely irrelevant to the eventual proof, is small. For example, tidying tactics are safe in this sense: by expressing the goal in a more convenient form, they open up new options without closing off any old ones. Since they are so safe, they come first in the order of priority. By contrast, expanding a definition is substantially less safe: sometimes it is possible to reason in a high-level way without expanding, and since we do not allow ‘de-expansion’ in this program , expanding a definition is closing off the option of such high-level arguments, so we are reluctant to do it unless we have convinced ourselves that high-level arguments are not available. For example, if there is an assumption of the form ‘Thus, expansion has a fairly low priority. Having said that, some expansions, such as elementary expansions or expansions of pre-existential assumptions, are considerably safer, so those ones have higher priority.Somewhere in between are the other reasoning tactics. Here it becomes more complicated to apply the general principle, even as an informal guide, because the safety of a tactic depends heavily on context. In particular, forwards reasoning is in general fairly unsafe—if you have a lot of information and do not know which statements are relevant, then the probability that any given deduction will form part of the eventual proof may be quite small— but it is much safer when it comes to routine problems, which tend not to suffer from the problem of irrelevant information.when it is safe, humans tend to prefer forwards reasoning to backwards reasoning [It seems that easoning , 30, thoThis aspect of our program is, however, unstable, for the reason just given. When humans are faced with several possibilities for forwards reasoning, they will often switch to backwards reasoning in order to lessen the risk of making irrelevant deductions, but our program does not yet have any facility for making this kind of judgment.One other feature of the ordering of reasoning tactics is that we prefer pure reasoning tactics to library reasoning tactics. That is because in general an assumption is more likely to be relevant than a library statement, though if enough of the premises of a library statement are present as assumptions, that is a fairly strong argument for its relevance.x becomes obvious and it remains to carry out the relatively easy task of verifying that x has been chosen.At the bottom of the list of priorities is the process of creating metavariables. That is because humans tend to regard it as a last resort. When mathematicians need to prove statements of the form We will not say much more here about how we chose the priority order, but we have two brief further points. First, although our reasons are not completely precise, we found that in practice they were adequate, in the sense that they suggested an order before we started, and we found that we did not have to modify the order when we tried further problems . Secondly, when it comes to the finer detail of the ordering, there may not be that much to choose between different tactics. However, conflicts rarely arise between different tactics that are not distinguished by any of the above criteria, so in practice these finer details have little if any effect on what the program actually does.Now that we have discussed how the program works, let us look at another example, which involves most of the tactics we have discussed and shows how the order of priority works in practice. The problem to be solved is the following.A and B be open subsets of a metric space X. Prove that Let The initial goal is represented as follows.A is open’) and the second is an existential elimination . After that process, it arrives at the following.y.After this, deleteUnmatchable causes the program to remove the statements not allow the program to remove the assumption we have just used (and this is a good example of a situation where deletion would be a very bad idea). However, it cannot use the assumption with the new target. The program then uses backwardsLibraryReasoning again and this time it does remove the assumption, on the grounds that the variable x that appears in the assumption does not appeat in any other statement. After that, it has reached the following state.a and b is a positive real number that is less than or equal to both a and b.The highest priority tactic the program can now apply is elementaryExpansionOfTarget, so it does that, and automatically splits the resulting conjunctive statement (rather than using the tactic splitConjunctiveTarget).Once the program had generated the write-ups for several problems, we wanted to test whether they could pass for write-ups written by a human mathematician. In this section we describe an informal experiment that we carried out for this purpose.We began by asking two mathematicians, one an undergraduate and one a PhD student, to write out proofs for five problems for which our program had generated proof write-ups. We did not tell either of them why we were making this unusual request, and we did not ask them to make their write-ups as good as possible. One of the problems was to show that the inverse image of an open set under a continuous function is open, and one of our volunteers decided to prove the converse, so that he could use the topological definition of continuity to prove another of the assertions—that a composition of continuous functions is continuous. We had to ask him to rewrite the latter and give the epsilon-delta proof, since we wanted the differences between the write-ups to be a matter of style rather than substance.We had another problem of this kind, which was that both our volunteers made frequent use of open balls. For example, their expansion of ‘The program’s write-ups were not designed to be indistinguishable from human write-ups: we merely wanted them to be acceptable as human write-ups. Therefore, we left in certain features, such as ending every proof with the words, ‘we are done’, that we could with a little trouble have changed. . The third criticism would be harder to deal with, but in future versions, we plan to switch to having two styles of write up: a ‘proof write-up’ and a more detailed ‘proof-discovery account’. For the first style we will let the program work out the values of bulleted variables, then simply declare those values when the variable is first mentioned after being converted to a metavariable. This will correspond to the human practice of writing something like ‘Let It would be easy to modify the program so that none of these criticisms apply, so they do not point to fundamentally non-human aspects of how it thinks. To change the first, we would just have to use a library containing open-balls expansions of definitions such as ‘Although our program’s output came in for quite a bit of criticism, so did the write-ups by the undergraduate and PhD student—it seems that the readers were harsh judges. However, for most of the problems, the human write-ups were found preferable to the program’s.After the success (as we considered it) of this experiment, we dared to try a direct test. We published a new post, this time explaining that one proof was by a program, one by an undergraduate and one by a PhD student, and inviting readers to vote on which one they thought was by the program. For each problem, the write-ups were numbered (a), (b) and (c). There were seven options for the voting: one could choose between (a), (b) and (c), but also choose between ‘The computer-generated output is definitely We did not reveal the results of the voting so far, or anybody’s comments, until the experiment was ended and the voting was closed. However, there was a different kind of dependence between answers, which was that people had the opportunity to look for clues that two different write-ups were from the same source. Given that we had not tried to remove stylistic ‘tics’ from our program’s write-ups, this put the program at a significant disadvantage. It was clear from the comments that many people had noticed that for each problem exactly one write-up ended with the words ‘we are done’.http://gowers.wordpress.com/2013/04/14/answers-results-of-polls-and-a-brief-description-of-the-program/.Despite this, the program did reasonably well at fooling people that it was human. The typical pattern was that roughly half the voters would correctly guess which output was by the program, with slightly under half of that half saying that the output was definitely by the program. The undergraduate would always ‘come second’, and there would always be a fair number of people who said that they had no idea which output was written by the computer. There were surprisingly many votes for ‘The computer-generated output is definitely mathematicians that it was human. It could be that the more mathematically experienced readers found the program’s output easy to distinguish, while the votes for the human write-ups came from people who were not used to reading mathematical proofs. However, we feel justified in concluding that the program’s output is not obviously written by a computer program, and that was our objective.One thing this experiment could not tell us, except to a limited extent through the comments, was whether the program was good at fooling https://github.com/mg262/research/raw/master/robotone.zip; the readme file inside the archive contains instructions on compiling and running the prover. Note that although the problems and library are logically separated from the rest of the program, they are is currently stored as pure data in a Haskell module and compiled with the rest of the code.The prover was written in Haskell, and contains about 3300 lines of source code. Readers who wish to replicate the evaluation or try the prover on other problems can obtain the source code at Note that the shell script that invokes the prover also runs LaTeX on its output, and that this accounts for nearly all of its running time; the actual Haskell program runs too fast to measure  one has forgotten to include. Note that this is a benefit of our strategy of not backtracking: there is a definite single final state in which the program is ‘stuck’, and examining that state is invariably helpful.In the short term, we would like to make a number of small improvements to the program so that it handles a greater range of problems satisfactorily. In the longer term, we would like to enlarge significantly the set of problems that our program, or some new version of it, is capable of solving. To do this, we will have to enable the program to handle certain kinds of deductions that it currently handles either not at all or only in a rather rudimentary way. In particular, an immediate target is to give the program the means to deal with second-order quantification, which would allow it to solve easy compactness problems, and also problems that require the construction of ‘obvious’ sequences.At a more basic level, the program does not currently solve problems that involve proof by contraposition or contradiction. It is not hard to add tactics that allow it to cope with a few problems of this kind, but it is trickier to do so while not letting it apply those tactics in inappropriate contexts. More work is needed to understand what triggers the ‘contradiction move’ in human mathematicians, but we expect to be able to add this facility in the near future.The program is also not as good as we would like at handling equality substitutions. The situation here is similar: we can obviously add tactics that perform such substitutions (and have done so in the current version of the program), but it is more challenging to understand when humans make such substitutions. It is also tricky to come up with a general understanding of how they choose which out of two equal variables or complex terms to eliminate. At its most general, the problem of how to handle equality is well known to be hard, but our immediate aim would be a program that can handle the easy cases of that problem competently and in a human way.Related to this, we need to do more work in order to enable the program to solve problems that require the arithmetic structure of the real numbers, rather than just the order structure. For example, the prover does not yet solve problems such as showing that the limit of the sum of two convergent sequences is the sum of the limits.x, y and z with In the longer term, we would of course like the program to be able to solve non-routine problems. A major step up in problem-solving sophistication is needed when one is required to carry out mathematical constructions, especially when they are far from unique. This is true even for very easy problems. Consider for example the problem of finding an infinite set of positive integers that contains no three distinct numbers


Cyrtotrachelus buqueti using inverted fluorescence microscopy (IFM), scanning electron microscopy (SEM), and a mechanical testing system. IFM was used to investigate the distribution of resilin in the hind wing, and SEM was performed to assess the functional characteristics and cross-sectional microstructure of the wings. Moreover, mechanical properties regarding the intersecting location of folding lines and the bending zone (BZ) were examined. Resilin, a rubber-like protein, was found in several mobile joints and in veins walls that are connected to the wing membranes. Taken together, structural data, unfolding motions, and results of tensile testing suggest two conclusions on resilin in the hind wing of C. buqueti: firstly, the resilin distribution is likely associated with specific folding mechanisms of the hind wings, and secondly, resilin occurs at positions where additional elasticity is needed, such as in the bending zone, in order to prevent structural damage during repeated folding and unfolding of the hind wings. The functional significance of resilin joints may shed light on the evolutionary relationship between morphological and structural hind wing properties.Research data of the microstructure and surface morphology of insect wings have been used to help design micro air vehicles (MAV) and coating materials. The present study aimed to examine the microstructure and morphology of the hind wings of  With the changing nature of the environment, the flying skills of insects are also evolving. Coleopteran insects’ fore wings are often both thickened and stiff, which can prevent their hind wings from being destroyed during ground locomotion and can reduce their abdomen's motility during flight. Additionally, the flexible wing of insects has a kind of complex venation structure that can control the torsion under the aerodynamic forces, which is also the main reason that the size of the hind wings is larger than that of elytra. Previous studies have shown that insects in flight adjust their flight attitudes through the thorax muscles to distort the wing base, and no muscle exists in the wing membranes and veins, instead of resilin is located in some specific areas  and media posterior (MP 1+2) were examined in cross-sections using scanning electron microscopy, which showed that the veins were structured in layers. Furthermore, we assessed the mechanical properties in the locations of intersecting folding lines and the bending zone. This may indicate that the hind wings of C. buqueti have a special advantage that related to the design of bionic flapping-wing micro air vehicle. They include not only the small size and the ability to fold when not in use, but also inspiring engineers to design advanced composite materials.In the present study, the functional morphology and microstructures of hind wings of 2.2.1.Cyrtotrachelus buqueti were used for morphological investigations of the hind wings , the hind wings were washed using filtered phosphate-buffered saline (PBS) to remove dust and were stored at room temperature. For SEM was performed, the insects were dehydrated using 70% ethanol and were then placed in an incubator for at least 12 h. Drying may cause wrinkling of the wing surface, however, this process is important for obtaining high-quality images. After drying, the hind wings of each individual were removed for morphological examinations.Specimens of the bamboo weevil nd wings . All spe2.2.C. buqueti, the four dried hind wings were separately mounted between two coverslips with the anhydrous ethanol medium to flatten them. We observed the samples under an inverted fluorescence microscope  in one of three bands of wavelengths: the green , blue  and an ultraviolet ray (UV) band . We conducted a detailed investigation on the ventral and dorsal of each hind wing. The temperature and humidity in the laboratory kept at 25°C and 70%, respectively.This method can reveal the presence of the intrinsic resilin, a peculiar protein, without the need for antibody labeling or other chemical treatments. In order to more directly observe the distributions of resilin in the hind wings of 2.3.C. buqueti has strong flying ability. It not only can shuttle freely in the dense bamboo forest, but also can fly to the ground to drill and repair the nest. Therefore, in order to adapt to the change of the living environment quickly, the flying wings of C. buqueti play an irreplaceable role. In order to photograph the state changes of C. buqueti hind wings during deployment of the wings, a high-speed camera  was used to capture the video sequences of the hind wings unfolded at a speed of 2000 frames per second, which is the best tool for dynamic image acquisition and analysis.Through long-term field observations and insect's behavior experiments, it was found that 2.4.C. buqueti, high stability was expected. The wingspan is about 90–100 mm, which allows to provide sufficient lifting power and thrust during flight. The main veins  may play an important supporting role when the hind wings are subjected to bending deformation due to aerodynamic effects. According to previous studies , as shown in 3.3.1.Cyrtotrachelus buqueti has two pairs of wings, and each pair includes an elytron and a hind wing  line and the median flexion (MF) line. As confirmed by observations, the hind wings of C. buqueti are typically stowed under the elytra when folded, and rest on the thorax surface in a curved shape.Slightly different from  and BZ2 , and BZ13.2.IFM observation of UV-bands confirmed resilin occurrence in several areas of the hind wings. Auto-fluorescence of resilin was observed in locations that needed to be flexible, such as the MB  totally takes about 0.1820 s. When taking off, the locking mechanism of the When the hind wings were fully unfolded, the angle between the leading edge and the abdomen was about 80° . The vei3.4.C. buqueti manifest that the wing vein is a kind of anisotropic biological material with the organization of laminar structure. They have a cross-section of a layered structure as shown in In this work, at the micro level, the SEM observation of the main veins RA and MP1+2 (convex veins) of the hind wings of 3.5.C. buqueti were obtained by multiple sets of tensile testing. The force-displacement curves are shown in The mechanical properties of the bending zone and the intersection of folding lines in the hind wings of 4.C. buqueti is a versatile insects that are a skillful driller and are able to effectively chew in holes and sucks on the bamboo. They can fly freely in dense bamboo forests, which shows that they have evolved special wings to cope with complex conditions. Therefore, from the perspective of biomechanics, the hind wing of C. buqueti is not only an outstanding biological material, but also provides an idea for the design of a new concept micro-bionic flapping-wing aircraft. Moreover, the macroscopic and microscopic structures of their hind wings have important guiding significance for the design of lightweight structures and materials.The C. buqueti. The main work includes: (1) fluorescent inverted microscope was used to observe the presence of resilin for its hind wings; (2) the unfolding process of the hind wing was recorded and analyzed by high-speed cameras; (3) microstructural features of the veins were observed using SEM; (4) the force versus displacement curves of the intersection of the folding lines and the bending zone were obtained. The results show that resilin in the hing wings of C. buqueti is mainly distributed in the edge of the wing veins and some specific areas  between the two basal planes. In order to form a plane when unfolded (Q = 180°), the sum of the four angles at the origin must be 360°. When the opening angle is less than 180°, the produced shape is not plane. When the opening angle exceeded 180°, the structure assumed a position formation mirroring the first stable position  is a thick-walled cylinder with one layer  which was densely structure, a middle layer, and a lower layer (lower epidermis) which was also densely structured (C. buqueti and resilin play an important role for the resistance to torsional deflection and overall elasticity.Previous studies confirmed several microstructural properties in insects: Wang et al.  and Chenructured . The midructured , which lructured . Insect ructured . MoreoveC. buqueti can undergo substantial passive bending deformation by aerodynamic forces without sustaining damage. Moreover, elastic energy stored in resilin may partially contribute to the mechanical properties of hind wings. In the meantime, from the above experiments, it was find that there were a large number of resilin patches in the intersection of folding lines and the bending zone. Therefore, we can determine that the flying wings of C. buqueti have strong mechanical properties. This resilin may reduce the alternating stress generated by the wings during flight and folding or unfolding.The mechanical test of the bending zone and the intersection of folding lines showed that, for the bending zone and the intersection of the fold lines, the samples broke under a specific range of loads. The results showed that the two tested areas were able to withstand mechanical strains of 50–100 times of the insects’ body weight. The mechanical properties of flying wings in may be useful for the design of bionic wings. The flying wings of 5.C. buqueti need to be relatively large in order to ensure flight capability and so must be folded to completely fit under the fore wing. The Coleoptera evolved a unique solution for unfolding and folding of the hind wings. The preliminary studies on the morphology characteristics and mechanical properties of the hind wing of C. buqueti were completed in this paper. We combined four technologies; IFM, high-speed photography, SEM, together with the tensile testing. The microstructures indicated that the hollow structure of the veins can reduce the flight weight of the insect. The function of resilin located in specific areas may reduce the alternating mechanical stress generated by the wings during flight and repeated folding and unfolding. In the past decade, research on resilin has received extensive attention. The mechanical properties of resilin with potentially greater application in various fields. Mechanical tests confirmed superior mechanical properties of the hind wings. Therefore, our results may be of importance for the field of aeronautics regarding the folding and unfolding of aircraft deforming wings, and perhaps for astronautics regarding the folding and unfolding of satellite solar panels.The hind wings of 6.The field of my research involved morphology, lightweight biomaterials, biomimetic structural design and bionic engineering.


Purpose: Strong mentorship has been shown to improve mentee productivity, clinical skills, medical knowledge, and career preparation. We conducted a survey to evaluate resident satisfaction with mentorship within their radiation oncology residency programs.Methods and Materials: In January 2019, 126 radiation oncology residents training at programs in the northeastern United States were asked to anonymously complete the validated Munich Evaluation of Mentoring Questionnaire (MEMeQ). Results of residents with a formal mentoring program were compared to those without a formal program.Results: Overall response rate was 42%(n = 53). Participants were 25% post-graduate year two (PGY-2), 21% PGY-3, 26% PGY-4, and 28% PGY-5. Only 38% of residents reported participation in a formal mentoring program, while 62% had no formal program, and 13% reported having no mentor at all. Residents participating in a formal mentoring program reported strikingly higher rates of overall satisfaction with mentoring compared to those who were not . Overall, 38% of residents were either satisfied/very satisfied with their mentoring experience, while 49% of residents were unsatisfied/very unsatisfied.Conclusion: Residents participating in a formal mentorship program are significantly more likely to be satisfied with their mentoring experience than those who are not. Our results suggest that radiation oncology residency programs should strongly consider implementing formal mentorship programs. Strong mentorship has been shown to improve mentee productivity, clinical skills, medical knowledge, and career preparation –5. A recDespite the reportedly strong correlation between mentorship and mentee success, there has been relatively little research performed regarding mentorship in radiation oncology residency. In this study, we conducted a survey to evaluate resident satisfaction with mentorship within their radiation oncology residency programs. We hypothesized that a formal mentorship program  improves overall satisfaction with the mentorship experience.Surveymonkey.com, San Mateo, CA USA) to each program's chief residents, to be forwarded to the residents presently enrolled in their training program. The survey was accessible for a 4-week period from January 1, 2019 to January 31, 2019. No personal information was collected. This study was conducted according to the Rutgers Cancer Institute of New Jersey institutional review board guidelines. Responses were compared based on reported participation in a formal vs. less than formal mentoring program. Univariate comparisons were performed using Chi-squared tests.The Munich Evaluation of Mentoring Questionnaire (MEMeQ) is a validated online questionnaire of 7 items which evaluates satisfaction with mentoring relationships . In Janun = 53). Participants were 25% post graduate year two (PGY-2), 21% PGY-3, 26% PGY-4, and 28% PGY-5. Thirty-one residents (59%) reported having one or two mentors, and seven residents (13%) reported having no mentor. The top three areas where mentoring was found to be helpful included research (94%), job opportunities (92%), and networking (84%). Other areas mentioned included guidance on public speaking/presentation (35%) and work-life balance (13%).Overall response rate was 42%  by 68% of residents, supportive/encouraging by 65%, providing guidance on course of study or career management by 60%, answering questions satisfactorily  by 60%, motivating by 53%, and accessible by 49%. Overall, 38% of the residents were satisfied with their mentoring experience while 49% were dissatisfied and the remainder were ambivalent . On furtp < 0.001). Mentors' accessibility  and ability to answer questions satisfactorily  also improved with formal mentoring programs (Twenty residents (38%) reported participation in a formal mentoring program with regular interval meetings. Residents participating in a formal mentoring program reported significantly higher rates of overall satisfaction with mentoring compared to those who were not  with their mentorship experience. Given the proven importance of mentoring in terms of resident satisfaction and future success, our results suggest that radiation oncology residency programs should strongly consider implementing formal mentorship programs if they have not already.All datasets generated for this study are included in the article/supplementary material.The studies involving human participants were reviewed and approved by Rutgers Cancer Institute of New Jersey. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The handling Editor declared a past co-authorship with several of the authors SJ and SK.


The rural interdisciplinary team training (RITT) program has provided in-person training for almost 2000 VA providers and staff at 125 rural clinics since 2011. The multi-modal workshop, accredited for 6.5 hours for a number of disciplines, focuses on the recognition of common issues facing older Veterans, red flags prompting further assessment, how to administer screening instruments and team-based approaches for improving patient outcomes. Participants develop an improvement action plan improvement project based on common challenges faced by clinic patients. A program evaluation found an increase in geriatrics knowledge and a modest improvement in teamwork after the education program. It also found that participants self-identified an enhanced ability on average to use red flags after the 6.5 hour training in areas such as polypharmacy, falls and caregiver stress. The action plans were often not implemented at follow-up. The evaluation results will be discussed, as well as challenges, limitations, and implications.


Toxoplasma gondii infection offer solutions, potentially addressing cost concerns and leading to better clinical outcomes through improved access to screening. Testing performance of a lateral flow immunochromatography-based Toxoplasma ICT IgG–IgM test for combined detection of Toxoplasma-specific IgG and IgM has been previously described using serum samples from the National Collaborative Chicago-Based Congenital Toxoplasmosis Study (NCCCTS) and other cohorts [Toxoplasmosis causes substantial morbidity and mortality on a global scale . This di cohorts –8. HereiSamples were obtained from consenting United States individuals, including seropositive individuals affiliated with the NCCCTS ,6 and obμL of blood, which was directly applied to the Toxoplasma ICT IgG–IgM–BK (LDBIO) test, followed by application of four drops of buffer, provided in the kit. Tests were interpreted at 20–30 minutes by individuals performing tests and photographed for later interpretation by two individuals unaware of the subjects’ identity and serologic status.Participants provided whole blood via fingerstick . ParticiEarlier, we described 100% sensitivity and specificity of a pink-line, serum-variant of this POC test . Given iAll participants provided written, informed consent. This study was performed in accordance with the rules and regulations of the University of Chicago Institutional Review Board (protocol #8793) and/or IRB approval in Morocco.T. gondii assessed using the whole blood-variant test and confirmed using their NCCCTS records (n = 71), or concurrent standard laboratory testing (n = 134). Nineteen seronegative pregnant women had fingerstick and venipuncture performed each month for the first three months and three more for the first month (two with pregnancy loss) in an ongoing pilot gestational screening program, adding 41 samples to make 244 total samples. Overall, 101 samples proved seropositive, including five from acutely infected individuals who had IgM/IgG antibodies against T. gondii, while 143 were seronegative. Seropositive is defined here as having detectable anti-Toxoplasma IgG and/or IgM. For a small subset of pediatric patients  who were undergoing venipuncture as part of routine clinical care, whole blood was obtained from the needle tip to avoid a second, potentially distressing fingerstick.A total of 205 persons , Table 2Toxoplasma antibodies infected at remote times and with lower titers were positive in the POC test in the range detected by gold-standard tests  and specificity of 100% (95% CI: 97.45%–100%) . Whole-brd tests .No participant refused a second fingerstick, although they were informed they could. Participants did not report significant discomfort associated with fingerstick. Patients and providers enthusiastically accepted the monthly gestational screening program.Toxoplasma infection has the potential to markedly change clinical approach to this infection by detecting seroconversion using small volumes of whole blood—obviating venipuncture and expensive equipment for serum separation—and performing with high sensitivity and specificity. Advantages and disadvantages are discussed in Point-of-care testing for Advantages:True POC test, useful in a standard obstetrician’s office and less invasive than conventional serologic testing . POC testing has the potential to significantly expand access to screening during gestation for this serious, potentially fatal infection, with spillover benefit in facilitating screening programs for other congenital infections and improvements in maternal fetal health and well-being and care. Our demonstration of the high performance of this whole-blood POC test and the test’s strong functionality at the POC, which has not been previously demonstrated, provide the proof of principle of its potential utility and widespread applicability in clinical settings. This new test also fulfills the World Health Organization criteria for the ideal POC test .Performance and interpretation of test take less than two minutes of operator time with results available to be interpreted within 20–30 minutes, facilitating appropriate clinical intervention so that it is ideal for following those who are seronegative to detect seroconversion.No need for expensive equipment/electricity, decreasing cost and enhancing utility, especially in developing countries.Disadvantages:T. gondii-specific IgM. It also requires follow-up testing for seroconversion when used for screening during gestation.Requires initial, alternative testing for those who are seropositive that distinguishes IgG from IgM and tests for avidity of antibody for those who have Requires a fingerstick, which occasionally may need to be performed more than once due to technical difficulties.Requires materials  and training to perform and interpret test.Toxoplasma-specific IgG and IgM, or IgG or IgM alone, are present. It does not identify natural IgM antibodies, which are false positives [While the cost effectiveness of gestational screening and antenatal treatment in the US is demonstrated for a wide range of parameters , it was ositives . For iniS1 Methods(DOCX)Click here for additional data file.S1 TableSerologic results for all included participants in the United States (A) and Morocco (B).(DOCX)Click here for additional data file.S1 Movie(MP4)Click here for additional data file.


Introduction: A number of effective physical activity programs for older adults exist, but are not widely delivered within community settings, such as the Cooperative Extension System. The purpose of this paper was to determine if an evidence-based intervention (EBI) developed in one state Extension system could be scaled-out to a new state system.Methods and results: The RE-AIM  framework was used to guide an iterative evaluation of three translational stages. Stage 1: Before program adoption, Extension health educators were surveyed and interviewed to assess physical activity programming perceptions and factors that may influence their decision to attend training or deliver the program in practice. Results indicated that a virtual, scalable training protocol would be necessary and that training needed to include hands-on instruction and be catered to those who were less confident in physical activity program delivery. Stage 2: Training attendees were surveyed pre- and post-training on factors related to the adoption-decision making process and contacted post-training to assess program delivery status. Training did not influence perceptions of the program, intent to deliver, or confidence in delivering the program. Stage 3: During program implementation, the program was evaluated through the RE-AIM framework by surveying across three key stakeholder groups: (1) program participants, (2) potential delivery personnel, and (3) Extension administrators. Findings indicate that the program has the potential to reach a large and representative proportion of the target audience, especially in rural areas. However, adoption and implementation rates among Extension health educators and community partners were low and data collection for effectiveness, implementation, and maintenance was a challenge.Conclusion: Overall, the results indicate initial struggles to translating and evaluating the program in a large, rural state. Implications for practice include making system-level changes to increase physical activity program adoption rates among Extension health educators and improve data collection and program evaluation through this community-based organization. More work is needed to identify infrastructure support and capacity to scale-out EBIs. Delivering (and sustaining) evidence-based interventions (EBI) in community practice is challenging. One challenge is that delivery personnel have to select an intervention and ensure its fit within the mission, values, and resources of the system they are delivering . County-One way that Extension has been challenged at scale-out is that rather than scaling core elements of “what works,” new or unique programs are introduced to the system. This is evidenced by a recent systematic review of open-access physical activity interventions for older adults which found that 17 unique open-access physical activity programs were offered by Extension professionals . Ten Extension health educators cover the state's 23 counties and the Wind River Indian Reservation . These educators have a broad reach, interacting with over 40,000 participants per year. In Wyoming, there is no Extension health or physical activity specialist. When a new county-based Extension health educator  was hired by University of Wyoming Extension (UWE), the educator contacted the Virginia Cooperative Extension (VCE) Exercise Specialist to inquire about potential research partnerships. The new county-based Extension health educator served as a knowledge broker  . In alig5. Ten ExLifelong Improvements through Fitness Together (LIFT) , 18 was n = 5) indicated that they would be interested in receiving training (either in person or via webinar) on delivery and evaluation of the LIFT program.Before offering a LIFT training, the exercise specialist and UWE knowledge broker conducted a pragmatic concurrent, transformative mixed-methods [i.e., equal emphasis on the quantitative and qualitative findings n = 115 meaning units) and facilitators (n = 157) for delivering physical activity programs and types of programs delivered (n = 71). See Table Thematic coding of the two interviews yielded 349 meaning units reporting perceived barriers  designed to target Extension health educators who were less confident in delivering physical activity programs and include hands-on instruction and teach-back. LIFT training was created as a 4 h “live” virtual format based on evidence-based methods on training , 25, leaIn alignment with Extension practices, pragmatic recruitment and feedback methods  were used. Nine participants completed the initial LIFT training in September 2017: the knowledge broker, one Extension health educator, and seven community partners . Following this training, additional community partners expressed interest in the training, and the research team offered another training in December 2017 to four community partners.Z = 0.000, p = 1.000) or inclusion of LIFT in a plan of work .Of the thirteen total training participants, nine (69%) completed surveys. Of the seven who completed pre-training surveys, trainees were predominantly middle-aged Caucasian females. All participants (100%) were very or completely confident in meeting physical activity recommendations. While all participants (100%) reported high physical activity levels through the International Physical Activity Questionnaire (IPAQ), only 57% reported meeting strength training guidelines. Pre-training, participants agreed that they intended to deliver LIFT and to include LIFT in a plan of work; mean values were both 4.00 (+0.707). The Wilcoxon signed-rank test showed that post-training surveys did not detect a statistically significant change in intent to deliver LIFT (The program characteristic that influences the adoption-decision making process of highest importance was “The program has been successful when tested in community settings” (mean rating of 4.00 = very important). While the majority of the factors (73%) had a mean rating between 3.00 and 3.99 (moderately important), four factors rated as only “somewhat important”: “I do not feel that the program is part of my job responsibility,” “I do not feel comfortable delivering the program,” “I am not physically active, so do not feel comfortable delivering a physical activity program,” and “I do not have the expertise that is needed to deliver the program.” By February 2018 (5 months after the first training and 2 months after the second), three LIFT programs were being delivered through UWE, and the comprehensive evaluation was initiated. Due to this low initial implementation rate, a survey for trainees who both did and did not implement the program was added to the measures for adoption.reach and effectiveness (LIFT participants), adoption and implementation , and system-level maintenance (UWE administrators). LIFT participants were recruited through senior centers (including participants in Strong Bones programs), newspaper articles, flyers, and word of mouth. Extension health educators and community partners were contacted via email after the LIFT training to complete a brief online survey. Extension administrators (N = 3) were also contacted via email to complete a brief online survey.The final step in this research was to evaluate both the individual-level impact and the system-level delivery of LIFT through the RE-AIM framework. The evaluation included three levels of respondents in an attempt to collect data on all RE-AIM dimensions: Data were collected on all RE-AIM dimensions except for individual-level maintenance, which was outside the scope of this work .The pre- and post-program surveys included IPAQ items to assess whether participants were meeting physical activity recommendations . A validThe primary adoption indicator was the total number and representativeness of those trained on LIFT program . In addition, all those eligible to deliver the LIFT program were asked to complete a survey assessing: (1) acceptability, appropriateness, and feasibility  of impleThis was assessed through process evaluations designed for delivery personnel to self-report the extent to which the program was delivered as intended and capture adaptations made during program delivery. The process evaluations contained five categories: warm-up activity, group-dynamics strategy, exercises, cool down, and overall program delivery.As a proxy measures for system-level maintenance, administrator perceptions were sought related to: (1) acceptability, appropriateness, and feasibility  of LIFT;t-test was used to compare mean age; Fisher's exact test was used to compare categorical variables due to the small sample size.Means and standard deviations of continuous variables and frequencies and proportions of nominal variables were calculated for the overall sample. Representativeness was calculated by comparing demographics  of LIFT participants to all older adults (age 65 and older) in Wyoming (city or county level census data was not available) . As raw 7.0) with seven (39%) classified as obese, seven (39%) classified as overweight, and four (22%) classified as normal weight. For each of the three delivery locations, proportion of LIFT participants was calculated as 17 out of 1,480 adults age 65 or older (1%) in Lander, 12 out of 35 (34%) in Pavillion, and 8 out of 281 (3%) in Guernsey. When comparing the representativeness of LIFT participants to older adults in Wyoming, there were no significant differences in terms of race (p = 1.000), ethnicity (p = 1.000), employment status (p = 1.000), education level (p = 0.297), and BMI (p = 0.324). There was a significant difference in age and gender: LIFT participants were younger  and more likely to be female (p = 0.000)  age of 67.8 ±4.9) years, were predominantly retired (78%), and were Caucasian females (100%). Participants had a mean (±SD) BMI of 29.9  and no significant difference in any of the other tests . As only five participants completed both pre- and post-program surveys, changes in physical activity levels were not included in this report.Of the 37 LIFT research participants, 10 completed both pre- and post-program functional fitness assessments. There was a statistically significant increase in the 30 s chair stand test  were compared to the demographics of those who attended LIFT training  and completed the pre-training survey  . ResultsOne of the three administrators completed the surveys; due to the low response rate, administrator survey results are also not included.Results of this work indicated initial struggles in scaling-out a previously tested Extension EBI to Wyoming Extension. All three stages were equally important in capturing the challenges and facilitators of the LIFT EBI scale-out; however, a lack of compliance with data completion , educators face barriers to adopting these programs, including role clarity, traditional delivery models, and organizational culture.Physical activity is not explicitly included as a focus area of Extension health educators' work. Including “physical activity promotion” in Extension health educators' job descriptions and changing position titles to be more inclusive of physical activity could help educators prioritize physical activity programming and de-implement other work duties that are not evidence-based . In addiIn addition to role clarity within the system, Extension professionals need guidance on how to leverage volunteers and community members to engage participants across a disperse region. That is, novel approaches for program delivery may be necessary to increase the penetration of an intervention across a state system, particularly a large rural one. For example, interview respondents mentioned time required for program delivery as a barrier to Strong Women, Strong Bones (and LIFT requires a similar time commitment); adaptations to the delivery model may encourage adoption. They also mentioned a lack of funds to cover fuel to reach geographically disparate communities. A train-the-trainer delivery model may help Extension health educators to adopt the program and then turn it over to community partners, as has been done by Washburn and colleagues for their version of LIFT . To addrFinally, the culture of the organization and the state can also affect physical activity program adoption. Wyoming is a politically conservative state with an individualistic and independent culture. This culture can impede health promotion efforts, as observed in the tobacco control efforts of the 2000s: “We are independent, we're rugged, we'll smoke if we want to, and do not want any government folks trying to tell us how to live healthier and live longer” . This beAlthough perceptions of the LIFT training were positive, participating in training did not change attendees' predictors of implementation, stage of change, or positive intent to deliver both the LIFT program and physical activity programming in general. This may be because those who attended already planned on delivering LIFT  both before and after training). As these are the top predictors of program implementation, it was expected that program implementation rates following training would be high. However, only 2 months had passed since the second LIFT training, so it is possible that those who attended training will deliver LIFT once they have determined a location, community partners, etc., as intent to deliver LIFT was high post-training. The optimal length of time post-training to assess implementation status is unknown; other studies of Extension-delivery physical activity programs have assessed delivery status after 1 year  or annually for 5 years . OverallMore work is needed to understand the adoption-implementation gap among community partners that occurred following the LIFT training. Training attendees had positive perceptions of the training and intentions to deliver LIFT post-training. Without responses to the follow-up survey, it is difficult to understand perceptions of the program or implementation barriers that occurred. Future research on system-level changes to promote physical activity programming through Extension, training on evidence-based programming for Extension health educators, physical activity program delivery methods that decrease educator time commitment, and barriers to community partner program implementation are needed to address low adoption and implementation rates.Extension is an open-access entity that values pragmatic outcomes; stringent collection of empirical data is more novel to this system and its personnel. Therefore, the magnitude of effect and fit of LIFT within the system is yet to be determined. Furthermore, it was difficult to determine if the program was implemented with fidelity; changes are needed to encourage or incentivize collecting these data from delivery personnel , 42. WhiAlthough effectiveness, adoption, and implementation data were challenging to collect, there were positive outcomes in terms of reach. First, LIFT demonstrated a strong reach among eligible older adults, particularly in small communities  from one community participated). In terms of representativeness, LIFT participants were more likely to be female; however, this is also not surprising as the Wyoming Extension system had previously delivered Strong Women, Strong Bones . The proThe most prominent limitation of this work is the small sample size for empirical data. However, in pragmatic settings large sample sizes are not always available; the purpose of this work was to report process and outcome data in order to aid in replicability and understanding the process of scaling an intervention that was adapted specifically for Extension. Due to low survey response rates, perceptions of LIFT that influence adoption and system level maintenance were not captured. These data may have offered insight into reasons for LIFT not being adopted and implemented  and predicted institutionalization of the program. Future work should consider other methods of collecting data to determine perceptions of LIFT and potentially adapt the program to improve adoption and implementation rates. The sample sizes for each portion of this study were small as they were limited by the organization structure of UWE . However, these educators have a large reach, as they cover the entire state and are tasked with providing community-based education to all Wyomingites.Finally, strategies to partner with delivery personnel could have been better used to enhance buy-in and ensure a good fit between LIFT and UWE as a delivery system . The resThis is the first study to follow the scaling-out of an Extension intervention for Extension professionals by Extension professionals that did not include place-based adaptations. For example, Sequin et al. used Extension as a dissemination model for the Strong Women, Strong Bones intervention but were not Extension professionals themselves . In anotApplying a planning and evaluation framework  has the potential to improve transparency and translation of best practices into community settings. However, many settings do not have the resources to capture these iterative, pragmatic data. The results of this study suggest that system-level changes are needed to increase physical activity program adoption rates among Extension health educators, reduce system-level barriers , and leverage partnerships to ensure programs can reach those most in need of intervention. Collecting ongoing effectiveness data will be a challenge, and pragmatic ways to indicate a public health impact need to be developed . These improvements in community-based and community-driven data collection may improve reach, effectiveness, adoption, implementation, and maintenance of interventions within Extension.The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.LB and SH conceived of the study, and participated in its design and coordination. LB led the manuscript preparation. NR contributed to data collection and analysis. TS contributed to data analysis. All authors read, contributed to, and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.