Common test generators fall into two categories. Generating test inputs at the unit level is fast, but can lead to false alarms when a function is called with inputs that would not occur in a system context. If a generated input at the system level causes a failure, this is a true alarm, as the input could also have come from the user or a third party; but system testing is much slower. In this paper, we introduce the concept of a test generation bridge, which joins the accuracy of system testing with the speed of unit testing. A Test Generation Bridge allows to combine an arbitrary system test generator with an arbitrary unit test generator. It does so by carving parameterized unit tests from system (test) executions. These unit tests run in a context recorded from the system test, but individual parameters are left free for the unit test generator to systematically explore. This allows symbolic test generators such as KLEE to operate on individual functions in the recorded system context. If the test generator detects a failure, we lift the failure-inducing parameter back to the system input; if the failure can be reproduced at the system level, it is reported as a true alarm. Our BASILISK prototype can extract and test units out of complex systems such as a Web Python SQLite C stack; in its evaluation, it achieves a higher coverage than a state-of-the-art system test generator.
To get deeper than scanning and parsing routines, though, one requires syntactically correct inputs. To this end, one can use formal specifications of the input language to generate inputs---for instance, leveraging as producers @cite_23 . The test generator @cite_22 uses its grammar for existing inputs as well and can thus combine existing with newly generated input fragments.
@cite_22 @cite_23
2250220874 2794070657 2620641089 2155033057
Natural Language Generation (NLG) is the task of automatically producing natural language text to describe information present in non-linguistic data. It involves three main subtasks: (i) selecting the relevant portion of input data; (ii) determining the words that will be used to verbalise the selected data; and (iii) mapping these words into natural language text. The latter task is known as Surface Realisation (SR). In my thesis, I study the SR task in the context of input data coming from Knowledge Bases (KB). I present two novel approaches to surface realisation from knowledge bases: a supervised approach and a weakly supervised approach. In the first, supervised, approach, I present a corpus-based method for inducing a Feature Based Lexicalized Tree Adjoining Grammar (FB-LTAG) from a parallel corpus of text and data. The resulting grammar includes a unification based semantics and can be used by an existing surface realiser to generate sentences from test data. I show that the induced grammar is compact and generalises well over the test data yielding results that are close to those produced by a handcrafted symbolic approach and which outperform an alternative statistical approach. In the weakly supervised approach, I explore a method for surface realisation from KB data which uses a supplied lexicon but does not require a parallel corpus. Instead, I build a corpus from heterogeneous sources of domain-related text and use it to identify possible lexicalisations of KB symbols (classes and relations) and their verbalisation patterns (frames). Based on the observations made, I build different probabilistic models which are used for selection of appropriate frames and syntax semantics linking while verbalising KB inputs. I evaluate the output sentences and analyse the issues relevant to learning from non-parallel corpora. In both these approaches, I use the data derived from an existing biomedical ontology as a reference input. The proposed methods are generic and can be easily adapted for input from other ontologies for which a parallel non-parallel corpora exists. Computer-generated text plays a pivotal role in various applications, but the quality of the generated text is much lower than that of human-generated text. The use of artificially generated “machine text” can thus negatively affect such practical applications as website generation and text corpora collection. A method for distinguishing computer- and human-generated text is thus needed. Previous methods extract fluency features from a limited internal corpus and use them to identify the generated text. We have extended this approach to also estimate fluency using an enormous external corpus. We have also developed a method for extracting and distinguishing the noises characteristically created by a person or a machine. For example, people frequently use spoken noise words (2morrow, wanna, etc.) and misspelled ones (comin, hapy, etc.) while machines frequently generate incorrect expressions (such as untranslated phrases). A method combining these fluency and noise features was evaluated using 1000 original English messages and 1000 artificial English ones translated from Spanish. The results show that this combined method had the highest accuracy (80.35 ) and the lowest equal error rate (19.44 ) compared with one of state-of-the-art methods, which uses syntactic parser. Moreover, experiments using texts in other languages produced similar results, demonstrated that our proposed method works consistently across various languages. Knowledge about how a program processes its inputs can help to understand the structure of the input as well as the structure of the program. In a JSON value like [1, true, "Alice"], for instance the integer value 1, the boolean value true and the string value "Alice" would be handled by different functions or stored in different variables. Our AUTOGRAM tool uses dynamic tainting to trace the data flow of each input character for a set of sample inputs and identifies syntactical entities by grouping input fragments that are handled by the same functions. The resulting context-free grammar reflects the structure of valid inputs and can be used for reverse engineering of formats and can serve as direct input for test generators. A video demonstrating AUTOGRAM is available at https: youtu.be Iqym60iWBBk. The recovery of structure from flat sequences of input data is a problem that almost all programs need to solve. Computer Science has developed a wide array of declarative languages for describing the structure of languages, usually based on the context-free grammar formalism, and there exist parser generators that produce efficient parsers for these descriptions. However, when faced with a problem involving parsing, most programmers opt for ad-hoc hand-coded solutions, or use parser combinator libraries to construct parsing functions. This paper develops a hybrid approach, treating grammars as collections of active right-hand sides, indexed by a set of non-terminals. Active right-hand sides are built using the standard monadic parser combinators and allow the consumed input to affect the language being parsed, thus allowing for the precise description of the realistic languages that arise in programming. We carefully investigate the semantics of grammars with active right-hand sides, not just from the point of view of language acceptance but also in terms of the generation of parse results. Ambiguous grammars may generate exponentially, or even infinitely, many parse results and these must be efficiently represented using Shared Packed Parse Forests (SPPFs). A particular feature of our approach is the use of Reynolds-style parametricity to ensure that the language that grammars describe cannot be affected by the representation of parse results.