upload lng

lng/L2SCA/LICENSE.txt

@@ -0,0 +1,339 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+                            NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.

lng/L2SCA/Makefile

@@ -0,0 +1,12 @@
+# This is a rudimentary Makefile for rebuilding the tregex distribution.
+# We actually use ant (q.v.) or a Java IDE.
+
+JAVAC = javac
+JAVAFLAGS = -O -d classes -encoding utf-8
+
+tregex:
+	mkdir -p classes
+	$(JAVAC) -classpath CLASSPATH:lib/AppleJavaExtensions.jar $(JAVAFLAGS) src/edu/stanford/nlp/*/*.java src/edu/stanford/nlp/*/*/*.java src/edu/stanford/nlp/*/*/*/*.java
+	cd classes ; jar -cfm ../stanford-tregex-`date +%Y-%m-%d`.jar ../src/edu/stanford/nlp/trees/tregex/gui/tregex-manifest.txt edu ; cd ..
+	cp stanford-tregex-`date +%Y-%m-%d`.jar stanford-tregex.jar
+

lng/L2SCA/README-L2SCA.txt

@@ -0,0 +1,94 @@
+This code is the L2 Syntactic Complexity Analyzer described in:
+
+Lu, Xiaofei (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4):474-496.
+
+Version 3.3.3, released June 30, 2016
+
+Copyright (C) 2016 Xiaofei Lu
+
+This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
+
+This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+To download the latest version of this software, follow the appropriate link at
+	http://www.personal.psu.edu/xxl13/download.html
+
+==============================================================================
+ABOUT
+
+L2 Syntactic Complexity Analyzer is designed to automate syntactic complexity analysis of written English language samples produced by advanced learners of English using fourteen different measures proposed in the second language development literature. The analyzer takes a written English language sample in plain text format as input and generates 14 indices of syntactic complexity of the sample. 
+
+The analyzer is implemented in python and runs on UNIX-like (LINUX, MAC OS, or UNIX) systems with Java 1.5 and python 2.5 or higher installed. The analyzer takes as input a plain text file, counts the frequency of the following 9 structures in the text: words (W), sentences (S), verb phrases (VP), clauses (C), T-units (T), dependent clauses (DC), complex T-units (CT), coordinate phrases (CP), and complex nominals (CN), and computes the following 14 syntactic complexity indices of the text: mean length of sentence (MLS), mean length of T-unit (MLT), mean length of clause (MLC), clauses per sentence (C/S), verb phrases per T-unit (VP/T),, clauses per T-unit (C/T), dependent clauses per clause (DC/C), dependent clauses per T-unit (DC/T), T-units per sentence (T/S), complex T-unit ratio (CT/T), coordinate phrases per T-unit (CP/T), coordinate phrases per clause (CP/C), complex nominals per T-unit (CN/T), and complex nominals per clause (CP/C).
+
+The analyzer calls the Stanford praser (Klein & Manning, 2003) to parse the input file and Tregex (Levy and Andrew, 2006) to query the parse trees. Both the Stanford parser and Tregex are bundled in this download and installation along with the appropriate licenses. 
+
+CONTENTS
+
+[1] Running the single file analyzer
+[2] Input format
+[3] Output format
+[4] Running the multiple file analyzer
+[5] A list of the files included in this package
+
+==============================================================================
+[1] Running the single file analyzer
+
+To run the single file analyzer, type the following at the command line:
+
+python analyzeText.py <input_file> <output_file>
+
+Note that the python script should be called from within this directory. To make sure everything runs correctly, run the following and compare your output with the sample1_output file in the samples/ subdirectory. 
+
+python analyzeText.py samples/sample1.txt samples/sample1_testing
+==============================================================================
+[2] Input format
+
+The input file should be a clean English text in plain text format (with a .txt suffix in the filename). Sample files can be found in the "samples" sub-directory.
+
+==============================================================================
+[3] Output format
+
+A name of the output file must be provided, but you can name it anything you like. 
+
+The first line in the output file is a comma-delimited list of 24 fields, including Filename, abbreviations of the 9 structures mentioned above, and abbreviations of the 14 syntactic complexity indices mentioned above. 
+
+The second line (and subsequent lines if analyzing multiple files in a directory) is a comma-delimited list of 24 values, including the name of the input file, the frequency counts of the 9 structures, and the values of the 14 syntactic complexity indices. 
+
+This format may be hard to read but allows for easy import to Excel or SPSS. 
+
+==============================================================================
+[4] Running the multiple file analyzer
+
+To run the multiple file analyzer, type the following at the command line:
+
+python analyzeFolder.py <path_to_input_file_folder> <output_file>
+
+path_to_input_file_folder is the path to the folder or directory that contains the text files you want to analyze (e.g., /home/inputFiles/). The path should end with a slash, as in the example. Only files that end with the .txt suffix will be analyzed. 
+
+Note that the python script should be called from within this directory. To make sure everything runs correctly, run the following and compare your output with the samples_output file in the samples/ subdirectory. 
+
+python analyzeFolder.py samples/ samples/samples_testing
+==============================================================================
+[5] A list of the files included in this package
+
+README-L2SCA.txt - this file
+
+analyzeText.py - the single file analyzer
+
+analyzeFolder.py - the multiple file analyzer
+
+samples/ - this directory includes the following sample files:
+
+sample1.txt: an English text in plain text format
+
+sample2.txt: another English text in plain text format
+
+sample1_output: sample output file generated by the single file analyzer
+
+samples_output: sample output file generated by the multiple file analyzer
+
+All files for Tregex 3.3.1
+
+Stanford parser 3.3.1

lng/L2SCA/README-gui.txt

@@ -0,0 +1,206 @@
+Tregex GUI v3.3.1 - 2014-01-04
+----------------------------------------------
+
+Copyright (c) 2003-2012 The Board of Trustees of 
+The Leland Stanford Junior University. All Rights Reserved.
+
+Original core Tregex code by Roger Levy and Galen Andrew.
+Original core Tsurgeon code by Roger Levy.
+TregexGUI by Anna Rafferty
+Support code, additional features, etc. by Chris Manning
+This release prepared by John Bauer.
+
+----------------------------
+TREGEX GRAPHICAL USER INTERFACE (GUI) README
+----------------------------
+
+The Tregex GUI is a graphical user interface for Tregex and Tsurgeon.
+You can access it by double- clicking on the jar file tregex.jar. For
+searching large treebanks, you may need to use more memory; the script
+run-tregex-gui.command includes this allocation of memory and can be run
+from the command line or double-click to run on a Mac. If you still have
+memory problems, you can allot more memory by opening the script in a
+text editor and changing "-mx300m" to include a bigger number (e.g.,
+"-mx512m").  Tregex requires Java 1.5+.  Further documentation for
+Tregex and Tsurgeon can be found in README-tregex.txt and
+README-tsurgeon.txt, respectively.
+
+----------------------------
+LOADING TREEBANKS/TREE FILES
+----------------------------
+
+To load a file containing Penn Treebank formatted trees, choose "Load trees..." from the file menu.  
+Multiple tree files and/or directories may be selected.  After selecting the tree files you wish to 
+load, press "Load with file filters" to choose what filters you would like to apply to the files.  
+All filters are run based on the name of the file.  Possible filtering options are:
+
+- Prefix: Load only files that start with the given character sequence
+
+- Extension: Load only files that end with the given character sequence
+
+- Has number in range: Loads only numbered files such that the number falls in the given range, inclusive.
+Ranges can be disjoint as long as multiple ranges are comma-separated (e.g., "100-500,550-700")
+
+File filters are combined such that all loaded files must obey all of
+the filters; only one filter of any given type should be specified.  
+
+Once the tree files are loaded, their names appear in the upper left hand panel "Tree files:".  
+Unchecking the check boxes next to the files causes the unchecked files not to be included in 
+searches/tsurgeon operations.  To remove all files from the tree panel, choose "Clear all files"
+from the Edit menu.
+
+----------------------------
+PERFORMING TREGEX SEARCHES
+----------------------------
+
+To perform a Tregex search, load the files you would like to search and type a Tregex pattern 
+in the "Pattern:" box in the top middle of the window.  Press "Help" beneath the Pattern box 
+for information about Tregex syntax.  After you have typed the pattern, press "Search" to 
+find all matches to the given pattern.
+
+By default, trees that contain at least one match are displayed in the "Matches:" panel in the 
+top right of the window, and the first matching tree is graphically displayed in the bottom 
+portion of the window.  Click on a match in the Match panel to display it graphically.  In the 
+graphical display, matched nodes in the tree are displayed in a different color than other nodes.  
+To display only the matched subtrees, choose "Preferences..." (Mac, from the Application menu) or 
+"Options..." (other OS, under Tools), and check "Show only matched portions of the tree".  You must 
+rerun the search to switch between showing only matched portions and showing full trees.
+
+In preferences, other display options can also be set, such as the colors, size, and font used by 
+the graphical display.
+
+----------------------------
+USING TSURGEON
+----------------------------
+
+Tsurgeon modifications can also be performed using Interactive Tregex.  To enable Tsurgeon, choose 
+"Preferences..." from the File menu and check "Enable Tsurgeon".  You can now run Tsurgeon scripts. 
+Tsurgeon commands must be paired with a Tregex pattern that names the nodes on which modifications 
+will be performed.  Type the Tregex pattern in the Pattern box, and type the modifications you would 
+like to make in the "Tsurgeon script:" box.  Then click "Run script" to perform the modifications.  
+Each Tsurgeon operation must appear on a separate line in the Tsurgeon script box.  Press "Help" for 
+some information about Tsurgeon operation syntax.
+
+
+----------------------------
+SAVING RESULTS
+----------------------------
+
+You can save the results of a Tregex search or Tsurgeon operation by choosing "Save matches..." from the 
+File menu.  This saves all trees in the Matches panel in Penn Treebank form. "Save matched sentences..." saves
+the matches in sentence String form, just as they show up in the matches panel.
+
+You can also save a log of the number of matches found for each pattern you have searched.  By clicking the
+"Statistics" button in the middle of the screen, below the Tsurgeon buttons, you can see a table of the patterns
+for which you have searched, the number of trees that each matched, and the number of overall matches that were
+found.  To save this information in a tab delimited text file, choose "Save statistics..." from the File menu. 
+
+All three save options save files in the encoding specified in the Preferences panel for loading tree files.
+
+----------------------------
+MULTILANGUAGE SUPPORT
+----------------------------
+
+Some multilanguage support is built into Tregex GUI, and most languages can be read by the GUI.  To enable
+this support, choose go to Preferences (Mac, under the application menu) or Options (other OS, under the Tools menu).
+Several options may need to be changed: tree reader factory, head finder, font, and encoding.  Several possible
+tree reader factories and head finders are provided; you may also specify your own. Two common languages you may be
+trying to use are Chinese or Arabic; any head finder or tree reader factory beginning with "Chinese" or "Arabic" will
+work for these languages, and additionally, CTBTreeReaderFactory is compatible with many Chinese treebanks.  Based on
+your choice of head finder and tree reader factory, the Tregex GUI will guess if you may need a different font and/or 
+text encoding. If a different text encoding is usually used for your selections, you will be prompted as to what text 
+encoding you would like to use.  This may also be specified directly in the Preferences panel.  
+
+
+
+----------------------------
+QUESTIONS
+----------------------------
+
+For more information on Tregex or Tsurgeon, read README-tregex.txt and README-tsurgeon.txt, and also look at the javadocs 
+suggested in those files.  For questions about this distribution, please contact Stanford's JavaNLP group at
+[email protected].  We provide assistance on a best-effort basis.
+
+----------------------------
+LICENSE
+----------------------------
+
+ Tregex GUI
+ Copyright (c) 2007-2011 The Board of Trustees of 
+ The Leland Stanford Junior University. All Rights Reserved.
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License
+ as published by the Free Software Foundation; either version 2
+ of the License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+
+ For more information, bug reports, fixes, contact:
+    Christopher Manning
+    Dept of Computer Science, Gates 1A
+    Stanford CA 94305-9010
+    USA
+    [email protected]
+    http://www-nlp.stanford.edu/software/tregex.shtml
+  
+
+----------------------------
+CHANGES
+----------------------------
+
+2014-01-04    3.3.1     Bugfix release, new createSubtree tsurgeon 
+                        operation 
+
+2013-11-12    3.3.0     Allow a TregexMatcher to have its own 
+                        HeadFinder, useful for the dependencies 
+
+2013-06-19    3.2.0     Fix for tsurgeon number reading bug 
+
+2013-04-04    2.0.6     Update to maintain compatibility 
+
+2012-11-11    2.0.5     Efficiency improvements 
+
+2012-07-09    2.0.4     Minor bug fixes 
+
+2012-05-22    2.0.3     Rebuilt to be compatible with everything.
+
+2012-03-09    2.0.2     Efficiency improvements
+
+2011-12-16    2.0.1     Fix bug in matchesAt, fix bug in category 
+                        function, add macros
+
+2011-09-14    2.0.0     Efficiency improvements, include semgrex.
+
+2011-05-15    1.4.4     Rebuilt to be compatible with everything.
+
+2011-05-15    1.4.3     Rebuilt to be compatible with everything.
+
+2011-04-17    1.4.2     Rebuilt to be compatible with tagger, parser, 
+                        and corenlp.
+
+2010-11-18    1.4.1     Small fixes and improvements (improved help 
+                        screens, multipattern Tsurgeon scripts with
+                        comments introduced by % supported, unclosed
+                        regex no longer crashes GUI, support character
+                        encodings in script files, fix bug in tregex
+                        matching immediate domination path, TregexGUI
+                        now shows filename and line number of each
+                        match in matches panel)
+
+2009-09-30      1.4     GUI slider for tree size, generalized relabel
+                        command (incompatibly), __ and @ now supported
+                        in path constraints; bugfixes.
+
+2008-05-06      1.1     Several bug fixes; addition of browse trees
+                        function, improved copy/paste and drag and
+                        drop support; misc. feature additions
+
+2007-09-20      1.0     Initial release

lng/L2SCA/README-tregex.txt

@@ -0,0 +1,429 @@
+Tregex v3.3.1 - 2014-01-04
+----------------------------------------------
+
+Copyright (c) 2003-2012 The Board of Trustees of 
+The Leland Stanford Junior University. All Rights Reserved.
+
+Original core Tregex code by Roger Levy and Galen Andrew.
+Original core Tsurgeon code by Roger Levy.
+GUI by Anna Rafferty
+Support code, additional features, etc. by Chris Manning
+This release prepared by John Bauer.
+
+This package contains Tregex and Tsurgeon.
+
+Tregex is a Tgrep2-style utility for matching patterns in trees.  It can
+be run in a graphical user interface, from the command line using the 
+TregexPattern main method, or used programmatically in java code via the 
+TregexPattern, TregexMatcher and TregexPatternCompiler classes.
+
+As of version 1.2, the Tsurgeon tree-transformation utility is bundled
+together with Tregex.  See the file README.tsurgeon for details.
+
+Java version 1.6 is required to use Tregex.  If you really want to use
+Tregex under an earlier version of Java, look into RetroWeaver:
+
+  http://retroweaver.sourceforge.net/
+  
+
+QUICKSTART
+-----------------------------------------------
+
+Programmatic use, command-line use, and GUI-use are supported.  To access the 
+graphical interface for Tsurgeon and Tregex, double-click the tregex.jar file. 
+Some help (particularly with syntax) is provided within the program; for further
+assistance, see README-gui.txt and the documentation mentioned below.
+
+A full explanation of pattern syntax and usage is given in the javadocs
+(particularly TregexPattern), and some of this information is also presented in
+the TREGEX SYNTAX section below.  As a quick example of usage,
+the following line will scan an English PennTreebank annotated corpus
+and print all nodes representing a verb phrase dominating a past-tense
+verb and a noun phrase.
+
+./tregex.sh 'VP < VBD < NP' corpus_dir
+
+
+CONTENTS
+-----------------------------------------------
+
+README-tregex.txt
+
+  This file.
+  
+  
+README-tsurgeon.txt
+
+  Documentation for Tsurgeon, a tool for modifying trees.
+  
+README-gui.txt
+
+  Documentation for the graphical interface for Tregex and Tsurgeon tools.
+  
+LICENSE.txt
+
+  Tregex is licensed under the GNU General Public License.
+
+stanford-tregex.jar
+
+  This is a JAR file containing all the Stanford classes necessary to
+  run tregex.
+
+src
+
+  A directory containing the Java 1.6 source code for the Tregex
+  distribution.
+
+javadoc
+
+  Javadocs for the distribution.  In particular, look at the javadocs
+  for the class edu.stanford.nlp.trees.tregex.TregexPattern.  The
+  first part of that class's javadoc describes syntax and semantics
+  for relations, node labels, node names, and variable groups.  The
+  docs for the main method describe command-line options.
+
+tregex.sh
+
+  a shell script for invoking the Tregex tree search tool.
+
+tsurgeon.sh
+
+  a shell script for invoking the Tsurgeon tree transformation tool.
+  
+run-tregex-gui.command
+  
+  A command file that can be double-clicked on a Mac to start the gui.
+  
+run-tregex-gui.bat
+ 
+  A bat file that can be double-clicked on a PC to start the gui.
+
+examples
+
+  a directory containing several sample files to show Tsurgeon operation:
+- atree
+  a sample natural-language tree in Penn Treebank annotation style.
+- exciseNP
+- renameVerb
+- relabelWithGroupName
+  Sample tree-transformation operation files for Tsurgeon.  See
+  README-tsurgeon.txt for more information about the contents of these
+  files.
+  
+  
+TREGEX 
+-----------------------------------------------
+Tregex Pattern Syntax and Uses
+
+Using a Tregex pattern, you can find only those trees that match the pattern you're 
+looking for.  The following table shows the symbols that are allowed in the pattern, 
+and below there is more information about using these patterns. 
+ 
+Table of Symbols and Meanings:
+A << B	
+   A dominates B  
+A >> B 
+   A is dominated by B  
+A < B 
+   A immediately dominates B  
+A > B 
+   A is immediately dominated by B  
+A $ B 
+   A is a sister of B (and not equal to B)   
+A .. B 
+   A precedes B 
+A . B 
+   A immediately precedes B 
+A ,, B 
+   A follows B 
+A , B 
+   A immediately follows B 
+A <<, B 
+   B is a leftmost descendent of A 
+A <<- B 
+   B is a rightmost descendent of A 
+A >>, B 
+   A is a leftmost descendent of B 
+A >>- B 
+   A is a rightmost descendent of B 
+A <, B 
+   B is the first child of A 
+A >, B 
+   A is the first child of B 
+A <- B 
+   B is the last child of A 
+A >- B 
+   A is the last child of B 
+A <` B 
+   B is the last child of A 
+A >` B 
+   A is the last child of B 
+A <i B 
+   B is the ith child of A (i > 0) 
+A >i B 
+   A is the ith child of B (i > 0) 
+A <-i B 
+   B is the ith-to-last child of A (i > 0) 
+A >-i B 
+   A is the ith-to-last child of B (i > 0) 
+A <: B 
+   B is the only child of A 
+A >: B 
+   A is the only child of B 
+A <<: B 
+   A dominates B via an unbroken chain (length > 0) of unary local trees. 
+A >>: B 
+   A is dominated by B via an unbroken chain (length > 0) of unary local trees. 
+A $++ B 
+   A is a left sister of B (same as $.. for context-free trees) 
+A $-- B 
+   A is a right sister of B (same as $,, for context-free trees) 
+A $+ B 
+   A is the immediate left sister of B (same as $. for context-free trees) 
+A $- B 
+   A is the immediate right sister of B (same as $, for context-free trees) 
+A $.. B 
+   A is a sister of B and precedes B 
+A $,, B 
+   A is a sister of B and follows B 
+A $. B 
+   A is a sister of B and immediately precedes B 
+A $, B 
+   A is a sister of B and immediately follows B 
+A <+(C) B 
+   A dominates B via an unbroken chain of (zero or more) nodes matching description C 
+A >+(C) B 
+   A is dominated by B via an unbroken chain of (zero or more) nodes matching description C 
+A .+(C) B 
+   A precedes B via an unbroken chain of (zero or more) nodes matching description C 
+A ,+(C) B 
+   A follows B via an unbroken chain of (zero or more) nodes matching description C 
+A <<# B 
+   B is a head of phrase A 
+A >># B 
+   A is a head of phrase B 
+A <# B 
+   B is the immediate head of phrase A 
+A ># B 
+   A is the immediate head of phrase B 
+A == B 
+   A and B are the same node 
+A : B
+   [this is a pattern-segmenting operator that places no constraints on the relationship between A and B] 
+
+ Label descriptions can be literal strings, which much match labels  exactly, or regular 
+ expressions in regular expression bars: /regex/.  Literal string matching proceeds as 
+ String equality. In order to prevent ambiguity with other Tregex symbols, only standard  
+ "identifiers" are allowed as literals, i.e., strings matching [a-zA-Z]([a-zA-Z0-9_])* . 
+ If you want to use other symbols, you can do so by using a regular expression instead of 
+ a literal string.  A disjunctive list of literal strings can be given separated by '|'. 
+ The special string '__' (two underscores) can be used to match any  node.  (WARNING!!  
+ Use of the '__' node description may seriously  slow down search.)  If a label description 
+ is preceeded by '@', the label will match any node whose basicCategory matches the description.  
+NB: A single '@' thus scopes over a disjunction  specified by '|': @NP|VP means things with basic category NP or VP.  
+
+Label description regular expressions are matched as find(),  as in Perl/tgrep;  
+you need to specify ^ or $ to constrain matches.    
+
+In a chain of relations, all relations are relative to the first node in the chain. 
+For example, (S < VP < NP) means an S over a VP and also over an NP.  If instead what 
+you want is an S above a VP above an NP, you should write  S < (VP < NP).  
+
+Nodes can be grouped using parentheses '(' and ')'  as in  S < (NP $++ VP)  to match an S 
+over an NP, where the NP has a VP as a right sister.  
+
+Boolean relational operators
+
+Relations can be combined using the '&' and '|' operators, negated with the '!' operator, 
+and made optional with the '?' operator.  Thus (NP < NN | < NNS)  will match an NP node 
+dominating either  an NN or an NNS.   (NP > S & $++ VP)  matches an NP that  is both under 
+an S and has a VP as a right sister.   
+
+Relations can be grouped using brackets '[' and ']'.  So the  expression 
+
+NP [< NN | < NNS] & > S 
+
+matches an NP that (1) dominates either an NN or an NNS, and (2) is under an S.  Without  
+brackets, & takes precedence over |, and equivalent operators are left-associative.  Also 
+note that & is the default combining operator if the  operator is omitted in a chain of 
+relations, so that the two patterns are equivalent: 
+   (S < VP < NP)
+   (S < VP & < NP)     
+   
+As another example,  (VP < VV | < NP % NP) can be written explicitly as  (VP [< VV | [< NP & % NP] ] ). 
+
+Relations can be negated with the '!' operator, in which case the expression will match 
+only if there is no node satisfying the relation.  For example  (NP !< NNP)  matches only 
+NPs not dominating  an NNP.  Label descriptions can also be negated with '!': (NP < !NNP|NNS) 
+matches  NPs dominating some node that is not an NNP or an NNS.  
+
+Relations can be made optional with the '?' operator.  This way the expression will match even 
+if the optional relation is not satisfied.  This is useful when used together with node naming 
+(see below).  
+
+
+Basic Categories
+
+In order to consider only the "basic category" of a tree label,  i.e. to ignore functional tags 
+or other annotations on the label,  prefix that node's description with the @ symbol.  For example   
+(@NP < @/NN.?/).   This can only be used for individual nodes;  if you want all nodes to use the 
+basic category, it would be more efficient  to use a TreeNormalizer to remove functional tags 
+before passing the tree to the TregexPattern.   
+
+
+Segmenting patterns
+
+The ":" operator allows you to segment a pattern into two pieces.  This can simplify your pattern 
+writing.  For example,  the pattern S : NP matches only those S nodes in trees that also have an NP node.   
+
+
+Naming nodes
+
+Nodes can be given names (a.k.a. handles) using '='.  A named node will be stored in a  map that 
+maps names to nodes so that if a match is found, the node  corresponding to the named node can 
+be extracted from the map.  For  example  (NP < NNP=name)  will match an NP dominating an NNP  
+and after a match is found, the map can be queried with the  name to retreived the matched node 
+using {@link TregexMatcher#getNode(Object o)}  with (String) argument "name" (not "=name").  Note 
+that you are not allowed to name a node that is under the scope of a negation operator (the 
+semantics would  be unclear, since you can't store a node that never gets matched to).  Trying to 
+do so will cause a ParseException to be thrown. Named nodes can be put within the scope of an 
+optional operator.   
+
+Named nodes that refer back to previous named nodes need not have a node  description -- this is 
+known as "backreferencing".  In this case, the expression  will match only when all instances of 
+the same name get matched to the same tree node.  For example, the pattern:
+
+(@NP <, (@NP $+ (/,/ $+ (@NP $+ /,/=comma))) <- =comma)  
+
+matches only an NP dominating exactly the sequence NP, NP; the mother NP cannot have any other 
+daughters. Multiple  backreferences are allowed.  If the node with no node description does not 
+refer  to a previously named node, there will be no error, the expression simply will  not match 
+anything.   
+
+Another way to refer to previously named nodes is with the "link" symbol: '~'.  A link is like a 
+backreference, except that instead of having to be <i>equal to</i> the  referred node, the 
+current node only has to match the label of the referred to node.  A link cannot have a node 
+description, i.e. the '~' symbol must immediately follow a  relation symbol.   
+
+
+Variable Groups
+
+If you write a node description using a regular expression, you can assign its matching groups to 
+variable names.  If more than one node has a group assigned to the same variable name, then matching 
+will only occur when all such groups  capture the same string.  This is useful for enforcing 
+coindexation constraints.  The syntax is:
+
+/ <regex-stuff> /#<group-number>%<variable-name>
+
+For example, the pattern (designed for Penn Treebank trees):    
+
+@SBAR < /^WH.*-([0-9]+)$/#1%index<<(__=empty < (/^-NONE-/< /^\\*T\\*-([0-9]+)$/#1%index))
+
+will match only such that the WH- node under the SBAR is coindexed with the trace node that gets the name empty.
+
+
+MISCELLANEOUS
+-----------------------------------------------
+
+Head Finders
+
+  To use the headship relations <# ># <<# >># correctly it is
+  important to specify a HeadFinder class appropriate to the trees
+  that you are searching.  For information about how to specify a
+  HeadFinder class at the command line or through the API, please read
+  the javadocs for the class
+  edu.stanford.nlp.trees.tregex.TregexPattern.  The following
+  HeadFinder classes are included with the Tregex distribution:
+
+  Penn Treebank of English (http://www.cis.upenn.edu/~treebank/):
+
+    edu.stanford.nlp.trees.CollinsHeadFinder (default)
+
+  Penn Treebank of Chinese (http://www.cis.upenn.edu/~chinese/):
+
+    edu.stanford.nlp.trees.international.pennchinese.ChineseHeadFinder
+
+  Penn Treebank of Arabic (http://www.ircs.upenn.edu/arabic/):
+
+    edu.stanford.nlp.trees.international.arabic.ArabicHeadFinder
+
+  NEGRA (http://www.coli.uni-saarland.de/projects/sfb378/negra-corpus/) 
+
+  and
+
+  TIGER (http://www.ims.uni-stuttgart.de/projekte/TIGER/TIGERCorpus/)
+
+  treebanks of German (these can use the same headfinder):
+
+    edu.stanford.nlp.trees.international.negra.NegraHeadFinder
+
+  Tuebingen Treebank of Written German (http://www.sfs.uni-tuebingen.de/en_tuebadz.shtml):
+
+    edu.stanford.nlp.trees.international.tuebadz.TueBaDZHeadFinder
+    
+    
+Tdiff
+
+  TregexGUI supports a constituent diff'ing method--similar to the UNIX diff command--for trees. To
+enable Tdiff:
+  1) Clear the tree file list: File -> Clear tree file list 
+  2) Enable Tdiff: Options -> Tdiff
+  3) Load two (2) files using the "File -> Load" dialog.
+  4) Select "Browse" on the main display
+  
+The GUI will display differences between each pair of trees in the two files. As such, the two files must 
+contain the same number of trees. 
+
+The first file in the tree file list is treated as the reference. Trees from the second file 
+will be displayed in the GUI, with bracketing differences highlighted in blue. Below the tree, 
+constituents in the reference tree that do not appear in the tree from the second file are shown
+as lines below each respective span.
+
+Tregex searches are supported and apply to the trees in the second file.
+
+This feature was designed for debugging and analyzing parser output.
+
+THANKS
+-----------------------------------------------
+
+Thanks to the members of the Stanford Natural Language Processing Lab
+for great collaborative work on Java libraries for natural language
+processing.
+
+  http://nlp.stanford.edu/javanlp/
+  
+LICENSE
+-----------------------------------------------
+
+ Tregex, Tsurgeon, and Interactive Tregex
+ Copyright (c) 2003-2012 The Board of Trustees of 
+ The Leland Stanford Junior University. All Rights Reserved.
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License
+ as published by the Free Software Foundation; either version 2
+ of the License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+
+ For more information, bug reports, fixes, contact:
+    Christopher Manning
+    Dept of Computer Science, Gates 1A
+    Stanford CA 94305-9010
+    USA
+    [email protected]
+    http://www-nlp.stanford.edu/software/tregex.shtml
+  
+
+CONTACT
+-----------------------------------------------
+
+For questions about this distribution, please contact Stanford's JavaNLP group at
+[email protected].  We provide assistance on a best-effort basis.

lng/L2SCA/README-tsurgeon.txt

@@ -0,0 +1,529 @@
+Tsurgeon v3.3.1 - 2014-01-04
+----------------------------------------------
+
+Copyright (c) 2003-2012 The Board of Trustees of 
+The Leland Stanford Junior University. All Rights Reserved.
+
+Original core Tregex code by Roger Levy and Galen Andrew.
+Original core Tsurgeon code by Roger Levy.
+GUI by Anna Rafferty
+Support code, additional features, etc. by Chris Manning
+This release prepared by John Bauer.
+
+This package contains Tregex and Tsurgeon.
+
+Tregex is a Tgrep2-style utility for matching patterns in trees.  It can
+be run in a graphical user interface, from the command line using the 
+TregexPattern main method, or used programmatically in java code via the 
+TregexPattern, TregexMatcher and TregexPatternCompiler classes.
+
+As of version 1.2, the Tsurgeon tree-transformation utility is bundled
+together with Tregex.  See the file README.tsurgeon for details.
+
+Java version 1.6 is required to use Tregex.  If you really want to use
+Tregex under an earlier version of Java, look into RetroWeaver:
+
+  http://retroweaver.sourceforge.net/
+  
+TSURGEON
+----------------------------------------------
+Tsurgeon is a tool for modifying trees that match a particular Tregex
+pattern.  Further documentation for Tregex and Tregex GUI can be found in 
+README-tregex.txt and README-gui.txt, respectively.
+
+----------------------------------------------
+
+Brief description:
+
+Takes some trees, tries to match one or more tregex expressions to
+each tree, and for each successful match applies some surgical
+operations to the tree.  Pretty-prints each resulting tree (after all
+successful match/operation sets have applied) to standard output.
+
+
+A simple example:
+
+  ./tsurgeon.csh -treeFile atree exciseNP renameVerb
+  
+-----------------------------------------
+RUNNING TREGEX
+-----------------------------------------
+
+Program Command Line Options:
+
+-treeFile <filename>
+
+  specify the name of the file that has the trees you want to transform.
+
+-po <matchPattern> <operation>
+
+  Apply a single operation to every tree using the specified match
+  pattern and the specified operation.
+
+-s
+
+  Prints the output trees one per line, instead of pretty-printed.
+
+The arguments are then Tsurgeon scripts.
+Each argument should be the name of a transformation file that contains a list of pattern
+and transformation operation list pairs.  That is, it is a sequence of pairs of a
+TregexPattern pattern on one or more lines, then a
+blank line (empty or whitespace), then a list of transformation operations one per line
+(as specified by Tsurgeon syntax below to apply when the pattern is matched,
+and then another blank line (empty or whitespace).
+Note the need for blank lines: The code crashes if they are not present as separators
+(although the blank line at the end of the file can be omitted).
+The script file can include comment lines, either whole comment lines or
+trailing comments introduced by %, which extend to the end of line.  A needed percent
+mark in partterns or operations can be escaped by a preceding backslash.
+
+-----------------------------------------
+TSURGEON SYNTAX
+-----------------------------------------
+
+Legal operation syntax and semantics (see Examples section for further detail):
+
+delete <name_1> <name_2> ... <name_m>
+
+  For each name_i, deletes the node it names and everything below it.
+
+prune <name_1> <name_2> ... <name_m>
+
+  For each name_i, prunes out the node it names.  Pruning differs from
+  deletion in that if pruning a node causes its parent to have no
+  children, then the parent is in turn pruned too.
+
+excise <name1> <name2>
+
+  The name1 node should either dominate or be the same as the name2
+  node.  This excises out everything from name1 to name2.  All the
+  children of name2 go into the parent of name1, where name1 was.
+
+relabel <name> <new-label>
+
+Relabels the node to have the new label. There are three possible forms
+for the new-label:
+  relabel nodeX VP - for changing a node label to an alphanumeric
+  string, relabel nodeX /''/ - for relabeling a node to something that
+  isn't a valid identifier without quoting, and relabel nodeX
+  /^VB(.*)$/verb\/$1/ - for regular expression based relabeling. In the
+  last case, all matches of the regular expression against the node
+  label are replaced with the replacement String. This has the semantics
+  of Java/Perl's replaceAll: you may use capturing groups and put them
+  in replacements with $n. Also, as in the example, you can escape a
+  slash in the middle of the second and third forms with \/ and \\.
+  This last version lets you make a new label that is an arbitrary 
+  String function of the original label and additional characters that
+  you supply.
+
+insert <name> <position>
+insert <tree> <position>
+
+  inserts the named node, or a manually specified tree (see below for
+  syntax), into the position specified.  Right now the only ways to
+  specify position are:
+
+     $+ <name>     the left sister of the named node
+     $- <name>     the right sister of the named node
+     >i <name>     the i_th daughter of the named node.
+     >-i <name>    the i_th daughter, counting from the right, of the named node.
+
+move <name> <position>
+
+  moves the named node into the specified position.  To be precise, it
+  deletes (*NOT* prunes) the node from the tree, and re-inserts it
+  into the specified position. See above for how to specify position
+
+replace <name1> <name2>
+
+  deletes name1 and inserts a copy of name2 in its place.
+
+adjoin <tree> <target-node>
+
+  adjoins the specified auxiliary tree (see below for syntax) into the
+  target node specified.  The daughters of the target node will become
+  the daughters of the foot of the auxiliary tree.
+  
+adjoinH <tree> <target-node>
+
+  similar to adjoin, but preserves the target node and makes it the root
+  of <tree>.  (It is still accessible as <code>name</code>.  The root of
+  the auxiliary tree is ignored.) 
+  
+adjoinF <tree> <target-node>
+
+  similar to adjoin, but preserves the target node and makes it the foot
+  of <tree>.  (It is still accessible as <code>name</code>, and retains
+  its status as parent of its children. The foot of the auxiliary tree
+  is ignored.) 
+  
+coindex <name_1> <name_2> ... <name_m>
+
+  Puts a (Penn Treebank style) coindexation suffix of the form "-N" on
+  each of nodes name_1 through name_m.  The value of N will be
+  automatically generated in reference to the existing coindexations
+  in the tree, so that there is never an accidental clash of
+  indices across things that are not meant to be coindexed.
+
+-----------------------------------------
+
+Syntax for trees to be inserted or adjoined:
+
+
+A tree to be adjoined in can be specified with LISP-like
+parenthetical-bracketing tree syntax such as those used for the Penn
+Treebank.  For example, for the NP "the dog" to be inserted you might
+use the syntax
+
+  (NP (Det the) (N dog))
+
+That's all that there is for a tree to be inserted.  Auxiliary trees
+(a la Tree Adjoining Grammar) must also have exactly one frontier node
+ending in the character "@", which marks it as the "foot" node for
+adjunction.  Final instances of the character "@" in terminal node labels
+will be removed from the actual label of the tree.
+
+For example, if you wanted to adjoin the adverb "breathlessly" into a
+VP, you might specify the following auxiliary tree:
+
+  (VP (Adv breathlessly) VP@ )
+
+All other instances of "@" in terminal nodes must be escaped (i.e.,
+appear as \@); this escaping will be removed by tsurgeon.
+
+In addition, any node of a tree can be named (the same way as in
+tregex), by appending =<name> to the node label.  That name can be
+referred to by subsequent tsurgeon operations triggered by the same
+match.  All other instances of "=" in node labels must be escaped
+(i.e., appear as \=); this escaping will be removed by tsurgeon.  For
+example, if you want to insert an NP trace somewhere and coindex it
+with a node named "antecedent" you might say
+
+  insert (NP (-NONE- *T*=trace)) <node-location>
+  coindex trace antecedent $
+  
+-----------------------------------------
+Examples of Tsurgeon operations:  
+
+Tree (used in all examples):
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+
+Apply delete:
+	VP < PP=prep
+	delete prep
+Result:
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+      (VP (VBN arrested)
+    (. .)))
+The PP node directly dominated by a VP is removed, as is
+everything under it.
+
+Apply prune:
+	S < (NP < NNP=noun)
+	prune noun
+Result:
+(ROOT
+  (S
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+The NNP node is removed, and since this results in the NP above it
+having no terminal children, the NP node is deleted as well.
+Note: This is different from delete in which the NP above the NNP 
+would remain.
+
+Apply excise:
+	VP < PP=prep
+	excise prep prep
+Result:
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+      (VP (VBN arrested)
+          (IN in)
+          (NP (NNP May)))))
+    (. .)))
+The PP node is removed, and all of its children are added in the
+place it was previously located.  Excise removes all the nodes from
+the first named node to the second named node, and the children of 
+the second node are added as children of the parent of the first node.
+Thus, for another example:
+	VP=verb < PP=prep
+	excise verb prep
+Result:
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+		(IN in)
+        (NP (NNP May)))
+    (. .)))
+
+    
+Apply relabel: 
+	VP=v < PP=prep
+	relabel prep verbPrep
+Result:
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (verbPrep (IN in)
+          (NP (NNP May)))))
+    (. .)))
+The label for the node called prep (PP) is changed to verbPrep.
+The other form of relabel uses regular expressions; consider the following
+operation:
+	/^VB.+/=v
+	relabel v /^VB(.*)$/ #1
+Result:
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (D was)
+      (VP (N arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+The Tregex pattern matches all nodes that begin "VB" and have at least one
+more character.  The Tsurgeon operation then matches the node label to the
+regular expression "^VB(.*)$" and selects the text matching the first part
+that is not completely specified in the pattern.  In this case, that is the
+part matching the wildcard (.*), which matches all characters after the VB.
+The node is then relabeled with that part of the text, causing, for example,
+"VBD" to be relabeled "D".  The "#1" specifies that the name of the node
+should be the first group in the regex.
+    
+Apply insert (shown here with inserting a node, but could also be a tree):
+	S < (NP < (NNP=name !$- DET))
+	insert (DET Ms.) $+ name
+Result:
+(ROOT
+  (S
+    (NP (DET Ms.)
+        (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+The pattern matches the NNP node that is directly dominated by an NP
+(which is directly dominated by an S) and is not a direct right sister
+of a DET.  Thus, the (DET Ms.) node is inserted immediately to the left
+of that NNP node, as specified by "$+ name".  "$+" is the location and
+"name" describes what node the location is with respect to.
+Note: Tsurgeon will re-search for matches after each run of the script;
+thus, cycles may occur, causing the program to not terminate.  The key
+is to write patterns that match prior to the changes you would like to
+make but that do not match afterwards.  If the clause "!$- DET" had been
+left out in this example, Tsurgeon would have matched the pattern after
+every insert operation, causing an infinite number of DETs to be added.
+
+Apply move:
+	VP=verb < PP=prep
+	move prep $- verb
+Result:
+(ROOT
+  (S
+    (NP (NNP Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+        (VP (VBN arrested)))
+    (PP (IN in)
+        (NP (NNP May)))
+    (. .)))
+The PP is moved out of the VP that dominates it and added as a direct right
+sister of the VP.  As for insert, "$-" specifies the location for prep while
+"verb" specifies what that location is relative to.
+Note: "move" is a macro operation that deletes the given node and then inserts 
+it. "move" does not use prune, and thus any branches that now lack terminals will
+remain rather than being removed.
+
+Apply replace:
+	S < (NP=name < NNP)
+	replace name (NP (DET A) (NN woman))
+Result:
+(ROOT
+  (S
+    (NP (DET A)
+    	(NN woman))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+"name" is matched to an NP that is dominated by an S and dominates an NNP, and
+a new subtree ("(NP (DET A) (NN woman))") is added in the place where "name" was.
+Note: This operation is vulnerable to falling into an infinite loop. See the note 
+concerning the "insert" operation and how patterns are matched.
+
+Apply adjoin:
+	S < (NP=name < NNP)
+	adjoin (NP (DET A) (NN woman) NP@) name
+Result:
+(ROOT
+  (S
+    (NP (DET A)
+    	(NN woman)
+    	(NP (NNP Maria_Eugenia_Ochoa_Garcia)))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+First, the NP is matched to the NP dominating the NNP tag. Then, the specified
+tree ("(NP (DET A) (NN woman) NP@)") is placed in that location.  The "@" symbol
+specifies that the children of the original NP node ("name") are to be placed
+as children of a new NP node that is directly to the right of (NN woman).  If
+the specified tree were "(NP (DET A) (NN woman) VP@)" then the child
+(NNP Maria_Eugenia_Ochoa_Garcia) would appear under a VP.  Exactly one "@" node 
+must appear in the specified tree in order to indicate where to place the node
+from the original tree.
+
+Apply adjoinH:
+	S < (NP=name < NNP)
+	adjoinH ((NP (DET A) (NN woman) NP@)) name
+Result:
+(ROOT
+  (S
+    (NP (NP (DET A)
+    		(NN woman)
+    		(NP (NNP Maria_Eugenia_Ochoa_Garcia))))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+This operation differs from adjoin in that it retains the named node (in this
+case, "name").  The named node is made the root of the specified tree, resulting
+in two NP nodes dominating the DET in this example whereas only one was present
+in the previous example.  Note that the specified tree is wrapped in an extra
+pair of parentheses in order to show the syntax for retaining the named node.
+If the extra parentheses were not there and the specified tree was, for example, 
+(VP (DET A) (NN woman) NP@), the VP would be ignored in order to retain an NP as
+the root.  Thus, in this case, "adjoinH (VP (DET A) (NN woman) NP@) name" and
+"adjoinH ((DET A) (NN woman) NP@) name" both produce the same tree:
+(ROOT
+  (S
+    (NP (DET A)
+    	(NN woman)
+    	(NP (NNP Maria_Eugenia_Ochoa_Garcia)))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+
+
+Apply adjoinF:
+	S < (NP=name < NNP)
+	adjoinF (NP(DET A) (NN woman) @) name
+Result: 
+(ROOT
+  (S
+    (NP (DET A)
+    	(NN woman)
+    	(NP (NNP Maria_Eugenia_Ochoa_Garcia)))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP (NNP May)))))
+    (. .)))
+This operation is very similar to adjoin and adjoinH, but this time the original
+named node ("name" in this case) is maintained as the root of the subtree that
+is adjoined.  Thus, no node label needs to be given in front of the "@" and if
+one is given, it will be ignored.  For instance, "adjoinF (NP(DET A) (NN woman) VP@) name"
+would still produce the same tree as above, despite the VP preceding the @.
+
+Apply coindex:
+	NP=node < NNP=name
+	coindex node name
+Result:
+(ROOT
+  (S
+    (NP-1 (NNP-1 Maria_Eugenia_Ochoa_Garcia))
+    (VP (VBD was)
+      (VP (VBN arrested)
+        (PP (IN in)
+          (NP-2 (NNP-2 May)))))
+    (. .)))
+This causes the named nodes to be numbered such that all nodes that are part
+of the same match have the same number and all matches have distinct new names.
+We had two instances of an NP dominating an NNP in this example, and they were
+renamed such that NP-i < NNP-i for each match, with 1 <= i <= number of matches.
+
+-----------------------------------------
+TSURGEON SCRIPTS
+-----------------------------------------
+Script format:
+
+Tsurgeon scripts are a combination of a Tregex pattern to match and a series
+of Tsurgeon operations to perform on that match.  The first line of a Tsurgeon
+script should be the Tregex pattern.  This should be followed by a blank line,
+and then each subsequent line may contain one Tsurgeon operation.  Tsurgeon
+operations should not be separated by blank lines.  The following is an example
+of  correctly formatted script:
+
+S < NP=node < NNP=name
+
+relabel node NP_NAME
+coindex node name
+
+
+Comments: 
+
+The character % introduces a comment that extends to the end of the
+line.  All other intended uses of % must be escaped as \% .
+
+-----------------------------------------
+CONTACT
+-----------------------------------------
+
+For questions about this distribution, please contact Stanford's JavaNLP group at
+[email protected].  We provide assistance on a best-effort basis.
+
+
+-----------------------------------------
+LICENSE
+-----------------------------------------
+
+ Tregex, Tsurgeon, and Interactive Tregex
+ Copyright (c) 2003-2011 The Board of Trustees of 
+ The Leland Stanford Junior University. All Rights Reserved.
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License
+ as published by the Free Software Foundation; either version 2
+ of the License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+
+ For more information, bug reports, fixes, contact:
+    Christopher Manning
+    Dept of Computer Science, Gates 1A
+    Stanford CA 94305-9010
+    USA
+    [email protected]
+    http://www-nlp.stanford.edu/software/tregex.shtml
+
+

lng/L2SCA/analyzeFolder.py

@@ -0,0 +1,148 @@
+"""
+This script analyzes all text files (with the .txt suffix only) in a single folder or directory.  
+
+It counts the occurrences of the following 9 structures in each text: words (W), sentences (S), verb phrases (VP), clauses (C), T-units (T), dependent clauses (DC), complex T-units (CT), coordinate phrases (CP), and complex nominals (CN). 
+
+These frequency counts are then used to compute the following 14 syntactic complexity indices of each text: mean length of sentence (MLS), mean length of T-unit (MLT), mean length of clause (MLC), clauses per sentence (C/S), verb phrases per T-unit (VP/T), clauses per T-unit (C/T), dependent clauses per clause (DC/C), dependent clauses per T-unit (DC/T), T-units per sentence (T/S), complex T-unit ratio (CT/T), coordinate phrases per T-unit (CP/T), coordinate phrases per clause (CP/C), complex nominals per T-unit (CN/T), and complex nominals per clause (CN/C). 
+
+To run the script, type the following at the command line:
+python analyzeText.py inputFileDirectory outputFileName
+
+inputFileDirectory is the path to the directory or folder that contains the text files you want to analyze (e.g., /home/inputFiles/). The path should end with a slash, as in the example. outputFileName is the name you want to assign to the output file. Both must be provided. 
+
+The first line of the output file will be a comma-delimited list of 24 fields (including Filename, abbreviations of the 9 structures, and abbreviations of the 14 syntactic complexity indices). The subsequent lines of the file will each provide a comma-delimited list of 24 values for one input file (including the name of the file, frequency counts of the 9 structures, and the values of the 14 syntactic complexity indices). This format may be hard to read but allows easy import to Excel or SPSS. 
+"""
+
+import sys, os, subprocess, glob, re
+
+#a function to divide two numbers from strings
+def division(x,y):
+    if float(x)==0 or float(y)==0:
+        return 0
+    return float(x)/float(y)
+
+#the following is a list of tregex patterns for various structures
+
+#sentence (S)
+s="'ROOT'"
+
+#verb phrase (VP)
+vp="'VP > S|SINV|SQ'"
+vp_q="'MD|VBZ|VBP|VBD > (SQ !< VP)'" 
+
+#clause (C)
+c="'S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])]'"
+
+#T-unit (T)
+t="'S|SBARQ|SINV|SQ > ROOT | [$-- S|SBARQ|SINV|SQ !>> SBAR|VP]'"
+
+#dependent clause (DC)
+dc="'SBAR < (S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])])'"
+
+#complex T-unit (CT)
+ct="'S|SBARQ|SINV|SQ [> ROOT | [$-- S|SBARQ|SINV|SQ !>> SBAR|VP]] << (SBAR < (S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP\
+|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])]))'"
+
+#coordinate phrase (CP)
+cp="'ADJP|ADVP|NP|VP < CC'"
+
+#complex nominal (CN)
+cn1="'NP !> NP [<< JJ|POS|PP|S|VBG | << (NP $++ NP !$+ CC)]'"
+cn2="'SBAR [<# WHNP | <# (IN < That|that|For|for) | <, S] & [$+ VP | > VP]'"
+cn3="'S < (VP <# VBG|TO) $+ VP'"
+
+#fragment clause
+fc="'FRAG > ROOT !<< (S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])])'"
+
+#fragment T-unit
+ft="'FRAG > ROOT !<< (S|SBARQ|SINV|SQ > ROOT | [$-- S|SBARQ|SINV|SQ !>> SBAR|VP])'"
+
+#list of patterns to search for
+patternlist=[s,vp,c,t,dc,ct,cp,cn1,cn2,cn3,fc,ft,vp_q]
+
+#location of the Stanford parser
+parserPath="stanford-parser-full-2014-01-04/lexparser.sh"
+
+#path to the directory or folder containing input files
+directoryPath=sys.argv[1]
+
+#output file name
+outputFile=open(sys.argv[2],"w")
+
+#write a list of 24 comma-delimited fields to the output file
+fields="Filename,W,S,VP,C,T,DC,CT,CP,CN,MLS,MLT,MLC,C/S,VP/T,C/T,DC/C,DC/T,T/S,CT/T,CP/T,CP/C,CN/T,CN/C"
+outputFile.write(fields+"\n")
+
+#process text files in the directory one by one
+for filename in glob.glob( os.path.join(directoryPath, '*.txt') ):
+    print("Processing "+filename+"...")
+
+    #Extract the name of the file being processed
+    output=filename.split('/')[-1]
+
+    #name a temporary file to hold the parse trees of the input file
+    parsedFile=filename+".parsed"
+
+    #parse the input file
+    command=parserPath + " " + filename + " > " + parsedFile
+    a=subprocess.getoutput(command).split('\n')[-1].split()
+
+    #list of counts of the patterns
+    patterncount=[]
+
+    #query the parse trees using the tregex patterns
+    for pattern in patternlist:
+        command = "./tregex.sh " + pattern + " " + parsedFile + " -C -o"
+        count = subprocess.getoutput(command).split('\n')[-1]
+        patterncount.append(int(count))
+
+    #update frequencies of complex nominals, clauses, and T-units
+    patterncount[7]=patterncount[-4]+patterncount[-5]+patterncount[-6]
+    patterncount[2]=patterncount[2]+patterncount[-3]
+    patterncount[3]=patterncount[3]+patterncount[-2]
+    patterncount[1]=patterncount[1]+patterncount[-1]
+        
+    #word count
+    infile=open(parsedFile,"r")
+    content=infile.read()
+    w=len(re.findall("\([A-Z]+\$? [^\)\(]+\)",content))
+    infile.close()
+        
+    #add frequencies of words and other structures to output string
+    output+=","+str(w) #number of words
+    for count in patterncount[:8]:
+        output+=","+str(count)
+
+    #list of frequencies of structures other than words
+    [s,vp,c,t,dc,ct,cp,cn]=patterncount[:8]
+
+    #compute the 14 syntactic complexity indices
+    mls=division(w,s)
+    mlt=division(w,t)
+    mlc=division(w,c)
+    c_s=division(c,s)
+    vp_t=division(vp,t)
+    c_t=division(c,t)
+    dc_c=division(dc,c)
+    dc_t=division(dc,t)
+    t_s=division(t,s)
+    ct_t=division(ct,t)
+    cp_t=division(cp,t)
+    cp_c=division(cp,c)
+    cn_t=division(cn,t)
+    cn_c=division(cn,c)
+
+    #add syntactic complexity indices to output string
+    for ratio in [mls,mlt,mlc,c_s,vp_t,c_t,dc_c,dc_t,t_s,ct_t,cp_t,cp_c,cn_t,cn_c]:
+        output+=","+str("%.4F" % ratio)
+
+    #write output string to output file
+    outputFile.write(output+"\n")
+
+    #delete the temporary file holding the parse trees
+    command="rm "+parsedFile
+    os.popen(command)
+
+outputFile.close()
+
+print("Done. Output was saved to " + sys.argv[2] +".")

lng/L2SCA/analyzeText.py

@@ -0,0 +1,146 @@
+"""
+This script analyzes a single plain text file.  
+
+It counts the occurrences of the following 9 structures in the text: words (W), sentences (S), verb phrases (VP), clauses (C), T-units (T), dependent clauses (DC), complex T-units (CT), coordinate phrases (CP), and complex nominals (CN). 
+
+These frequency counts are then used to compute the following 14 syntactic complexity indices of the text: mean length of sentence (MLS), mean length of T-unit (MLT), mean length of clause (MLC), clauses per sentence (C/S), verb phrases per T-unit (VP/T), clauses per T-unit (C/T), dependent clauses per clause (DC/C), dependent clauses per T-unit (DC/T), T-units per sentence (T/S), complex T-unit ratio (CT/T), coordinate phrases per T-unit (CP/T), coordinate phrases per clause (CP/C), complex nominals per T-unit (CN/T), and complex nominals per clause (CN/C). 
+
+To run the script, type the following at the command line:
+python analyzeText.py inputFileName outputFileName
+
+inputFileName is the name of your input text file. outputFileName is the name you want to assign to the output file. Both names must be provided.  
+
+The output file will contain 2 lines. The first line is a comma-delimited list of 24 fields (including Filename, abbreviations of the 9 structures, and abbreviations of the 14 syntactic complexity indices). The second line is a comma-delimited list of 24 values (including the name of the input file, frequency counts of the 9 structures, and the values of the 14 syntactic complexity indices). This format may be hard to read but allows easy import to Excel or SPSS. 
+"""
+
+import sys, os, subprocess, re, tempfile
+
+#a function to divide two numbers from strings
+def division(x,y):
+    if float(x)==0 or float(y)==0:
+        return 0
+    return float(x)/float(y)
+
+#the following is a list of tregex patterns for various structures
+
+#sentence (S)
+s="ROOT"
+
+#verb phrase (VP)
+vp="VP > S|SINV|SQ"
+vp_q="MD|VBZ|VBP|VBD > (SQ !< VP)"
+
+#clause (C)
+c="S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])]"
+
+#T-unit (T)
+t="S|SBARQ|SINV|SQ > ROOT | [$-- S|SBARQ|SINV|SQ !>> SBAR|VP]"
+
+#dependent clause (DC)
+dc="SBAR < (S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])])"
+
+#complex T-unit (CT)
+ct="S|SBARQ|SINV|SQ [> ROOT | [$-- S|SBARQ|SINV|SQ !>> SBAR|VP]] << (SBAR < (S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])]))"
+
+#coordinate phrase (CP)
+cp="ADJP|ADVP|NP|VP < CC"
+
+#complex nominal (CN)
+cn1="NP !> NP [<< JJ|POS|PP|S|VBG | << (NP $++ NP !$+ CC)]"
+cn2="SBAR [<# WHNP | <# (IN < That|that|For|for) | <, S] & [$+ VP | > VP]"
+cn3="S < (VP <# VBG|TO) $+ VP"
+
+#fragment clause
+fc="FRAG > ROOT !<< (S|SINV|SQ [> ROOT <, (VP <# VB) | <# MD|VBZ|VBP|VBD | < (VP [<# MD|VBP|VBZ|VBD | < CC < (VP <# MD|VBP|VBZ|VBD)])])"
+
+#fragment T-unit
+ft="FRAG > ROOT !<< (S|SBARQ|SINV|SQ > ROOT | [$-- S|SBARQ|SINV|SQ !>> SBAR|VP])"
+
+#list of patterns to search for
+patternlist=[s,vp,c,t,dc,ct,cp,cn1,cn2,cn3,fc,ft,vp_q]
+
+pre_path = 'lng/L2SCA'
+
+#location of the Stanford parser
+parserPath= os.path.join(pre_path, "stanford-parser-full-2014-01-04/lexparser.sh")
+
+def sca(input_text):
+    inputFile = '/tmp/%s.txt'%next(tempfile._get_candidate_names())
+    with open(inputFile, 'w') as f:
+        f.write(input_text + '\n')
+
+#extract the name of the file being processed
+    output = []
+
+#name a temporary file to hold the parse trees of the input file
+    parsedFile=inputFile+".parsed"
+
+#parse the input file
+    command=[parserPath, inputFile]
+    with open(parsedFile, 'w') as f:
+        subprocess.run(command, stdout = f,
+                stderr = subprocess.DEVNULL
+                )
+
+
+#list of counts of the patterns
+    patterncount=[]
+
+#query the parse trees using the tregex patterns
+    for pattern in patternlist:
+        command = [os.path.join(pre_path, "tregex.sh"), pattern, parsedFile, "-C", "-o"]
+        out = subprocess.run(command, check = True, stdout = subprocess.PIPE, stderr = subprocess.DEVNULL)
+        if len(out.stdout) > 0:
+            count = int(out.stdout)
+        else:
+            count = 0
+        patterncount.append(count)
+
+#update frequencies of complex nominals, clauses, and T-units
+    patterncount[7]=patterncount[-4]+patterncount[-5]+patterncount[-6]
+    patterncount[2]=patterncount[2]+patterncount[-3]
+    patterncount[3]=patterncount[3]+patterncount[-2]
+    patterncount[1]=patterncount[1]+patterncount[-1]
+
+#word count
+    infile=open(parsedFile,"r")
+    content=infile.read()
+    w=len(re.findall("\([A-Z]+\$? [^\)\(]+\)",content))
+    infile.close()
+
+#add frequencies of words and other structures to output string
+    output.append(int(w))
+    for count in patterncount[:8]:
+        output.append(int(count))
+        
+#list of frequencies of structures other than words
+    [s,vp,c,t,dc,ct,cp,cn]=patterncount[:8]
+
+#compute the 14 syntactic complexity indices
+    mls=division(w,s)
+    mlt=division(w,t)
+    mlc=division(w,c)
+    c_s=division(c,s)
+    vp_t=division(vp,t)
+    c_t=division(c,t)
+    dc_c=division(dc,c)
+    dc_t=division(dc,t)
+    t_s=division(t,s)
+    ct_t=division(ct,t)
+    cp_t=division(cp,t)
+    cp_c=division(cp,c)
+    cn_t=division(cn,t)
+    cn_c=division(cn,c)
+
+#add syntactic complexity indices to output string
+    for ratio in [mls,mlt,mlc,c_s,vp_t,c_t,dc_c,dc_t,t_s,ct_t,cp_t,cp_c,cn_t,cn_c]:
+        output.append(ratio)
+
+#list of 24 comma-delimited fields
+    # fields="Filename,W,S,VP,C,T,DC,CT,CP,CN,MLS,MLT,MLC,C/S,VP/T,C/T,DC/C,DC/T,T/S,CT/T,CP/T,CP/C,CN/T,CN/C"
+
+#delete the temporary file holding the parse trees
+    os.remove(inputFile)
+    os.remove(parsedFile)
+
+    return output

lng/L2SCA/examples/atree

@@ -0,0 +1 @@
+(VP (VP (VBZ Try) (NP (NP (DT this) (NN wine)) (CC and) (NP (DT these) (NNS snails)))) (PUNCT .))

lng/L2SCA/examples/exciseNP

@@ -0,0 +1,6 @@
+NP < (NP=np < NNS) < (NP=np1 < NN)
+
+excise np np
+excise np1 np1
+
+

lng/L2SCA/examples/relabelWithGroupName

@@ -0,0 +1,4 @@
+/^VB.+/=v
+
+relabel v /^VB(.*)$/ #1
+

lng/L2SCA/examples/renameVerb

@@ -0,0 +1,3 @@
+VBZ=vbz $ NP
+
+relabel vbz MYVERB

lng/L2SCA/lib/ABOUT-AppleJavaExtensions.txt

@@ -0,0 +1,29 @@
+AppleJavaExtensions.jar provides some stub methods to allow
+compilation of code which contains Java methods that reference Mac OS
+X specific Java APIs on any platform.  This is needed only for
+compilation of the class edu.stanford.nlp.trees.tregex.gui.OSXAdapter .
+Using this class and the links to Apple-specific technologies is
+required to allow the Mac version of Tregex to behave like a normal
+Mac application in responding to About and Preferences... menu items.
+
+This library is not needed or used at runtime on any platform.
+
+If you'd prefer not to have these complications in the source for your
+use on other platforms, simply delete both AppleJavaExtensions.jar and
+the file src/edu/stanford/nlp/trees/tregex/gui/OSXAdapter.java . The
+OSXAdapter class is loaded using reflection by the main TregexGUI
+class, so its absence will not cause any errors in compilation.
+
+The file README-AppleJavaExtensions.txt contains Apple's README and
+license information for AppleJavaExtensions.jar . More information on
+AppleJavaExtensions can be found at:
+
+  http://developer.apple.com/samplecode/AppleJavaExtensions/
+
+This issue of needing to include AppleJavaExtensions.jar occurs for
+many Java GUI programs which want to function well on Mac OS X,
+including NetBeans, FindBugs, etc.  Do a Google search on:
+
+  AppleJavaExtensions license
+
+to find examples.

lng/L2SCA/lib/AppleJavaExtensions.jar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0f46aaca0deba5d07490f66d420a58e5a17e4fe8b5118a3ae831207d953f52b
+size 4189

lng/L2SCA/lib/README-AppleJavaExtensions.txt

@@ -0,0 +1,46 @@
+AppleJavaExtensions
+v 1.2
+
+This is a pluggable jar of stub classes representing the new Apple eAWT and eIO APIs for Java 1.4 on Mac OS X.  The purpose of these stubs is to allow for compilation of eAWT- or eIO-referencing code on platforms other than Mac OS X.  The jar file is enclosed in a zip archive for easy expansion on other platforms.  
+
+These stubs are not intended for the runtime classpath on non-Mac platforms.  Please see the OSXAdapter sample for how to write cross-platform code that uses eAWT.
+
+Disclaimer: IMPORTANT:  This Apple software is supplied to you by Apple
+Computer, Inc. ("Apple") in consideration of your agreement to the
+following terms, and your use, installation, modification or
+redistribution of this Apple software constitutes acceptance of these
+terms.  If you do not agree with these terms, please do not use,
+install, modify or redistribute this Apple software.
+
+In consideration of your agreement to abide by the following terms, and
+subject to these terms, Apple grants you a personal, non-exclusive
+license, under Apple's copyrights in this original Apple software (the
+"Apple Software"), to use, reproduce, modify and redistribute the Apple
+Software, with or without modifications, in source and/or binary forms;
+provided that if you redistribute the Apple Software in its entirety and
+without modifications, you must retain this notice and the following
+text and disclaimers in all such redistributions of the Apple Software. 
+Neither the name, trademarks, service marks or logos of Apple Computer,
+Inc. may be used to endorse or promote products derived from the Apple
+Software without specific prior written permission from Apple.  Except
+as expressly stated in this notice, no other rights or licenses, express
+or implied, are granted by Apple herein, including but not limited to
+any patent rights that may be infringed by your derivative works or by
+other works in which the Apple Software may be incorporated.
+
+The Apple Software is provided by Apple on an "AS IS" basis.  APPLE
+MAKES NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION
+THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY AND FITNESS
+FOR A PARTICULAR PURPOSE, REGARDING THE APPLE SOFTWARE OR ITS USE AND
+OPERATION ALONE OR IN COMBINATION WITH YOUR PRODUCTS.
+
+IN NO EVENT SHALL APPLE BE LIABLE FOR ANY SPECIAL, INDIRECT, INCIDENTAL
+OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) ARISING IN ANY WAY OUT OF THE USE, REPRODUCTION,
+MODIFICATION AND/OR DISTRIBUTION OF THE APPLE SOFTWARE, HOWEVER CAUSED
+AND WHETHER UNDER THEORY OF CONTRACT, TORT (INCLUDING NEGLIGENCE),
+STRICT LIABILITY OR OTHERWISE, EVEN IF APPLE HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+
+Copyright © 2003-2006 Apple Computer, Inc., All Rights Reserved

lng/L2SCA/run-tregex-gui.bat

@@ -0,0 +1 @@
+java -mx300m -cp "stanford-tregex.jar;" edu.stanford.nlp.trees.tregex.gui.TregexGUI

lng/L2SCA/run-tregex-gui.command

@@ -0,0 +1,2 @@
+#!/bin/sh
+java -mx300m -cp `dirname $0`/stanford-tregex.jar edu.stanford.nlp.trees.tregex.gui.TregexGUI

lng/L2SCA/samples/my_sample.txt

@@ -0,0 +1 @@
+A few people in a restaurant setting, one of them is drinking orange juice.

lng/L2SCA/samples/sample1.txt

@@ -0,0 +1,10 @@
+Scores of properties are under extreme fire threat as a huge blaze
+continues to advance through Sydney's north-western suburbs. Fires
+have also shut down the major road and rail links between Sydney and
+Gosford.
+
+The promotional stop in Sydney was everything to be expected for a
+Hollywood blockbuster - phalanxes of photographers, a stretch limo to
+a hotel across the Quay - but with one difference. A line-up of
+masseurs was waiting to take the media in hand. Never has the term
+"massaging the media" seemed so accurate.

lng/L2SCA/samples/sample1_output

@@ -0,0 +1,2 @@
+Filename,W,S,VP,C,T,DC,CT,CP,CN,MLS,MLT,MLC,C/S,VP/T,C/T,DC/C,DC/T,T/S,CT/T,CP/T,CP/C,CN/T,CN/C
+sample1.txt,87,5,11,7,5,2,2,2,12,17.4000,17.4000,12.4286,1.4000,2.2000,1.4000,0.2857,0.4000,1.0000,0.4000,0.4000,0.2857,2.4000,1.7143

lng/L2SCA/samples/sample2.txt

@@ -0,0 +1 @@
+Naaman was the captain of the host of King of Syria.  He was a great and honourable man but he had a disease called Leprosy.  The little maid that served Naaman's wife told her if her master, Naaman, was with the prophet in Samaria, he would be healed.  So the King of Syria sent Naaman, along with a letter to the King of Israel.  When the King of Israel received the letter, he tore his clothes.  The King was upset because he knew that he could not heal Naaman.

lng/L2SCA/samples/samples_output

@@ -0,0 +1,3 @@
+Filename,W,S,VP,C,T,DC,CT,CP,CN,MLS,MLT,MLC,C/S,VP/T,C/T,DC/C,DC/T,T/S,CT/T,CP/T,CP/C,CN/T,CN/C
+sample1.txt,87,5,11,7,5,2,2,2,12,17.4000,17.4000,12.4286,1.4000,2.2000,1.4000,0.2857,0.4000,1.0000,0.4000,0.4000,0.2857,2.4000,1.7143
+sample2.txt,90,6,13,13,7,5,3,1,13,15.0000,12.8571,6.9231,2.1667,1.8571,1.8571,0.3846,0.7143,1.1667,0.4286,0.1429,0.0769,1.8571,1.0000

lng/L2SCA/stanford-parser-full-2014-01-04/LICENSE.txt

@@ -0,0 +1,339 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+                            NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.

lng/L2SCA/stanford-parser-full-2014-01-04/Makefile

@@ -0,0 +1,13 @@
+# This is a rudimentary Makefile for rebuilding the parser.
+# We actually use ant (q.v.) or a Java IDE.
+
+JAVAC = javac
+JAVAFLAGS = -O -d classes -encoding utf-8
+
+parser:
+	mkdir -p classes
+	$(JAVAC) $(JAVAFLAGS) src/edu/stanford/nlp/*/*.java \
+	    src/edu/stanford/nlp/*/*/*.java src/edu/stanford/nlp/*/*/*/*.java
+	cd classes ; jar -cfm ../stanford-parser-`date +%Y-%m-%d`.jar ../src/edu/stanford/nlp/parser/lexparser/lexparser-manifest.txt edu ; cd ..
+	cp stanford-parser-`date +%Y-%m-%d`.jar stanford-parser.jar
+	rm -rf classes

lng/L2SCA/stanford-parser-full-2014-01-04/ParserDemo.java

@@ -0,0 +1,100 @@
+
+import java.util.Collection;
+import java.util.List;
+import java.io.StringReader;
+
+import edu.stanford.nlp.process.Tokenizer;
+import edu.stanford.nlp.process.TokenizerFactory;
+import edu.stanford.nlp.process.CoreLabelTokenFactory;
+import edu.stanford.nlp.process.DocumentPreprocessor;
+import edu.stanford.nlp.process.PTBTokenizer;
+import edu.stanford.nlp.ling.CoreLabel;
+import edu.stanford.nlp.ling.HasWord;
+import edu.stanford.nlp.ling.Sentence;
+import edu.stanford.nlp.trees.*;
+import edu.stanford.nlp.parser.lexparser.LexicalizedParser;
+
+class ParserDemo {
+
+  /**
+   * The main method demonstrates the easiest way to load a parser.
+   * Simply call loadModel and specify the path of a serialized grammar
+   * model, which can be a file, a resource on the classpath, or even a URL.
+   * For example, this demonstrates loading from the models jar file, which
+   * you therefore need to include in the classpath for ParserDemo to work.
+   */
+  public static void main(String[] args) {
+    LexicalizedParser lp = LexicalizedParser.loadModel("edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz");
+    if (args.length > 0) {
+      demoDP(lp, args[0]);
+    } else {
+      demoAPI(lp);
+    }
+  }
+
+  /**
+   * demoDP demonstrates turning a file into tokens and then parse
+   * trees.  Note that the trees are printed by calling pennPrint on
+   * the Tree object.  It is also possible to pass a PrintWriter to
+   * pennPrint if you want to capture the output.
+   */
+  public static void demoDP(LexicalizedParser lp, String filename) {
+    // This option shows loading, sentence-segmenting and tokenizing
+    // a file using DocumentPreprocessor.
+    TreebankLanguagePack tlp = new PennTreebankLanguagePack();
+    GrammaticalStructureFactory gsf = tlp.grammaticalStructureFactory();
+    // You could also create a tokenizer here (as below) and pass it
+    // to DocumentPreprocessor
+    for (List<HasWord> sentence : new DocumentPreprocessor(filename)) {
+      Tree parse = lp.apply(sentence);
+      parse.pennPrint();
+      System.out.println();
+
+      GrammaticalStructure gs = gsf.newGrammaticalStructure(parse);
+      Collection tdl = gs.typedDependenciesCCprocessed();
+      System.out.println(tdl);
+      System.out.println();
+    }
+  }
+
+  /**
+   * demoAPI demonstrates other ways of calling the parser with
+   * already tokenized text, or in some cases, raw text that needs to
+   * be tokenized as a single sentence.  Output is handled with a
+   * TreePrint object.  Note that the options used when creating the
+   * TreePrint can determine what results to print out.  Once again,
+   * one can capture the output by passing a PrintWriter to
+   * TreePrint.printTree.
+   */
+  public static void demoAPI(LexicalizedParser lp) {
+    // This option shows parsing a list of correctly tokenized words
+    String[] sent = { "This", "is", "an", "easy", "sentence", "." };
+    List<CoreLabel> rawWords = Sentence.toCoreLabelList(sent);
+    Tree parse = lp.apply(rawWords);
+    parse.pennPrint();
+    System.out.println();
+
+    // This option shows loading and using an explicit tokenizer
+    String sent2 = "This is another sentence.";
+    TokenizerFactory<CoreLabel> tokenizerFactory =
+        PTBTokenizer.factory(new CoreLabelTokenFactory(), "");
+    Tokenizer<CoreLabel> tok =
+        tokenizerFactory.getTokenizer(new StringReader(sent2));
+    List<CoreLabel> rawWords2 = tok.tokenize();
+    parse = lp.apply(rawWords2);
+
+    TreebankLanguagePack tlp = new PennTreebankLanguagePack();
+    GrammaticalStructureFactory gsf = tlp.grammaticalStructureFactory();
+    GrammaticalStructure gs = gsf.newGrammaticalStructure(parse);
+    List<TypedDependency> tdl = gs.typedDependenciesCCprocessed();
+    System.out.println(tdl);
+    System.out.println();
+
+    // You can also use a TreePrint object to print trees and dependencies
+    TreePrint tp = new TreePrint("penn,typedDependenciesCollapsed");
+    tp.printTree(parse);
+  }
+
+  private ParserDemo() {} // static methods only
+
+}

lng/L2SCA/stanford-parser-full-2014-01-04/ParserDemo2.java

@@ -0,0 +1,88 @@
+
+import java.io.IOException;
+import java.io.StringReader;
+import java.util.*;
+
+import edu.stanford.nlp.ling.CoreLabel;
+import edu.stanford.nlp.ling.HasWord;
+import edu.stanford.nlp.ling.Label;
+import edu.stanford.nlp.ling.Word;
+import edu.stanford.nlp.process.DocumentPreprocessor;
+import edu.stanford.nlp.process.Tokenizer;
+import edu.stanford.nlp.trees.*;
+import edu.stanford.nlp.parser.lexparser.LexicalizedParser;
+
+class ParserDemo2 {
+
+  /** This example shows a few more ways of providing input to a parser.
+   *
+   *  Usage: ParserDemo2 [grammar [textFile]]
+   */
+  public static void main(String[] args) throws IOException {
+    String grammar = args.length > 0 ? args[0] : "edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz";
+    String[] options = { "-maxLength", "80", "-retainTmpSubcategories" };
+    LexicalizedParser lp = LexicalizedParser.loadModel(grammar, options);
+    TreebankLanguagePack tlp = lp.getOp().langpack();
+    GrammaticalStructureFactory gsf = tlp.grammaticalStructureFactory();
+
+    Iterable<List<? extends HasWord>> sentences;
+    if (args.length > 1) {
+      DocumentPreprocessor dp = new DocumentPreprocessor(args[1]);
+      List<List<? extends HasWord>> tmp =
+        new ArrayList<List<? extends HasWord>>();
+      for (List<HasWord> sentence : dp) {
+        tmp.add(sentence);
+      }
+      sentences = tmp;
+    } else {
+      // Showing tokenization and parsing in code a couple of different ways.
+      String[] sent = { "This", "is", "an", "easy", "sentence", "." };
+      List<HasWord> sentence = new ArrayList<HasWord>();
+      for (String word : sent) {
+        sentence.add(new Word(word));
+      }
+      String sent2 = ("This is a slightly longer and more complex " +
+                      "sentence requiring tokenization.");
+      // Use the default tokenizer for this TreebankLanguagePack
+      Tokenizer<? extends HasWord> toke =
+        tlp.getTokenizerFactory().getTokenizer(new StringReader(sent2));
+      List<? extends HasWord> sentence2 = toke.tokenize();
+      List<List<? extends HasWord>> tmp =
+        new ArrayList<List<? extends HasWord>>();
+      tmp.add(sentence);
+      tmp.add(sentence2);
+      sentences = tmp;
+    }
+
+    for (List<? extends HasWord> sentence : sentences) {
+      Tree parse = lp.parse(sentence);
+      parse.pennPrint();
+      System.out.println();
+      GrammaticalStructure gs = gsf.newGrammaticalStructure(parse);
+      List<TypedDependency> tdl = gs.typedDependenciesCCprocessed();
+      System.out.println(tdl);
+      System.out.println();
+
+      System.out.println("The words of the sentence:");
+      for (Label lab : parse.yield()) {
+        if (lab instanceof CoreLabel) {
+          System.out.println(((CoreLabel) lab).toString("{map}"));
+        } else {
+          System.out.println(lab);
+        }
+      }
+      System.out.println();
+      System.out.println(parse.taggedYield());
+      System.out.println();
+
+    }
+
+    // This method turns the String into a single sentence using the
+    // default tokenizer for the TreebankLanguagePack.
+    String sent3 = "This is one last test!";
+    lp.parse(sent3).pennPrint();
+  }
+
+  private ParserDemo2() {} // static methods only
+
+}

lng/L2SCA/stanford-parser-full-2014-01-04/README.txt

@@ -0,0 +1,280 @@
+Stanford Lexicalized Parser v3.3.1 - 2014-01-04
+-----------------------------------------------
+
+Copyright (c) 2002-2012 The Board of Trustees of The Leland Stanford Junior
+University. All Rights Reserved.
+
+Original core parser code by Dan Klein.  Support code, additional
+modules, languages, features, internationalization, compaction, typed
+dependencies, etc. by Christopher Manning, Roger Levy, Teg Grenager,
+Galen Andrew, Marie-Catherine de Marneffe, Jenny Finkel, Spence Green,
+Bill MacCartney, Anna Rafferty, Huihsin Tseng, Pi-Chuan Chang,
+Wolfgang Maier, Richard Eckart, Richard Socher, and John Bauer.
+
+This release prepared by John Bauer.
+
+This package contains 4 parsers: a high-accuracy unlexicalized PCFG; a
+lexicalized dependency parser; a factored model, where the estimates
+of dependencies and an unlexicalized PCFG are jointly optimized to
+give a lexicalized PCFG treebank parser; and an RNN parser, where
+recursive neural networks trained with semantic word vectors are used
+to score parse trees.  Also included are grammars for various
+languages for use with these parsers.
+
+For more information about the parser API, point a web browser at the
+included javadoc directory (use the browser's Open File command to open
+the index.html file inside the javadoc folder).  Start by looking at the
+Package page for the edu.stanford.nlp.parser.lexparser package, and then
+look at the page for the LexicalizedParser class documentation therein,
+particularly documentation of the main method.
+
+Secondly, you should also look at the Parser FAQ on the web:
+
+    http://nlp.stanford.edu/software/parser-faq.shtml
+
+This software requires Java 6 (JDK 1.6.0+).  (You must have installed it
+separately. Check that the command "java -version" works and gives 1.6+.)
+
+
+QUICKSTART
+
+UNIX COMMAND-LINE USAGE
+
+On a Unix system you should be able to parse the English test file with the
+following command:
+
+    ./lexparser.sh data/testsent.txt
+
+This uses the PCFG parser, which is quick to load and run, and quite accurate.
+
+[Notes: it takes a few seconds to load the parser data before parsing
+begins; continued parsing is quicker. To use the lexicalized parser, replace
+englishPCFG.ser.gz with englishFactored.ser.gz in the lexparser.sh script
+and use the flag -mx600m to give more memory to java.]
+
+WINDOWS GUI USAGE
+
+On a Windows system, assuming that java is on your PATH, you should be able
+to run a parsing GUI by double-clicking on the lexparser-gui.bat icon,
+or giving the command lexparser-gui in this directory from a command prompt.
+
+Click Load File, Browse, and navigate to and select testsent.txt in
+the top directory of the parser distribution.  Click Load Parser,
+Browse, and select the models jar, also in the top directory of the
+parser distribution.  From the models jar, select englishPCFG.ser.gz.
+Click Parse to parse the first sentence.
+
+OTHER USE CASES
+
+The GUI is also available under Unix:
+
+    lexparser-gui.sh
+
+Under Mac OS X, you can double-click on lexparser-gui.command to invoke the
+GUI.  The command-line version works on all platforms.	Use lexparser.bat
+to run it under Windows.  The GUI is only for exploring the parser. It does
+not allow you to save output.  You need to use the command-line program or
+programmatic API to do serious work with the parser.
+
+ADDITIONAL GRAMMARS
+
+The parser is supplied with several trained grammars. There are English
+grammars based on the standard LDC Penn Treebank WSJ training sections 2-21
+(wsj*), and ones based on an augmented data set, better for questions,
+commands, and recent English and biomedical text (english*).
+
+All grammars are located in the included models jar. (If you'd like to have
+grammar files like in older versions of the parser, you can get them by
+extracting them from the jar file with the 'jar -xf' command.)
+
+MULTILINGUAL PARSING
+In addition to the English grammars, the parser comes with trained grammars
+for Arabic, Chinese, French, and German. To parse with these grammars, run
+
+    lexparser-lang.sh
+
+with no arguments to see usage instructions. You can change language-specific
+settings passed to the parser by modifying lexparser_lang.def.
+
+You can also train and evaluate new grammars using:
+
+    lexparser-lang-train-test.sh
+
+To see how we trained the grammars supplied in this distribution, see
+
+    bin/makeSerialized.csh
+
+You will not be able to run this script (since it uses Stanford-specific file
+paths), but you should be able to see what we did.
+
+Arabic
+Trained on parts 1-3 of the Penn Arabic Treebank (ATB) using the
+pre-processing described in (Green and Manning, 2010). The default input
+encoding is UTF-8 Arabic script. You can convert text in Buckwalter encoding to UTF-8
+with the package edu.stanford.nlp.international.arabic.Buckwalter which is included 
+in stanford-parser.jar.
+
+The parser *requires* segmentation and tokenization of raw text per the ATB standard
+prior to parsing. You can generate this segmentation and tokenization with the Stanford 
+Word Segmenter, which is available separately at:
+
+  http://nlp.stanford.edu/software/segmenter.shtml
+
+Chinese
+There are Chinese grammars trained just on mainland material from
+Xinhua and more mixed material from the LDC Chinese Treebank. The default
+input encoding is GB18030.
+
+French
+Trained on the functionally annotated section of the French Treebank
+(FTB) using the pre-processing described in (Green et al., 2011). For raw text input,
+a tokenizer is enabled by default that produces FTB tokenization. To disable this
+tokenizer, use the "-tokenized" option. To tokenize raw text separately, see
+the usage information in edu.stanford.nlp.international.french.process.FrenchTokenizer.
+
+German
+Trained on the Negra corpus. Details are included in (Rafferty and
+Manning, 2008).
+
+TREEBANK PREPROCESSING
+
+The pre-processed versions of the ATB described
+in (Green and Manning, 2010) and the FTB described in (Green et al.,
+2011) can be reproduced using the TreebankPreprocessor included in this
+release. The configuration files are located in /conf. For example,
+to create the ATB data, run:
+
+    bin/run-tb-preproc -v conf/atb-latest.conf
+
+Note that you'll need to update the conf file paths to your local treebank
+distributions as the data is not distributed with the parser. You'll
+also need to set the classpath in the cmd_line variable of run-tb-preproc.
+
+The TreebankPreprocessor conf files support various options, which are
+documented in
+
+    edu.stanford.nlp.international.process.ConfigParser
+
+EVALUATION METRICS
+
+The Stanford parser comes with Java implementations of the following
+evaluation metrics:
+
+    Dependency Labeled Attachment
+
+    Evalb         (Collins, 1997)
+      -Includes per-category evaluation with the -c option
+
+    Leaf Ancestor (Sampson and Babarczy, 2003)
+      -Both micro- and macro-averaged score
+
+    Tagging Accuracy
+
+See the usage instructions and javadocs in the requisite classes located in
+edu.stanford.nlp.parser.metrics.
+
+LICENSE
+
+// StanfordLexicalizedParser -- a probabilistic lexicalized NL CFG parser
+// Copyright (c) 2002-2012 The Board of Trustees of
+// The Leland Stanford Junior University. All Rights Reserved.
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License
+// as published by the Free Software Foundation; either version 2
+// of the License, or (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+//
+// For more information, bug reports, fixes, contact:
+//    Christopher Manning
+//    Dept of Computer Science, Gates 1A
+//    Stanford CA 94305-9010
+//    USA
+//    [email protected]
+//    http://nlp.stanford.edu/downloads/lex-parser.shtml
+
+
+---------------------------------
+CHANGES
+---------------------------------
+
+2014-01-04    3.3.1     Bugfix release, dependency improvements 
+
+2013-11-12    3.3.0     Remove the attr dependency, add imperatives to 
+                        English training data 
+
+2013-06-19    3.2.0     New RNN model for WSJ and English with 
+                        improved test set accuracy, rel dependency 
+                        removed 
+
+2013-04-05    2.0.5     Dependency improvements, ctb7 model, -nthreads 
+                        option 
+
+2012-11-12    2.0.4     Dependency speed improvements; other 
+                        dependency changes 
+
+2012-07-09    2.0.3     Minor bug fixes 
+
+2012-05-22    2.0.2     Supports adding extra data in non-tree format
+
+2012-03-09    2.0.1     Caseless English model added, ready for maven
+
+2012-01-11    2.0.0     Threadsafe!
+
+2011-09-14    1.6.9     Added some imperatives to the English 
+                        training data; added root dependency.
+
+2011-06-15    1.6.8     Added French parser and leaf ancestor
+                        evaluation metric; reorganized distribution;
+                        new data preparation scripts; rebuilt grammar
+                        models; other bug fixes
+
+2011-05-15    1.6.7     Minor bug fixes
+
+2011-04-17    1.6.6     Compatible with tagger, corenlp and tregex.
+
+2010-10-30    1.6.5     Further improvements to English Stanford 
+                        Dependencies and other minor changes
+
+2010-08-16    1.6.4     More minor bug fixes and improvements to English 
+                        Stanford Dependencies and question parsing
+
+2010-07-09    1.6.3     Improvements to English Stanford Dependencies and 
+                        question parsing, minor bug fixes
+
+2010-02-25    1.6.2     Improvements to Arabic parser models, 
+                        and to English and Chinese Stanford Dependencies
+
+2008-10-19    1.6.1     Slightly improved Arabic, German and
+                        Stanford Dependencies
+
+2007-08-18      1.6     Added Arabic, k-best PCCFG parsing; 
+                        improved English grammatical relations
+
+2006-05-30    1.5.1     Improved English and Chinese grammatical relations; 
+                        fixed UTF-8 handling
+
+2005-07-20      1.5     Added grammatical relations output; 
+                        fixed bugs introduced in 1.4
+
+2004-03-24      1.4     Made PCFG faster again (by FSA minimization); 
+                        added German support
+
+2003-09-06      1.3     Made parser over twice as fast; 
+                        added tokenization options
+
+2003-07-20      1.2     Halved PCFG memory usage; 
+                        added support for Chinese
+
+2003-03-25      1.1     Improved parsing speed; included GUI, 
+                        improved PCFG grammar
+
+2002-12-05      1.0     Initial release

lng/L2SCA/stanford-parser-full-2014-01-04/README_dependencies.txt

@@ -0,0 +1,194 @@
+STANFORD DEPENDENCIES.  Stanford Parser v3.3.1
+-----------------------------------------------------------
+
+The manual for the English version of the Stanford Dependencies
+representation:
+
+    StanfordDependenciesManual.pdf
+
+should be consulted for the current set of dependency representations
+and the correct commands for generating Stanford Dependencies together
+with any of the Stanford Parser, another parser, or a treebank.
+
+A typed dependencies representation is also available for Chinese.  For
+the moment the documentation consists of the code, and a brief
+presentation in this paper:
+
+Pi-Chuan Chang, Huihsin Tseng, Dan Jurafsky, and Christopher
+D. Manning. 2009.  Discriminative Reordering with Chinese Grammatical
+Relations Features.  Third Workshop on Syntax and Structure in Statistical
+Translation.
+
+
+--------------------------------------
+ORIGINAL DEPENDENCIES SCHEME
+
+For an overview of the original typed dependencies scheme, please look
+at:
+
+  Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D.
+  Manning. 2006. Generating Typed Dependency Parses from Phrase
+  Structure Parses. 5th International Conference on Language Resources
+  and Evaluation (LREC 2006).
+  http://nlp.stanford.edu/~manning/papers/LREC_2.pdf
+
+For more discussion of the design principles, please see:
+
+  Marie-Catherine de Marneffe and Christopher D. Manning. 2008. The
+  Stanford typed dependencies representation. In Proceedings of the
+  workshop on Cross-Framework and Cross-Domain Parser Evaluation, pp. 1-8.
+  http://nlp.stanford.edu/~manning/papers/dependencies-coling08.pdf
+
+These papers can be cited as references for the English Stanford
+Dependencies.
+
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v3.3.1
+
+A couple of fixes/improvements were made in the dependency conversion,
+and one change was made to the taxonomy of relations.
+
+ - The partmod and infmod relations were deleted, and replaced with
+vmod for reduced, non-finite verbal modifiers. The distinction between
+these two relations can be recovered from the POS tag of the dependent.
+ - A couple of improvements were made to the conversion, the largest
+ one being recognizing pobj inside a PP not headed by something tagged
+ as IN or TO.
+
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v3.3
+
+Some fixes/improvements were made in the dependency conversion, and one
+change was made to the taxonomy of relations.
+
+ - For currency amount expressions with a currency symbol like "$", it
+   had previously been the case that "$" was the head, and then each
+   number word modified it as a number. We realized that this was
+   unnecessarily inconsistent. For the expression "two thousand dollars",
+   "dollars" is the head, but "thousand" is a num modifier of it, and
+   number is used for the parts of a number multi-word expression only.
+   This analysis is now also used for cases with a currency symbol. E.g.,
+   "for $ 52.7 million": prep(for, $) num($, million) number(million, 52.7).
+   Similarly, for "the $ 2.29 billion value", we changed the analysis from
+   num(value, $) number($, billion) to amod(value, $) num($, billion).
+   This corresponds to hwat you got for "a two dollar value".
+   This is actually the most common change (at least on WSJ newswire!).
+ - Remove the attr relation. Some cases disappear by making the question
+   phrase of WHNP be NP questions the root. Others (predicative NP
+   complements) become xcomp.
+ - Less aggressive labeling of participial form VPs as xcomp. More of them
+   are correctly labeled partmod (but occasionally a true xcomp is also
+   mislabeled as partmod).
+ - Small rule changes to recognize a few more ccomp and parataxis.
+
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v3.2, JUNE 2013
+
+Various small fixes were made to the dependencies conversion,
+and one change to the taxonomy of relations:
+ - rel was removed. rel was originally used as the relation for an
+    overt relativizer in a relative clause. But it was never a real
+    grammatical relation, and we gradually started labeling easy cases
+    as nsubj or dobj. In this release, rel is removed, pobj cases are
+    also labeled, and the remaining hard cases are labeled as dep.
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v2.0.5, MARCH 2013
+
+We have begun a more major effort to improve the suitability and coverage of
+Stanford Dependencies on less formal text types, and to clean up a couple of
+the more quirky dependencies in the original set. These changes are still
+ongoing, but in this first installment, we have removed 3 dependencies and
+added 2:
+ - abbrev was removed, and is now viewed as just a case of appos.
+ - complm was removed, and is now viewed as just a case of mark.
+    (This is consistent with an HPSG-like usage of mark.)
+ - purpcl was removed, and is now viewed as just a case of advcl.
+ - discourse was added. The lack of a dependency type for
+    interjections was an omission even in the early versions, but it
+    became essential as we expanded our consideration of informal
+    text types. It is used for interjections, fillers, discourse markers
+    and emoticons.
+  - goeswith was added. In badly edited text, it is used to join the
+    two parts of a word.
+
+A few other changes and improvements were also made, including improvements
+in the recognition of advcl. There has been a reduction of "dep" dependencies
+of about 14% on newswire (and higher on more informal text genres).
+
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v2.0.4, NOVEMBER 2012
+
+A few minor changes and fixes were made: HYPH is now recognized, and treated
+as punctuation and clausal complements of adjectives (including comparatives)
+are recognized as ccomp.
+
+--------------------------------------
+
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v1.6.9
+
+This version adds an explicit root dependency in the set of dependencies
+returned.  In the past, there had been no explicit representation of the
+root of the sentence in the set of dependencies returned, except in the
+CoNLL format output, which always showed the root.  Now, there is always
+an explicit extra dependency that marks the sentence root, using a fake
+ROOT pseudoword with index 0.  That is, the root is marked in this way:
+    root(ROOT-0, depends-3)
+Otherwise there were only a couple of minute changes in the dependencies
+produced (appositions are now recognized in WHNPs!).
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- v1.6.8
+
+This version includes only small fixes, principally addressing some gaps
+in the correct treatment of dependencies in inverted sentence (SQ and SINV)
+constructions, and some errors in the treatment of copulas in the presence of
+temporal NPs.
+
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- NOVEMBER 2010 - JANUARY 2011
+
+Two changes were made to the taxonomy of dependencies.
+ - measure (phrase modifier) was generalized and replaced by
+npadvmod (noun phrase adverbial modifier) which includes measure
+phrases and other adverbial uses of noun phrases.  Temporal NPs
+(tmod) are now a subtype of npadvmod in the dependency hierarchy.
+ - mwe (multi-word expression) is introduced for certain common
+function word dependencies for which another good analysis isn't
+easy to come by (and which were frequently dep before) such as
+"instead of" or "rather than".
+
+A new option has ben added to allow the copula to be treated as
+the head when it has an adjective or noun complement.
+
+The conversion software will now work fairly well with the
+David Vadas version of the treebank with extra noun phrase
+structure.  (A few rare cases that are handled with the standard
+treebank aren't yet handled, but you will get better dependencies
+for compound nouns and multiword adjectival modifiers, etc.)
+
+Considerable improvements were made in the coverage of named
+dependencies. You should expect to see only about half as many generic
+"dep" dependencies as in version 1.6.4.
+
+--------------------------------------
+CHANGES IN ENGLISH TYPED DEPENDENCIES CODE -- JUNE-AUGUST 2010
+
+No new dependency relations have been introduced.
+
+There have been some significant improvements in the generated
+dependencies, principally covering:
+ - Better resolution of nsubj and dobj long distance dependencies
+   (but v1.6.4 fixes the overpercolation of dobj in v1.6.3)
+ - Better handling of conjunction distribution in CCprocessed option
+ - Correction of bug in v1.6.2 that made certain verb dependents noun
+   dependents.
+ - Better dependencies are generated for question structures (v1.6.4)
+ - Other minor improvements in recognizing passives, adverbial
+   modifiers, etc.
+

lng/L2SCA/stanford-parser-full-2014-01-04/bin/makeSerialized.csh

@@ -0,0 +1,242 @@
+#!/bin/csh -f
+
+# This is the file we use to make the serialized grammars for the parser.
+# If you are on the Stanford NLP machines, you can use it to remake the 
+# serialized parsers (such as when there have been incompatible software
+# changes).  Don't forget to klog first so you can access the AFS corpora.
+#
+# If you are not on the Stanford NLP machines, then the script won't work 
+# for you as is, since it contains hard-coded paths to various treebanks.
+# But it may still be useful to inspect it to see what options we used to
+# generate the various supplied grammars.
+#
+# NOTE: Output files in this script should ALWAYS use relative paths, so
+# that you can copy this script and run it in a different directory and
+# it will write output files there.
+#
+# usage:
+# cd /u/nlp/data/lexparser   # to have files output in "usual" location
+# ./makeSerialized.csh
+#
+## Uncomment this bit to run it with older parser version
+# setenv CLASSPATH /u/nlp/distrib/lexparser-2004-03-24/javanlp.jar:
+
+if ( ! $?JAVANLP_HOME) then
+  echo 'JAVANLP_HOME is not set'
+  echo 'Add a line like setenv JAVANLP_HOME $HOME/javanlp to your environment'
+  exit
+endif
+
+set wsjptb=/afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj
+# now ctb6
+set ctb=/afs/ir/data/linguistic-data/Chinese-Treebank/6/data/utf8/bracketed
+# now ctb7!
+set ctb7train=/u/nlp/data/chinese/ctb7/train.mrg
+set ctb7test=/u/nlp/data/chinese/ctb7/test.mrg
+set negra=/afs/ir/data/linguistic-data/NEGRA/penn-format-train-dev-test
+
+set host=`hostname | cut -d. -f1`
+
+if ( ! -r $wsjptb) then
+  echo "Can't read WSJ PTB.  Maybe you forgot to klog??"
+  exit
+endif
+
+mv -f serializedParsers.log serializedParsers.bak
+uptime > serializedParsers.log
+echo "Classpath is $CLASSPATH" >> serializedParsers.log
+
+# English WSJ 2-21 PCFG binary and text grammars
+
+( echo "Running wsjPCFG (goodPCFG) on $host -server" ; time java -server -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -goodPCFG -saveToSerializedFile wsjPCFG.ser.gz -saveToTextFile wsjPCFG.txt -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+
+# English noTagSplit no rule compaction PCFG text grammar
+( echo "Running wsjPCFG-noTagSplit-noCompact on $host -server" ; time java -server -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -goodPCFG -noTagSplit -saveToTextFile wsjPCFG-noTagSplit.txt -compactGrammar 0 -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+
+# English WSJ 2-21 Factored binary
+
+## Not yet clear that goodFactored is better than -ijcai03 -- not on dev set
+# ( echo "Running wsjFactored (goodFactored) on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDAtsv" -goodFactored -saveToSerializedFile wsjFactored.ser.gz -saveToTextFile wsjFactored.txt -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+( echo "Running wsjFactored (ijcai03 correctTags) on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -ijcai03 -v -printStates -compactGrammar 0 -correctTags -saveToSerializedFile wsjFactored.ser.gz -saveToTextFile wsjFactored.txt -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+( echo "Running wsjFactored (ijcai03 replication) on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -ijcai03 -v -printStates -compactGrammar 0 -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+
+
+## "General English" models 
+
+# english{Factored|PCFG} is currently trained on:
+# - WSJ sections 1-21
+# - Genia as reformatted by Andrew Clegg, his training split
+# - 2 English Chinese Translation Treebank and 3 English Arabic Translation 
+#   Treebank files backported to the original treebank annotation standards
+#   (by us) 
+# - 95 sentences parsed by us (mainly questions and imperatives; a few from 
+#   recent newswire).
+
+# /u/nlp/data/genia/sentences_cleaned.tree
+
+# "General English" Factored binary
+
+
+
+( echo "Running englishFactored (from treebank) on $host server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -wordFunction edu.stanford.nlp.process.AmericanizeFunction -evals "factDA,tsv" -ijcai03 -saveToSerializedFile englishFactored.ser.gz -maxLength 40 -train /afs/ir/data/linguistic-data/Treebank/Treebank3Stanford/parsed/mrg/wsj 100-2199,9000-9099 -train2 /u/nlp/data/lexparser/extraTrain 1-4000 0.5 -taggedFiles tagSeparator=_,/u/nlp/data/pos-tagger/english/train-tech-english -testTreebank /afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj/22 2200-2219 ) >>& ./serializedParsers.log
+
+# "General English" PCFG binary 
+
+( echo "Running englishPCFG (from treebank) on $host server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -wordFunction edu.stanford.nlp.process.AmericanizeFunction -evals "factDA,tsv" -goodPCFG -saveToSerializedFile englishPCFG.ser.gz -maxLength 40 -train /afs/ir/data/linguistic-data/Treebank/Treebank3Stanford/parsed/mrg/wsj 100-2199,9000-9099  -train2 /u/nlp/data/lexparser/extraTrain 1-4000 0.5 -taggedFiles tagSeparator=_,/u/nlp/data/pos-tagger/english/train-tech-english  -testTreebank /afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj/22 2200-2219 ) >>& ./serializedParsers.log
+
+
+# "General English" PCFG, case insensitive, binary
+
+( echo "Running caseless englishPCFG (from treebank) on $host server" ; time java -mx4g edu.stanford.nlp.parser.lexparser.LexicalizedParser -wordFunction edu.stanford.nlp.process.LowercaseAndAmericanizeFunction -evals factDA,tsv -goodPCFG -saveToSerializedFile englishPCFG.caseless.ser.gz -maxLength 40 -train /afs/ir/data/linguistic-data/Treebank/Treebank3Stanford/parsed/mrg/wsj 100-2199,9000-9099 -train2 /u/nlp/data/lexparser/extraTrain 1-4000 0.5 -taggedFiles tagSeparator=_,/u/nlp/data/pos-tagger/english/train-tech-english  -testTreebank /afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj/22 2200-2219 ) >>& ./serializedParsers.log
+
+
+# English WSJ 2-21 PCFG simplified grammar
+# This dumbed down parser is used by the RNN parser.  
+# See /scr/nlp/data/dvparser for more details.
+( echo "Running wsj pcfg (simplified for use in the RNN parser) on $host -server" ; time java -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -goodPCFG -noRightRec -dominatesV 0 -baseNP 0 -saveToSerializedFile wsjPCFG.nocompact.simple.ser.gz -maxLength 40  -compactGrammar 0 -train /afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj 200-2199 -testTreebank /afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj 2200-2219 ) >>& ./serializedParsers.log
+
+# English with extras PCFG simplified grammar
+# This dumbed down parser is used by the RNN parser.  
+# See /scr/nlp/data/dvparser for more details.
+( echo "Running english pcfg (simplified for use in the RNN parser) on $host -server" ; time java -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -wordFunction edu.stanford.nlp.process.AmericanizeFunction -evals "factDA,tsv" -goodPCFG -noRightRec -dominatesV 0 -baseNP 0 -saveToSerializedFile englishPCFG.nocompact.simple.ser.gz -maxLength 40  -compactGrammar 0 -train /afs/ir/data/linguistic-data/Treebank/Treebank3Stanford/parsed/mrg/wsj 100-2199,9000-9099  -train2 /u/nlp/data/lexparser/extraTrain 1-4000 0.5 -taggedFiles tagSeparator=_,/u/nlp/data/pos-tagger/english/train-tech-english -testTreebank /afs/ir/data/linguistic-data/Treebank/3/parsed/mrg/wsj 2200-2219 ) >>& ./serializedParsers.log
+
+
+# Xinhua Mainland Chinese PCFG binary
+
+( echo "Running xinhuaPCFG on $host -server" ; time java -server -mx800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -chinesePCFG -saveToSerializedFile xinhuaPCFG.ser.gz -maxLength 40 -train $ctb 026-270,301-499,600-999 -test $ctb 001-025 ) >>& ./serializedParsers.log
+# new train list (Galen and Huihsin): 026-270,301-499,555-589,597-1041
+# newer train list (Galen and Huihsin): 026-270,301-499,600-999
+# this is all Xinhua minus Stanford devel and Bikel test
+
+# Xinhua Mainland Chinese Factored binary
+
+( echo "Running xinhuaFactored on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -acl03chinese -scTags -saveToSerializedFile xinhuaFactored.ser.gz -maxLength 40 -train $ctb 026-270,301-499,600-999 -test $ctb 001-025 ) >>& ./serializedParsers.log
+
+# Mixed dialect Chinese on lots of data (with chineseFactored)
+
+( echo "Running chineseFactored on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -chineseFactored -saveToSerializedFile chineseFactored.ser.gz -maxLength 40 -train $ctb7train -test $ctb7test ) >>& ./serializedParsers.log
+# new train list (Galen and Huihsin): 026-270,301-499,555-589,597-1041
+# newer train list (Galen and Huihsin): 026-270,301-499,600-999
+# this is all Xinhua minus Stanford devel and Bikel test
+# CTB files 001-499, 555-589,597-1000 are from newswire of
+# XinHua.
+# Files 500-554 are Information Services Department of HKSAR.
+# Files 590-596 and 1001-1151 are Sinorama articles, more of literature
+#   nature and from Taiwan.
+# Files 2000-3145 are ACE broadcast news (from where?).  We only use a few for now.
+
+# Mixed dialect Chinese PCFG on lots of data
+
+( echo "Running chinesePCFG on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -chinesePCFG -useUnicodeType -saveToSerializedFile chinesePCFG.ser.gz -maxLength 40 -train $ctb7train -test $ctb7test ) >>& ./serializedParsers.log
+# new train list (Galen and Huihsin): 026-270,301-499,555-589,597-1041
+# newer train list (Galen and Huihsin): 026-270,301-499,600-999
+# this is all Xinhua minus Stanford devel and Bikel test
+
+
+# Chinese parser for unsegmented Chinese
+
+( echo "Running xinhuaFactoredSegmenting on $host -server" ; time java -server -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -segmentMarkov -train $ctb 26-270,301-499,600-999 -sctags -acl03chinese -saveToSerializedFile xinhuaFactoredSegmenting.ser.gz ) >>& ./serializedParsers.log
+java -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -encoding utf-8 xinhuaFactoredSegmenting.ser.gz /u/nlp/data/lexparser/chinese-onesent-unseg-utf8.txt >>& ./serializedParsers.log
+
+
+# It used to be the case that explicitly saying tLPP on command line was 
+# needed for file encoding.  But it has been fixed.
+# ( echo "Running xinhuaFactored from serialized check on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -maxLength 40 -loadFromSerializedFile xinhuaFactored.ser.gz -test $ctb 001-025 ) >>& ./serializedParsers.log
+# This now works
+( echo "Running xinhuaFactored from serialized (check without specifying -tLPP) on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -maxLength 40 -loadFromSerializedFile xinhuaFactored.ser.gz -test $ctb 001-025 ) >>& ./serializedParsers.log
+
+( echo "Running chinesePCFG (simplified for use in the RNN parser) on $host -server" ; time java -server -mx4g edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.ChineseTreebankParserParams -chineseFactored -PCFG -compactGrammar 0 -saveToSerializedFile chinesePCFG-simple.ser.gz -maxLength 40 -train $ctb7train -test $ctb7test ) >>& ./serializedParsers.log
+
+# German Factored binary from Negra (version 2)
+# $negra 3 is the dev set 
+
+( echo "Running germanFactored on $host -server" ; time java -server -mx5g edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.NegraPennTreebankParserParams -hMarkov 1 -vMarkov 2 -vSelSplitCutOff 300 -uwm 1 -unknownSuffixSize 2 -maxLength 40 -nodeCleanup 2 -saveToSerializedFile germanFactored.ser.gz -train $negra 1 -test $negra 3 ) >>& ./serializedParsers.log
+
+# German PCFG from Negra (version 2)
+
+( echo "Running germanPCFG on $host -server" ; time java -server -mx2g edu.stanford.nlp.parser.lexparser.LexicalizedParser -v -evals tsv -tLPP edu.stanford.nlp.parser.lexparser.NegraPennTreebankParserParams -PCFG -hMarkov 1 -vMarkov 2 -vSelSplitCutOff 300 -uwm 1 -unknownSuffixSize 1 -maxLength 40 -nodeCleanup 2 -saveToSerializedFile germanPCFG.ser.gz -train $negra 1 -test $negra 3 ) >>& ./serializedParsers.log
+
+# German Dependency parser
+# This requires normalizing the dependency output to strip boundary symbol.
+# ( echo "Running germanDep on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -tLPP edu.stanford.nlp.parser.lexparser.NegraPennTreebankParserParams -dep -hMarkov 1 -maxLength 40 -saveToSerializedFile germanDep.ser.gz -train $negra 1 -test $negra 3 ) >>& ./serializedParsers.log
+
+
+
+########
+# The languages below this line use TreebankPreprocessor for pre-processing prior to training
+########
+set mydir=`pwd`
+set data_dir=/u/nlp/data/lexparser/trees
+set tree_pipe=$JAVANLP_HOME/projects/core/scripts/run-tb-preproc
+set train_sh=$JAVANLP_HOME/projects/core/scripts/lexparser-lang-train-test.sh
+
+if( ! -e $data_dir ) then
+  mkdir $data_dir
+endif
+
+########
+# ARABIC
+########
+set ar_data_dir=$data_dir/Arabic
+set ar_conf_file=$JAVANLP_HOME/projects/core/src/edu/stanford/nlp/international/arabic/pipeline/configurations/atb-latest.conf
+set ar_train_args="Arabic 40 $ar_data_dir/2-Unvoc-All.utf8.txt $ar_data_dir/2-Unvoc-Dev.utf8.txt BASELINE_ar -saveToSerializedFile arabicFactored.ser.gz"
+
+if( ! -e $ar_data_dir ) then
+  mkdir $ar_data_dir
+endif
+
+echo Running $tree_pipe -p $ar_data_dir -v $ar_conf_file >>& ./serializedParsers.log
+$tree_pipe -p $ar_data_dir -v $ar_conf_file >& $ar_data_dir/build.log
+
+echo "" >>& ./serializedParsers.log
+( echo "Training Arabic Factored grammar using baseline feature set" ; time $train_sh $ar_train_args ) >>& ./serializedParsers.log
+
+
+########
+# FRENCH
+########
+set fr_data_dir=$data_dir/French
+set fr_conf_file=$JAVANLP_HOME/projects/core/src/edu/stanford/nlp/international/french/pipeline/configurations/ftb-latest.conf
+set fr_train_args="French 40 $fr_data_dir/FTB-All.utf8.txt $fr_data_dir/FTB-Dev.utf8.txt BASELINE_fr -saveToSerializedFile frenchFactored.ser.gz"
+
+if( ! -e $fr_data_dir ) then
+  mkdir $fr_data_dir
+endif
+
+echo Running $tree_pipe -p $fr_data_dir -v $fr_conf_file >>& ./serializedParsers.log
+$tree_pipe -p $fr_data_dir -v $fr_conf_file >& $fr_data_dir/build.log
+
+echo "" >>& ./serializedParsers.log
+echo time $train_sh $fr_train_args >>& ./serializedParsers.log
+( echo "Training French Factored grammar using baseline feature set" ; time $train_sh $fr_train_args ) >>& ./serializedParsers.log
+
+
+
+
+## English just to check parser code regression (not saved)
+
+## Just for reference
+( echo "Running wsjPCFG (acl03pcfg replication) on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -acl03pcfg -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+
+## See if same results from serialized parser
+( echo "Running wsjFactored (ijcai03 from serialized) on $host -server" ; time java -server -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -v -maxLength 40 -loadFromSerializedFile wsjFactored.ser.gz -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+# ( echo "Running wsjFactored (ijcai03 with nodeprune) on $host -server" ; time java -server -mx1800m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -ijcai03 -v -compactGrammar 0 -nodePrune true -maxLength 40 -train $wsjptb 200-2199 -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+
+## See if same results from text grammar parser
+( echo "Running wsjFactored (ijcai03 from textGrammar) on $host -server" ; time java -server -mx1500m edu.stanford.nlp.parser.lexparser.LexicalizedParser -evals "factDA,tsv" -v -maxLength 40 -loadFromTextFile wsjFactored.txt -testTreebank $wsjptb 2200-2219 ) >>& ./serializedParsers.log
+
+uptime >> serializedParsers.log
+
+mv -f serializedParsersPerformance.last serializedParsersPerformance.2ndlast
+mv -f serializedParsersPerformance.current serializedParsersPerformance.last
+echo -n "Parser run by $USER on " > serializedParsersPerformance.current
+date >> serializedParsersPerformance.current
+grep 'N: 253\|N: 393\|Done testing on treebank\|Running \| summary ' serializedParsers.log >> serializedParsersPerformance.current
+echo >> serializedParsersPerformance.current
+echo >> serializedParsersPerformance.current
+
+cat serializedParsersPerformance.current >> serializedParsersPerformance.txt
+
+cp -f serializedParsers.last serializedParsers.2ndlast
+cp -f serializedParsers.current serializedParsers.last
+cp -f serializedParsers.log serializedParsers.current

lng/L2SCA/stanford-parser-full-2014-01-04/bin/run-tb-preproc

@@ -0,0 +1,65 @@
+#!/usr/bin/env python
+#
+# Convenience script for running
+# edu.stanford.nlp.trees.treebank.TreebankPreprocessor.
+#
+# This package automatically generates the Arabic and French
+# parser training data from the respective source distributions.
+#
+# See the README for more details.
+#
+# author:  Spence Green
+##############################
+
+import sys
+from optparse import OptionParser
+import os
+import subprocess
+from time import sleep
+
+def run_treebank_pipeline(opts,conf_file):
+    cmd_line = 'java -Xmx%s -Xms%s edu.stanford.nlp.trees.treebank.TreebankPreprocessor' % (opts.jmem,opts.jmem)
+
+    if opts.verbose:
+        cmd_line = cmd_line + ' -v'
+
+    if opts.extra:
+        cmd_line = cmd_line + ' ' + opts.extra
+
+    if opts.output_path:
+        cmd_line = cmd_line + ' -p ' + opts.output_path
+
+    cmd_line = cmd_line + ' ' + conf_file
+
+    p = call_command(cmd_line)
+
+    while p.poll() == None:
+        out_str = p.stdout.readline()
+        if out_str != '':
+            print out_str[:-1]
+
+# TODO: this will not handle spaces in the input or output paths
+def call_command(command):
+    process = subprocess.Popen(command.split(' '), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+    return process
+
+def main():
+    usage = 'usage: %prog [opts] conf_file'
+    parser = OptionParser(usage=usage)
+    parser.add_option('-m','--java-mem',dest='jmem',default='500m',help='Set JVM memory heap size (e.g. 500m)')
+    parser.add_option('-v','--verbose',dest='verbose',action='store_true',default=False,help='Verbose mode')
+    parser.add_option('-o','--options',dest='extra',help='Pass options directly to TreebankPreprocessor')
+    parser.add_option('-p','--output-path',dest='output_path',help="Destination directory for the output")
+
+    (opts,args) = parser.parse_args()
+
+    if len(args) != 1:
+        parser.print_help()
+        sys.exit(-1)
+
+    conf_file = args[0]
+
+    run_treebank_pipeline(opts,conf_file)
+
+if __name__ == '__main__':
+    main()

lng/L2SCA/stanford-parser-full-2014-01-04/build.xml

@@ -0,0 +1,190 @@
+<!-- build.xml file for ant for JavaNLP -->
+
+<!-- A "project" describes a set of targets that may be requested
+     when Ant is executed.  The "default" attribute defines the
+     target which is executed if no specific target is requested,
+     and the "basedir" attribute defines the current working directory
+     from which Ant executes the requested task.  This is normally
+     set to the current working directory.
+-->
+
+<project name="JavaNLP" default="compile" basedir=".">
+
+  <property name="build.home"    value="${basedir}/classes"/>
+  <property name="build.tests"    value="${basedir}/classes"/>
+  <property name="docs.home"     value="${basedir}/docs"/>
+  <property name="src.home"      value="${basedir}/src"/>
+  <property name="javadoc.home"  value="${basedir}/javadoc"/>
+
+
+<!--  ==================== Compilation Control Options ==================== -->
+
+<!--
+
+  These properties control option settings on the Javac compiler when it
+  is invoked using the <javac> task.
+
+  compile.debug        Should compilation include the debug option?
+
+  compile.deprecation  Should compilation include the deprecation option?
+
+  compile.optimize     Should compilation include the optimize option?
+
+  compile.source       Source version compatibility
+
+  compile.target       Target class version compatibility
+
+-->
+
+  <property name="compile.debug"       value="true"/>
+  <property name="compile.deprecation" value="false"/>
+  <property name="compile.optimize"    value="true"/>
+  <property name="compile.source"      value="1.6" />
+  <property name="compile.target"      value="1.6" />
+
+
+
+
+<!-- ==================== All Target ====================================== -->
+
+<!--
+
+  The "all" target is a shortcut for running the "clean" target followed
+  by the "compile" target, to force a complete recompile.
+
+-->
+
+  <target name="all" depends="clean,compile"
+   description="Clean build and dist directories, then compile"/>
+
+
+
+<!-- ==================== Clean Target ==================================== -->
+
+<!--
+
+  The "clean" target deletes any previous "build" and "dist" directory,
+  so that you can be ensured the application can be built from scratch.
+
+-->
+
+  <target name="clean" description="Delete old classes">
+    <delete dir="${build.home}/edu"/>
+  </target>
+
+
+<!-- ==================== Classpath Targets ==================================== -->
+
+<!--
+
+  Sets the classpath for this project properly. We now always use the
+  lib dir within javanlp.
+
+-->
+
+  <target name="classpath" description="Sets the classpath">
+      <path id="compile.classpath">
+        <fileset dir="${basedir}">
+          <include name="*.jar"/>
+          <exclude name="stanford-parser*"/>
+        </fileset>
+      </path>
+  </target>
+
+
+
+
+
+<!-- ==================== Compile Target ================================== -->
+
+<!--
+
+  The "compile" target transforms source files (from your "src" directory)
+  into object files in the appropriate location in the build directory.
+  This example assumes that you will be including your classes in an
+  unpacked directory hierarchy under "/WEB-INF/classes".
+
+-->
+
+  <target name="compile" depends="prepare,classpath"
+   description="Compile Java sources">
+
+    <!-- Compile Java classes as necessary -->
+    <mkdir    dir="${build.home}"/>
+    <javac srcdir="${src.home}"
+          destdir="${build.home}"
+            debug="${compile.debug}"
+         encoding="utf-8" 
+      deprecation="${compile.deprecation}"
+         optimize="${compile.optimize}"
+	   source="${compile.source}"
+           target="${compile.target}"
+includeantruntime="false">
+        <classpath refid="compile.classpath"/>
+      <compilerarg value="-Xmaxerrs"/>
+      <compilerarg value="20"/>
+      <!-- <compilerarg value="-Xlint"/> -->
+    </javac>
+
+    <!-- Copy application resources -->
+<!--
+    <copy  todir="${build.home}/WEB-INF/classes">
+      <fileset dir="${src.home}" excludes="**/*.java"/>
+    </copy>
+-->
+
+  </target>
+
+
+<!-- ==================== Javadoc Target ================================== -->
+
+<!--
+
+  The "javadoc" target creates Javadoc API documentation for the Java
+  classes included in your application.  Normally, this is only required
+  when preparing a distribution release, but is available as a separate
+  target in case the developer wants to create Javadocs independently.
+
+-->
+
+  <target name="javadoc" depends="compile"
+   description="Create Javadoc API documentation">
+
+    <mkdir          dir="${javadoc.home}"/>
+    <javadoc sourcepath="${src.home}"
+                destdir="${javadoc.home}"
+              maxmemory="768m"
+                 author="true"
+                 source="1.6"
+                Overview="${src.home}/edu/stanford/nlp/overview.html"
+                Doctitle="Stanford JavaNLP API Documentation"
+             Windowtitle="Stanford JavaNLP API"
+           packagenames="*">
+      <bottom><![CDATA[<FONT SIZE=2><A HREF=\"http://nlp.stanford.edu\">Stanford NLP Group</A></FONT>]]></bottom>
+      <link href="http://java.sun.com/j2se/1.6.0/docs/api/"/>
+    </javadoc>
+
+  </target>
+
+
+<!-- ==================== Prepare Target ================================== -->
+
+<!--
+
+  The "prepare" target is used to create the "build" destination directory,
+  and copy the static contents of your web application to it.  If you need
+  to copy static files from external dependencies, you can customize the
+  contents of this task.
+
+  Normally, this task is executed indirectly when needed.
+
+-->
+
+  <target name="prepare">
+
+    <!-- Create build directories as needed -->
+    <mkdir  dir="${build.home}"/>
+
+  </target>
+
+</project>

lng/L2SCA/stanford-parser-full-2014-01-04/conf/atb-latest.conf

@@ -0,0 +1,209 @@
+###########################
+# Baseline ATB Newswire Datasets
+#
+# This file creates the three data sets used in the current
+# line of Arabic parsing research:
+#
+#   (1) Raw (no Bies mapping) / Unvocalized  ("Raw")
+#   (2) Bies + DT / Unvocalized              ("Unvoc")
+#   (3) Bies + DT / Vocalized                ("Voc")
+#   (4) Bies + DT / Unvocalized              ("NoDashTags")
+#       -No traces or phrasal tag decorations. For training the Berkeley parser.
+#
+# Note that "Bies + DT" refers to the enhancement to the Bies mappings
+# proposed by Kulick et al. (2006).
+#
+# The training/dev/test set is the "Mona Diab split" from the 2005 JHU
+# workshop on parsing Arabic dialects (Chiang et al., 2006).
+#
+#
+# IMPORTANT: All paths should reference the base Arabic data directory
+#
+#       /u/nlp/data/Arabic
+#
+###########################
+
+NAME=1 Raw Train
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/train
+OUTPUT_ENCODING=UTF8
+FLAT=true
+
+;;
+
+NAME=1 Raw Dev
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/dev
+OUTPUT_ENCODING=UTF8
+FLAT=true
+
+;;
+
+NAME=1 Raw Test
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/test
+OUTPUT_ENCODING=UTF8
+FLAT=true
+
+;;
+
+NAME=2 Unvoc All
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+
+;;
+
+NAME=2 Unvoc Train
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/train
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+
+;;
+
+NAME=2 Unvoc Dev
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/dev
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+
+;;
+
+NAME=2 Unvoc Test
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/test
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+
+;;
+
+NAME=3 Voc Train
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/with-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/with-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/with-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/train
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+LEXMAPPER=edu.stanford.nlp.international.arabic.pipeline.UnvocLexicalMapper
+FLAT=true
+
+;;
+
+NAME=3 Voc Dev
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/with-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/with-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/with-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/dev
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+LEXMAPPER=edu.stanford.nlp.international.arabic.pipeline.UnvocLexicalMapper
+FLAT=true
+
+;;
+
+NAME=3 Voc Test
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/with-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/with-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/with-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/test
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+USEDET=true
+LEXMAPPER=edu.stanford.nlp.international.arabic.pipeline.UnvocLexicalMapper
+FLAT=true
+
+;;
+
+
+NAME=4 Unvoc Train NoDashTags
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/train
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+NODASHTAGS=true
+ADDROOT=true
+USEDET=true
+
+;;
+
+NAME=4 Unvoc Dev NoDashTags
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/dev
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+NODASHTAGS=true
+ADDROOT=true
+USEDET=true
+
+;;
+
+NAME=4 Unvoc Test NoDashTags
+TYPE=edu.stanford.nlp.international.arabic.pipeline.ATBArabicDataset
+PATHp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/data/penntree/without-vowel
+PATHp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/data/penntree/without-vowel
+PATHp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/data/penntree/without-vowel
+SPLIT=/u/nlp/data/Arabic/splits/mona-diab-split/test
+OUTPUT_ENCODING=UTF8
+MAPPINGp1=/u/nlp/data/Arabic/ldc/atb-latest/p1/docs/atb1-v4.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp2=/u/nlp/data/Arabic/ldc/atb-latest/p2/docs/atb2-v3.0-taglist-conversion-to-PennPOS-forrelease.lisp
+MAPPINGp3=/u/nlp/data/Arabic/ldc/atb-latest/p3/docs/atb3-v3.1-taglist-conversion-to-PennPOS-forrelease.lisp
+NODASHTAGS=true
+ADDROOT=true
+USEDET=true
+
+;;

lng/L2SCA/stanford-parser-full-2014-01-04/conf/ftb-latest.conf

@@ -0,0 +1,44 @@
+###########################
+# Baseline FTB Datasets
+#
+# IMPORTANT: All paths should reference the base Arabic data directory
+#
+#       /u/nlp/data/FrenchTreebank/versionJune2010
+#
+###########################
+
+NAME=FTB All
+TYPE=edu.stanford.nlp.international.french.pipeline.FTBDataset
+PATH=/u/nlp/data/FrenchTreebank/versionJune2010/corpus-fonctions
+OUTPUT_ENCODING=UTF8
+TVISITOR=edu.stanford.nlp.international.french.pipeline.FTBCorrectorVisitor
+FLAT=true
+
+;;
+
+NAME=FTB Train
+TYPE=edu.stanford.nlp.international.french.pipeline.FTBDataset
+PATH=/u/nlp/data/FrenchTreebank/versionJune2010/corpus-fonctions
+SPLIT=$JAVANLP_HOME/projects/core/src/edu/stanford/nlp/international/french/pipeline/splits/candito.train
+OUTPUT_ENCODING=UTF8
+TVISITOR=edu.stanford.nlp.international.french.pipeline.FTBCorrectorVisitor
+
+;;
+
+NAME=FTB Dev
+TYPE=edu.stanford.nlp.international.french.pipeline.FTBDataset
+PATH=/u/nlp/data/FrenchTreebank/versionJune2010/corpus-fonctions
+SPLIT=$JAVANLP_HOME/projects/core/src/edu/stanford/nlp/international/french/pipeline/splits/candito.dev
+OUTPUT_ENCODING=UTF8
+TVISITOR=edu.stanford.nlp.international.french.pipeline.FTBCorrectorVisitor
+
+;;
+
+NAME=FTB Test
+TYPE=edu.stanford.nlp.international.french.pipeline.FTBDataset
+PATH=/u/nlp/data/FrenchTreebank/versionJune2010/corpus-fonctions
+SPLIT=$JAVANLP_HOME/projects/core/src/edu/stanford/nlp/international/french/pipeline/splits/candito.test
+OUTPUT_ENCODING=UTF8
+TVISITOR=edu.stanford.nlp.international.french.pipeline.FTBCorrectorVisitor
+
+;;

lng/L2SCA/stanford-parser-full-2014-01-04/data/arabic-onesent-utf8.txt

@@ -0,0 +1 @@
+و نشر العدل من خلال قضاء مستقل .

lng/L2SCA/stanford-parser-full-2014-01-04/data/chinese-onesent-gb18030.txt

@@ -0,0 +1 @@
+���� ϣ�� ���� û�� ���� ������ �ƻ� ��

lng/L2SCA/stanford-parser-full-2014-01-04/data/chinese-onesent-unseg-gb18030.txt

@@ -0,0 +1 @@
+����Ժ��ǰ��������֪ͨ��Ҫ�������ʵ��ʵ��֤�г���Ӧ�ĸ������ߣ�ά����ʳƷ�۸��ȶ���

lng/L2SCA/stanford-parser-full-2014-01-04/data/chinese-onesent-unseg-utf8.txt

@@ -0,0 +1 @@
+国务院日前发出紧急通知，要求各地切实落实保证市场供应的各项政策，维护副食品价格稳定。

lng/L2SCA/stanford-parser-full-2014-01-04/data/chinese-onesent-utf8.txt

@@ -0,0 +1 @@
+俄国 希望 伊朗 没有 制造 核武器 计划 。

lng/L2SCA/stanford-parser-full-2014-01-04/data/english-onesent.txt

@@ -0,0 +1 @@
+The quick brown fox jumped over the lazy dog.

lng/L2SCA/stanford-parser-full-2014-01-04/data/french-onesent.txt

@@ -0,0 +1 @@
+Le gouvernement se résout donc à renvoyer la balle dans le camp de partenaires qui ont amplement fait la preuve de leur incapacité à gérer le système de santé .

lng/L2SCA/stanford-parser-full-2014-01-04/data/german-onesent.txt

@@ -0,0 +1 @@
+Wir haben nichts zu tun .

lng/L2SCA/stanford-parser-full-2014-01-04/data/pos-sentences.txt

@@ -0,0 +1,7 @@
+People can butter their bread with a knife .
+People can butter/VB their bread with a knife .
+People can butter/NN their bread with a knife .
+People/NNS can/MD butter/VB their/PRP$ bread/NN with/IN a/DT knife/NN ./.
+People/NNS can/VB butter/NN their/PRP$ bread/NN with/IN a/DT knife/NN ./.
+People/NNS can/NN butter/NN their/PRP$ bread/NN with/IN a/DT knife/NN ./.
+People/NN can/NN butter/NN their/NN bread/NN with/NN a/NN knife/NN ./NN

lng/L2SCA/stanford-parser-full-2014-01-04/data/testsent.txt

@@ -0,0 +1,10 @@
+Scores of properties are under extreme fire threat as a huge blaze
+continues to advance through Sydney's north-western suburbs. Fires
+have also shut down the major road and rail links between Sydney and
+Gosford.
+
+The promotional stop in Sydney was everything to be expected for a
+Hollywood blockbuster - phalanxes of photographers, a stretch limo to
+a hotel across the Quay - but with one difference. A line-up of
+masseurs was waiting to take the media in hand. Never has the term
+"massaging the media" seemed so accurate.

lng/L2SCA/stanford-parser-full-2014-01-04/ejml-0.23.jar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0250933fe8cc6a44eb098016d4dadaba7746a27efc3d5a7f4f4c9bf247cfe09
+size 211938

lng/L2SCA/stanford-parser-full-2014-01-04/lexparser-gui.bat

@@ -0,0 +1,3 @@
+:: runs the parser GUI
+:: usage lexparser-gui [parserDataFilename [textFileName]]
+java -mx800m -cp "*" edu.stanford.nlp.parser.ui.Parser

lng/L2SCA/stanford-parser-full-2014-01-04/lexparser-gui.command

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+#
+# Runs the Lexicalized Parser GUI.  You can just run this script and then
+# load a grammar and file to be parsed from the menus or you can specify
+# them on the command line.
+#
+# Usage: ./lexparser-gui.sh [parserDataFilename [textFileName]]
+#
+
+
+scriptdir=`dirname $0`
+
+java -mx800m -cp "$scriptdir/*" edu.stanford.nlp.parser.ui.Parser $*

lng/L2SCA/stanford-parser-full-2014-01-04/lexparser-gui.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+#
+# Runs the Lexicalized Parser GUI.  You can just run this script and then
+# load a grammar and file to be parsed from the menus or you can specify
+# them on the command line.
+#
+# Usage: ./lexparser-gui.sh [parserDataFilename [textFileName]]
+#
+
+
+scriptdir=`dirname $0`
+
+java -mx800m -cp "$scriptdir/*" edu.stanford.nlp.parser.ui.Parser $*

lng/L2SCA/stanford-parser-full-2014-01-04/lexparser-lang-train-test.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+#
+# Defines standard configurations for training and evaluating the
+# multilingual parsers (Arabic, Chinese, German, French). You can
+# also train and test the English parsers with this script.
+#
+# For details on the language-specific options, see the javadocs and
+# lexparser_lang.def.
+#
+
+# Memory limit
+mem=6g
+
+if [ ! $# -ge 5 ]; then
+   echo Usage: `basename $0` lang len train_file test_file out_file features
+   echo
+   echo '  lang       : Language to parse (Arabic, English, Chinese, German, French)'
+   echo '  len        : Maximum length of the sentences to parse'
+   echo '  train_file : Training treebank file'
+   echo '  test_file  : Test treebank file (for evaluation)'
+   echo '  out_file   : Prefix for the output filename'
+   echo '  features   : Variable length list of optional parser features'
+   echo
+   echo 'Parser memory limit is currently:' "$mem"
+   echo   
+   exit
+fi
+
+# Setup command-line options
+lang=$1
+len=$2
+train_path=$3
+test_file=$4
+out_file=$5
+
+shift 5
+
+# Language-specific configuration
+scriptdir=`dirname $0`
+echo $JAVANLP_HOME
+source $JAVANLP_HOME/projects/core/scripts/lexparser_lang.def
+
+# Setting classpath
+#CLASSPATH="$CLASSPATH":"$scriptdir/*"
+
+# Run the Stanford parser
+java -Xmx"$mem" -cp "$scriptdir/*:$CLASSPATH" edu.stanford.nlp.parser.lexparser.LexicalizedParser -maxLength "$len" \
+-tLPP "$tlp" $lang_opts $* -writeOutputFiles \
+-outputFilesExtension "$out_file"."$len".stp -outputFormat "penn" \
+-outputFormatOptions "removeTopBracket,includePunctuationDependencies" -train "$train_path" -test "$test_file"