<commit_before>.\" <NAME>
.TH relit 1 "8 April 2016"
.hw relit
.SH NAME
relit \- Reverse literate programming tool
.SH SYNOPSIS
.B relit 
[\fB-a\fR]
[\fB-f\fR]
[\fIdefinitions...\fR]
[\fB--\fR]
.IR sourcefilenames...
.SH DESCRIPTION
.B relit
is a 'reverse literate programming' tool. It is designed to process LaTeX and TeX files (or troff, etc) and flexibly extract and assemble complete source code. Reverse literate programming encourages putting the entire compilable source code in publishable documents (such as peer reviewed publications), in any suitable order, and perhaps with unit tests and other support (e.g., makefiles) in the same file but hidden from the reader. 
.SH OPTIONS
Arguments can be in any order; the files are always processed last. Within files, definitions and uses of names can be in any order; name definitions do not need to be in the files where they are used.
.TP
-a 
List all definitions
.TP
-f 
List all files processed to standard output
.TP
\fIname\fR=\fIvalue\fR
Define a name. If there are spaces in the \fIvalue\fR it must be enclosed in quote marks.
.TP
--
Treat all subsequent parameters as file names, following the same convention as \fBrm\fR (1).
.SH DESCRIPTION
The
.B relit 
tool generally processes TeX or LaTeX files.
.LP
% define 
.I name 
.I re
, 
.I re 
[ ,
.I tag
]
.IP
Define 
.I name 
to be the following lines inclusive that match between the two regular expressions.
.LP
% generate 
.I name 
.I re
, 
.I re 
[
.I tag
]
.IP
Generate a file, 
.I name 
to be the following lines inclusive that match between the two regular expressions.
.IP
Within the text matched for %define and %generate, names of the form <\fIname\fR> are expanded by their definitions. 
.LP
% set-tag 
.I value
.IP
Define a default tag for any subsequent %define or %generate without an explicit tag. A tag value can 
include <\fInames\fR>. By using <\fIname\fR> as a tag, the name, and hence the tag, can be set in the command line. Default tags can be changed repeatedly as required.
.PP
Where \fIre\fR can have the form:
.TP
\&. 
Match the current line (i.e., if the first \fIre\fR, the immediately next line after the % command, and if the second \fIre\fR, the line the first \fIre\fR matches).
.TP
/ \fIregular expression\fR / 
Match the regular expression, starting the search on the line the meaning of '.' (see above). See \fBre_format\fR (7).
.LP
REs can be followed by +\fInumber\fR or -\fInumber\fR, to adjust the line match (however it is an error to search backwards).
.SH EXAMPLE
.nf
%generate t.c /demo/, /define/-1
<demo>
%define demo ., /Hello/, Boo!
Hello world
.fi
.PP
will create a file t.c containing "Hello world" and a file t.c-tagged.txt (which would not be created if there were no tags) containing "Boo!Hello world".
.PP
Typically, a %generate command, placed after LaTeX's \\end{document}, will be used to create a makefile to keep track of how to make, compile and unit test the source code files. Typically, tags will be \\seen{} and \\unseen{}, appropriately defined.
.SH BUGS
Regular expressions cannot contain /
.PP
% cannot be changed (it would be nice to be able to process makefiles or C programs too).
.PP
On second thoughts, if the \fBrelit\fR commands were in this format:
.TP
\\relit{command name re, re, tag} - say, in the original format
they would look more like Latex. In normal use, the command could be defined to disappear, but if the author wished it could be defined like \\marginpar, so that the author can more easily keep track of where the commands are used. Perhaps we should have both? The advantage of the % notation is that it requires no new definitions to start using it.<commit_msg>Added files via upload<commit_after>.\" <NAME>
.TH relit 1 "26 April 2016"
.hw relit
.SH NAME
relit \- Reverse literate programming tool
.SH SYNOPSIS
.B relit 
[\fB-a\fR]
[\fB-f\fR]
[\fB-n\fR]
[\fB-v\fR]
[\fIdefinitions...\fR]
[\fB--\fR]
.IR sourcefilenames...
.SH DESCRIPTION
.B relit
is a 'reverse literate programming' tool. It is designed to process LaTeX and TeX files (or potentially troff, etc) and flexibly extract and assemble complete source code. Reverse literate programming encourages putting the entire compilable source code in publishable documents (such as peer reviewed publications), in any suitable order, and perhaps with unit tests and other support (e.g., makefiles) in the same file but hidden from the reader. 
.SH OPTIONS
Arguments can be in any order; the files are always processed last. Within files, definitions and uses of names can be in any order; name definitions do not need to be in the files where they are used.
.TP
-a 
List all definitions (overrides -n)
.TP
-f 
List all files generated to standard output
.TP
-n 
List all names (more concise output than -a)
.TP
-v
Verbose mode (show all \fBrelit\fR commands)
.TP
\fIname\fR=\fIvalue\fR
Define a name. If there are spaces in the \fIvalue\fR it must be enclosed in quote marks.
.TP
--
Treat all subsequent parameters as file names, following the same convention as \fBrm\fR (1).
.SH DESCRIPTION
The
.B relit 
tool generally processes TeX or LaTeX files. Commands are written either as comments (as text following the % character, so they are completely ignored by LaTeX) or they can be written as ordinary LaTeX commands, which LaTeX (or TeX) can then also process for any further purposes in the usual way. In normal use, \\relit can be defined to disappear, but if the author wishes it can be defined like \\marginpar, so that the author can easily keep track of where relit commands are used. The advantage of the % notation is that it requires no new LaTeX definitions to start using it immediately.
.LP
% define \fIname\fR \fIre\fR, \fIre\fR [, \fItag\fR]
\fBor\fR
\\relit[\fItag\fR]{define \fIname\fR \fIre\fR, \fIre\fR}
.IP
Define 
.I name 
to be the following lines that match between the two regular expressions inclusive. The name can be optionally tagged. (In the LaTeX form, if the tag is specified, the square brackets are required.)
.LP
% generate \fIname\fR \fIre\fR, \fIre\fR [, \fItag\fR]
\fBor\fR
\\relit[\fItag\fR]{generate \fIname\fR \fIre\fR, \fIre\fR}
.IP
Generate a file called 
.I name 
to be the following lines that match between the two regular expressions inclusive.
.IP
Within the text matched for %define and %generate, names of the form <\fIname\fR> are expanded by their definitions. 
.LP
% set-tag \fIvalue\fR
\fBor\fR
\\relit{set-tag \fIvalue\fR}
.IP
Define a default tag for any subsequent defines or generates without an explicit tag. A tag value can 
include <\fInames\fR>. By using <\fIname\fR> as a tag, the name, and hence the tag, can be set in the command line. Default tags can be changed repeatedly as required.
.PP
Where \fIre\fR can have the form:
.TP
\&. 
Match the current line (i.e., if the first \fIre\fR, the immediately next line after the % command, and if the second \fIre\fR, the line the first \fIre\fR matches). Hence the pair .,. means the next line. However, in the case of the \\relit command form, the first . includes all text from the closing } to the end of line: hence .+1,. would mean the next line.
.TP
/ \fIregular expression\fR / 
Match the regular expression, starting the search on the line the meaning of '.' (see above). See \fBre_format\fR (7).
.LP
REs can be followed by +\fInumber\fR or -\fInumber\fR, to adjust the line match.
.SH EXAMPLE
.nf
%generate t.c /demo/, /define/-1
<demo>
%define demo ., /Hello/, Boo!
Hello world
.fi
.PP
will create a file t.c containing "Hello world" and a file t.c-tagged.txt (which would not be created if there were no tags) containing "Boo!Hello world".
.PP
Typically, a %generate command, placed after LaTeX's \\end{document}, will be used to create a makefile to keep track of how to make, compile and unit test the source code files. Typically, tags will be \\seen{} and \\unseen{}, appropriately defined.
.SH BUGS
Regular expressions cannot contain / (or } in the \\relit form)
.PP
The fixed strings, and % and \\relit in particular, cannot be changed (it might be nice to be able to process files using other conventions, such as makefiles or C programs, too).
.SH AUTHOR
<NAME>, <EMAIL>, 2016
{'fraction_non_alphanumeric': 0.08424145953226399, 'fraction_numerical': 0.0029386555650789763, 'mean_word_length': 3.5965109735509286, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6209936', 'n_tokens_mistral': 2596, 'n_tokens_neox': 2443, 'n_words': 1303}