<commit_before>.TH GREP 1
.SH NAME
grep \- search the channel logs
.SH SYNOPSIS
.B grep
[ -ci ] [ -E | -P | -F ] [ -m count ] [ -f channel ]
[ -C context_lines | -A context_lines_after -B context_lines_before ]
[ -t yyyy-MM-dd ] [ -T yyyy-MM-dd ]
[--]
.IR search_string...
.SH DESCRIPTION
.B grep
searches the logs for a channel or user based on the search string. It then returns up to the first 500 results in chronological order. If there is only one line of output it is returned inline, otherwise the contents are posted to dpaste.
.SH DATA FORMAT
UNIX_TIMESTAMP NICK USERNAME HOSTNAME ("message" | "action" | "notice") MESSAGE_CONTENT

The fields are separated by a "\\r":
.SH OPTIONS
.IP -c
Output the number of matching lines instead of the text found.
.IP -i
Makes the search case-insensitive
.IP -E
Extended regex mode. Makes special characters behave as you are used to. See the real grep man page for more details.
.IP -P
Perl regex mode for the lulz. See real grep for more info.
.IP -F
Search for literal matches of the provided string.
.IP "-m count"
Only return at most count lines.
.IP "-f channel"
The channel to search the logs for. Does not support searching other users' logs.
.IP "-C context_lines"
Return context_lines number of lines before and after each match. The sections surrounding each match will be separated by lines of "---". If two context sections overlap they will merge together. -C takes precedence over -A and -B.
.IP "-A context_lines_after"
Return context_lines_after number of lines after each matched line.
.IP "-B context_lines_before"
Return context_lines_before number of lines before each matched line.
.IP "-t yyyy-MM-dd"
Only return matches from log files dated after yyyy-MM-dd. The first two numbers of the year can be ommited and any of the year, month or date can be replaced with a "*" to match any year, month or date respectively.
.IP "-T yyyy-MM-dd"
Only return matches from log files dated before yyyy-MM-dd. The first two numbers of the year can be ommited and any of the year, month or date can be replaced with a "*" to match any year, month or date respectively.
.IP search_string
All of the remaining text is passed as the regex to grep. Note that it does not handle or need quotes at this point(this will be fixed eventually).
.SH EXAMPLES
To search for every line containing Wednesday from the current channel's logs, case insensitive:
.PP
.nf
.RS
\&./grep -i Wednesday
.RE
.fi
.PP
If you also want the line before and two lines afterwards you can do:
.PP
.nf
.RS
\&./grep -i -B 1 -A 2 Wednesday
.RE
.fi
.PP
To only get the lines from a specific user you can do something of the form:
.PP
.nf
.RS
\&./grep -i -B 1 -A 2 ^[0-9]\\+\\rnickname\\r.*Wednesday
.RE
.fi
.PP
There is a lot going on above. First and foremost \\r searches for the record separator in the data. So we have used this to specify that we want the nickname field to contain nickname.
.PP
Secondly note how + needed to be escaped. The default matching mode requires you to escape the following characters unless you want them to match literally:
.PP
.nf
.RS
? + { | ( )
.RE
.fi
.PP
If you don't want to have to escape special characters use the -E option, which enables egrep style matching.
.PP
Finally grep does not have a \\d expression for digits so you have to do something like [0-9].
.SS Searching for Numbers
To search for a specific number in the logs you will probably have to use something like the following:
.PP<commit_msg>adding missing special character<commit_after>.TH GREP 1
.SH NAME
grep \- search the channel logs
.SH SYNOPSIS
.B grep
[ -ci ] [ -E | -P | -F ] [ -m count ] [ -f channel ]
[ -C context_lines | -A context_lines_after -B context_lines_before ]
[ -t yyyy-MM-dd ] [ -T yyyy-MM-dd ]
[--]
.IR search_string...
.SH DESCRIPTION
.B grep
searches the logs for a channel or user based on the search string. It then returns up to the first 500 results in chronological order. If there is only one line of output it is returned inline, otherwise the contents are posted to dpaste.
.SH DATA FORMAT
UNIX_TIMESTAMP NICK USERNAME HOSTNAME ("message" | "action" | "notice") MESSAGE_CONTENT

The fields are separated by a "\\r":
.SH OPTIONS
.IP -c
Output the number of matching lines instead of the text found.
.IP -i
Makes the search case-insensitive
.IP -E
Extended regex mode. Makes special characters behave as you are used to. See the real grep man page for more details.
.IP -P
Perl regex mode for the lulz. See real grep for more info.
.IP -F
Search for literal matches of the provided string.
.IP "-m count"
Only return at most count lines.
.IP "-f channel"
The channel to search the logs for. Does not support searching other users' logs.
.IP "-C context_lines"
Return context_lines number of lines before and after each match. The sections surrounding each match will be separated by lines of "---". If two context sections overlap they will merge together. -C takes precedence over -A and -B.
.IP "-A context_lines_after"
Return context_lines_after number of lines after each matched line.
.IP "-B context_lines_before"
Return context_lines_before number of lines before each matched line.
.IP "-t yyyy-MM-dd"
Only return matches from log files dated after yyyy-MM-dd. The first two numbers of the year can be ommited and any of the year, month or date can be replaced with a "*" to match any year, month or date respectively.
.IP "-T yyyy-MM-dd"
Only return matches from log files dated before yyyy-MM-dd. The first two numbers of the year can be ommited and any of the year, month or date can be replaced with a "*" to match any year, month or date respectively.
.IP search_string
All of the remaining text is passed as the regex to grep. Note that it does not handle or need quotes at this point(this will be fixed eventually).
.SH EXAMPLES
To search for every line containing Wednesday from the current channel's logs, case insensitive:
.PP
.nf
.RS
\&./grep -i Wednesday
.RE
.fi
.PP
If you also want the line before and two lines afterwards you can do:
.PP
.nf
.RS
\&./grep -i -B 1 -A 2 Wednesday
.RE
.fi
.PP
To only get the lines from a specific user you can do something of the form:
.PP
.nf
.RS
\&./grep -i -B 1 -A 2 ^[0-9]\\+\\rnickname\\r.*Wednesday
.RE
.fi
.PP
There is a lot going on above. First and foremost \\r searches for the record separator in the data. So we have used this to specify that we want the nickname field to contain nickname.
.PP
Secondly note how + needed to be escaped. The default matching mode requires you to escape the following characters unless you want them to match literally:
.PP
.nf
.RS
? + { } | ( )
.RE
.fi
.PP
If you don't want to have to escape special characters use the -E option, which enables egrep style matching.
.PP
Finally grep does not have a \\d expression for digits so you have to do something like [0-9].
.SS Searching for Numbers
To search for a specific number in the logs you will probably have to use something like the following:
.PP
{'fraction_non_alphanumeric': 0.0731426929156488, 'fraction_numerical': 0.003448771375197586, 'mean_word_length': 3.458680333119795, 'pattern_counts': {'":': 2, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '881560', 'n_tokens_mistral': 2089, 'n_tokens_neox': 2012, 'n_words': 1170}