<commit_before><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Introduction</title>
</head>
<body>

<h1>What is it?</h1>
<p>It is a library that allows the parameter names of methods and constructors to be accessed at runtime. Normally this information is dropped by the compiler. In effect, methods like '<span
  style="font-family: monospace;">doSometing(mypkg.Person <strong>toMe</strong>)</span>'
currently look like '<span style="font-family: monospace;">doSomething(mypkg.Person
<strong>???</strong>)</span>' to people using Java's reflection to inspect methods.&nbsp; </p>
<p>To
  date parameter name access has not been very useful to Java application developers, but
  with the advent of advanced scripting languages and web action
  frameworks for the JVM it is of increasing importance to be able to
  leverage a method's parameter names. Scripting languages like <a
  href="http://groovy.codehaus.org/">Groovy</a> &amp; <a
  href="http://jruby.codehaus.org/">JRuby</a>, web action frameworks
  like <a href="http://waffle.codehaus.org">Waffle</a> and <a href="#"
  title="http://www.vraptor.org/">VRaptor</a> (that verge on the
  tranparent) and the compelling <a href="http://grails.codehaus.org/">Grails</a>. SOAP and REST designs could also benefit.</p>
<p>ParaNamer allows you to generate and use parameter name info for
versions of Java prior to JDK 7.0.
Parameter name access was scheduled for JDK 6.0, bit slipped as the development team ran out of time to implement it. It is important to note that there is no guarantee that it will get into JDK 7.0 either, and that if it does it may bot be simple to use.</p>
<p>Paranamer is Open Source, and licensed as BSD. It is compatible with commercial/proprieatry, GPL, Apache use.<br>
</p>
<h1>Accessing Parameter Name data</h1>
<p>There is a
method called <span class="code-object">'lookupParameterNames</span>' that returns an array of strings for a method or constructor.</p>
<div class="code">
  <div class="codeHeader"><b>MySomethingOrOther.java</b></div>
  <div class="codeContent">
    <pre class="code-java"><span
  class="code-object">
 Method method = Foo.class.getMethod(...);<br>
 Paranamer paranamer = new CachingParanamer();<br> <br> String[] parameterNames = paranamer.lookupParameterNames(method)</span><br></pre>
  </div>
</div>
<p>The method throws a ParameterNamesNotFoundException if no parameter names
are found. </p>
<p>Instead of handling that exception, you could determine if parameter name data is available, but first calling areParameterNamesAvailable().<br>
</p>
<p>ParaNamer does not have any runtime jar dependecies while looking
  up parameter info previously generated that's been zipped into a jar.</p>
<h2>DefaultParanamer</h2>
<p>DefaultParanamer tries to read parameter name data from an extra static field on the class. This field need to be added after compilation of the class, and before you put the resulting classes in a jar.</p>
<p>The static field essentially looks like -</p>
<div class="code">
<div class="codeContent">
<pre class="code-java">    private static final String __PARANAMER_DATA = "v1.0 \n"<br>        + "&lt;init&gt; \n"<br>        + "&lt;init&gt; com.foo.Bar bar \n"<br>        + "toString \n"<br>        + "setName java.lang.String,java.lang.String givenName,familyName \n";<br>        + "setDateOfBirth int,int,int day,month,year \n";</pre></div>
</div>
<p>Clearly the method's source needs to be analysed and lines added per
method to that __PARANAMER_DATA field.&nbsp;&nbsp; See below.</p>
<h2>BytecodeReadingParanamer</h2>
<p>If generating meta data for parameter names at compile time
  is not for you, try class BytecodeReadingParanamer as a runtime only solution. This uses a cut down forked version of ASM to extract debug information from a class at runtime. As it happens this is the fallback implementation for  CachingParanamer when DefaultParanamer reports that there is no meta data for a class.</p>
<h2> CachingParanamer</h2>
<p>CachingParanamer  stores the results of each parameter name lookup, so that second and subsequent invocations will be far quicker.</p>
<p>Its also interesting to note that CachingParanamertries tries to
  use DefaultParanamer and BytecodeReadingParanamer. The latter is a fallback strategy if the former cannot find
  parameter name data.</p>
<h1>Creating Parameter Name data</h1>
<h2>Optional Generation of Parameter Name data with Ant<br>
</h2>
<p>This for DefaultParanamer of course.</p>
<p>You need to download:</p>
<ul>
  <li><a
    href="http://repository.codehaus.org/com/thoughtworks/paranamer/paranamer/">latest
  paranamer jar</a></li>
  <li><a
    href="http://repository.codehaus.org/com/thoughtworks/paranamer/paranamer-generator/">latest
  paranamer-generator jar</a></li>
  <li><a
    href="http://repository.codehaus.org/com/thoughtworks/paranamer/paranamer-ant/">latest
  paranamer-ant jar</a></li>
  <li><a href="http://repo1.maven.org/maven2/qdox/qdox/">latest
  qdox jar</a> (1.6 or above)</li>
</ul>
and declare in your Ant script the following after &lt;javac/&gt;
(remember to add to the taskdef classpath all the above jars):
<br>
<div class="code">
<div class="codeContent"><pre class="code-java">&lt;taskdef name="paranamer" <br>       classname="com.thoughtworks.paranamer.ant.ParanamerGeneratorTask"/&gt;<br>&lt;paranamer sourceDirectory="src/java" outputDirectory="target/classes"/&gt;<br></pre>
</div>
</div>
<p>Classes are changed to have an extra static String member variable that contains the member functions and their parameter names. Be sure to zip them up in to you final jar.<br>
</p>
<h2>Optional Generation of Parameter Name data with Maven 2<br>
</h2>
For Maven, it is simpler. Just add this to the
&lt;build&gt;&lt;plugins&gt; section of your pom.xml:
<br>
<br>
<div class="code">
<div class="codeContent"><pre style="margin-left: 40px;">&lt;plugin&gt;<br>&nbsp;&nbsp;&nbsp; &lt;groupId&gt;com.thoughtworks.paranamer&lt;/groupId&gt;<br>&nbsp;&nbsp;&nbsp; &lt;artifactId&gt;paranamer-maven-plugin&lt;/artifactId&gt;<br>&nbsp;&nbsp; &nbsp;&lt;executions&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;execution&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;id&gt;run&lt;/id&gt;  &lt;!-- id is optional --&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;configuration&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;sourceDirectory&gt;<span
  style="font-weight: bold;">${basedir}/src/java</span>&lt;/sourceDirectory&gt;<br>&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;outputDirectory&gt;${basedir}/target/classes&lt;/outputDirectory&gt;<br>&nbsp;&nbsp;&nbsp;  &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;/configuration&gt;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;<br> &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;goals&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;goal&gt;run&lt;/goal&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;/goals&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;/execution&gt;<br>&nbsp;&nbsp;&nbsp; &lt;/executions&gt;<br>&lt;/plugin&gt;</pre>
</div>
</div>
<br>
The jar file will automatically be made with parameter name data.<br>
<h2>Using Paranamer in your application without depending on Paranamer's jar</h2>
<p>There are already too many jar's for day to day Java development
right?&nbsp; Simply consume the runtime paranamer jar into your project's jar using the Maven2 'shade' plugin.<br>
</p>
<div class="code">
<div class="codeHeader"><b>Put in your POM.xml file</b></div>
<div class="codeContent"><pre class="code-java"><span
  class="code-object"> &lt;build&gt;<br>    &lt;plugins&gt;<br>      &lt;plugin&gt;<br>        &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>        &lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;<br>        &lt;executions&gt;<br>          &lt;execution&gt;<br>            &lt;phase&gt;package&lt;/phase&gt;<br>            &lt;goals&gt;<br>              &lt;goal&gt;shade&lt;/goal&gt;<br>            &lt;/goals&gt;<br>            &lt;configuration&gt;<br>              &lt;shadedArtifactId&gt;artifact-shaded&lt;/shadedArtifactId&gt;<br>              &lt;relocations&gt;<br>                &lt;relocation&gt;<br>                  &lt;pattern&gt;com.thoughtworks.paranamer&lt;/pattern&gt;<br>                  &lt;shadedPattern&gt;com.yourcompany.yourapp.paranamer&lt;/shadedPattern&gt;<br>                &lt;/relocation&gt;<br>              &lt;/relocations&gt;<br>            &lt;/configuration&gt;<br>          &lt;/execution&gt;<br>        &lt;/executions&gt;<br>      &lt;/plugin&gt;<br>    &lt;/plugins&gt;<br>  &lt;/build&gt;<br> </span><br></pre>
</div>
</div>

<h1>How Paranamer works</h1>

<h2>What if a parameter names changes?</h2>
<p>The general answer to this question is that you should not ship something to third parties where they are going to hard-core your parameter names into their application. For you own in-house stuff, accessing parameter names is harmless. You should be able to ripple though the source code of even large applications, and change say &quot;badSpeeldWord&quot; to &quot;badlySpelledWorld&quot; if you need to.</p>
<p>In a later version we'll store previous parameter names too. It won't be
  be magic, you'll have to code the previous parameter names in your
  source with a doclet tag. The lookup mechanism will change too.
   It will likely leverage a doclet tag.<br>
</p>
<h1>ParaNamer's Future</h1>
<p>ParaNamer will go away in the future when Java gets Reflection
access to Method parameter names. That should look like:<br>
</p>
<div class="code">
<div class="codeContent"><pre style="margin-left: 40px;"
  class="code-java">Method method = Foo.class.getMethod(....)<br>String[] parameterNames = method.getParameterNames();<br></pre>
</div>
<p>But for now, we are thinking of the following directions :-</p>
<ol>
  <li>Plugins for Intellij IDEA, Eclipse, NetBeans to make
  parameter name data as part of their compilation processes</li>
  <li>Storage and retireval of previous parameter names</li>
</ol>
</div>

<h1>Projects using it</h1>
<a href="http://oval.sourceforge.net/">Oval - Object Validation Framework</a>,<br>
<a href="http://vraptor.org/">VRaptor - Web Framework</a>,<br>
<a href="http://waffle.codehaus.org/">Waffle - Web Framework</a><br>
<h1>Project Information</h1>
<p>The unit tests for ParaNamer illustrate some more way to use
it: <a
  href="http://svn.paranamer.codehaus.org/paranamer/trunk/paranamer/src/test/com/thoughtworks/paranamer/ParanamerExampleTestCase.java">http://svn.paranamer.codehaus.org/paranamer/trunk/paranamer/src/test/com/thoughtworks/paranamer/ParanamerExampleTestCase.java</a></p><commit_msg>one more project using this<commit_after><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Introduction</title>
</head>
<body>

<h1>What is it?</h1>
<p>It is a library that allows the parameter names of methods and constructors to be accessed at runtime. Normally this information is dropped by the compiler. In effect, methods like '<span
  style="font-family: monospace;">doSometing(mypkg.Person <strong>toMe</strong>)</span>'
currently look like '<span style="font-family: monospace;">doSomething(mypkg.Person
<strong>???</strong>)</span>' to people using Java's reflection to inspect methods.&nbsp; </p>
<p>To
  date parameter name access has not been very useful to Java application developers, but
  with the advent of advanced scripting languages and web action
  frameworks for the JVM it is of increasing importance to be able to
  leverage a method's parameter names. Scripting languages like <a
  href="http://groovy.codehaus.org/">Groovy</a> &amp; <a
  href="http://jruby.codehaus.org/">JRuby</a>, web action frameworks
  like <a href="http://waffle.codehaus.org">Waffle</a> and <a href="#"
  title="http://www.vraptor.org/">VRaptor</a> (that verge on the
  tranparent) and the compelling <a href="http://grails.codehaus.org/">Grails</a>. SOAP and REST designs could also benefit.</p>
<p>ParaNamer allows you to generate and use parameter name info for
versions of Java prior to JDK 7.0.
Parameter name access was scheduled for JDK 6.0, bit slipped as the development team ran out of time to implement it. It is important to note that there is no guarantee that it will get into JDK 7.0 either, and that if it does it may bot be simple to use.</p>
<p>Paranamer is Open Source, and licensed as BSD. It is compatible with commercial/proprieatry, GPL, Apache use.<br>
</p>
<h1>Accessing Parameter Name data</h1>
<p>There is a
method called <span class="code-object">'lookupParameterNames</span>' that returns an array of strings for a method or constructor.</p>
<div class="code">
  <div class="codeHeader"><b>MySomethingOrOther.java</b></div>
  <div class="codeContent">
    <pre class="code-java"><span
  class="code-object">
 Method method = Foo.class.getMethod(...);<br>
 Paranamer paranamer = new CachingParanamer();<br> <br> String[] parameterNames = paranamer.lookupParameterNames(method)</span><br></pre>
  </div>
</div>
<p>The method throws a ParameterNamesNotFoundException if no parameter names
are found. </p>
<p>Instead of handling that exception, you could determine if parameter name data is available, but first calling areParameterNamesAvailable().<br>
</p>
<p>ParaNamer does not have any runtime jar dependecies while looking
  up parameter info previously generated that's been zipped into a jar.</p>
<h2>DefaultParanamer</h2>
<p>DefaultParanamer tries to read parameter name data from an extra static field on the class. This field need to be added after compilation of the class, and before you put the resulting classes in a jar.</p>
<p>The static field essentially looks like -</p>
<div class="code">
<div class="codeContent">
<pre class="code-java">    private static final String __PARANAMER_DATA = "v1.0 \n"<br>        + "&lt;init&gt; \n"<br>        + "&lt;init&gt; com.foo.Bar bar \n"<br>        + "toString \n"<br>        + "setName java.lang.String,java.lang.String givenName,familyName \n";<br>        + "setDateOfBirth int,int,int day,month,year \n";</pre></div>
</div>
<p>Clearly the method's source needs to be analysed and lines added per
method to that __PARANAMER_DATA field.&nbsp;&nbsp; See below.</p>
<h2>BytecodeReadingParanamer</h2>
<p>If generating meta data for parameter names at compile time
  is not for you, try class BytecodeReadingParanamer as a runtime only solution. This uses a cut down forked version of ASM to extract debug information from a class at runtime. As it happens this is the fallback implementation for  CachingParanamer when DefaultParanamer reports that there is no meta data for a class.</p>
<h2> CachingParanamer</h2>
<p>CachingParanamer  stores the results of each parameter name lookup, so that second and subsequent invocations will be far quicker.</p>
<p>Its also interesting to note that CachingParanamertries tries to
  use DefaultParanamer and BytecodeReadingParanamer. The latter is a fallback strategy if the former cannot find
  parameter name data.</p>
<h1>Creating Parameter Name data</h1>
<h2>Optional Generation of Parameter Name data with Ant<br>
</h2>
<p>This for DefaultParanamer of course.</p>
<p>You need to download:</p>
<ul>
  <li><a
    href="http://repository.codehaus.org/com/thoughtworks/paranamer/paranamer/">latest
  paranamer jar</a></li>
  <li><a
    href="http://repository.codehaus.org/com/thoughtworks/paranamer/paranamer-generator/">latest
  paranamer-generator jar</a></li>
  <li><a
    href="http://repository.codehaus.org/com/thoughtworks/paranamer/paranamer-ant/">latest
  paranamer-ant jar</a></li>
  <li><a href="http://repo1.maven.org/maven2/qdox/qdox/">latest
  qdox jar</a> (1.6 or above)</li>
</ul>
and declare in your Ant script the following after &lt;javac/&gt;
(remember to add to the taskdef classpath all the above jars):
<br>
<div class="code">
<div class="codeContent"><pre class="code-java">&lt;taskdef name="paranamer" <br>       classname="com.thoughtworks.paranamer.ant.ParanamerGeneratorTask"/&gt;<br>&lt;paranamer sourceDirectory="src/java" outputDirectory="target/classes"/&gt;<br></pre>
</div>
</div>
<p>Classes are changed to have an extra static String member variable that contains the member functions and their parameter names. Be sure to zip them up in to you final jar.<br>
</p>
<h2>Optional Generation of Parameter Name data with Maven 2<br>
</h2>
For Maven, it is simpler. Just add this to the
&lt;build&gt;&lt;plugins&gt; section of your pom.xml:
<br>
<br>
<div class="code">
<div class="codeContent"><pre style="margin-left: 40px;">&lt;plugin&gt;<br>&nbsp;&nbsp;&nbsp; &lt;groupId&gt;com.thoughtworks.paranamer&lt;/groupId&gt;<br>&nbsp;&nbsp;&nbsp; &lt;artifactId&gt;paranamer-maven-plugin&lt;/artifactId&gt;<br>&nbsp;&nbsp; &nbsp;&lt;executions&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;execution&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;id&gt;run&lt;/id&gt;  &lt;!-- id is optional --&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;configuration&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;sourceDirectory&gt;<span
  style="font-weight: bold;">${basedir}/src/java</span>&lt;/sourceDirectory&gt;<br>&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;outputDirectory&gt;${basedir}/target/classes&lt;/outputDirectory&gt;<br>&nbsp;&nbsp;&nbsp;  &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;/configuration&gt;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;<br> &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;goals&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;goal&gt;run&lt;/goal&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;/goals&gt;<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &lt;/execution&gt;<br>&nbsp;&nbsp;&nbsp; &lt;/executions&gt;<br>&lt;/plugin&gt;</pre>
</div>
</div>
<br>
The jar file will automatically be made with parameter name data.<br>
<h2>Using Paranamer in your application without depending on Paranamer's jar</h2>
<p>There are already too many jar's for day to day Java development
right?&nbsp; Simply consume the runtime paranamer jar into your project's jar using the Maven2 'shade' plugin.<br>
</p>
<div class="code">
<div class="codeHeader"><b>Put in your POM.xml file</b></div>
<div class="codeContent"><pre class="code-java"><span
  class="code-object"> &lt;build&gt;<br>    &lt;plugins&gt;<br>      &lt;plugin&gt;<br>        &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;<br>        &lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;<br>        &lt;executions&gt;<br>          &lt;execution&gt;<br>            &lt;phase&gt;package&lt;/phase&gt;<br>            &lt;goals&gt;<br>              &lt;goal&gt;shade&lt;/goal&gt;<br>            &lt;/goals&gt;<br>            &lt;configuration&gt;<br>              &lt;shadedArtifactId&gt;artifact-shaded&lt;/shadedArtifactId&gt;<br>              &lt;relocations&gt;<br>                &lt;relocation&gt;<br>                  &lt;pattern&gt;com.thoughtworks.paranamer&lt;/pattern&gt;<br>                  &lt;shadedPattern&gt;com.yourcompany.yourapp.paranamer&lt;/shadedPattern&gt;<br>                &lt;/relocation&gt;<br>              &lt;/relocations&gt;<br>            &lt;/configuration&gt;<br>          &lt;/execution&gt;<br>        &lt;/executions&gt;<br>      &lt;/plugin&gt;<br>    &lt;/plugins&gt;<br>  &lt;/build&gt;<br> </span><br></pre>
</div>
</div>

<h1>How Paranamer works</h1>

<h2>What if a parameter names changes?</h2>
<p>The general answer to this question is that you should not ship something to third parties where they are going to hard-core your parameter names into their application. For you own in-house stuff, accessing parameter names is harmless. You should be able to ripple though the source code of even large applications, and change say &quot;badSpeeldWord&quot; to &quot;badlySpelledWorld&quot; if you need to.</p>
<p>In a later version we'll store previous parameter names too. It won't be
  be magic, you'll have to code the previous parameter names in your
  source with a doclet tag. The lookup mechanism will change too.
   It will likely leverage a doclet tag.<br>
</p>
<h1>ParaNamer's Future</h1>
<p>ParaNamer will go away in the future when Java gets Reflection
access to Method parameter names. That should look like:<br>
</p>
<div class="code">
<div class="codeContent"><pre style="margin-left: 40px;"
  class="code-java">Method method = Foo.class.getMethod(....)<br>String[] parameterNames = method.getParameterNames();<br></pre>
</div>
<p>But for now, we are thinking of the following directions :-</p>
<ol>
  <li>Plugins for Intellij IDEA, Eclipse, NetBeans to make
  parameter name data as part of their compilation processes</li>
  <li>Storage and retireval of previous parameter names</li>
</ol>
</div>

<h1>Projects using it</h1>
<a href="http://oval.sourceforge.net/">Oval - Object Validation Framework</a>,<br>
<a href="http://vraptor.org/">VRaptor - Web Framework</a>,<br>
<a href="http://waffle.codehaus.org/">Waffle - Web Framework</a><br>
<a href="http://mandragora.sourceforge.net">Mandragora</a><br>

<h1>Project Information</h1>
<p>The unit tests for ParaNamer illustrate some more way to use
it: <a
  href="http://svn.paranamer.codehaus.org/paranamer/trunk/paranamer/src/test/com/thoughtworks/paranamer/ParanamerExampleTestCase.java">http://svn.paranamer.codehaus.org/paranamer/trunk/paranamer/src/test/com/thoughtworks/paranamer/ParanamerExampleTestCase.java</a></p>
{'fraction_non_alphanumeric': 0.15172063843528, 'fraction_numerical': 0.004612971676353907, 'mean_word_length': 5.070848501820218, 'pattern_counts': {'":': 0, '<': 514, '<?xml version=': 0, '>': 514, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4258043', 'n_tokens_mistral': 7263, 'n_tokens_neox': 6905, 'n_words': 2067}