<commit_before><!DOCTYPE html>
<html lang="en">
	<head>
		<link href="http://gmpg.org/xfn/11" rel="profile">
		<meta http-equiv="X-UA-Compatible" content="IE=edge">
		<meta http-equiv="content-type" content="text/html; charset=utf-8">

		<!-- Enable responsiveness on mobile devices-->
		<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1">

		<title>Software</title>

		<!-- CSS -->
		<link href="//fonts.googleapis.com/" rel="dns-prefetch">
		<link href="//fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic|Abril+Fatface|PT+Sans:400,400italic,700&amp;subset=latin,latin-ext" rel="stylesheet">

		<link rel="stylesheet" href="/theme/css/poole.css" />
		<link rel="stylesheet" href="/theme/css/hyde.css" />
		<link rel="stylesheet" href="/theme/css/syntax.css" />
		<link rel="stylesheet" href="/theme/font-awesome/css/font-awesome.min.css">

		<!-- RSS -->
		<link rel="alternate" type="application/rss+xml" title="RSS" href="/atom.xml">
	</head>

	<body class="theme-base-0d">
<div class="sidebar">
	<div class="container sidebar-sticky">
		<div class="sidebar-about">

			<h1>
				<a href="/">
					<img class="profile-picture" src="/images/njudd.jpg">
					<NAME>
				</a>
			</h1>
			<p class="lead"></p>
			<p class="lead"> </p>
			<p></p>
		</div>
		<nav class="sidebar-nav">
		  <a href="/">Home</a> | <a href="/files/ncj-cv.pdf">CV</a> | <a href="/blog/">Blog</a> | <a href="/software.html">Software</a> |
		</nav>
		<nav class="sidebar-nav">
			<a class="sidebar-nav-item" href="mailto:<EMAIL>">
				<i class="fa fa-envelope"></i>
			</a>
			<a class="sidebar-nav-item" href="http://www.github.com/nclarkjudd">
				<i class="fa fa-github"></i>
			</a>
			<a class="sidebar-nav-item" href="http://www.twitter.com/nclarkjudd">
				<i class="fa fa-twitter"></i>
			</a>
			<a class="sidebar-nav-item" href="http://www.linkedin.com/in/nclarkjudd">
				<i class="fa fa-linkedin"></i>
			</a>
			<a class="sidebar-nav-item" href="">
				<i class="fa fa-feed"></i>
			</a>
		</nav>
	</div>
</div>		<div class="content container">
    <h1>Software</h1>
    

    <p>If you're a researcher in the social sciences or a journalist working on news in the public interest, then I encourage you to contact me about how software I have developed for my own research can help you in your work.</p>
<p>Over the course of the past few years, I have developed and continue to maintain a small number of open source software packages. More information is available at my <a href="https://github.com/nclarkjudd?tab=repositories">GitHub profile</a>, but here are some highlights of interest:</p>
<h2>congressional-record</h2>
<p>Originally developed by programmers at Sunlight Labs, <a href="https://www.github.com/unitedstates/congressional-record">congressional-record</a> is a parser that extracts structured data and text from the Congressional Record, a near-verbatim, official account of proceedings on the floor of the United States Congress. The core purpose of this parser is to extract text entered into the Record by members of Congress --- either in a floor speech, inserted as a supplement, or submitted as an extension of remarks --- and correctly assign each speech to a speaker. I began work on this project in late 2014 and began releasing code in 2015. In 2017 I became the primary maintainer for this repository and a contributor to <a href="https://theunitedstates.io/">The United States Project</a>, a shared commons of data and tools for the United States.</p>
<p>My fork of this software includes changes to the parser that substantially eliminate false negatives, that is, speeches that should be recognized as a new speech and assigned to a speaker. I also make additional use of metadata released with the speeches by the Library of Congress to increase the number of correct assignments of speeches to speakers. In addition, I rewrote the code to produce output in JSON or CSV, rather than XML.</p>
<h3>Usage</h3>
<p><img alt="&quot;Women in Congress&quot;" src="images/women-in-congress.png"> The data journalist <strong><NAME></strong> used my code in <a href="https://pudding.cool/2018/07/women-in-congress/">this visualization of gender representation in Congress</a>.</p>
<p>I'm happy to see other researchers using this code in their research:</p>
<ul>
<li><a href="http://shoub.web.unc.edu/"><NAME></a>, for her dissertation, "Shifting Debate to Shift Policy."</li>
</ul>
<h2>poindexter</h2>
<p><a href="https://github.com/nclarkjudd/poindexter">Poindexter</a> downloads, cleans and parses data on so-called "527" political organizations from the Internal Revenue Service. The IRS releases this data in a form that is not immediately accessible using open-source tools; poindexter fixes that. I originally developed this software while working as a summer fellow at The Sunlight Foundation.</p>

		</div>
		<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){<commit_msg>minor formatting chg<commit_after><!DOCTYPE html>
<html lang="en">
	<head>
		<link href="http://gmpg.org/xfn/11" rel="profile">
		<meta http-equiv="X-UA-Compatible" content="IE=edge">
		<meta http-equiv="content-type" content="text/html; charset=utf-8">

		<!-- Enable responsiveness on mobile devices-->
		<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1">

		<title>Software</title>

		<!-- CSS -->
		<link href="//fonts.googleapis.com/" rel="dns-prefetch">
		<link href="//fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic|Abril+Fatface|PT+Sans:400,400italic,700&amp;subset=latin,latin-ext" rel="stylesheet">

		<link rel="stylesheet" href="/theme/css/poole.css" />
		<link rel="stylesheet" href="/theme/css/hyde.css" />
		<link rel="stylesheet" href="/theme/css/syntax.css" />
		<link rel="stylesheet" href="/theme/font-awesome/css/font-awesome.min.css">

		<!-- RSS -->
		<link rel="alternate" type="application/rss+xml" title="RSS" href="/atom.xml">
	</head>

	<body class="theme-base-0d">
<div class="sidebar">
	<div class="container sidebar-sticky">
		<div class="sidebar-about">

			<h1>
				<a href="/">
					<img class="profile-picture" src="/images/njudd.jpg">
					<NAME>
				</a>
			</h1>
			<p class="lead"></p>
			<p class="lead"> </p>
			<p></p>
		</div>
		<nav class="sidebar-nav">
		  <a href="/">Home</a> | <a href="/files/ncj-cv.pdf">CV</a> | <a href="/blog/">Blog</a> | <a href="/software.html">Software</a> |
		</nav>
		<nav class="sidebar-nav">
			<a class="sidebar-nav-item" href="mailto:<EMAIL>">
				<i class="fa fa-envelope"></i>
			</a>
			<a class="sidebar-nav-item" href="http://www.github.com/nclarkjudd">
				<i class="fa fa-github"></i>
			</a>
			<a class="sidebar-nav-item" href="http://www.twitter.com/nclarkjudd">
				<i class="fa fa-twitter"></i>
			</a>
			<a class="sidebar-nav-item" href="http://www.linkedin.com/in/nclarkjudd">
				<i class="fa fa-linkedin"></i>
			</a>
			<a class="sidebar-nav-item" href="">
				<i class="fa fa-feed"></i>
			</a>
		</nav>
	</div>
</div>		<div class="content container">
    <h1>Software</h1>
    

    <p>If you're a researcher in the social sciences or a journalist working on news in the public interest, then I encourage you to contact me about how software I have developed for my own research can help you in your work.</p>
<p>Over the course of the past few years, I have developed and continue to maintain a small number of open source software packages. More information is available at my <a href="https://github.com/nclarkjudd?tab=repositories">GitHub profile</a>, but here are some highlights of interest:</p>
<h2>congressional-record</h2>
<p>Originally developed by programmers at Sunlight Labs, <a href="https://www.github.com/unitedstates/congressional-record">congressional-record</a> is a parser that extracts structured data and text from the Congressional Record, a near-verbatim, official account of proceedings on the floor of the United States Congress. The core purpose of this parser is to extract text entered into the Record by members of Congress --- either in a floor speech, inserted as a supplement, or submitted as an extension of remarks --- and correctly assign each speech to a speaker. I began work on this project in late 2014 and began releasing code in 2015. In 2017 I became the primary maintainer for this repository and a contributor to <a href="https://theunitedstates.io/">The United States Project</a>, a shared commons of data and tools for the United States.</p>
<p>My fork of this software includes changes to the parser that substantially eliminate false negatives, that is, speeches that should be recognized as a new speech and assigned to a speaker. I also make additional use of metadata released with the speeches by the Library of Congress to increase the number of correct assignments of speeches to speakers. In addition, I rewrote the code to produce output in JSON or CSV, rather than XML.</p>
<h3>Usage</h3>
<p><img alt="&quot;Women in Congress&quot;" src="images/women-in-congress.png"> The data journalist <strong>Durand D'Souza</strong> used my code in <a href="https://pudding.cool/2018/07/women-in-congress/">this visualization of gender representation in Congress</a>.</p>
<p>I'm happy to see other researchers using this code in their research:</p>
<ul>
<li><strong><a href="http://shoub.web.unc.edu/"><NAME></a></strong>, for her dissertation, "Shifting Debate to Shift Policy."</li>
</ul>
<h2>poindexter</h2>
<p><a href="https://github.com/nclarkjudd/poindexter">Poindexter</a> downloads, cleans and parses data on so-called "527" political organizations from the Internal Revenue Service. The IRS releases this data in a form that is not immediately accessible using open-source tools; poindexter fixes that. I originally developed this software while working as a summer fellow at The Sunlight Foundation.</p>

		</div>
		<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
{'fraction_non_alphanumeric': 0.15287517531556802, 'fraction_numerical': 0.011220196353436185, 'mean_word_length': 4.4403269754768395, 'pattern_counts': {'":': 0, '<': 240, '<?xml version=': 0, '>': 240, 'https://': 10, 'lorem ipsum': 0, 'www.': 8, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5235284', 'n_tokens_mistral': 3457, 'n_tokens_neox': 3091, 'n_words': 1024}