<commit_before>---
layout: post_notitle
title: OpenNews Code and projects
section: code
---
<img src="/media/img/code.jpg" class="topline">

<h2>{{ page.title }}</h2>
<p class="bodybig">OpenNews supports developers inside and outside of newsrooms in creating code that helps journalism thrive on the open web. We believe that the code being written in news transforms not only the industry, but the web itself.</p>
<div id="fellowcode">
<h3>Projects by the Knight-Mozilla Fellows</h3>
<p>Our <a href="/fellowships/community.html">Fellows</a> spend ten months hacking in some of the best newsrooms in the world, following their passions and creating compelling open-source projects. Here are just a few of the many things <a href="https://github.com/opennews">they've developed</a>:</p>
<ul class="fellowcode">
<li><a href="http://tabula.nerdpower.org/" class="fellowhack">Tabula</a>
<br>Tabula is a tool to extract tabular data from PDFs. The project was created by 2013 Knight-Mozilla Fellow <NAME>, with the help of 2013 Knight-Mozilla Fellow <NAME> and his colleague at ProPublica <NAME>. 
<br>Key libraries : 
<br><a href="http://tabula.nerdpower.org/">On the web</a> | <a href="https://github.com/jazzido/tabula">On GitHub</a>
</li>

<li><a href="http://hyperaud.io/" class="fellowhack">Hyperaudio</a>
<br><NAME>' Hyperaudio is a tool to facilitate the easy assemblage of audio and video programs from their underlying transcripts. The ongoing aims are to create something usable that works with both audio and video and allows transitions and overlaying to be specified via in-pad natural language instructions and to build up a library of material and to integrate with other third-parties such as Amara (formerly Universal Subtitles). Hyperaudio is currently a project with <a href="https://webfwd.org/portfolio">Mozilla's WebFWD</a>. In addition to <NAME>, 2012 Knight-Mozilla Fellow <NAME> and <NAME> are on the development team.
<br><a href="http://hyperaud.io/">On the web</a> | <a href="https://github.com/hyperaudio-dev">On GitHub</a>
</li>

<li><a href="https://dataset.readthedocs.org/en/latest/" class="fellowhack">Dataset</a>
<br>Dataset is a tool from 2013 Knight-Mozilla Fellow <NAME> and <NAME> to make it easier to manage databases in Python. Dataset makes it easier to import and export databases: "databases for lazy people."
<br><a href="https://dataset.readthedocs.org/en/latest/">On the web</a> | <a href="https://github.com/pudo/dataset">On GitHub</a>
</li>

<li><a href="https://github.com/veltman/learninglunches" class="fellowhack">Learning Lunches</a>
<br>2013 Knight-Mozilla Fellow <NAME> began organizing informal discussions with colleagues about technical topics. He's shared the materials from these discussions on GitHub. Topics have included databases, maps, and web scraping. 
<br><a href="https://github.com/veltman/learninglunches">On GitHub</a>
</li>

</ul>
</div>
<h3>Code from Code Sprints</h3>
We developed Code Sprints to help create some of the small, simple tools that can have a big impact in newsrooms.

<p>Our Code Sprint projects include:</p>
<ul>
<li><a href="https://github.com/jlord/sheetsee.js">Sheetsee.js</a>: Easy data visualizations using a simple spreadsheet backend.
<li><a href="https://github.com/open-city/dedupe">Dedupe</a>: A library for deduplication, entity resolution, record linkage, and author disambiguation of big datasets.
<li><a href="http://treasury.io/">Treasury.io</a>: A parser and API for the daily cash balance updates from the US Treasury.
<li><a href="https://github.com/wnyc/election_california_2012">California Election Parser</a>: A parser for election data used by over 200 California news sites in 2012.
</ul>

We'd love to develop more Code Sprints. <a href="/codesprints.html">Learn more about the program and apply</a>.

<h3>Code from Hack Days</h3>
We've sponsored more than 40 <a href="/hackdays.html">hack days</a> around the world where journalists and developers have worked with data from censuses, elections, campaign finance, and more. 

Some projects that got their start at hack days include:
<ul>
<li><a href="http://www.civomega.com/">CivOmega</a>: this project got its start at the 2013 Knight-MIT-Mozilla hack day. It allows people to ask questions of legislative data and was recently awarded a Sunlight Foundation OpenGov Grant.</li>
<li><a href="http://hackdash.org/">HackDash</a>: was originally developed at the 2012 Hacks/Hackers Buenos Aires Media Party and a year later, this tool for organizing hackathon projects powered the hack day at the 2013 event.</li>
<li><a href="http://newsdiffs.org/">NewsDiffs</a>: began at the 2012 Knight-MIT-Mozilla hack day as a way to track changes to articles and headlines. It now tracks an archives changes to articles on five news sites.</li>
<li><a href="http://treasury.io/">Treasury.io</a>: began as a project called FMS parser at the Bicoastal Datafest. It was developed by the CSV Soundsystem hacker team, which includes 2013 Knight-Mozilla Fellow <NAME>, to help track the US government's virtual checkbook.</li></ul>

<h3>Code Convenings</h3>

Starting in 2014, OpenNews plans to gather groups of journalism developers and open-source contributors to collaborate on shared codebases and libraries so that we can stop continually reinventing the wheel on needed infrastructure, like election parsers, opsec, visualizations, and more. We'll have more information soon and look forward to this new way to create code that will help news organizations.<commit_msg>updated code page<commit_after>---
layout: post_notitle
title: OpenNews Code and projects
section: code
---
<img src="/media/img/code.jpg" class="topline">

<h2>{{ page.title }}</h2>
<p class="bodybig">OpenNews supports developers inside and outside of newsrooms in creating code that helps journalism thrive on the open web. We believe that the code being written in news transforms not only the industry, but the web itself.</p>
<div id="fellowcode">
<h3>Projects by the Knight-Mozilla Fellows</h3>
<p>Our <a href="/fellowships/community.html">Fellows</a> spend ten months hacking in some of the best newsrooms in the world, following their passions and creating compelling open-source projects. Here are just a few of the many things <a href="https://github.com/opennews">they've developed</a>:</p>
<ul class="fellowcode">
<li><a href="http://tabula.nerdpower.org/" class="fellowhack">Tabula</a>
<br>Tabula is a tool to extract tabular data from PDFs. The project was created by 2013 Knight-Mozilla Fellow <NAME>, with the help of 2013 Knight-Mozilla Fellow <NAME> and his colleague at ProPublica <NAME>.
<br>Key libraries :
<br><a href="http://tabula.nerdpower.org/">On the web</a> | <a href="https://github.com/jazzido/tabula">On GitHub</a>
</li>

<li><a href="http://hyperaud.io/" class="fellowhack">Hyperaudio</a>
<br><NAME>' Hyperaudio is a tool to facilitate the easy assemblage of audio and video programs from their underlying transcripts. The ongoing aims are to create something usable that works with both audio and video and allows transitions and overlaying to be specified via in-pad natural language instructions and to build up a library of material and to integrate with other third-parties such as Amara (formerly Universal Subtitles). Hyperaudio is currently a project with <a href="https://webfwd.org/portfolio">Mozilla's WebFWD</a>. In addition to <NAME>, 2012 Knight-Mozilla Fellow <NAME> and <NAME> are on the development team.
<br><a href="http://hyperaud.io/">On the web</a> | <a href="https://github.com/hyperaudio-dev">On GitHub</a>
</li>

<li><a href="https://dataset.readthedocs.org/en/latest/" class="fellowhack">Dataset</a>
<br>Dataset is a tool from 2013 Knight-Mozilla Fellow <NAME> and <NAME> to make it easier to manage databases in Python. Dataset makes it easier to import and export databases: "databases for lazy people."
<br><a href="https://dataset.readthedocs.org/en/latest/">On the web</a> | <a href="https://github.com/pudo/dataset">On GitHub</a>
</li>

<li><a href="https://github.com/veltman/learninglunches" class="fellowhack">Learning Lunches</a>
<br>2013 Knight-Mozilla Fellow <NAME> began organizing informal discussions with colleagues about technical topics. He's shared the materials from these discussions on GitHub. Topics have included databases, maps, and web scraping.
<br><a href="https://github.com/veltman/learninglunches">On GitHub</a>
</li>

</ul>
</div>
<h3>Code from Code Sprints</h3>
We developed Code Sprints to help create some of the small, simple tools that can have a big impact in newsrooms.

<p>Our Code Sprint projects include:</p>
<ul>
<li><a href="https://github.com/jlord/sheetsee.js">Sheetsee.js</a>: Easy data visualizations using a simple spreadsheet backend.
<li><a href="https://github.com/open-city/dedupe">Dedupe</a>: A library for deduplication, entity resolution, record linkage, and author disambiguation of big datasets.
<li><a href="http://treasury.io/">Treasury.io</a>: A parser and API for the daily cash balance updates from the US Treasury.
<li><a href="https://github.com/wnyc/election_california_2012">California Election Parser</a>: A parser for election data used by over 200 California news sites in 2012.
</ul>

We'd love to develop more Code Sprints. <a href="/codesprints.html">Learn more about the program and apply</a>.

<h3>Code from Hack Days</h3>
We've sponsored more than 40 <a href="/hackdays.html">hack days</a> around the world where journalists and developers have worked with data from censuses, elections, campaign finance, and more.

Some projects that got their start at hack days include:
<ul>
<li><a href="http://www.civomega.com/">CivOmega</a>: this project got its start at the 2013 Knight-MIT-Mozilla hack day. It allows people to ask questions of legislative data and was recently awarded a Sunlight Foundation OpenGov Grant.</li>
<li><a href="http://hackdash.org/">HackDash</a>: was originally developed at the 2012 Hacks/Hackers Buenos Aires Media Party and a year later, this tool for organizing hackathon projects powered the hack day at the 2013 event.</li>
<li><a href="http://newsdiffs.org/">NewsDiffs</a>: began at the 2012 Knight-MIT-Mozilla hack day as a way to track changes to articles and headlines. It now tracks an archives changes to articles on five news sites.</li>
<li><a href="http://treasury.io/">Treasury.io</a>: began as a project called FMS parser at the Bicoastal Datafest. It was developed by the CSV Soundsystem hacker team, which includes 2013 Knight-Mozilla Fellow <NAME>, to help track the US government's virtual checkbook.</li></ul>

<h3>Code Convenings</h3>

Twice a year, OpenNews brings together groups of newsroom developers and other open-source contributors to collaborate on shared codebases and libraries. Whether it is spending time open-sourcing work already built in the newsroom so that others can benefit from it too or working on new projects together, we believe that collaborating on shared codebases helps move newsroom development--and the web--forward. 
{'fraction_non_alphanumeric': 0.11232802317161478, 'fraction_numerical': 0.011766835626357712, 'mean_word_length': 5.094318808604523, 'pattern_counts': {'":': 0, '<': 229, '<?xml version=': 0, '>': 229, 'https://': 24, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5761145', 'n_tokens_mistral': 3341, 'n_tokens_neox': 3082, 'n_words': 1341}