<commit_before># atom-finder [![Build Status](https://travis-ci.org/dgopstein/atom-finder.svg?branch=master)](https://travis-ci.org/dgopstein/atom-finder)

A Clojure library for finding [Atoms of Confusion](https://atomsofconfusion.com) in C projects.

Facilities for:

 * Parsing C/C++ with Eclipse CDT library
 * Finding specific patterns in an AST
 * Traversing version histories through git
 * Parsing commit logs for bug/patch IDs

## Project Structure Overview

The majority of interesting files in this project are located in the top-level
`src` directory. Secondarily, test files are in located `test` and jars in
`resources`. Under `src` there are several important directories:

* `atom_finder` - Clojure files for parsing C/C++ files and searching for atoms
* `analysis` - R files for statistically analyzing the results of clojure
* `conf` - Configuration variables to customize each runtime environment

## Working with Clojure

This project is primariy written in Clojure(JVM), and uses many Java libraries.
In order to run this project you should install:

 * [Leiningen](https://leiningen.org/) - The Clojure build manager.<commit_msg>More directory description<commit_after># atom-finder [![Build Status](https://travis-ci.org/dgopstein/atom-finder.svg?branch=master)](https://travis-ci.org/dgopstein/atom-finder)

A Clojure library for finding [Atoms of Confusion](https://atomsofconfusion.com) in C projects.

Facilities for:

 * Parsing C/C++ with Eclipse CDT library
 * Finding specific patterns in an AST
 * Traversing version histories through git
 * Parsing commit logs for bug/patch IDs

## Project Structure Overview

The majority of interesting files in this project are located in the top-level
`src` directory. Secondarily, test files are in located `test` and jars in
`resources`. Under `src` there are several important directories:

* `atom_finder` - Clojure files for parsing C/C++ files and searching for atoms
* `analysis` - R files for statistically analyzing the results of the code mining
* `conf` - Configuration variables to customize each runtime environment

The most important, and complicated directory is `src/atom_finder` which
contains all the code analyze source code and repositories. Within the top-level
of `atom-finder` is code which is both specific to this project, but also
reusable between different analyses. Below the top-level are several other
useful directories:

* `classifier` - Every file in this directory is used to determine whether an
  individual AST node is a particular atom of confusion
* `questions` - Every file in this directory corresponds to one of our
  (published, or potential) research hypotheses. These files implicitly use the
  classifier infrastructure to observe patterns.
* `tree_diff` - Tree diffing was a difficult enough problem that took several
  iterations to get working. Each evolution is it's own sub-namespace in this
  directory. Ultimately only `difflib` ended up being used.
* `util` - The most reusable and general functions. Most of these files are
  potentially useful in other projects outside this one.

## Working with Clojure

This project is primariy written in Clojure(JVM), and uses many Java libraries.
In order to run this project you should install:

 * [Leiningen](https://leiningen.org/) - The Clojure build manager.
{'fraction_non_alphanumeric': 0.07295106574602221, 'fraction_numerical': 0.0, 'mean_word_length': 4.715265866209262, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3119857', 'n_tokens_mistral': 909, 'n_tokens_neox': 855, 'n_words': 415}