<commit_before>Yuno: Y U NO have testing framework?
====================================

If you're still running test cases by hand, stop everything you're doing and *read this right now*.

No, stop eating that. Hold your breath. It'll be worth it.

Yuno is a framework that will run whatever tests you want and tell you what exploded. It was made for [Compilers](http://cseweb.ucsd.edu/~ricko/CSE131/) at UCSD, but it's good for any case where lots of different inputs need to run against a spec. It's cross-platform, customizable, and helps you get a better grade.


A few features
--------------------
Run all the tests in Phase 1, with diff output for failures:

    yuno run phase 1 --diff

Run a custom set of tests, then save them as a suite for later:

    yuno run files phase*/**/*-good.rc --save known_good

Run a suite you made earlier (or got from someone else):

    yuno run suite crazy_edge_cases

See everything that still needs fixing:

    yuno show failing

And if your perhaps-not-sober partner made some changes, it can help you with regressions:
```
$ yuno run all
....output....
--------------
Ran 100 tests

  2 passed
  98 failed
      yuno (show|run) failed

- 62 regressions
      (list)
```


How it works
------------

A Yuno test case has two parts: a source file (testname.rc) fed to the compiler and an answer file (testname.ans.out) to be compared with the compiler's output. If the two files match, the test case passes. If not, it fails.

Each pair of files lives somewhere in the test repo, arranged in whatever structure you want. For Compilers, the layout usually looks like this:

    testcases/
        phase1/
            check1/
                MyFirstTest.rc
                MyFirstTest.ans.out
            check2/
                (etc)
        phase2/
            (etc)

This layout also works with older testing frameworks, so the repo can be versioned independently and no one is locked in to a specific tester.

When you want to run your tests, give `yuno run` the right command to find them and remember the results are all your fault.

Download and install
--------------------

### Python check

Yuno requires Python 2.7.x (not 2.6 or 3.x). It's a long, sad, highly fragmented story. If you don't already have it, or you have a different version, you should [get a copy](http://www.python.org/download/releases/2.7/) and install it somewhere nice. You can check your default Python install's version like this: `$ python --version`.

**ProTip:** for ieng6 users, Python 2.7 lives in `/software/common/python-2.7/bin/python2.7`.

### The framework

If you're using Git, just:

    $ cd /your/project
    $ git clone https://github.com/bulatb/yuno.git

If not, download the [project zip](https://github.com/bulatb/yuno/archive/master.zip) and extract it wherever you want.

**Note**: For a hassle-free install that works out of the box, your folder structure should look like this:

    /your/project/ (anywhere)
        .eclipsecrap, etc/
        bin/
            (.class files)
        testcases/
            (test repo; see below)
        yuno/
            (yuno.py, etc)

### CSE 131: the test repo

Historically it's been a good idea to maintain a central, class-wide test repo for everyone to share their tests. Someone usually steps up to manage it and make sure everything stays good. To get yourself a working copy, or just see the folder structure it should have, see the [GitHub project here](https://github.com/bulatb/compilers-testcases).

If your care cup is [especially empty](http://pinterest.com/pin/135178426287092414/), clone that repo next to Yuno and skip down down to the manual.

    $ cd /your/project<commit_msg>Added note about permissions and multiple Python versions.<commit_after>Yuno: Y U NO have testing framework?
====================================

If you're still running test cases by hand, stop everything you're doing and *read this right now*.

No, stop eating that. Hold your breath. It'll be worth it.

Yuno is a framework that will run whatever tests you want and tell you what exploded. It was made for [Compilers](http://cseweb.ucsd.edu/~ricko/CSE131/) at UCSD, but it's good for any case where lots of different inputs need to run against a spec. It's cross-platform, customizable, and helps you get a better grade.


A few features
--------------------
Run all the tests in Phase 1, with diff output for failures:

    yuno run phase 1 --diff

Run a custom set of tests, then save them as a suite for later:

    yuno run files phase*/**/*-good.rc --save known_good

Run a suite you made earlier (or got from someone else):

    yuno run suite crazy_edge_cases

See everything that still needs fixing:

    yuno show failing

And if your perhaps-not-sober partner made some changes, it can help you with regressions:
```
$ yuno run all
....output....
--------------
Ran 100 tests

  2 passed
  98 failed
      yuno (show|run) failed

- 62 regressions
      (list)
```


How it works
------------

A Yuno test case has two parts: a source file (testname.rc) fed to the compiler and an answer file (testname.ans.out) to be compared with the compiler's output. If the two files match, the test case passes. If not, it fails.

Each pair of files lives somewhere in the test repo, arranged in whatever structure you want. For Compilers, the layout usually looks like this:

    testcases/
        phase1/
            check1/
                MyFirstTest.rc
                MyFirstTest.ans.out
            check2/
                (etc)
        phase2/
            (etc)

This layout also works with older testing frameworks, so the repo can be versioned independently and no one is locked in to a specific tester.

When you want to run your tests, give `yuno run` the right command to find them and remember the results are all your fault.

Download and install
--------------------

### Python check

Yuno requires Python 2.7.x (not 2.6 or 3.x). It's a long, sad, highly fragmented story. If you don't already have it, or you have a different version, you should [get a copy](http://www.python.org/download/releases/2.7/) and install it somewhere nice. You can check your default Python install's version like this: `$ python --version`. ProTip: for ieng6 users, Python 2.7 lives in `/software/common/python-2.7/bin/python2.7`.

To run Yuno as `yuno.py` on non-Windows machines, you may need to make it executable:

    $ chmod +x yuno.py

If you have multiple Python installs and don't want to fiddle with shebangs, it might be good to make an alias for the runtime you want:

    $ alias yuno="/path/to/python2.7 yuno.py"

### The framework

If you're using Git, just:

    $ cd /your/project
    $ git clone https://github.com/bulatb/yuno.git

If not, download the [project zip](https://github.com/bulatb/yuno/archive/master.zip) and extract it wherever you want.

**Note**: For a hassle-free install that works out of the box, your folder structure should look like this:

    /your/project/ (anywhere)
        .eclipsecrap, etc/
        bin/
            (.class files)
        testcases/
            (test repo; see below)
        yuno/
            (yuno.py, etc)

### CSE 131: the test repo

Historically it's been a good idea to maintain a central, class-wide test repo for everyone to share their tests. Someone usually steps up to manage it and make sure everything stays good. To get yourself a working copy, or just see the folder structure it should have, see the [GitHub project here](https://github.com/bulatb/compilers-testcases).

If your care cup is [especially empty](http://pinterest.com/pin/135178426287092414/), clone that repo next to Yuno and skip down down to the manual.

    $ cd /your/project
{'fraction_non_alphanumeric': 0.1006485084306096, 'fraction_numerical': 0.013748378728923476, 'mean_word_length': 2.830601092896175, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6732402', 'n_tokens_mistral': 2236, 'n_tokens_neox': 2090, 'n_words': 1061}