<commit_before>
Concepts
========

.. warning:: This documentation is work in progress and there will be areas that are lacking.

Motivation
----------

    "Have a look at this piece of code that I’m writing--I’m sure it has been written before.
    I wouldn't be surprised to find it verbatim somewhere on GitHub." - `@kr1 <https://github.com/kr1>`_

Every piece of functionality in a software project
requires code that lies somewhere in the wide reusability spectrum that goes
form extremely custom and strongly tied to the specific implementation
to completely generic and highly reusable.

On the *custom* side of the spectrum there is all the code that defines the
features of the software and all the choices of its implementation. That one is code that need
to be written.

On the other hand a seasoned software developer is trained to spot
pieces of functionality that lie far enough on the *generic* side of the range
that with high probability a library already implements it
**and documents it well enough to be discovered with an internet search**.

In between the two extremes there is a huge gray area populated by pieces of functionality
that are not *generic* enough to obviously deserve a place in a library, but are
*common* enough that must have been already implemented by someone else for their
software. This kind of code is doomed to be re-implemented again and again
for the simple reason that **there is no way to search code by functionality**...

Or is it?


Test-driven code search
-----------------------

pytest-nodev is a pytest plugin that enables *test-driven code search* and
consequently a software development strategy called
*test-driven reuse* or TDR that we call *nodev*,
that is an extension of the well known *test-driven development* or TDD.

The idea is that once the developer has written the tests that define the behaviour of a new
function to a degree sufficient to validate the implementation they are going to write
it is good enough to validate
any implementation. Running the tests on a large set of functions may result in a *hit*, that is
a function that already implements their feature.

Due to its nature the approach is better suited for discovering smaller functions
with a generic signature.


Tests validation
----------------

Another use for pytest-nodev is, with a bit of additional work, to validate a project test suite.
If a test passes with an unexpected object there are two possibilities,
either the test is not strict enough and allows for false positives and needs to be updated,
or the *hit* is actually a function you could use instead of your implementation.


Keywords:

 * Source code *search by feature*, *search by functionality*, *search by specification* or *nodev*
 * *Feature-specification test* and test suite or *Requirement-specification test*
 * *Test-driven reuse* or *test-driven code search* or *test-driven source code search*


Bibliography
------------

- "CodeGenie: a tool for test-driven source code search", <NAME> *et al*,
  Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion,
  917--918, **2007**, ACM, http://dx.doi.org/10.1145/1297846.1297944

- "Code conjurer: Pulling reusable software out of thin air", <NAME> *et al*,
  IEEE Software, (25) 5 45-52, **2008**, IEEE, http://dx.doi.org/10.1109/MS.2008.110 ---
  `PDF <http://cosc612.googlecode.com/svn/Research%20Paper/Code%20Conjurer.pdf>`__

- "Finding Source Code on the Web for Remix and Reuse", <NAME> *et al*, 251, **2013** ---
  `PDF <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.2645&rep=rep1&type=pdf>`__

- "Test-Driven Reuse: Improving the Selection of Semantically Relevant Code", <NAME>,
  Ph.D. thesis, **2014**, UNIVERSITY OF CALGARY ---
  `PDF <http://lsmr.org/docs/nurolahzade_phd_2014.pdf>`__
<commit_msg>Ensure ASCII.<commit_after>
Concepts
========

.. warning:: This documentation is work in progress and there will be areas that are lacking.

Motivation
----------

    "Have a look at this piece of code that I?m writing--I'm sure it has been written before.
    I wouldn't be surprised to find it verbatim somewhere on GitHub." - `@kr1 <https://github.com/kr1>`_

Every piece of functionality in a software project
requires code that lies somewhere in the wide reusability spectrum that goes
form extremely custom and strongly tied to the specific implementation
to completely generic and highly reusable.

On the *custom* side of the spectrum there is all the code that defines the
features of the software and all the choices of its implementation. That one is code that need
to be written.

On the other hand a seasoned software developer is trained to spot
pieces of functionality that lie far enough on the *generic* side of the range
that with high probability a library already implements it
**and documents it well enough to be discovered with an internet search**.

In between the two extremes there is a huge gray area populated by pieces of functionality
that are not *generic* enough to obviously deserve a place in a library, but are
*common* enough that must have been already implemented by someone else for their
software. This kind of code is doomed to be re-implemented again and again
for the simple reason that **there is no way to search code by functionality**...

Or is it?


Test-driven code search
-----------------------

pytest-nodev is a pytest plugin that enables *test-driven code search* and
consequently a software development strategy called
*test-driven reuse* or TDR that we call *nodev*,
that is an extension of the well known *test-driven development* or TDD.

The idea is that once the developer has written the tests that define the behaviour of a new
function to a degree sufficient to validate the implementation they are going to write
it is good enough to validate
any implementation. Running the tests on a large set of functions may result in a *hit*, that is
a function that already implements their feature.

Due to its nature the approach is better suited for discovering smaller functions
with a generic signature.


Tests validation
----------------

Another use for pytest-nodev is, with a bit of additional work, to validate a project test suite.
If a test passes with an unexpected object there are two possibilities,
either the test is not strict enough and allows for false positives and needs to be updated,
or the *hit* is actually a function you could use instead of your implementation.


Keywords:

 * Source code *search by feature*, *search by functionality*, *search by specification* or *nodev*
 * *Feature-specification test* and test suite or *Requirement-specification test*
 * *Test-driven reuse* or *test-driven code search* or *test-driven source code search*


Bibliography
------------

- "CodeGenie: a tool for test-driven source code search", <NAME> *et al*,
  Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion,
  917--918, **2007**, ACM, http://dx.doi.org/10.1145/1297846.1297944

- "Code conjurer: Pulling reusable software out of thin air", <NAME> *et al*,
  IEEE Software, (25) 5 45-52, **2008**, IEEE, http://dx.doi.org/10.1109/MS.2008.110 ---
  `PDF <http://cosc612.googlecode.com/svn/Research%20Paper/Code%20Conjurer.pdf>`__

- "Finding Source Code on the Web for Remix and Reuse", <NAME> *et al*, 251, **2013** ---
  `PDF <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.2645&rep=rep1&type=pdf>`__

- "Test-Driven Reuse: Improving the Selection of Semantically Relevant Code", <NAME>,
  Ph.D. thesis, **2014**, UNIVERSITY OF CALGARY ---
  `PDF <http://lsmr.org/docs/nurolahzade_phd_2014.pdf>`__

{'fraction_non_alphanumeric': 0.08657840083073728, 'fraction_numerical': 0.023883696780893044, 'mean_word_length': 3.98705501618123, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3694702', 'n_tokens_mistral': 2205, 'n_tokens_neox': 2012, 'n_words': 1085}