<commit_before>
cyber-dojo.rb failure messages are
FAILED: ....
Check what [docker] does and mimic that.

- - - - - - - - - - - - - - - - - - - - -

cyber-dojo volume create --name=VOL --git=URL
this URL cannot be on the local disk, since by default
its path will not be visible inside the docker container
(which on OSX is running elsewhere, on the default VM)

- - - - - - - - - - - - - - - - - - - - -

default-pulled images are
%w( gcc_assert gpp_assert csharp_nunit java_junit python_pytest ruby_mini_test )

- - - - - - - - - - - - - - - - - - - - -

test/languages/test_languages_manifests.rb
  on startup the app should run the tests/checks
  to verify 3 setup volumes are well-formed.

- - - - - - - - - - - - - - - - - - - - -

Now that data is externalized I can arrange for it to
always be at the same locations inside the web container!
I don't need CYBER_DOJO_EXERCISES_ROOT et al.
But it is worth keeping them anyway for setting up
a local (not inside docker) web server?
No I don't need them for that either.

caches_dir needs to be ENV_VAR
Its used in app_controllers/setup_test.rb
Or rather a ROOT_DIR is needed. Really its RAILS_ROOT

Can these just be off RAILS_ROOT?
  caches/  YES
  downloads/  YES
  katas/ YES. Even if server uses non data-container /var/www/cyber-dojo/katas
              this can be mapped to /usr/src/cyber-dojo/katas
  data/  NO. not for local dev-server


CYBER_DOJO_DATA_ROOT
 can be anywhere inside web server, docker-compose.yml should just work
  as it volume-mounts to data-root automatically.
 points to data-repos on local server

get tests running and passing on local server first?

- - - - - - - - - - - - - - - - - - - - -

Looking to make some smaller alpine-linux language images.
Specifically, C#-NUnit and Java-JUnit

Java
frolvlad/alpine-oraclejdk8:slim
  How big is it? ~166.9 MB  (vs 843.7 MB)
  docker run --rm frolvlad/alpine-oraclejdk8:slim sh -c "javac -version"
    javac 1.8.0_92
  There is a package called junit (branch=edge, repo=testing)

- - - - - - - - - - - - - - - - - - - - -

app_controllers tests has this warning
/usr/lib/ruby/gems/2.3.0/gems/activesupport-4.1.0/lib/active_support/values/time_zone.rb:285: warning: circular argument reference - now
Tried upgrading to Ruby 4.2.6
solves this as long Gemfile.lock is not part of the docker context
(remove it from web's dockerignore file)
This causes a few app_controller deprecation warnings
and a load of app_controller test fails which seem to be due to bad URLs

- - - - - - - - - - - - - - - - - - - - -

Having Gemfile as part of web's docker image context is unfortunate.
It means the Alpine apk update stuff repeats unnecessarily if, for
example, you bump the rails version in Gemfile.

- - - - - - - - - - - - - - - - - - - - -

Occasional test failures...
  ForkerControllerTest#test_'EAE021',
    when language no longer exists, the fork fails, and the reason given is language
  SetupControllerTest#test_'EB77D9',
    show_exercises page uses cached exercises that are runnable

  Tests are defintely not unit-tests and almost certainly have order
  dependencies. State of cache files. How to resolve?

  I could reinstate FakeDisk, leave env-vars for data ROOTs, then
  fill FakeDisk with known content, then do access to dojo methods
  such as dojo.exercises

  For now, deleting caches at start of app_controller tests.

- - - - - - - - - - - - - - - - - - - - -

Does the Tennis-Ruby exercise still crash the web server?
$ docker pull cyberdojofoundation/ruby_test_unit
And try it.

- - - - - - - - - - - - - - - - - - - - -

Note: need a check that each docker image has a user in it called nobody
      (I don't think it's uid is important)

- - - - - - - - - - - - - - - - - - - - -

If the server is up you can still do [docker rmi image]
So I still need proper nil error handling on the setup pages

- - - - - - - - - - - - - - - - - - - - -

Cribs

ruby ./cyber-dojo.rb volume create --name=default-languages    --git=https://github.com/cyber-dojo/default-languages.git
ruby ./cyber-dojo.rb volume create --name=default-exercises    --git=https://github.com/cyber-dojo/default-execises.git
ruby ./cyber-dojo.rb volume create --name=default-instructions --git=https://github.com/cyber-dojo/default-instructions.git

- - - - - - - - - - - - - - - - - - - - -

)) ./cyber-dojo catalog

     $ ./cyber-dojo volume create --name=VOL --git=URL
       This will check that the URL adhered to the api,
       viz it specified whether it was languages/exercises/instructions
       If ok it will add a label, --label=cyber-dojo-volume=languages

       manifest.json in the github source will specify
          'type': (languages/exercises/instructions)
       and if not instructions type
          'lhs-column-title': 'name',
          'rhs-column-title': 'language'

       Then volume can be mounted read-only.

     $ ./cyber-dojo volume inspect VOL [VOL...]

       This would do the old catalog listing.
       This should show the origin (URL) of the volume.
       Need to make the languages checker work by being
       passed a path (there are now 2 manifest locations, exercises and languages)
       Then [volume inspect] call will then do
         docker run --rm -v VOL:/data sh -c "app/lib/check_setup_volume.rb /data"
           check_setup_volume.rb will parse ARGV etc and delegates to setup_volume_checker.rb

       test/languages/test_languages_manifest.rb
         Rework to utility rather than test with asserts.

       Ownership of all files should be cyber-dojo
           (comes in from a volume)

       Manifest must specify at least one file as auto_pull:true
           do I really want this?
           How about simply making this part of the app startup.
           And check the state inside the app and disable home page [create]
           buttons if their setup page would have nothing to show (no matching docker image)

     What should volume manifest.json file contain?
        type: languages/exercises/instructions
        col-names: [ TEXT, TEXT ]
        auto_pull: true/false  (default for all images?)

     cyber-dojo.rb could do with some documentation showing example output
     of various internal commands.

- - - - - - - - - - - - - - - - - - - - -

)) finish the what/how separation

     Union of collection. Do in later release.
     ./cyber-dojo up --exercises=alpha,beta
        With meta-data the command could simplify to [./cyber-dojo up a,b] ?? no.

        unioning affects the UX quite a lot.
        It also introduces the possiblity of manifests from different collections
        having an identical display_name...

        DONT add the collection name to the path because then it will *NOT* replace
        the default it will union with it...

        Unioning means I will somehow need to create multiple docker-compose yml files
        from a 'template'
        Don't really want multiple yml files - much simpler to pull just one
        from the web image when doing a [./cyber-dojo update]

     give collections a defined api and meta-data
       each collection manifest specifies if is for languages/exercises/instructions
       each volume label adds the URL it was created from
       each language manifest specifies whether its docker-image should be auto pulled. default=no

- - - - - - - - - - - - - - - - - - - - -

)) cyber-dojo data repos should be data only; move .rb files somewhere else.

- - - - - - - - - - - - - - - - - - - - -

)) add another docker-compose.yml for main prod server
     prod server uses volume-mount for katas (not data-container)
     so I will need to use an docker-compose extension .yml file for
     specifying the katas (like I'll be doing for exercises/languages/instructions)

- - - - - - - - - - - - - - - - - - - - -

)) get latest image running on GCE
   Did it with hacked cyber-dojo script file that does not extract docker-compose.yml file
   from web container and mounts katas as a volume rather than a data-container.
   Had to make sure katas folder had user-rights matching user inside the web container.
   $ userdel cyber-dojo
   $ adduser --disabled-password --disabled-login --uid 19661 cyber-dojo
   $ cd /var/www/cyber-dojo
   $ sudo chown -R cyber-dojo:cyber-dojo katas
   $ cd ~
   $ sudo ./cyber-dojo pull all
   $ sudo ./cyber-dojo up

- - - - - - - - - - - - - - - - - - - - -

)) run tests in travis (via exec into web container)
   this will probably require some language images to be pulled
   these should be Alpine'd

- - - - - - - - - - - - - - - - - - - - -

)) get travis to push images to cdf hub
   push images to cyberdojofoundation hub if tests pass

- - - - - - - - - - - - - - - - - - - - -

)) ./cyber-dojo update
     needs fixing

- - - - - - - - - - - - - - - - - - - - -

)) documentation

- - - - - - - - - - - - - - - - - - - - -

)) submodule meta-data should be able to provide the
   name of the left and right columns.
      manifest.json
      {
          'type': 'exercises',
          'lhs-column-title': 'name',
          'rhs-column-title': 'language'
      }
   It would be nice if you could provide some kind of overview text
   to display which gave general instructions on the collection;
   in this case that they are all refactoring exercises.
   Would a simple readme.txt file in the repos root do?

   If multiple collections can be unioned, eg
      cyber-dojo up exercises=refactoring,james
   how will they be distinguished from each other?

   Ask James if he can put his 3 exercises onto his github account
   and I will use it as part of documentation.

- - - - - - - - - - - - - - - - - - - - -

)) add confirmation diagnostic to each [cyber-dojo] command?

- - - - - - - - - - - - - - - - - - - - -

Pull the images only when the
$ ./cyber-dojo up ...
command is issued?

----------------------------------------

Day with James :-)


mkdir tryout
cd tryout
git clone https://github.com/JonJagger/cyber-dojo.git
cd cyber-dojo
git branch drop-katas-data-container origin/drop-katas-data-container
git checkout drop-katas-data-container

Edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
point to new locations (under tryout)

Built a new web image

Tried to create a new server ontop of old one.
Could not do [./cyber-dojo restart]
>>>>>>It complained that cdf-nginx and cdf-web already existed.
Manually deleted them.

Started server.
Failed to create a kata because of permissions.
   (Note that katas is being volume-mounted (not volumes-from) in docker-compose.yml)
There is no cyber-dojo user.
On my dev default server
cat /etc/passwd
cyber-dojo:x:19661:19661:Linux User,,,:/home/cyber-dojo:/bin/sh

And this user does not exist on James's machine (Ubuntu)
$ adduser --disabled-password --no-create-home --uid 19661 cyber-dojo
Then
$ cd app/data
$ chown -R cyber-dojo katas

Then could create kata :-)
But problem with no root /cpputest folder (same as I had on my dev macbook)

$ docker run --rm -it cyberdojofoundation/cpp_cpputest sh
$ cd /
$ ls -al

Indeed there is no root folder /cpputest

Tried to recreate gpp_cpputest docker image
rebuilt image
didn't work
There is no user called cyber-dojo *inside* the gpp_cpputest container
Problem is that the image gpp_cpputest is not based FROM user-base
and so does not have cyber-dojo user inside it.
>>>>>>>I think this is not relevant. The user that runs inside the test container
is nobody. The cyber-dojo user is needed only inside the web container
(and also possibly on the host server for volume-mount interactions)


Switched create page to show plain languages
Make katas using C++, CppUTest (and C, CppUTest)
was ok, edited cyber-dojo.sh, /cpputest was there, got a red traffic-light
on the server went to katas/50/5C05.../manifest.json
language:C++ (g++)-CppUTest

Switched create page back to custom exercises.
Made new kata
katas/21/AD6F.../manifest.json
language:C

Aha!!!!
Manually edited the katas manifest.json
to "C++ (g++)-CppUTest"
and it worked. Woooooohoooooo

So the problem is the creation of the kata's manifest.
Looked at the code and fixed up the errors.
Works for both setup cases now: languages and exercises.

Retro: would be nice if you can find out the name of the
container/image from inside the container.

-------------

Tried again.

new server
installed docker
pulled from gihub
pulled new branch
edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
built a new web image
>>>>>>>>tried to create kata - failed with nil Rails error.
Look into this on my own server.

--------------

Features James mentioned during our chat
- - - - - - - - - - - - - - - - - - - -

revert button is in diff dialog again

on enter page, when kata id is entered and buttons become enabled
it would be nice to see the language and exercise names in the footer.

feature: sed in the editor (search and replace)

feature: copy one animal to a new one (keeping whole traffic-light history)

feature: fork giving two options, viz create new exercise-starting-point.

feature: replace duration on dashboard with time-ticks on top of minute columns.

feature: hovering over a traffic-light gives more detail, viz for changed files (not output)
         name of file, no of deleted lines, no of added lines

feature: traffic-lights have some visual indication if there is a
         "suspiciously large change"
         eg more than 1 file changed
         eg more than 3 lines changed in a non-test file

feature: hyperlink of compiler error on output to file and line number

feature: one one error in C/C++ projects

feature: colour syntax



<commit_msg>notes update<commit_after>
cyber-dojo.rb failure messages are
FAILED: ....
Check what [docker] does and mimic that.

- - - - - - - - - - - - - - - - - - - - -

cyber-dojo volume create --name=VOL --git=URL
this URL cannot be on the local disk, since by default
its path will not be visible inside the docker container
(which on OSX is running elsewhere, on the default VM)
Could handle this in cyber-dojo.sh and somehow mount the local path
inside the docker container.

- - - - - - - - - - - - - - - - - - - - -

default-pulled images are
%w( gcc_assert gpp_assert csharp_nunit java_junit python_pytest ruby_mini_test )

- - - - - - - - - - - - - - - - - - - - -

test/languages/test_languages_manifests.rb
  on startup the app should run the tests/checks
  to verify 3 setup volumes are well-formed.

- - - - - - - - - - - - - - - - - - - - -

Now that data is externalized I can arrange for it to
always be at the same locations inside the web container!
I don't need CYBER_DOJO_EXERCISES_ROOT et al.
But it is worth keeping them anyway for setting up
a local (not inside docker) web server?
No I don't need them for that either.

caches_dir needs to be ENV_VAR
Its used in app_controllers/setup_test.rb
Or rather a ROOT_DIR is needed. Really its RAILS_ROOT

Can these just be off RAILS_ROOT?
  caches/  YES
  downloads/  YES
  katas/ YES. Even if server uses non data-container /var/www/cyber-dojo/katas
              this can be mapped to /usr/src/cyber-dojo/katas
  data/  NO. not for local dev-server


CYBER_DOJO_DATA_ROOT
 can be anywhere inside web server, docker-compose.yml should just work
  as it volume-mounts to data-root automatically.
 points to data-repos on local server

get tests running and passing on local server first?

- - - - - - - - - - - - - - - - - - - - -

Looking to make some smaller alpine-linux language images.
Specifically, C#-NUnit and Java-JUnit

Java
frolvlad/alpine-oraclejdk8:slim
  How big is it? ~166.9 MB  (vs 843.7 MB)
  docker run --rm frolvlad/alpine-oraclejdk8:slim sh -c "javac -version"
    javac 1.8.0_92
  There is a package called junit (branch=edge, repo=testing)

- - - - - - - - - - - - - - - - - - - - -

app_controllers tests has this warning
/usr/lib/ruby/gems/2.3.0/gems/activesupport-4.1.0/lib/active_support/values/time_zone.rb:285: warning: circular argument reference - now
Tried upgrading to Ruby 4.2.6
solves this as long Gemfile.lock is not part of the docker context
(remove it from web's dockerignore file)
This causes a few app_controller deprecation warnings
and a load of app_controller test fails which seem to be due to bad URLs

- - - - - - - - - - - - - - - - - - - - -

Having Gemfile as part of web's docker image context is unfortunate.
It means the Alpine apk update stuff repeats unnecessarily if, for
example, you bump the rails version in Gemfile.

- - - - - - - - - - - - - - - - - - - - -

Occasional test failures...
  ForkerControllerTest#test_'EAE021',
    when language no longer exists, the fork fails, and the reason given is language
  SetupControllerTest#test_'EB77D9',
    show_exercises page uses cached exercises that are runnable

  Tests are defintely not unit-tests and almost certainly have order
  dependencies. State of cache files. How to resolve?

  I could reinstate FakeDisk, leave env-vars for data ROOTs, then
  fill FakeDisk with known content, then do access to dojo methods
  such as dojo.exercises

  For now, deleting caches at start of app_controller tests.

- - - - - - - - - - - - - - - - - - - - -

Does the Tennis-Ruby exercise still crash the web server?
$ docker pull cyberdojofoundation/ruby_test_unit
And try it.

- - - - - - - - - - - - - - - - - - - - -

Note: need a check that each docker image has a user in it called nobody
      (I don't think it's uid is important)

- - - - - - - - - - - - - - - - - - - - -

If the server is up you can still do [docker rmi image]
So I still need proper nil error handling on the setup pages

- - - - - - - - - - - - - - - - - - - - -

Cribs

ruby ./cyber-dojo.rb volume create --name=default-languages    --git=https://github.com/cyber-dojo/default-languages.git
ruby ./cyber-dojo.rb volume create --name=default-exercises    --git=https://github.com/cyber-dojo/default-execises.git
ruby ./cyber-dojo.rb volume create --name=default-instructions --git=https://github.com/cyber-dojo/default-instructions.git

- - - - - - - - - - - - - - - - - - - - -

)) ./cyber-dojo catalog

     $ ./cyber-dojo volume create --name=VOL --git=URL
       This will check that the URL adhered to the api,
       viz it specified whether it was languages/exercises/instructions
       If ok it will add a label, --label=cyber-dojo-volume=languages

       manifest.json in the github source will specify
          'type': (languages/exercises/instructions)
       and if not instructions type
          'lhs-column-title': 'name',
          'rhs-column-title': 'language'

       Then volume can be mounted read-only.

     $ ./cyber-dojo volume inspect VOL [VOL...]

       This would do the old catalog listing.
       This should show the origin (URL) of the volume.
       Need to make the languages checker work by being
       passed a path (there are now 2 manifest locations, exercises and languages)
       Then [volume inspect] call will then do
         docker run --rm -v VOL:/data sh -c "app/lib/check_setup_volume.rb /data"
           check_setup_volume.rb will parse ARGV etc and delegates to setup_volume_checker.rb

       test/languages/test_languages_manifest.rb
         Rework to utility rather than test with asserts.

       Ownership of all files should be cyber-dojo
           (comes in from a volume)

       Manifest must specify at least one file as auto_pull:true
           do I really want this?
           How about simply making this part of the app startup.
           And check the state inside the app and disable home page [create]
           buttons if their setup page would have nothing to show (no matching docker image)

     What should volume manifest.json file contain?
        type: languages/exercises/instructions
        col-names: [ TEXT, TEXT ]
        auto_pull: true/false  (default for all images?)

     cyber-dojo.rb could do with some documentation showing example output
     of various internal commands.

- - - - - - - - - - - - - - - - - - - - -

)) finish the what/how separation

     Union of collection. Do in later release.
     ./cyber-dojo up --exercises=alpha,beta
        With meta-data the command could simplify to [./cyber-dojo up a,b] ?? no.

        unioning affects the UX quite a lot.
        It also introduces the possiblity of manifests from different collections
        having an identical display_name...

        DONT add the collection name to the path because then it will *NOT* replace
        the default it will union with it...

        Unioning means I will somehow need to create multiple docker-compose yml files
        from a 'template'
        Don't really want multiple yml files - much simpler to pull just one
        from the web image when doing a [./cyber-dojo update]

     give collections a defined api and meta-data
       each collection manifest specifies if is for languages/exercises/instructions
       each volume label adds the URL it was created from
       each language manifest specifies whether its docker-image should be auto pulled. default=no

- - - - - - - - - - - - - - - - - - - - -

)) cyber-dojo data repos should be data only; move .rb files somewhere else.

- - - - - - - - - - - - - - - - - - - - -

)) add another docker-compose.yml for main prod server
     prod server uses volume-mount for katas (not data-container)
     so I will need to use an docker-compose extension .yml file for
     specifying the katas (like I'll be doing for exercises/languages/instructions)

- - - - - - - - - - - - - - - - - - - - -

)) get latest image running on GCE
   Did it with hacked cyber-dojo script file that does not extract docker-compose.yml file
   from web container and mounts katas as a volume rather than a data-container.
   Had to make sure katas folder had user-rights matching user inside the web container.
   $ userdel cyber-dojo
   $ adduser --disabled-password --disabled-login --uid 19661 cy<PASSWORD>-dojo
   $ cd /var/www/cyber-dojo
   $ sudo chown -R cyber-dojo:cyber-dojo katas
   $ cd ~
   $ sudo ./cyber-dojo pull all
   $ sudo ./cyber-dojo up

- - - - - - - - - - - - - - - - - - - - -

)) run tests in travis (via exec into web container)
   this will probably require some language images to be pulled
   these should be Alpine'd

- - - - - - - - - - - - - - - - - - - - -

)) get travis to push images to cdf hub
   push images to cyberdojofoundation hub if tests pass

- - - - - - - - - - - - - - - - - - - - -

)) ./cyber-dojo update
     needs fixing

- - - - - - - - - - - - - - - - - - - - -

)) documentation

- - - - - - - - - - - - - - - - - - - - -

)) submodule meta-data should be able to provide the
   name of the left and right columns.
      manifest.json
      {
          'type': 'exercises',
          'lhs-column-title': 'name',
          'rhs-column-title': 'language'
      }
   It would be nice if you could provide some kind of overview text
   to display which gave general instructions on the collection;
   in this case that they are all refactoring exercises.
   Would a simple readme.txt file in the repos root do?

   If multiple collections can be unioned, eg
      cyber-dojo up exercises=refactoring,james
   how will they be distinguished from each other?

   Ask James if he can put his 3 exercises onto his github account
   and I will use it as part of documentation.

- - - - - - - - - - - - - - - - - - - - -

)) add confirmation diagnostic to each [cyber-dojo] command?

- - - - - - - - - - - - - - - - - - - - -

Pull the images only when the
$ ./cyber-dojo up ...
command is issued?

----------------------------------------

Day with James :-)


mkdir tryout
cd tryout
git clone https://github.com/JonJagger/cyber-dojo.git
cd cyber-dojo
git branch drop-katas-data-container origin/drop-katas-data-container
git checkout drop-katas-data-container

Edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
point to new locations (under tryout)

Built a new web image

Tried to create a new server ontop of old one.
Could not do [./cyber-dojo restart]
>>>>>>It complained that cdf-nginx and cdf-web already existed.
Manually deleted them.

Started server.
Failed to create a kata because of permissions.
   (Note that katas is being volume-mounted (not volumes-from) in docker-compose.yml)
There is no cyber-dojo user.
On my dev default server
cat /etc/passwd
cyber-dojo:x:19661:19661:Linux User,,,:/home/cyber-dojo:/bin/sh

And this user does not exist on James's machine (Ubuntu)
$ adduser --disabled-password --no-create-home --uid 19661 cyber-dojo
Then
$ cd app/data
$ chown -R cyber-dojo katas

Then could create kata :-)
But problem with no root /cpputest folder (same as I had on my dev macbook)

$ docker run --rm -it cyberdojofoundation/cpp_cpputest sh
$ cd /
$ ls -al

Indeed there is no root folder /cpputest

Tried to recreate gpp_cpputest docker image
rebuilt image
didn't work
There is no user called cyber-dojo *inside* the gpp_cpputest container
Problem is that the image gpp_cpputest is not based FROM user-base
and so does not have cyber-dojo user inside it.
>>>>>>>I think this is not relevant. The user that runs inside the test container
is nobody. The cyber-dojo user is needed only inside the web container
(and also possibly on the host server for volume-mount interactions)


Switched create page to show plain languages
Make katas using C++, CppUTest (and C, CppUTest)
was ok, edited cyber-dojo.sh, /cpputest was there, got a red traffic-light
on the server went to katas/50/5C05.../manifest.json
language:C++ (g++)-CppUTest

Switched create page back to custom exercises.
Made new kata
katas/21/AD6F.../manifest.json
language:C

Aha!!!!
Manually edited the katas manifest.json
to "C++ (g++)-CppUTest"
and it worked. Woooooohoooooo

So the problem is the creation of the kata's manifest.
Looked at the code and fixed up the errors.
Works for both setup cases now: languages and exercises.

Retro: would be nice if you can find out the name of the
container/image from inside the container.

-------------

Tried again.

new server
installed docker
pulled from gihub
pulled new branch
edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
built a new web image
>>>>>>>>tried to create kata - failed with nil Rails error.
Look into this on my own server.

--------------

Features James mentioned during our chat
- - - - - - - - - - - - - - - - - - - -

revert button is in diff dialog again

on enter page, when kata id is entered and buttons become enabled
it would be nice to see the language and exercise names in the footer.

feature: sed in the editor (search and replace)

feature: copy one animal to a new one (keeping whole traffic-light history)

feature: fork giving two options, viz create new exercise-starting-point.

feature: replace duration on dashboard with time-ticks on top of minute columns.

feature: hovering over a traffic-light gives more detail, viz for changed files (not output)
         name of file, no of deleted lines, no of added lines

feature: traffic-lights have some visual indication if there is a
         "suspiciously large change"
         eg more than 1 file changed
         eg more than 3 lines changed in a non-test file

feature: hyperlink of compiler error on output to file and line number

feature: one one error in C/C++ projects

feature: colour syntax




{'fraction_non_alphanumeric': 0.09922713453719643, 'fraction_numerical': 0.004834987729387202, 'mean_word_length': 2.3940825459970165, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 46, 'https://': 8, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 152, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7172688', 'n_tokens_mistral': 8589, 'n_tokens_neox': 8190, 'n_words': 3368}