<commit_before>
How to get the data folder(s) onto the server running the docker daemon?

1. create a temporary folder
2. git clone repos into the temp folder
3. create a temporary docker image using the folder as its context
4. issue a docker command to extract the folder from the docker-container

I did before for [cyber-dojo backup] command.
However, that happened inside the web container (in cyber-dojo.rb)
So I think the 'scp' utility (see below) will also need to be
embedded inside cyber-dojo.rb

SCP utility
-----------
sh script to do this will use one parameter,
the name of source folder to copy (on host)

I may also need a [cyber-dojo] script command
to determine if the docker server already has a
given folder - I only want to do the default
folder copying once.

cyber-dojo up exercises=Xpath
cyber-dojo up languages=Lpath

where Lpath and Xpath are local folders.

Or

cyber-dojo languages path
cyber-dojo exercises path
cyber-dojo instructions path

This approach means I don't have to name the destination folder
on the docker server.
What about copying new exercises back from the docker-server, eg for backup

cyber-dojo languages    [push|pull] path
cyber-dojo exercises    [push|pull] path
cyber-dojo instructions [push|pull] path

where path could be a github url, which would do a git clone to get the source.

----------------------------------------

Day with James :-)


mkdir tryout
cd tryout
git clone https://github.com/JonJagger/cyber-dojo.git
cd cyber-dojo
git branch drop-katas-data-container origin/drop-katas-data-container
git checkout drop-katas-data-container

Edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
point to new locations (under tryout)

Built a new web image

Tried to create a new server ontop of old one.
Could not do [./cyber-dojo restart]
It complained that cdf-nginx and cdf-web already existed.
Manually deleted them.

Started server.
Failed to create a kata because of permissions.
There is no cyber-dojo user.
On my dev default server
cat /etc/passwd
cyber-dojo:x:19661:19661:Linux User,,,:/home/cyber-dojo:/bin/sh

And this user does not exist on James's machine (Ubuntu)
$ adduser --disabled-password --no-create-home --uid 19661 cyber-dojo
Then
$ cd app/data
$ chown -R cyber-dojo katas

Then could create kata :-)
But problem with no root /cpputest folder (same as I had on my dev macbook)

$ docker run --rm -it cyberdojofoundation/cpp_cpputest sh
$ cd /
$ ls -al

Indeed there is no root folder /cpputest


Tried to recreate gpp_cpputest docker image

>>>>>>There are no Dockerfile's underneath /app/data/languages
>>>>>>.gitignore seems to be the culprit.


Manually added Dockerfile for C++ (g++ 5.3)/CppuTest/_docker_context
rebuilt image
didn't work
There is no user called cyber-dojo *inside* the gpp_cpputest container
Problem is that the image gpp_cpputest is not based FROM user-base
and so does not have cyber-dojo user inside it.




Switched create page to show plain languages
Make katas using C++, CppUTest (and C, CppUTest)
was ok, edited cyber-dojo.sh, /cpputest was there, got a red traffic-light
on the server went to katas/50/5C05.../manifest.json
language:C++ (g++)-CppUTest

Switched create page back to custom exercises.
Made new kata
katas/21/AD6F.../manifest.json
language:C

Aha!!!!
Manually edited the katas manifest.json
to "C++ (g++)-CppUTest"
and it worked. Woooooohoooooo

So the problem is the creation of the kata's manifest.
Looked at the code and fixed the errors.
Works for both setup cases now: languages and exercises.


-------------

Tried again.

new server
installed docker
pulled from gihub
pulled new branch
edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
built a new web image
tried to create kata - failed with nil Rails error.

Look into this on my own server.

Remember
>>>>>>There are no Dockerfile's underneath /app/data/languages
>>>>>>.gitignore seems to be the culprit
>>>>>>It has Dockerfile instead of /Dockerfile (eg)

--------------

Features James mentioned during our chat
- - - - - - - - - - - - - - - - - - - -

revert button is in diff dialog again

on enter page, when kata id is entered and buttons become enabled
it would be nice to see the language and exercise names in the footer.

feature: sed in the editor (search and replace)

feature: copy one animal to a new one (keeping whole traffic-light history)

feature: fork giving two options, viz create new exercise-starting-point.

feature: replace duration on dashboard with time-ticks on top of minute columns.

feature: hovering over a traffic-light gives more detail, viz for changed files (not output)
         name of file, no of deleted lines, no of added lines

feature: traffic-lights have some visual indication if there is a
         "suspiciously large change"
         eg more than 1 file changed
         eg more than 3 lines changed in a non-test file
<commit_msg>added more notes after chatting to James<commit_after>
How to get the data folder(s) onto the server running the docker daemon?

1. create a temporary folder
2. git clone repos into the temp folder
3. create a temporary docker image using the folder as its context
4. issue a docker command to extract the folder from the docker-container

I did before for [cyber-dojo backup] command.
However, that happened inside the web container (in cyber-dojo.rb)
So I think the 'scp' utility (see below) will also need to be
embedded inside cyber-dojo.rb

SCP utility
-----------
sh script to do this will use one parameter,
the name of source folder to copy (on host)

I may also need a [cyber-dojo] script command
to determine if the docker server already has a
given folder - I only want to do the default
folder copying once.

cyber-dojo up exercises=Xpath
cyber-dojo up languages=Lpath

where Lpath and Xpath are local folders.

Or

cyber-dojo languages path
cyber-dojo exercises path
cyber-dojo instructions path

This approach means I don't have to name the destination folder
on the docker server.
What about copying new exercises back from the docker-server, eg for backup

cyber-dojo languages    [push|pull] path
cyber-dojo exercises    [push|pull] path
cyber-dojo instructions [push|pull] path

where path could be a github url, which would do a git clone to get the source.

----------------------------------------

Day with James :-)


mkdir tryout
cd tryout
git clone https://github.com/JonJagger/cyber-dojo.git
cd cyber-dojo
git branch drop-katas-data-container origin/drop-katas-data-container
git checkout drop-katas-data-container

Edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
point to new locations (under tryout)

Built a new web image

Tried to create a new server ontop of old one.
Could not do [./cyber-dojo restart]
It complained that cdf-nginx and cdf-web already existed.
Manually deleted them.

Started server.
Failed to create a kata because of permissions.
There is no cyber-dojo user.
On my dev default server
cat /etc/passwd
cyber-dojo:x:19661:19661:Linux User,,,:/home/cyber-dojo:/bin/sh

And this user does not exist on James's machine (Ubuntu)
$ adduser --disabled-password --no-create-home --uid 19661 cyber-dojo
Then
$ cd app/data
$ chown -R cyber-dojo katas

Then could create kata :-)
But problem with no root /cpputest folder (same as I had on my dev macbook)

$ docker run --rm -it cyberdojofoundation/cpp_cpputest sh
$ cd /
$ ls -al

Indeed there is no root folder /cpputest


Tried to recreate gpp_cpputest docker image

>>>>>>There are no Dockerfile's underneath /app/data/languages
>>>>>>.gitignore seems to be the culprit.


Manually added Dockerfile for C++ (g++ 5.3)/CppuTest/_docker_context
rebuilt image
didn't work
There is no user called cyber-dojo *inside* the gpp_cpputest container
Problem is that the image gpp_cpputest is not based FROM user-base
and so does not have cyber-dojo user inside it.




Switched create page to show plain languages
Make katas using C++, CppUTest (and C, CppUTest)
was ok, edited cyber-dojo.sh, /cpputest was there, got a red traffic-light
on the server went to katas/50/5C05.../manifest.json
language:C++ (g++)-CppUTest

Switched create page back to custom exercises.
Made new kata
katas/21/AD6F.../manifest.json
language:C

Aha!!!!
Manually edited the katas manifest.json
to "C++ (g++)-CppUTest"
and it worked. Woooooohoooooo

So the problem is the creation of the kata's manifest.
Looked at the code and fixed the errors.
Works for both setup cases now: languages and exercises.


-------------

Tried again.

new server
installed docker
pulled from gihub
pulled new branch
edited app/docker/cyber-dojo to make two hard-wired env-vars for paths
built a new web image
tried to create kata - failed with nil Rails error.

Look into this on my own server.

Remember
>>>>>>There are no Dockerfile's underneath /app/data/languages
>>>>>>.gitignore seems to be the culprit
>>>>>>It has Dockerfile instead of /Dockerfile (eg)

--------------

Features James mentioned during our chat
- - - - - - - - - - - - - - - - - - - -

revert button is in diff dialog again

on enter page, when kata id is entered and buttons become enabled
it would be nice to see the language and exercise names in the footer.

feature: sed in the editor (search and replace)

feature: copy one animal to a new one (keeping whole traffic-light history)

feature: fork giving two options, viz create new exercise-starting-point.

feature: replace duration on dashboard with time-ticks on top of minute columns.

feature: hovering over a traffic-light gives more detail, viz for changed files (not output)
         name of file, no of deleted lines, no of added lines

feature: traffic-lights have some visual indication if there is a
         "suspiciously large change"
         eg more than 1 file changed
         eg more than 3 lines changed in a non-test file

feature: hyperlink of compiler error on output to file and line number

feature: one one error in C/C++ projects

feature: colour syntax


{'fraction_non_alphanumeric': 0.08233173076923077, 'fraction_numerical': 0.006209935897435897, 'mean_word_length': 3.5552007299270074, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 63, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4078266', 'n_tokens_mistral': 3076, 'n_tokens_neox': 2893, 'n_words': 1426}