<jupyter_start><jupyter_text>Conda EnvironmentsThis notebook automatically configures a conda environment. This notebook is typically what you run before anything else in the project. In that way, you are set up with a controlled and isolated environment for further development/research.  Imports<jupyter_code>import os
from os.path import dirname<jupyter_output><empty_output><jupyter_text>InitializationSome important variables to be used afterwards.<jupyter_code>name = "also_anomaly_detector"

root_dir = dirname(dirname(os.getcwd()))

fn_conda_dep = 'dependencies-deploy.yaml'
fn_conda_dev = 'dependencies-develop.yaml'<jupyter_output><empty_output><jupyter_text>IntroductionThis cookiecutter is set up for optimal use with conda, for local dependency managment. The takeaway is this; for local dependency managment, we rely on conda and nothing else.Note that this has nothing to do with remote dependency managment. This is what you need to take care of when preparing a release of your code which goes via PyPi or alternatives. We treat that as an independent problem. Mixing remote and local dependency managment tends to add complexity instead of removing it.  WorkflowWe distinguish between `deployment` and `development` environments. Of course, in research this is not always that useful or clear. We follow this rule of thumb:```Everything that should end up in pip release, goes in deployment```Which still justifies keeping the two separated.  Deployment EnvironmentThis environment is whatever an end user may need in order to use your package.<jupyter_code>%%bash -s "$name" "$root_dir" "$fn_conda_dep"

source ~/.bashrc

conda env list
%%bash -s "$name" "$root_dir" "$fn_conda_dep"

source ~/.bashrc

cd $2

conda env create -f $3 -n $1<jupyter_output>Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

Downloading and Extracting Packages
python-3.8.5         | 49.3 MB   | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Installing pip dependencies: ...working... Ran pip subprocess with arguments:
['/home/zissou/miniconda3/envs/also_anomaly_detector/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/zissou/repos/also_anomaly_detector/condaenv.7vzngcf5.requirements.txt']
Pip subprocess output:
Collecting numpy
  Downloading numpy-1.19.2-cp38-cp38-manylinux2010_x86[...]<jupyter_text>Installation of `AFFE` At some point, this will become better.<jupyter_code>%%bash -s "$name" "$root_dir"

source ~/.bashrc

cd $2

conda activate $1

pip install git+https://github.com/eliavw/affe.git<jupyter_output>Collecting git+https://github.com/eliavw/affe.git
  Cloning https://github.com/eliavw/affe.git to /tmp/pip-req-build-y71wvnbo
Requirement already satisfied: numpy in /home/zissou/miniconda3/envs/also_anomaly_detector/lib/python3.8/site-packages (from affe==0.0.0) (1.19.2)
Requirement already satisfied: scipy in /home/zissou/miniconda3/envs/also_anomaly_detector/lib/python3.8/site-packages (from affe==0.0.0) (1.5.2)
Requirement already satisfied: pandas in /home/zissou/miniconda3/envs/also_anomaly_detector/lib/python3.8/site-packages (from affe==0.0.0) (1.1.2)
Collecting lxml
  Using cached lxml-4.5.2-cp38-cp38-manylinux1_x86_64.whl (5.4 MB)
Collecting requests
  Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
Processing /home/zissou/.cache/pip/wheels/91/cf/b0/0c9998060b55ca80ea7a50a8639c3bdc6ba886eeff014bc9ac/psutil-5.7.2-cp38-cp38-linux_x86_64.whl
Processing /home/zissou/.cache/pip/wheels/93/7f/7d/78ec535a4340ef2696aad8b17fe8bb063d56301bd62881b069/dill-0.3.2-py3-none-any.whl[...]<jupyter_text>Development environment<jupyter_code>%%bash -s "$name" "$root_dir" "$fn_conda_dev"

source ~/.bashrc

cd $2

conda activate $1

conda env update -n $1 -f $3<jupyter_output>Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

Downloading and Extracting Packages
attrs-20.2.0         | 42 KB     | ########## | 100% 
pyzmq-19.0.2         | 445 KB    | ########## | 100% 
zeromq-4.3.2         | 511 KB    | ########## | 100% 
mkdocs-material-5.5. | 1.9 MB    | ########## | 100% 
bleach-3.2.1         | 112 KB    | ########## | 100% 
pygments-2.7.1       | 671 KB    | ########## | 100% 
tqdm-4.49.0          | 55 KB     | ########## | 100% 
cffi-1.14.3          | 225 KB    | ########## | 100% 
traitlets-5.0.4      | 175 KB    | ########## | 100% 
pyrsistent-0.17.3    | 90 KB     | ########## |[...]<jupyter_text>Install also_anomaly_detectorInstalling the source code of this repo as a package itself. This is often quite useful.<jupyter_code>%%bash -s "$name" "$root_dir" "$fn_conda_dev"

source ~/.bashrc

cd $2

conda activate $1

python setup.py develop<jupyter_output>running develop
running egg_info
creating src/also_anomaly_detector.egg-info
writing src/also_anomaly_detector.egg-info/PKG-INFO
writing dependency_links to src/also_anomaly_detector.egg-info/dependency_links.txt
writing requirements to src/also_anomaly_detector.egg-info/requires.txt
writing top-level names to src/also_anomaly_detector.egg-info/top_level.txt
writing manifest file 'src/also_anomaly_detector.egg-info/SOURCES.txt'
reading manifest file 'src/also_anomaly_detector.egg-info/SOURCES.txt'
writing manifest file 'src/also_anomaly_detector.egg-info/SOURCES.txt'
running build_ext
Creating /home/zissou/miniconda3/envs/also_anomaly_detector/lib/python3.8/site-packages/also-anomaly-detector.egg-link (link to src)
Adding also-anomaly-detector 0.0.0 to easy-install.pth file

Installed /home/zissou/repos/also_anomaly_detector/src
Processing dependencies for also-anomaly-detector==0.0.0
Searching for pandas==1.1.2
Best match: pandas 1.1.2
Adding pandas 1.1.2 to easy-install.pth file

Usi[...]<jupyter_text>Jupyter kernelExpose the environment to your jupyter.<jupyter_code>%%bash -s "$name" "$root_dir" "$fn_conda_dep"

source ~/.bashrc

cd $2

conda activate $1
python -m ipykernel install --user --name $1 --display-name "$1"<jupyter_output>Installed kernelspec also_anomaly_detector in /home/zissou/.local/share/jupyter/kernels/also_anomaly_detector
{'fraction_non_alphanumeric': 0.11835184102863823, 'fraction_numerical': 0.041057860900058446, 'mean_word_length': 4.4716227018385295, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '121758', 'license': 'MIT', 'n_tokens_mistral': 2514, 'n_tokens_neox': 2302, 'n_words': 654, 'path': 'note/deploy/conda.ipynb', 'repo_name': 'eliavw/also-anomaly-detector', 'size': '6844'}