<commit_before>---
layout: default
---

# Cheatsheet

I'll probably forget these are here and look them up on stack exchange... ..sigh..

## Python

- Best way to install tensorflow such that all cuda dependencies are handled correctly:
```bash
conda create tf-env python=3.6
conda activate tf-env
conda install -c anaconda tensorflow-gpu=1.12
```

- Remove output cells from a jupyter notebook:
```bash
jupyter nbconvert notebook.ipynb --to notebook --ClearOutputPreprocessor.enabled=True --stdout > notebook_clear.ipynb
```

- Assign arguments of a class to self:
```python
class Foo():
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)
#
foo = Foo(a=1,b=2)
print(foo.a)  # 1
```

- Call a function using a string:
```python
def foo(arg):
    pass
# One way
locals()['foo'](arg)
# Another way
eval('foo')(arg)
```

- Use matplotlib on a server: `plt.switch_backend('agg')` in the import block.
- Tensorflow/keras control gpu usage:
```python
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.allow_growth=True
sess = tf.Session(config=config)
set_session(sess)
```
```python
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.5
set_session(tf.Session(config=config))
```

- Use tqdm with multiprocessing:
```python
pool = multiprocessing.Pool(n_processes=4)
for _ in tqdm(pool.imap(map_function, input_iterable), total=len(input_iterable)):
  pass
```

- Make a movie with opencv.  A list of video codecs is [here](https://gist.github.com/takuma7/44f9ecb028ff00e2132e).
```python
from cv2 import VideoWriter, VideoWriter_fourcc
# Init
video_dest = 'video.mp4'
fps = 1
im_rows, im_cols = 100, 100
codec = VideoWriter_fourcc(*'mp4v')
video = VideoWriter(video_dest, codec, fps, (im_cols,im_rows))
# Main loop
for frame in range(nframes):
    im = np.random.random((im_rows, im_cols, 3))*255
    im = im.astype(np.uint8)
    video.write(im)
video.release()
```

- Print over the current line:
```python
import sys<commit_msg>commit<commit_after>---
layout: default
---

# Cheatsheet

I'll probably forget these are here and look them up on stack exchange... ..sigh..

## Python

- Best way to install tensorflow such that all cuda dependencies are handled correctly:
```bash
conda create tf-env python=3.6
conda activate tf-env
conda install -c anaconda tensorflow-gpu=1.12
```

- Remove output cells from a jupyter notebook:
```bash
jupyter nbconvert notebook.ipynb --to notebook --ClearOutputPreprocessor.enabled=True --stdout > notebook_clear.ipynb
```

- Assign arguments of a class to self:
```python
class Foo():
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)
#
foo = Foo(a=1,b=2)
print(foo.a)  # 1
```

- Call a function using a string:
```python
def foo(arg):
    pass
# One way
locals()['foo'](arg)
# Another way
eval('foo')(arg)
```

- Use matplotlib on a server: `plt.switch_backend('agg')` in the import block.
- Tensorflow/keras control gpu usage:
```python
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.allow_growth=True
sess = tf.Session(config=config)
set_session(sess)
```
```python
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.5
set_session(tf.Session(config=config))
```

- Use tqdm with multiprocessing:
```python
pool = multiprocessing.Pool(n_processes=4)
for _ in tqdm(pool.imap(map_function, input_iterable), total=len(input_iterable)):
  pass
```

- Make a movie with opencv.  A list of video codecs is [here](https://gist.github.com/takuma7/44f9ecb028ff00e2132e).
```python
from cv2 import VideoWriter, VideoWriter_fourcc
# Init
video_dest = 'video.mp4'
fps = 1
im_rows, im_cols = 100, 100
codec = VideoWriter_fourcc(*'mp4v')
video = VideoWriter(video_dest, codec, fps, (im_cols,im_rows))
# Main loop
for frame in range(nframes):
  im = np.random.random((im_rows, im_cols, 3))*255
  im = im.astype(np.uint8)
  video.write(im)
video.release()
```

- Print over the current line:
```python
import sys
{'fraction_non_alphanumeric': 0.13044524669073407, 'fraction_numerical': 0.018772563176895306, 'mean_word_length': 4.895035460992908, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '598513', 'n_tokens_mistral': 1527, 'n_tokens_neox': 1432, 'n_words': 397}