<jupyter_start><jupyter_text>storemagic - Don't lose your variables in Jupyter Notebook> A post on IPython extension 'storemagic' to keep your important variables in persistent storage. Use this trick to keep them safe and stop pulling your hair (if there are any left) - toc: true - badges: true- comments: true- categories: [jupyter, python]- keyword: [jupyter, notebook, python, tip, IPython, store, storemagic]- image: images/copied_from_nb/images/2022-05-30-storemagic-jupyter-notebook.jpeg ![](images/2022-05-30-storemagic-jupyter-notebook.jpeg)  Enviornment<jupyter_code>!jupyter --version<jupyter_output>Selected Jupyter core packages...
IPython          : 7.29.0
ipykernel        : 6.4.1
ipywidgets       : 7.6.5
jupyter_client   : 6.1.12
jupyter_core     : 4.8.1
jupyter_server   : 1.4.1
jupyterlab       : 3.2.1
nbclient         : 0.5.3
nbconvert        : 6.1.0
nbformat         : 5.1.3
notebook         : 6.4.5
qtconsole        : 5.1.1
traitlets        : 5.1.0<jupyter_text>AboutThis post is about a very important (and highly underrated) Jupyter notebook extension "storemagic". We will see how we can use it to store our important data, and recover safely even when the unforgivable happens. IntroductionAll Jupyter Notebook lovers remember their petrified moments when they are struck by a [dark terminal] force, their heart skips a beat, and they stare blanky at the screen. When they know that the unforgivable has happened but they still timidly move the cursor to the last cell and press Enter with the faintest hope that it might be still alive. But fate has chosen something else for us, and we poor people are left cursing ourselves as to why we did not save our precious work!I have been to those despair valleys (many times) till I stumbled upon 'storemagic'. It seems like archaic magic as so few people know or use it, but I can guarantee once you get used to it you will barely write more than five lines in your notebooks without using it. So let's see how it works. Store a variableUse `%store` magic for lightweight persistence. It stores variables, aliases and macros in IPython’s database. Let's create a variable and then store it using this magic.<jupyter_code>##
# create a variable
var_hello = "hello world!"

# store this variable
%store var_hello<jupyter_output>Stored 'var_hello' (str)<jupyter_text>Now I am going to intentionally restart the kernel. We can check that our created variable is now gone from the memory.<jupyter_code>var_hello<jupyter_output><empty_output><jupyter_text>But no worries. We have it stored safely using our magic. So let's get it back.<jupyter_code>## 
# get the variable back from store
%store -r var_hello<jupyter_output><empty_output><jupyter_text>Okay, we have our variable back and (with a sigh of relief) we can use it again.<jupyter_code>var_hello<jupyter_output><empty_output><jupyter_text>Let's create a few more variables and do some more magic with them.<jupyter_code>##
# create variables
var_foo = [1,2,3,4]
var_bar = {'a':var_hello}

# store multiple variables
%store var_foo var_bar<jupyter_output>Stored 'var_foo' (list)
Stored 'var_bar' (dict)<jupyter_text>Check all stored varaiblesWe can check all the varaibles stored using the following magic command.<jupyter_code>%store<jupyter_output>Stored variables and their in-db values:
var_bar               -> {'a': 'hello world!'}
var_foo               -> [1, 2, 3, 4]
var_hello             -> 'hello world!'<jupyter_text>Remove a variable from storeTo remove a variable from our storage is also straight forward. Put its name after `%store -d` flag<jupyter_code>##
# remove 'var_hello'
%store -d var_hello<jupyter_output><empty_output><jupyter_text>'var_hello' is now gone. Forever ...<jupyter_code>##
# check the remaining variables in store
%store<jupyter_output>Stored variables and their in-db values:
var_bar             -> {'a': 'hello world!'}
var_foo             -> [1, 2, 3, 4]<jupyter_text>Remove all variables from storeIf you want to remove all the varaibles from store and start clean then use `-z` flag<jupyter_code>##
# remove all variables
%store -z<jupyter_output><empty_output><jupyter_text>Kaboom! all variables are gone.<jupyter_code>##
# check store
%store<jupyter_output>Stored variables and their in-db values:
{'fraction_non_alphanumeric': 0.08108741504569955, 'fraction_numerical': 0.016170611670963207, 'mean_word_length': 4.388888888888889, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '229582', 'license': 'Apache-2.0', 'n_tokens_mistral': 1392, 'n_tokens_neox': 1327, 'n_words': 518, 'path': '_notebooks/2022-05-30-storemagic-jupyter-notebook.ipynb', 'repo_name': 'hassaanbinaslam/myblog', 'size': '4267'}