<jupyter_start><jupyter_text>TM351 VM Installation TestThis notebook provides a series of tests to ensure that the Docker container is running correctly.Run each cell in turn by clicking the play button or keyboard shortcut `shift-return`. (A full list of keyboard shortcuts can be found from the *Help* menu or via the keyboard shortcut `ESC-h`.)The cells should all run without error.  VersionsDisplay the VM build version and build time, as well as database service versions and `pandas` version.<jupyter_code>!cat /opt/version.txt
! psql --version
! mongod --version
import pandas as pd
pd.__version__<jupyter_output><empty_output><jupyter_text>Test Core PackagesTest loading and use of essential packages:<jupyter_code>from ou_tm351_py import test_install

test_install()
import pandas as pd
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Test plot:<jupyter_code>#When this cell is run, a simple line chart should be displayed
plt.plot([1,2,3,4])
plt.ylabel('some numbers')
plt.show()<jupyter_output><empty_output><jupyter_text>Database testsCheck that the database services are running as required:<jupyter_code>#SET DATABASE CONNECTION STRINGS
PGCONN='postgresql://testuser:testpass@localhost:5432/testdb'
MONGOHOST='localhost'
MONGOPORT=27017
MONGOCONN='mongodb://{MONGOHOST}:{MONGOPORT}/'.format(MONGOHOST=MONGOHOST,MONGOPORT=MONGOPORT)<jupyter_output><empty_output><jupyter_text>PostgreSQLCheck the connection to the PostgreSQL server:<jupyter_code>from sqlalchemy import create_engine
engine = create_engine(PGCONN)
#Run a simple query on a default table
from pandas import read_sql_query as psql

psql("SELECT table_schema,table_name FROM information_schema.tables \
    ORDER BY table_schema,table_name LIMIT 3;", engine)
#A table containing three rows should appear<jupyter_output><empty_output><jupyter_text>SQL Cell MagicWe can use cell magics to allow the writing of SQL statements within a code cell flagged appropriately.To invoke the cell magic in a cell, we need to run the following (though we could perhaps autoload this in every notebook?)In the following example, magic SQL cells will be configured to run as a the root user:<jupyter_code>%load_ext sql
%sql {PGCONN}
%%sql
SELECT table_schema,table_name FROM information_schema.tables ORDER BY table_schema,table_name LIMIT 1;<jupyter_output>* postgresql://testuser:***@localhost:5432/testdb
1 rows affected.<jupyter_text>Test the ability to pull the result of a SQL query directly into a dataframe:<jupyter_code>demo=%sql SELECT table_schema FROM information_schema.tables LIMIT 3
demo<jupyter_output>* postgresql://testuser:***@localhost:5432/testdb
3 rows affected.<jupyter_text>MongDBTest that the MongoDB database is running...This example also shows how to connect to the database:<jupyter_code>import pymongo
from pymongo import MongoClient
#If connecting to the default port, you can omit the second (port number) parameter
# Open a connection to the Mongo server, open the accidents database and name the collections of accidents and labels
c = pymongo.MongoClient(MONGOCONN)<jupyter_output><empty_output><jupyter_text>By default, this database should contain an accidents database along with any default databases:<jupyter_code>c.list_database_names()
db = c.accidents
accidents = db.accidents
accidents.find_one()<jupyter_output><empty_output><jupyter_text>Chart TestsViewing data in charts provides a handy way of actually looking at your data:<jupyter_code>import seaborn
from numpy.random import randn

data = randn(75)
plt.hist(data);
#Running this cell should produce a histogram.
import numpy as np

ts = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))
ts = ts.cumsum()
ts.plot();
#Running this cell should produce a line chart.<jupyter_output><empty_output><jupyter_text>MapsSeveral of the data investigations may benefit from displaying data on a map.Test that the mapping functions work:<jupyter_code>import folium
#Note - this will not display a map if you are offline.

#A network connection is required to retrieve the map tiles
osmap = folium.Map(location=[52.01, -0.71], zoom_start=13,height=500,width=800)
folium.Marker([52.0250, -0.7056],
              popup='The <b>Open University</b> campus.').add_to(osmap)

osmap
#Example of how to explicitly save a map as an HTML file
osmap.save('test.html')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07995444191343963, 'fraction_numerical': 0.01685649202733485, 'mean_word_length': 6.649825783972125, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '226240', 'license': 'MIT', 'n_tokens_mistral': 1401, 'n_tokens_neox': 1331, 'n_words': 461, 'path': 'TM351 VM Installation Test.ipynb', 'repo_name': 'ouseful-testing/nbval-test', 'size': '4390'}