<jupyter_start><jupyter_text>As an example of the building and presenting a short analysis of data...  using [RISE: Reveal.js - Jupyter/IPython Slideshow Extension](https://rise.readthedocs.io/en/stable/index.html) <div class="alert alert-info" style="font-family:Helvetica,                                     Ariel, Garamond;                                     font-size:20px">    Some examples we might be interested in:            Using APIs to gather data from the web (Requests)        Relationships between things (NetworkX)               Faker for generating random fake data with certain characteristics         Numpy and SciPy        Biopython /Bioconda        Pandas for dataframes (csv and spreadsheet replacement)      Pandas is a good place to start...      You can leave me out   Something I don't currently miss is the certainty that I should be reading things.> Can you really keep up if you wanted to be unbiased and **"on top of the literature"**---Timeline export in is an easy option from the [Pubmed search page](https://pubmed.ncbi.nlm.nih.gov/)so we can export total publications per year with:> ("1945"[Date - Publication] : "2020"[Date - Publication])and I might be interested on one of many areas I felt I should be reading:> circadian rhythms AND ("1945"[Date - Publication] : "2020"[Date - Publication])  The first thing we will do is import a library that handles data input, output and manipulation incredably well - [Pandas](https://pandas.pydata.org/)<jupyter_code>import pandas as pd
# a DataFrame from a local .CSV file
df_circ = pd.read_csv('./PubMed_Timeline_Results_by_Year_Circadian.csv',skiprows=1, index_col=['Year'])
df_circ.head(3)
df_circ.Count.plot()<jupyter_output><empty_output><jupyter_text>and all of these cells remain live (unless you export to html).  So you can modify during presentations, if you are feeling boldlet us see how  many papers we need to review and possibly read each day<jupyter_code>df_circ.assign(Per_day = df_circ.Count / 365)<jupyter_output><empty_output><jupyter_text>Publications are increasing in every field?If we want to see context (overall publication rates) we can import this data and then join the two dataframes<jupyter_code>#Import a second dataframe from .csv file
df_all = pd.read_csv('./PubMed_Timeline_Results_by_Year_All.csv',skiprows=1, index_col=['Year'])
df_all.head()<jupyter_output><empty_output><jupyter_text>and then join the two dataframes<jupyter_code>df_combine = df_all.join(df_circ, on = 'Year', lsuffix='_all')
df_combine.head()

df_final = df_combine.assign(Percentage= (df_combine.Count / df_combine.Count_all)*100)
df_final.Percentage.plot()<jupyter_output><empty_output><jupyter_text>We can export our processed data in a range range of formats<jupyter_code>df_final.to_excel('./final_publications_timeline.xlsx', sheet_name='circadian')<jupyter_output><empty_output><jupyter_text>We can use this library and Pandas to create and dataframe of testing data***Our fake DataFrame contains a name, country and coordinates (not related to each other), some random numbers, an e-mail address, timestamp, a paragraph of random text, and a 'bs'  (short for business, but then this is all 'bs')***<jupyter_code># import out libraries, faker to make data and numpy (numerical python) for random numbers
import faker as fk
import numpy as np
# let's create a fake data generator for here   
fake_demo =fk.Faker('en_GB')
# and have a look at some of the options for generating data
fake_demo.first_name_female()
# country
fake_demo.country()
# or geographical co-ordinates
fake_demo.latlng()
 # and maybe a timestamp
fake_demo.iso8601()<jupyter_output><empty_output><jupyter_text>We can also create a range of other data to test our analysis / model<jupyter_code>def create_fake_row(num=1):
    """Our fake DataFrame contains a name, country and coordinates (not related to each other),
    some random numbers, an e-mail address, timestamp, a paragraph of random text,
    and a 'bs'  (short for business, but then this is all 'bs')"""
    out = [{"name":fake_demo.name(),
            "LatLong":fake_demo.latlng(),
            "Country":fake_demo.country(),  # will not match LatLong
            "email":fake_demo.email(),
            "date_time":fake_demo.iso8601(),
            "paragraph":fake_demo.paragraph(),
            "bs":fake_demo.bs(),
            "randomdata":np.random.randint(1,1000)} for x in range(num)]
    return out
df_fake = pd.DataFrame(create_fake_row(1000))

df_fake.head()
#Split LatLong column up
df_fake =df_fake.join(pd.DataFrame(df_fake.LatLong.values.tolist(), columns=['Lat', 'Long']))<jupyter_output><empty_output><jupyter_text>we can split columns into new columns to help useage later if needed<jupyter_code>fake_density =df_fake.Country.value_counts()

fake_density<jupyter_output><empty_output><jupyter_text>Next we are going to import some of the choices we have for exploring geographical data<jupyter_code>import geopandas as gpd
import matplotlib
%matplotlib inline
gpd.datasets.available
world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
world.head()
fake_demo_mapped = world.join(fake_density, on='name',how='inner')

fake_demo_mapped.head()
#  A matplotlib option 
base = world.plot(alpha=0.1, color='gray', edgecolor='black', figsize=(16,16))
fake_demo_mapped.plot(column = 'Country',cmap='Greens', legend=False, ax=base)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09870609981515711, 'fraction_numerical': 0.009426987060998152, 'mean_word_length': 4.780982905982906, 'pattern_counts': {'":': 8, '<': 38, '<?xml version=': 0, '>': 41, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '596320', 'license': 'MIT', 'n_tokens_mistral': 1706, 'n_tokens_neox': 1636, 'n_words': 533, 'path': 'Notebook1B_presenting_data.ipynb', 'repo_name': 'LozRiviera/LAB_Intro_Jupyter', 'size': '5410'}