<jupyter_start><jupyter_text>Presenting Data> Your job as a data scientist is to communicate. Often you are communicating insight from distilling lots of data (models, stats, EDA, narative/editorialization) // making data accessible/self-service (run queries, dashboards that auto update) // communicating performance/diagnostics of complex models/processes.* Why Visualize    * Point Stats vs distributions        * much more information in visual (or other senses) channel    * Making Data Interpretable/Accessible (don't need/want to understand theory necessarily)        * Dashboards (model/SQL API/interface)            * streaming data            * linked charts        * Reproducible analysis/fact checking        * Knowldege sharing    * Narrative/editorialize    * General Process    * EDA vs Explanatory* What to communicate    * Results of analysis    * Insights found through EDA    * Models performance/diagnostics (residuals, ROC plots, etc.)        * builds trust in model        * communicates uncertainty* How to communicate   * Visual encodings   * Chart Types   * Maps   * Data Narratives        * Adding Context/Editorialization<jupyter_code>from bokeh.plotting import figure, show, output_notebook
output_notebook()<jupyter_output><empty_output><jupyter_text>Sanity CheckDoes it work? http://bokeh.pydata.org/en/latest/docs/user_guide/quickstart.htmluserguide-quickstart<jupyter_code># prepare some data
x = [1, 2, 3, 4, 5]
y = [6, 7, 2, 4, 5]

# output to static HTML file
# output_file("lines.html", title="line plot example")

# create a new plot with a title and axis labels
p = figure(title="simple line example", x_axis_label='x', y_axis_label='y')

# add a line renderer with legend and line thickness
p.line(x, y, legend="Temp.", line_width=15)

# show the results
show(p)<jupyter_output><empty_output><jupyter_text>Grammar of Graphics> “the fundamental principles or rules of an art or science” (OED Online 1989). A good grammar will allow us to gaininsight into the composition of complicated graphics, and reveal unexpected connectionsbetween seemingly different graphics (Cox 1978). A grammar provides a strong foundationfor understanding a diverse range of graphics. A grammar may also help guide us on whata well-formed or correct graphic looks like, but there will still be many grammaticallycorrect but nonsensical graphics. >> -- Wickham ([A Layered Grammar of Graphics](http://byrneslab.net/classes/biol607/readings/wickham_layered-grammar.pdf)) Components [Visual index](http://ggplot.yhathq.com/docs/index.html) (Courtesy of yHat)* Layers    * data -> aesthetic mapping    * statistical transforms    * geometric objects (`geom`)    * position    * different "view" of the same data* Scales* Coordinate system* Faceting_can change each in relative isolation_ ![](http://r4ds.had.co.nz/images/visualization-grammar-3.png)([source](http://r4ds.had.co.nz/visualize.html))  Histogram: binning stat + bar geom```rggplot(data = diamonds, mapping = aes(price)) +    layer(geom = "bar", stat = "bin",    mapping = aes(y = ..count..))```  In Python* http://bokeh.pydata.org/en/0.11.1/docs/gallery/histogram.html* http://bokeh.pydata.org/en/0.11.1/docs/gallery/histograms_chart.html<jupyter_code>from bokeh.charts import Histogram
from bokeh.sampledata.autompg import autompg as df

df.sort('cyl', inplace=True)

hist = Histogram(df, values='hp', title="HP Distribution", legend='top_right')

show(hist)
import numpy as np
from bokeh.models import HoverTool, BoxSelectTool

TOOLS = [BoxSelectTool(), HoverTool()]

# create our canvas
p1 = figure(title="HP Distribution", background_fill_color="#E8DDCB", tools=TOOLS)

# stat
hist, edges = np.histogram(df.hp, density=True, bins=50)

# geom
p1.quad(top=hist, bottom=0, left=edges[:-1], right=edges[1:],
        fill_color="#036564", line_color="#033649")

show(p1)<jupyter_output><empty_output><jupyter_text>[Customizing Tooltips](http://bokeh.pydata.org/en/0.11.1/docs/user_guide/tools.htmlsetting-tool-visuals)<jupyter_code>df.sort('cyl', inplace=True)

hist = Histogram(df, values='hp', color='cyl',
                 title="HP Distribution by Cylinder Count", legend='top_right')

show(hist)<jupyter_output><empty_output><jupyter_text>Let's Use some real (interesting) data!<jupyter_code>from bokeh.models import GeoJSONDataSource
from bokeh.plotting import figure
from bokeh.sampledata.sample_geojson import geojson


geo_source = GeoJSONDataSource(geojson=geojson)

p = figure()
p.circle(x='x', y='y', alpha=0.9, source=geo_source)
show(p)
import pandas as pd
# more time/compute intensive to parse dates. but we know we definitely have/need them
df = pd.read_csv('data/sf_listings.csv', parse_dates=['last_review'], infer_datetime_format=True)
df_reviews = pd.read_csv('data/reviews.csv', parse_dates=['date'], infer_datetime_format=True)
# index DataFrame on listing_id in order to join datasets
reindexed_df = df_reviews.set_index('listing_id')
reindexed_df.head()
# remember the original id in a column to group on
df['listing_id'] = df['id']
df_listing = df.set_index('id')
df_listing.head()
# join the listing information with the review information
review_timeseries = df_listing.join(reindexed_df)

print review_timeseries.columns
review_timeseries.head()
# lets try a pivot table...
reviews_over_time = pd.crosstab(review_timeseries.date, review_timeseries.neighbourhood)
reviews_over_time.head()
# smooth by resampling by month
reviews_over_time.resample('M').mean()[['Mission', 'South of Market', 'Noe Valley']].plot(figsize=(12,6))
TOOLS = "pan,wheel_zoom,box_zoom,reset,save,hover"
d = reviews_over_time.resample('M').mean()

p = figure(x_axis_type="datetime", tools=TOOLS)

p.line(d.index, d['Mission'])
show(p)
import bokeh.charts as charts

line = charts.Line(d, y=['Mission', 'South of Market', 'Noe Valley'],
            color=['Mission', 'South of Market', 'Noe Valley'],
            title="Interpreter Sample Data", ylabel='Duration', legend=True, tools=TOOLS)

show(line)
from bokeh.models.widgets import Select
from bokeh.io import output_file, show, vform

select = Select(title="Option:", value="foo", options=list(reviews_over_time))

show(vform(select))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11737164998385534, 'fraction_numerical': 0.010332579916047788, 'mean_word_length': 5.308553971486762, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '566072', 'license': 'MIT', 'n_tokens_mistral': 2026, 'n_tokens_neox': 1925, 'n_words': 539, 'path': 'presenting-data.ipynb', 'repo_name': 'hopelessoptimism/data-science-workshops', 'size': '6194'}