<jupyter_start><jupyter_text>CARTO Data Observatory This is a basic template notebook to start exploring your new Geography from CARTO's Data Observatory via the Python library [CARTOframes](https://carto.com/cartoframes).You can find more details about how to use CARTOframes in the [Quickstart guide](https://carto.com/developers/cartoframes/guides/Quickstart/).  Setup  InstallationMake sure that you have the latest version installed. Please, find more information in the [Installation guide](https://carto.com/developers/cartoframes/guides/Installation/).<jupyter_code>!pip install -U cartoframes
# Note: a kernel restart is required after installing the library

import cartoframes

cartoframes.__version__<jupyter_output><empty_output><jupyter_text>Credentials In order to be able to use the Data Observatory via CARTOframes, you need to set your CARTO account credentials first.<jupyter_code>from cartoframes.auth import set_default_credentials

username = 'YOUR_USERNAME'
api_key  = 'YOUR_API_KEY'  # Master API key. Do not make this file public!

set_default_credentials(username, api_key)<jupyter_output><empty_output><jupyter_text>**⚠️ Note about credentials**For security reasons, we recommend storing your credentials in an external file preventing publishing them by accident within your notebook. You can get more information in the section *Setting your credentials* of the [Authentication guide](https://carto.com/developers/cartoframes/guides/Authentication/).  Geography operations  Metadata explorationIn this section we will explore some basic information regarding the Geography you have licensed. More information on how to explore metadata associated to a Geography is available in the [Data discovery guide](https://carto.com/developers/cartoframes/guides/Data-discovery/).In order to access the Geography and its associeted metadata, you need to provide the "ID" which is a unique identifier of that Geography. The IDs of your Geography are available from Your Subscriptions page in the CARTO Dashboard and via the Discovery methods in CARTOFrames.<jupyter_code>from cartoframes.data.observatory import Geography

geography = Geography.get('YOUR_ID')
# Retrieve some general metadata about the Geography
geography.to_dict()<jupyter_output><empty_output><jupyter_text>Access the dataNow that we have explored some basic information about the Geography, we will proceed to download a sample of the Geography into a dataframe so we can operate it in Python. Geographies can be downloaded in full or by applying a filter with a SQL query. More info on how to download the Geography or portions of it is available in the [Data discovery guide](https://carto.com/developers/cartoframes/guides/Data-discovery/).<jupyter_code># Filter by SQL query
query = "SELECT * FROM $geography$ LIMIT 50"

geography_df = geography.to_dataframe(sql_query=query)<jupyter_output><empty_output><jupyter_text>**Note about SQL filters**Our SQL filtering queries allow for any PostgreSQL and PostGIS operation, so you can filter the rows (by a WHERE condition) or the columns (using the SELECT). Some common examples are filtering the Geography by bounding box or filtering by column value: ```SELECT * FROM $geography$ WHERE ST_IntersectsBox(geom, -74.044467,40.706128,-73.891345,40.837690)``````SELECT do_area, geom FROM $geography$```A good tool to get the bounding box details for a specific area is [bboxfinder.com](http://bboxfinder.com/0.000000,0.000000,0.000000,0.000000).<jupyter_code># First 10 rows of the Geography sample
geography_df.head()<jupyter_output><empty_output><jupyter_text>VisualizationNow that we have downloaded some data into a dataframe we can leverage the visualization capabilities of CARTOframes to build an interactive map.More info about building visualizations with CARTOframes is available in the [Visualization guide](https://carto.com/developers/cartoframes/guides/Visualization/).<jupyter_code>from cartoframes.viz import Layer

Layer(geography_df, geom_col='geom')<jupyter_output><empty_output><jupyter_text>**Note about variables**CARTOframes allows you to make data-driven visualizations from your Geography variables (columns) via the Style helpers. These functions provide out-of-the-box cartographic styles, legends, popups and widgets.Style helpers are also highly customizable to reach your desired visualization setting simple parameters. The helpers collection contains functions to visualize by color and size, and also by type: category, bins and continuous, depending on the type of the variable.<jupyter_code>from cartoframes.viz import color_bins_style

Layer(geography_df, color_bins_style('do_area'), geom_col='geom')<jupyter_output><empty_output><jupyter_text>Upload to CARTO accountIn order to operate with the data in CARTO Builder or to build a CARTOFrames visualization reading the data from a table in the Cloud instead of having it in your local environment (with its benefits in performance), you can load the dataframe as a table in your CARTO account.More info in the [Data Management guide](https://carto.com/developers/cartoframes/guides/Data-management/).<jupyter_code>from cartoframes import to_carto

to_carto(geography_df, 'my_geography', geom_col='geom')
# Build a visualization reading the data from your CARTO account
Layer('my_geography')<jupyter_output><empty_output><jupyter_text>Save to fileFinally, you can also export the data into a CSV file. More info in the [Data discovery guide](https://carto.com/developers/cartoframes/guides/Data-discovery/).<jupyter_code># Filter by SQL query
query = "SELECT * FROM $geography$ LIMIT 50"

geography.to_csv('my_geography.csv', sql_query=query)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07095187917105725, 'fraction_numerical': 0.011591148577449948, 'mean_word_length': 6.376943005181347, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '363883', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1645, 'n_tokens_neox': 1554, 'n_words': 612, 'path': 'docs/examples/data_observatory/do_geography_notebook_template.ipynb', 'repo_name': 'CartoDB/cartoframes', 'size': '5694'}