<jupyter_start><jupyter_text>Using Intake to Build Data Catalogs for Data ScienceTopics included in this notebook:* [What is Intake](intake)* [Intake Example - Using URIs to OmniSci Databases and SQL Expressions](example1)  What is Intake ***[Intake](https://intake.readthedocs.io) - Taking the pain out of data access and distribution***[Intake](https://intake.readthedocs.io) is a lightweight package for finding, investigating,   loading and disseminating data. It is useful for a variety of different groups and acts as   a common platform that everyone can use to smooth the progression of data from developers   and providers to users.    It is a cataloging system for listing data sources, their metadata and parameters, and   referencing which of driver should load each. In short, [Intake](https://intake.readthedocs.io)   allows the user to specify data sources via human-readable YAML catalogs, and then   transparently load them and begin analyzing data.For this example, we'll use an intake plugin for accessing OmniSci databases.   Intake Example - Using URIs to OmniSci Databases and SQL ExpressionsAs an example, we'll use a catalog which uses a URI to an OmniSci Database and SQL expressions.We begin by loading the catalog file:<jupyter_code>import intake
uri_catalog = intake.open_catalog('../catalog_uri.yml')<jupyter_output><empty_output><jupyter_text>We can interactively inspect the items in the catalog:<jupyter_code>list(uri_catalog)<jupyter_output><empty_output><jupyter_text>We can also display the individual catalog items to get more information:<jupyter_code>uri_catalog.flights<jupyter_output><empty_output><jupyter_text>With the catalog loaded, we can read full datasets into memory.<jupyter_code>fault_df = uri_catalog.faults.read()
fault_df.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06847045831032579, 'fraction_numerical': 0.0005521811154058532, 'mean_word_length': 6.023255813953488, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '91849', 'license': 'MIT', 'n_tokens_mistral': 523, 'n_tokens_neox': 498, 'n_words': 203, 'path': 'notebooks/03_Using_Intake_to_Build_Data_Catalogs_for_Data_Science.ipynb', 'repo_name': 'Quansight/omnisci-examples', 'size': '1811'}