<jupyter_start><jupyter_text>[![AWS Data Wrangler](_static/logo.png "AWS Data Wrangler")](https://github.com/awslabs/aws-data-wrangler) 25 - Redshift - Loading Parquet files with Spectrum  Enter your bucket name:<jupyter_code>import getpass
bucket = getpass.getpass()
PATH = f"s3://{bucket}/files/"<jupyter_output>···········································<jupyter_text>Mocking some Parquet Files on S3<jupyter_code>import awswrangler as wr
import pandas as pd

df = pd.DataFrame({
    "col0": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
    "col1": ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"],
})

df
wr.s3.to_parquet(df, PATH, max_rows_by_file=2, dataset=True, mode="overwrite");<jupyter_output><empty_output><jupyter_text>Crawling the metadata and adding into Glue Catalog<jupyter_code>wr.s3.store_parquet_metadata(
    path=PATH,
    database="aws_data_wrangler",
    table="test",
    dataset=True,
    mode="overwrite"
)<jupyter_output><empty_output><jupyter_text>Running the CTAS query to load the data into Redshift storage<jupyter_code>con = wr.redshift.connect(connection="aws-data-wrangler-redshift")
query = "CREATE TABLE public.test AS (SELECT * FROM aws_data_wrangler_external.test)"
with con.cursor() as cursor:
    cursor.execute(query)<jupyter_output><empty_output><jupyter_text>Running an INSERT INTO query to load MORE data into Redshift storage<jupyter_code>df = pd.DataFrame({
    "col0": [10, 11],
    "col1": ["k", "l"],
})
wr.s3.to_parquet(df, PATH, dataset=True, mode="overwrite");
query = "INSERT INTO public.test (SELECT * FROM aws_data_wrangler_external.test)"
with con.cursor() as cursor:
    cursor.execute(query)<jupyter_output><empty_output><jupyter_text>Checking the result<jupyter_code>query = "SELECT * FROM public.test"
wr.redshift.read_sql_table(con=con, schema="public", table="test")
con.close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16989247311827957, 'fraction_numerical': 0.013978494623655914, 'mean_word_length': 6.534412955465587, 'pattern_counts': {'":': 4, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '502110', 'license': 'Apache-2.0', 'n_tokens_mistral': 747, 'n_tokens_neox': 696, 'n_words': 144, 'path': 'tutorials/025 - Redshift - Loading Parquet files with Spectrum.ipynb', 'repo_name': 'mateogianolio/aws-data-wrangler', 'size': '1860'}