<jupyter_start><jupyter_text>Overview- beutifulll soup を使ってkaggleのdiscussionを取得することを試みる。- 参考: https://github.com/fkubota/arxiv_notification/tree/master/nb  Const  Import everything I need :)<jupyter_code>import datetime
from bs4 import BeautifulSoup
import requests
from fastprogress import progress_bar<jupyter_output><empty_output><jupyter_text>Execution<jupyter_code>url = 'https://www.kaggle.com/c/birdsong-recognition/discussion/158933'
response = requests.get(url)
html = response.text
print(html)<jupyter_output><!DOCTYPE html>
<html lang="en">
<head>
    <title>Cornell Birdcall Identification | Kaggle</title>
    <meta charset="utf-8" />
    <meta name="robots" content="index, follow" />
    <meta name="description" content="Build tools for bird population monitoring" />
    <meta name="turbolinks-cache-control" content="no-cache" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=5.0, minimum-scale=1.0">
    <meta name="theme-color" content="#008ABC" />
    <script nonce="l5WH1nQs6K7Ywe258yuHVQ==" type="text/javascript">
        if ('serviceWorker' in navigator) {
            navigator.serviceWorker.getRegistrations()
                .then(function(registrations) {
                    for (let registration of registrations) {
                        registration.unregister();
                    }
                })
                .catch(function(err) {
                    console.error("Service worker unregister failed: ", err);
                }[...]
{'fraction_non_alphanumeric': 0.12664041994750655, 'fraction_numerical': 0.015748031496062992, 'mean_word_length': 3.283707865168539, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '42843', 'license': 'MIT', 'n_tokens_mistral': 498, 'n_tokens_neox': 463, 'n_words': 84, 'path': '000_playground/0011_kaggle_discussion_scraping.ipynb', 'repo_name': 'fkubota/rkkubotay-gmail.com', 'size': '1524'}