Update web.py

web.py

@@ -249,7 +249,7 @@ def web_data():
         P("This section provides a complete discussion on the filtering applied to the 99 Common Crawl snapshots that comprise the web data section of TxT360. The section is split into the following topic areas: "),
         Ul(
             Li("Web Data Processing Summary", style = "margin-bottom: 5px"),
-            Li("Document Preperation", style = "margin-bottom: 5px"),
+            Li("Document Preparation", style = "margin-bottom: 5px"),
             Li("Line-Level Filtering", style = "margin-bottom: 5px"),
             Li("Local Deduplication", style = "margin-bottom: 5px"),
             Li("Each section is complete with code and comparisons to Dolma,", D_cite(bibtex_key="soldaini2024dolma"),
@@ -263,9 +263,9 @@ def web_data():
         H3("TxT360 CommonCrawl Filtering vs Other Pretraining Datasets"),
         P("The following section provides explicit details covering the reasoning and decisions behind each of the filters we applied. The table below provides a high-level comparison of TxT360's filtering compared to other commonly used pretraining datasets."),
         table_div_filter_data,
-        P("The table below provides a comparison of the quality filters that have been applied to each dataset. Of note, TxT360 does not use any machine learning (ML) based filters. ML filters are a useful and effecient filtering processing that should be consider for any filtering project. However, we are leaving that option to TxT360's end users."),
+        P("The table below provides a comparison of the quality filters that have been applied to each dataset. Of note, TxT360 does not use any machine learning (ML) based filters. ML filters are a useful and efficient filtering processing that should be consider for any filtering project. However, we are leaving this to future work."),
         table_div_qf_filter_data,
-        P("Our filtering rate is illustrated below. Before deduplication, our filtering rate is comparable to RefinedWeb. During global deduplication, we removed approximately 85.89% of the data, significantly higher than previous works, indicating a large number of duplicates across dumps. "),
+        P("Our filtering rate is illustrated below. Before deduplication, our filtering rate is comparable to RefinedWeb. During global deduplication, we removed approximately 85.89% of the data, significantly higher than previous works, indicating a large number of duplicates across snapshots. "),
         Img(src="images/filter_rate.jpg", height = "300", width = "600" ),
         P("Note: All percentages are based on the number of documents. The gray bars represent the relative percentages of removed documents at each step, while the colorful bars represent the percentages of retained documents relative to the total number of documents in the raw Common Crawl."),
         id="section2",),
@@ -278,9 +278,8 @@ def web_data():
         WARC files contain the raw data from the crawl, which store the full HTTP response and request metadata. 
         WET files contain plaintexts extracted by Common Crawl. In line with previous works""",D_cite(bibtex_key="thepile"),D_cite(bibtex_key="refinedweb"),D_cite(bibtex_key="gopher"),D_cite(bibtex_key="fineweb") ,""" , 
         we found WET files to include boilerplate content like navigation menus, ads, and other irrelevant texts. 
-        Accordingly, our pipeline starts from raw WARC files, reads with the warcio library, and extracts texts using trafilatura.
         """),
-        P("We directly read WARC files instead of WET files and extracted text using Trafilatura. Similar to RefinedWeb, we avoid using Machine Learning (ML)-based metrics for filtering documents to prevent bias introduced by ML models. Importantly, we apply global deduplication across the entire dataset, whereas previous works only use local deduplication. Note that although The Pile also employed global deduplication on its web data (Pile-CC), this accounted for just 0.6\% of 74 snapshots."),
+        P("We directly read WARC files with the warcio library instead of WET files and extracted text using Trafilatura. Similar to RefinedWeb, we avoid using Machine Learning (ML)-based metrics for filtering documents to prevent bias introduced by ML models. Importantly, we apply global deduplication across the entire dataset, whereas previous works only use local deduplication. Note that although The Pile also employed global deduplication on its web data (Pile-CC), this accounted for just 0.6\% of 74 snapshots."),
 
         Details(
                 Summary("Text Extraction Examples"),
@@ -338,7 +337,7 @@ def web_data():
         
         P(B("URL Filtering: "), """
         The following section details the decisions behind utilizing the UT1 blocklist. We chose to use the UT1 blocklist as a simple method for filtering
-        out potentially harmful content such as adult content. We also excluded URLs that contained the digital version of the curated curated data (e.g. wikipedia.org) to avoid duplication.
+        out potentially harmful content such as adult content. We also excluded URLs that contained the digital version of the curated data (e.g. wikipedia.org) to avoid duplication.
         """),
         
         P(B("URL Blocklist: "), """
@@ -579,7 +578,7 @@ def web_data():
         work, """, D_cite(bibtex_key="gopher"), D_cite(bibtex_key="refinedweb"), D_cite(bibtex_key="dolma"), """ we choose to remove any document with excessive line, paragraph, or n-gram repetitions.
         """),
         P(B("Fraction of Characters in Repeated Lines: "), """
-        Following Gopher,""", D_cite(bibtex_key="gopher"), """ we remove documents containing mupltiple, short duplicate passages, as well as those with few, 
+        Following Gopher,""", D_cite(bibtex_key="gopher"), """ we remove documents containing multiple, short duplicate passages, as well as those with few, 
         but longer duplicate passages. To achieve this goal, we calculate over the document both the fraction of passages 
         that are duplicates, and the fraction of characters contained within those duplicated passages.
         """),