first

Dockerfile

@@ -0,0 +1,16 @@
+# Read the doc: https://huggingface.co/docs/hub/spaces-sdks-docker
+# you will also find guides on how best to write your Dockerfile
+
+FROM python:3.11
+
+RUN useradd -m -u 1000 user
+USER user
+ENV PATH="/home/user/.local/bin:$PATH"
+
+WORKDIR /app
+
+COPY --chown=user ./requirements.txt requirements.txt
+RUN pip install --no-cache-dir --upgrade -r requirements.txt
+
+COPY --chown=user . /app
+CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -1,12 +1,11 @@
 ---
-title: About Me
-emoji: 📊
-colorFrom: red
-colorTo: gray
-sdk: gradio
-sdk_version: 4.24.0
-app_file: app.py
-pinned: false
+title: Me
+emoji: 🤠
+colorFrom: green
+colorTo: yellow
+sdk: docker
+pinned: true
+license: apache-2.0
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

about.py

@@ -0,0 +1,126 @@
+from app import *
+from datetime import datetime
+
+
+def page():
+    # me = Image('https://live.staticflickr.com/65535/53939742438_6ca1a4b3eb.jpg', alt='Nicholas Broad and his dog, Maya in Maine', caption="it'sa me", left=False, width=300)
+    h2s = 'Hi there 👋', 'Work Experience', "Education", "Volunteering",'Hobbies'
+    txts = [Markdown(intro), Markdown(work), Markdown(education), Markdown(volunteering), Markdown(hobbies)]
+    secs = Sections(h2s, txts)
+    return BstPage(0, "", *secs)
+
+birthdate = datetime(1994, 3, 18)
+age_in_years = (datetime.now() - birthdate).days // 365
+
+maya_birthdate = datetime(2019, 3, 18)
+maya_age_in_years = (datetime.now() - maya_birthdate).days // 365
+
+intro = f"""
+<div class="row">
+<div class="col-md-7">
+
+I'm Nicholas Broad. I'm currently {age_in_years} years old, living in San Francisco, California. I'm an open-minded, curious, and self-motivated person who enjoys learning new things and working on interesting projects. I like to think of myself as friendly, humble, and a bit goofy. I live with my beautiful dog, <a href="https://www.flickr.com/photos/131470140@N06/albums/72177720295849325/">Maya</a>, who is {maya_age_in_years} years old. You can often find us at parks in SF playing fetch. Maya will almost always be wearing a bandana, as all dogs should.
+</div>
+<div class="col-md-4">
+<a data-flickr-embed="true" href="https://www.flickr.com/photos/131470140@N06/53939742438/in/album-72177720300307682/" title="Nico and Maya"><img src="https://live.staticflickr.com/65535/53939742438_6ca1a4b3eb.jpg" width="333" height="500" alt="Nico and Maya" class="img-fluid rounded"/></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script>
+</div>
+</div>
+"""
+
+hf_start_date = datetime(2021, 12, 6)
+time_at_hf = (datetime.now() - hf_start_date)
+years_at_hf = time_at_hf.days // 365
+months_at_hf = (time_at_hf.days - years_at_hf * 365) // 30
+
+work = f"""
+I am currently working at [Hugging Face](https://hf.co) as a Machine Learning Engineer. I have been there {years_at_hf} years and {months_at_hf} months as a member of the monetization team. I have helped companies such as Johnson & Johnson, Grammarly, Optum, Writer, Uber, and Liberty Mutual improve their machine learning capabilities. 
+
+Prior to working at Hugging Face, I was a data scientist at GSK, where I created a search engine to make it easier to understand Medical Voice of Customer data (what the clinicians who use the products think of them).
+
+
+I believe in being transparent about my work journey, so I will share more than most people typically do. Here is all of my work experience since I was 16.
+
+
+* Machine Learning Engineer @ [Hugging Face](https://hf.co) (2021 - Present)
+    * Member of the success team within the monetization team.
+    * Helping enterprises and startups improve their machine learning capabilities. 
+        * This involves teaching how to use Hugging Face libraries such as transformers, accelerate, datasets, peft, trl, and diffusers.
+        * Typical use cases:
+            * Fine-tuning a model on a custom dataset.
+            * Serving fine-tuned models (both LLM and non-LLMs).
+            * Pre-training from scratch.
+            * RAG using LLMs and embeddings.
+            * Document understanding using vision-language models.
+            * Curating a high-quality instruction dataset.
+    * Manage relations with ~$2M worth of customers. 
+    * Most senior member of post-sales MLE team.
+    * Hired 5 other team mebers.
+    * Author on [BLOOM paper](https://huggingface.co/papers/2211.05100) for minor help with datasets.
+* Data Scientist @ GSK (2020 - 2021)
+    * Created a search engine to make it easier to understand what clinicians think of the products.
+* Data Science Fellowship @ SharpestMinds (2019 - 2020)
+    * Fellowship is a generous term. It was a 3-month bootcamp where I taught myself all about data science and machine learning while being mentored by an industry professional.
+* High School / Middle School Tutor @ AJ Tutoring (2018 - 2019)
+    * Shared my love for physics, math, and chemistry with students in the Bay Area.
+    * I even worked with the daughter of Google's CEO. 
+    * One of my students was in [the scandal where parents paid to get their kids into college](https://en.wikipedia.org/wiki/Varsity_Blues_scandal).
+* Dog Walker (2017-2019)
+    * I ran with my neighbors dog 3 times a week for 3 years.
+    * We did over 2000 miles together.
+* Image Sensor Intern @ ON Semiconductor (2017)
+    * Quit after 5 weeks because it was a waste of my time. Unrealistic expectations, no guidance, and not much learning.
+* Research Assistant Internship @ Stanford University (2016)
+    * Worked on thermionic energy harvesters.
+    * Spent too much time in the cleanroom alone at odd hours around dangerous chemicals and tools.
+* Financial Manager @ [Muwekma Tah-Ruk House](https://nacc.stanford.edu/our-community/muwekma-tah-ruk-native-theme-house) (2015-2016)
+    * Budgeted $100k for 35 students in the house for food and events.
+* Research Internship @ Lawrence Livermore Laboratory (2015)
+    * Fabrication of implantable neural devices.
+* Research Internship @ Stanford University (2014)  
+    * Fabricating sensors that work in extreme environments.
+    * Worked with graphene, high bandgap semiconductors, and cool equipment like scanning electron microscopes and x-ray diffraction.
+    * Co-author on 2 papers:
+        * [Irradiation effects of graphene-enhanced gallium nitride (GaN) metal-semiconductor-metal (MSM) ultraviolet photodetectors](https://www.spiedigitallibrary.org/conference-proceedings-of-spie/9491/949107/Irradiation-effects-of-graphene-enhanced-gallium-nitride-GaN-metal-semiconductor/10.1117/12.2178091.short), 
+        * [Impact of gamma irradiation on GaN/sapphire surface acoustic wave resonators](https://ieeexplore.ieee.org/abstract/document/6932293)
+* Admin Assistant @ Stanford Graduate School of Education (2014)
+    * Did various small tasks to help the school run smoothly.
+* Research Internship @ Stanford University (2013)
+    * Worked on helping underrepresented students start learning Computer Science.
+* Soccer Coach @ Los Gatos Soccer (2010-2012)
+    * Coached soccer basics for kids during the summer.
+"""
+
+volunteering = """All of my volunteering is tutoring.
+
+* [Lucile Packard Children's Hospital](https://med.stanford.edu/phm/clinical/lpch.html) (2013 - 2014)
+    * Tutored kids who had been in the hospital for a long time.
+* [East Palo Alto Tennis and Tutoring](https://www.epatt.org/) (2019)
+    * Tutored high school students in math and science.
+* [Student U](https://studentudurham.org/) (2019-2020)
+    * Tutored high school students in math and science.
+"""
+
+education = """
+* Coterminal M.S. in Electrical Engineering @ Stanford University (2016-2018)
+    * Did not graduate.
+    * Decided that I wasn't interested in the classes I was taking (physics, analog devices, signal processing, networks).
+    * I was also sick of doing research in a cleanroom.
+* B.S. in Electrical Engineering @ Stanford University (2012-2016)
+    * Focus on semiconductor device physics.
+* Los Gatos High School (2008-2012)
+    * Voted "Best All Around" by senior class.
+    * Played Varsity Soccer for 3 years.
+        * [Captain of the Varsity Soccer Team (2011-2012).](https://www.maxpreps.com/ca/los-gatos/los-gatos-wildcats/athletes/nicky-broad/soccer/stats/?careerid=11o9q24ridm22)
+        * [Co-MVP of the league (2012).](https://www.removepaywall.com/search?url=https://www.mercurynews.com/2012/03/19/broad-mason-are-league-mvps-in-soccer/)
+        * [Honorary mention in CCS (2012).](https://www.removepaywall.com/search?url=https://www.mercurynews.com/2012/03/27/all-mercury-news-boys-soccer-honors/)
+"""
+
+hobbies = """My main hobbies are going on walks with my dog, playing soccer, doing [Kaggle competitions](https://www.kaggle.com/nbroad), reading, [photography](https://flickr.com/photos/131470140@N06/), and taiko. This last one, taiko, is probably the most interesting, though I am on a bit of a hiatus. I played taiko for 12 years, starting with [San Jose Junior Taiko](https://www.taikoconservatory.org/juniortaiko) and continuing with [Stanford Taiko](https://taiko.stanford.edu/). Here is a video of me performing in Stanford's Bing Concert Hall. 
+"""
+
+four_sizzlin_beets = """<div class="embed-responsive embed-responsive-16by9"><iframe width="560" height="315" src="https://www.youtube.com/embed/0Q_4sffQZBA?si=4ArGNw1wy5ps2ra0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe></div>"""
+
+stanford_instagram_embed = """<blockquote class="instagram-media" data-instgrm-captioned data-instgrm-permalink="https://www.instagram.com/p/njAzUvj7Cs/?utm_source=ig_embed&amp;utm_campaign=loading" data-instgrm-version="14" style=" background:#FFF; border:0; border-radius:3px; box-shadow:0 0 1px 0 rgba(0,0,0,0.5),0 1px 10px 0 rgba(0,0,0,0.15); margin: 1px; max-width:540px; min-width:326px; padding:0; width:99.375%; width:-webkit-calc(100% - 2px); width:calc(100% - 2px);"><div style="padding:16px;"> <a href="https://www.instagram.com/p/njAzUvj7Cs/?utm_source=ig_embed&amp;utm_campaign=loading" style=" background:#FFFFFF; line-height:0; padding:0 0; text-align:center; text-decoration:none; width:100%;" target="_blank"> <div style=" display: flex; flex-direction: row; align-items: center;"> <div style="background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 40px; margin-right: 14px; width: 40px;"></div> <div style="display: flex; flex-direction: column; flex-grow: 1; justify-content: center;"> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 100px;"></div> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 60px;"></div></div></div><div style="padding: 19% 0;"></div> <div style="display:block; height:50px; margin:0 auto 12px; width:50px;"><svg width="50px" height="50px" viewBox="0 0 60 60" version="1.1" xmlns="https://www.w3.org/2000/svg" xmlns:xlink="https://www.w3.org/1999/xlink"><g stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"><g transform="translate(-511.000000, -20.000000)" fill="#000000"><g><path d="M556.869,30.41 C554.814,30.41 553.148,32.076 553.148,34.131 C553.148,36.186 554.814,37.852 556.869,37.852 C558.924,37.852 560.59,36.186 560.59,34.131 C560.59,32.076 558.924,30.41 556.869,30.41 M541,60.657 C535.114,60.657 530.342,55.887 530.342,50 C530.342,44.114 535.114,39.342 541,39.342 C546.887,39.342 551.658,44.114 551.658,50 C551.658,55.887 546.887,60.657 541,60.657 M541,33.886 C532.1,33.886 524.886,41.1 524.886,50 C524.886,58.899 532.1,66.113 541,66.113 C549.9,66.113 557.115,58.899 557.115,50 C557.115,41.1 549.9,33.886 541,33.886 M565.378,62.101 C565.244,65.022 564.756,66.606 564.346,67.663 C563.803,69.06 563.154,70.057 562.106,71.106 C561.058,72.155 560.06,72.803 558.662,73.347 C557.607,73.757 556.021,74.244 553.102,74.378 C549.944,74.521 548.997,74.552 541,74.552 C533.003,74.552 532.056,74.521 528.898,74.378 C525.979,74.244 524.393,73.757 523.338,73.347 C521.94,72.803 520.942,72.155 519.894,71.106 C518.846,70.057 518.197,69.06 517.654,67.663 C517.244,66.606 516.755,65.022 516.623,62.101 C516.479,58.943 516.448,57.996 516.448,50 C516.448,42.003 516.479,41.056 516.623,37.899 C516.755,34.978 517.244,33.391 517.654,32.338 C518.197,30.938 518.846,29.942 519.894,28.894 C520.942,27.846 521.94,27.196 523.338,26.654 C524.393,26.244 525.979,25.756 528.898,25.623 C532.057,25.479 533.004,25.448 541,25.448 C548.997,25.448 549.943,25.479 553.102,25.623 C556.021,25.756 557.607,26.244 558.662,26.654 C560.06,27.196 561.058,27.846 562.106,28.894 C563.154,29.942 563.803,30.938 564.346,32.338 C564.756,33.391 565.244,34.978 565.378,37.899 C565.522,41.056 565.552,42.003 565.552,50 C565.552,57.996 565.522,58.943 565.378,62.101 M570.82,37.631 C570.674,34.438 570.167,32.258 569.425,30.349 C568.659,28.377 567.633,26.702 565.965,25.035 C564.297,23.368 562.623,22.342 560.652,21.575 C558.743,20.834 556.562,20.326 553.369,20.18 C550.169,20.033 549.148,20 541,20 C532.853,20 531.831,20.033 528.631,20.18 C525.438,20.326 523.257,20.834 521.349,21.575 C519.376,22.342 517.703,23.368 516.035,25.035 C514.368,26.702 513.342,28.377 512.574,30.349 C511.834,32.258 511.326,34.438 511.181,37.631 C511.035,40.831 511,41.851 511,50 C511,58.147 511.035,59.17 511.181,62.369 C511.326,65.562 511.834,67.743 512.574,69.651 C513.342,71.625 514.368,73.296 516.035,74.965 C517.703,76.634 519.376,77.658 521.349,78.425 C523.257,79.167 525.438,79.673 528.631,79.82 C531.831,79.965 532.853,80.001 541,80.001 C549.148,80.001 550.169,79.965 553.369,79.82 C556.562,79.673 558.743,79.167 560.652,78.425 C562.623,77.658 564.297,76.634 565.965,74.965 C567.633,73.296 568.659,71.625 569.425,69.651 C570.167,67.743 570.674,65.562 570.82,62.369 C570.966,59.17 571,58.147 571,50 C571,41.851 570.966,40.831 570.82,37.631"></path></g></g></g></svg></div><div style="padding-top: 8px;"> <div style=" color:#3897f0; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:550; line-height:18px;">View this post on Instagram</div></div><div style="padding: 12.5% 0;"></div> <div style="display: flex; flex-direction: row; margin-bottom: 14px; align-items: center;"><div> <div style="background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(0px) translateY(7px);"></div> <div style="background-color: #F4F4F4; height: 12.5px; transform: rotate(-45deg) translateX(3px) translateY(1px); width: 12.5px; flex-grow: 0; margin-right: 14px; margin-left: 2px;"></div> <div style="background-color: #F4F4F4; border-radius: 50%; height: 12.5px; width: 12.5px; transform: translateX(9px) translateY(-18px);"></div></div><div style="margin-left: 8px;"> <div style=" background-color: #F4F4F4; border-radius: 50%; flex-grow: 0; height: 20px; width: 20px;"></div> <div style=" width: 0; height: 0; border-top: 2px solid transparent; border-left: 6px solid #f4f4f4; border-bottom: 2px solid transparent; transform: translateX(16px) translateY(-4px) rotate(30deg)"></div></div><div style="margin-left: auto;"> <div style=" width: 0px; border-top: 8px solid #F4F4F4; border-right: 8px solid transparent; transform: translateY(16px);"></div> <div style=" background-color: #F4F4F4; flex-grow: 0; height: 12px; width: 16px; transform: translateY(-4px);"></div> <div style=" width: 0; height: 0; border-top: 8px solid #F4F4F4; border-left: 8px solid transparent; transform: translateY(-4px) translateX(8px);"></div></div></div> <div style="display: flex; flex-direction: column; flex-grow: 1; justify-content: center; margin-bottom: 24px;"> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; margin-bottom: 6px; width: 224px;"></div> <div style=" background-color: #F4F4F4; border-radius: 4px; flex-grow: 0; height: 14px; width: 144px;"></div></div></a><p style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; line-height:17px; margin-bottom:0; margin-top:8px; overflow:hidden; padding:8px 0 7px; text-align:center; text-overflow:ellipsis; white-space:nowrap;"><a href="https://www.instagram.com/p/njAzUvj7Cs/?utm_source=ig_embed&amp;utm_campaign=loading" style=" color:#c9c8cd; font-family:Arial,sans-serif; font-size:14px; font-style:normal; font-weight:normal; line-height:17px; text-decoration:none;" target="_blank">A post shared by Stanford University (@stanford)</a></p></div></blockquote> <script async src="//www.instagram.com/embed.js"></script>"""
+
+
+hobbies += four_sizzlin_beets + "<br><br>Stanford Instagram also featured me on their page.<br><br>" + stanford_instagram_embed

app.py

@@ -0,0 +1,42 @@
+from fh_bootstrap import *
+from itertools import chain
+from markdown import markdown
+
+md_exts='codehilite', 'smarty', 'extra', 'sane_lists'
+def Markdown(s, exts=md_exts, **kw): return Div(NotStr(markdown(s, extensions=exts)), **kw)
+
+ghurl = 'https://github.com/nbroad1881'
+hf_logo_svg = "https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo.svg"
+li_url = "https://www.linkedin.com/in/nicholas-m-broad/"
+kaggle_url = "https://www.kaggle.com/nbroad"
+fh_url = "https://fastht.ml/"
+fh_logo = 'assets/fasthtml_logo.svg'
+
+def BstPage(selidx, title, *c):
+    navitems = [('Home', '/'), ('Blog', '/blog')]
+    
+    ra_items = (
+        A(Image(src="/assets/hf-logo.svg", width=28, height=28, cls="my-0 px-0 mx-0 py-0", left=False, pad=0), cls="ms-2 my-0 px-1 btn-lg btn", role="button", href="https://hf.co/nbroad"), 
+        Icon('fab fa-github', dark=False, sz='lg', href=ghurl, cls='ms-2 px-2'),
+        Icon("fab fa-linkedin", dark=False, sz='lg', href=li_url, cls='ms-2 px-2'),
+        Icon("fab fa-kaggle", dark=False, sz='lg', href=kaggle_url, cls='ms-2 px-2'),
+        Icon("fab fa-youtube", dark=False, sz='lg', href="https://www.youtube.com/@nicholasbroad1881", cls='ms-2 px-2'),
+        Icon("fab fa-twitter", dark=False, sz='lg', href="https://twitter.com/nbroad1881", cls='ms-2 px-2'),
+                )
+    ftlinks = [A(k, href=v, cls='nav-link px-2 text-muted')
+        for k,v in dict(Home='/').items()]
+    return (
+        Title(title),
+        Script('initTOC()'),
+        Container(
+            Navbar('nav', selidx, items=navitems, ra_items=ra_items, cls='navbar-light bg-secondary rounded-lg',
+                image=f'', hdr_href="fhurl", placement=PlacementT.Default, expand=SizeT.Md, toggle_left=False),
+            Toc(Container(H1(title, cls='pb-2 pt-1'), *c, cls='mt-3')),
+            BstFooter('Made using FastHTML', File(fh_logo), img_href=fh_url, cs=ftlinks),
+        typ=ContainerT.Xl, cls='mt-3', data_bs_spy='scroll', data_bs_target='#toc'))
+
+def Sections(h2s, texts):
+    colors = 'yellow', 'pink', 'teal', 'blue'
+    div_cls = 'py-2 px-3 mt-4 bg-light-{} rounded-lg'
+    return chain([Div(H2(h2, id=f'sec{i+1}', cls=div_cls.format(colors[i%4])), Div(txt, cls='px-2'))
+                  for i,(h2,txt) in enumerate(zip(h2s, texts))])

assets/hl-styles.css

@@ -0,0 +1,74 @@
+pre { line-height: 125%; }
+td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }
+span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }
+td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }
+span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }
+.codehilite .hll { background-color: #ffffcc }
+.codehilite { background: #f8f8f8; }
+.codehilite .c { color: #3D7B7B; font-style: italic } /* Comment */
+.codehilite .err { border: 1px solid #FF0000 } /* Error */
+.codehilite .k { color: #008000; font-weight: bold } /* Keyword */
+.codehilite .o { color: #666666 } /* Operator */
+.codehilite .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */
+.codehilite .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */
+.codehilite .cp { color: #9C6500 } /* Comment.Preproc */
+.codehilite .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */
+.codehilite .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */
+.codehilite .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */
+.codehilite .gd { color: #A00000 } /* Generic.Deleted */
+.codehilite .ge { font-style: italic } /* Generic.Emph */
+.codehilite .gr { color: #E40000 } /* Generic.Error */
+.codehilite .gh { color: #000080; font-weight: bold } /* Generic.Heading */
+.codehilite .gi { color: #008400 } /* Generic.Inserted */
+.codehilite .go { color: #717171 } /* Generic.Output */
+.codehilite .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
+.codehilite .gs { font-weight: bold } /* Generic.Strong */
+.codehilite .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
+.codehilite .gt { color: #0044DD } /* Generic.Traceback */
+.codehilite .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
+.codehilite .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
+.codehilite .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
+.codehilite .kp { color: #008000 } /* Keyword.Pseudo */
+.codehilite .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
+.codehilite .kt { color: #B00040 } /* Keyword.Type */
+.codehilite .m { color: #666666 } /* Literal.Number */
+.codehilite .s { color: #BA2121 } /* Literal.String */
+.codehilite .na { color: #687822 } /* Name.Attribute */
+.codehilite .nb { color: #008000 } /* Name.Builtin */
+.codehilite .nc { color: #0000FF; font-weight: bold } /* Name.Class */
+.codehilite .no { color: #880000 } /* Name.Constant */
+.codehilite .nd { color: #AA22FF } /* Name.Decorator */
+.codehilite .ni { color: #717171; font-weight: bold } /* Name.Entity */
+.codehilite .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */
+.codehilite .nf { color: #0000FF } /* Name.Function */
+.codehilite .nl { color: #767600 } /* Name.Label */
+.codehilite .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
+.codehilite .nt { color: #008000; font-weight: bold } /* Name.Tag */
+.codehilite .nv { color: #19177C } /* Name.Variable */
+.codehilite .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
+.codehilite .w { color: #bbbbbb } /* Text.Whitespace */
+.codehilite .mb { color: #666666 } /* Literal.Number.Bin */
+.codehilite .mf { color: #666666 } /* Literal.Number.Float */
+.codehilite .mh { color: #666666 } /* Literal.Number.Hex */
+.codehilite .mi { color: #666666 } /* Literal.Number.Integer */
+.codehilite .mo { color: #666666 } /* Literal.Number.Oct */
+.codehilite .sa { color: #BA2121 } /* Literal.String.Affix */
+.codehilite .sb { color: #BA2121 } /* Literal.String.Backtick */
+.codehilite .sc { color: #BA2121 } /* Literal.String.Char */
+.codehilite .dl { color: #BA2121 } /* Literal.String.Delimiter */
+.codehilite .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
+.codehilite .s2 { color: #BA2121 } /* Literal.String.Double */
+.codehilite .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */
+.codehilite .sh { color: #BA2121 } /* Literal.String.Heredoc */
+.codehilite .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */
+.codehilite .sx { color: #008000 } /* Literal.String.Other */
+.codehilite .sr { color: #A45A77 } /* Literal.String.Regex */
+.codehilite .s1 { color: #BA2121 } /* Literal.String.Single */
+.codehilite .ss { color: #19177C } /* Literal.String.Symbol */
+.codehilite .bp { color: #008000 } /* Name.Builtin.Pseudo */
+.codehilite .fm { color: #0000FF } /* Name.Function.Magic */
+.codehilite .vc { color: #19177C } /* Name.Variable.Class */
+.codehilite .vg { color: #19177C } /* Name.Variable.Global */
+.codehilite .vi { color: #19177C } /* Name.Variable.Instance */
+.codehilite .vm { color: #19177C } /* Name.Variable.Magic */
+.codehilite .il { color: #666666 } /* Literal.Number.Integer.Long */

assets/styles.css

@@ -0,0 +1,41 @@
+pre {
+    background-color: #f8f9fa;
+    border: 1px solid #e9ecef;
+    border-radius: 4px;
+    padding: 1rem;
+    margin-bottom: 1rem;
+}
+
+:root, [data-bs-theme=light] {
+    --bs-secondary: #169873;
+    --bs-secondary-rgb: 22, 152, 115;
+    --bs-primary: #c6f97f;
+    --bs-light-yellow: #faaf5c;
+    --bs-light-pink: #f49fbc;
+    --bs-light-red: #ff6060;
+    --bs-light-teal: #B8E1FF;
+    --bs-light-blue: #A9FFF7;
+    --bs-light-green: #94FBAB;
+}
+
+nav.navbar { --bs-btn-hover-bg: rgba(255,255,255,0.2); }
+.nav-link:hover { color: rgba(255,255,0,0.6); }
+.nav-link.active { font-weight: bold; }
+
+@media (min-width: 992px) {
+    .rounded-tl-lg { border-top-left-radius: 1.5rem !important; }
+}
+
+.rounded-lg {
+    border-top-left-radius: 1rem !important;
+    border-top-right-radius: 1rem !important;
+}
+
+blockquote {
+    border-left: 5px solid #007bff;
+    padding-left: 15px;
+    margin-left: 0;
+    background-color: #f8f9fa;
+    padding: 15px;
+    padding-bottom: 5px;
+}

blog.py

@@ -0,0 +1,64 @@
+from app import *
+import yaml
+import functools
+
+from pathlib import Path
+from datetime import datetime, date
+
+file_path = Path(__file__).parent
+
+NUM_RECENT_BLOGS = 20
+
+def full_page():
+
+    secs = Sections(["Recent Blogs"], [Div(*[blog_preview(blog_id) for blog_id in sorted_blogs[:NUM_RECENT_BLOGS]])])
+    return BstPage(0, '', *secs)
+
+
+def blog_preview(blog_id):
+    details = all_blogs[blog_id]
+
+    return Div(
+            A(H3(details[0]["title"]), href=f"/blog/{blog_id}"),
+            P(details[0].get("date_published", "")),
+            P(details[0].get("desc", "")+"...")
+        )
+    
+
+
+@functools.lru_cache()
+def get_blogs():
+    blogs = (file_path / "blogs").rglob("*.md")
+
+    blog_dict = {}
+
+    for blog in blogs:
+        with open(blog, 'r') as f:
+            id_ = blog.stem
+            text = f.read()
+            if "---" not in text:
+                continue
+            metadata, markdown = text.split("---", 2)[1:]
+            metadata = yaml.safe_load(metadata)
+            metadata["id"] = id_
+            blog_dict[id_] = (metadata, markdown)
+
+    blog_dict = {k:v for k, v in blog_dict.items() if v[0].get("date_published", "") != ""}
+
+    sorted_blogs = [x[0]["id"] for x in sorted(blog_dict.values(), key=lambda x: x[0].get("date_published"), reverse=True)]
+
+    return blog_dict, sorted_blogs
+
+
+all_blogs, sorted_blogs = get_blogs()
+
+
+
+def single_blog(blog_id):
+    """
+    Return a single blog post.
+
+    """
+    details = all_blogs[blog_id]
+
+    return BstPage(0, details[0]["title"], Markdown(details[1]))

blogs/2020/sentimentr.md

@@ -0,0 +1,58 @@
+---
+title: Sentimentr
+desc: A tool to visualize bias in news headlines about presidential candidates
+published: true
+date_published: 2020-01-12
+tags: nlp
+---
+
+<!-- {% include figure image_path="/assets/images/i-voted.jpg" alt="I voted"%} -->
+
+With presidential primaries just around the corner, I thought it would be interesting to see if I could tell if there is a consistent bias toward one candidate or another.  Could I quantitatively show that Fox News has more favorable headlines about Trump and CNN showing the opposite? 
+
+<!-- This year also broke the previous record with 20 candidates vying for the nomination. I thought it would be interesting to see if the general success or failure of a candidate could be seen in news headlines.  To make the task easier, I limited the candidate pool to the top 5 candidates: Biden, Sanders, Warren, Harris, and Buttigieg. At the time I started Harris had not dropped out, and since there was a reasonable amount of data about Harris, I decided to keep her in the final results.
+ -->
+The ideal news source is unbiased and not focusing all of their attention on one candidate; however we live in a time where "fake news" has entered everyone's daily vernacular. Unfortunately, there is scorn going both ways between liberals and conservatives with both claiming that their side knows the truth and lambasting the other side for being deceived and following villainous leaders. 
+
+I gathered thousands of headlines from CNN, Fox News, and The New York Times that contain the keywords Trump, Biden, Sanders, Warren, Harris, or Buttigieg.  I had to exclude many headlines that contained the names of multiple candidates because it would require making multiple models that are each tailored to one single candidate.
+
+Here are a few instances that have contain different candidates in the same headline that would make it difficult to measure a single sentiment for each candidate.
+
+*  *Here's why Trump keeps pumping up Bernie Sanders*  
+*  *Buttigieg on Trump: 'Senate is the jury today, but we are the jury tomorrow'*
+*  *Elizabeth Warren sought to 'raise a concern' with Bernie Sanders in post-debate exchange, Sanders campaign says*
+
+For this reason I decided to drop all headlines with the names of multiple candidates for this analysis.  Thankfully, I still ended up with over 5,000 articles. Take a look at the distribution of articles for each candidate and for each news source.
+
+<!-- {% include figure image_path="/assets/images/graphs/article-count-raw-and-opinion.png" alt="Linear Graph of Article Counts" caption="Bar plots of how many articles had a given candidate's name in it. Top is a raw count of total articles. Bottom separates it by news group."%} -->
+
+<!--{% include figure image_path="/assets/images/graphs/article-count-log.png" alt="Logarithmic Graph of Article Counts" caption="Logarithmic barplots of how many articles had a given candidate's name in it. Blue represents CNN, orange is Fox News, and green is The New York Times."%}
+-->
+Trump is by far the most talked-about candidate and for good reason: he is the sitting president and the sole republican candidate. After Trump in the ranking goes Biden, then Sanders and Warren are about the same then finally Harris and Buttigieg. 
+
+I was surprised at the sheer volume of CNN articles and also The New York Times' tiny quantity.  
+
+
+## Sentiment Analysis Models
+
+I used 3 different sentiment analysis models: two of which were pre-made packages. VADER and TextBlob are python packages that offer sentiment analysis trained on different subjects. VADER is a lexicon approach based off of social media tweets that contain emoticons and slang.  TextBlob is a Naive Bayes approach trained on IMDB review data.  My model is an LSTM with a base languange model based off of the [AWD-LSTM](https://arxiv.org/abs/1708.02182). I then trained its language model on news articles. Following that, I trained it on hand-labeled (by me 😤) article headlines. 
+
+Here are the average scores for each candidate.
+
+<!-- {% include figure image_path="/assets/images/graphs/news-barplot-average-scores.png" alt="Sentiment scores over time" caption="Bar plots of average sentiment scores separated by model and candidate."%} -->
+
+And then looking average scores over time.
+
+<!-- {% include figure image_path="/assets/images/graphs/average-4wk-scores-over-time-top-6.png" alt="Sentiment scores over time" caption="Line plots of sentiment scores separated by model and candidate."%} -->
+
+I should also note that these scores have been smoothed by a sliding average with a window size of 4 weeks.  Without smoothing it looks much more chaotic.  Smoothing hopefully shows longterm trends.  Even with smoothing it is a bit hard to tell if there are any consistent trends. The next thing I tried was to superimpose debate dates onto the democratic candidates to see if the candidate's performance could be seen after a debate. In some cases, there does seem to be a rise or drop in scores after a debate, but whether they are correlated remains unknown.
+
+
+<!-- {% include figure image_path="/assets/images/graphs/average-4wk-scores-over-time-top-6-with-debates.png" alt="Sentiment scores over time" caption="Line plots of sentiment scores separated by model and candidate with debates superimposed over."%} -->
+
+
+<!-- Out of 477,                                                                                                                      
+vader got 261,
+textblob got 151,
+lstm got 233 
+ -->

blogs/2021/harry_potter_rag.md

@@ -0,0 +1,10 @@
+---
+title: Facebook's RAG
+desc: How well does it do on Harry Potter trivia?
+published: true
+date_published: 2021-09-22
+tags: nlp
+---
+
+
+I posted this on Medium to see what it would be like to publish there. [Link here](https://medium.com/@nicholas_59841/testing-facebooks-rag-model-on-harry-potter-trivia-e95c316a6487)

blogs/2021/kaggle_chaii.md

@@ -0,0 +1,75 @@
+---
+title: Reflections on Kaggle competition [chaii - Hindi and Tamil Question Answering]
+desc: Reflecting on what worked and what didn't
+published: true
+date_published: 2021-01-05
+tags: kaggle nlp
+---
+
+On the evening of November 15, 2021 I eagerly sat at my computer, counting down the seconds until 7pm. As soon as it did, I hit refresh and my heart skipped a beat as I saw my name at the 17th spot on the leaderboard. I got a silver medal in a competition with over 900 teams! I was over the moon with excitement, and I achieved the rank of Kaggle Expert in the competitions category.
+
+I'm sure this is a common experience for Kagglers when they do well in a competition for the first time. This also might be a description of the moment when Kagglers officially become addicted. And as far as addictions go, this one is at least educational, but I'm sure many spouses and partners have complained about how much time Kaggle consumes.
+
+It's been a couple of months since the competition ended, and I've come down from my high enough to reflect on what I learned from the competition. This post is quite detailed so feel free to skip around to what interests you.
+
+## Competition Details 🏆
+
+Google Research India hosted the [chaii - Hindi and Tamil Question Answering](https://www.kaggle.com/c/chaii-hindi-and-tamil-question-answering) competition to create question answering models for two Indian languages used by millions of people. Hindi is a relatively high resource language, but Tamil is more in the mid-tier category. When there aren't as many high-quality public datasets or models, it can be hard for the community and industry to make useful AI applications.
+
+### Question Answering
+
+Given a passage of text and a question about the passage, a question answering (QA) model should find the span of text that best answers that question. This is an extractive process, so the answer must be in one contiguous section of text. There is a bit of ambiguity between QA and machine reading comprehension (MRC), so click [here to read an in-depth explanation about the differences](https://alessandro-lombardini.medium.com/question-and-answering-vs-machine-reading-comprehension-qa-vs-mrc-acf599536fe1).
+
+## The power of SQuAD 💪
+
+SQuAD is a popular dataset for training English QA models, but interestingly enough, training on English SQuAD helps models do better in languages that are nothing like English, such as Hindi and Tamil. Training a model on SQuAD before training on the chaii dataset resulted in significant improvements. 
+
+## Model choices
+
+Encoder-only models like RoBERTa were heavily used, and while there were a few attempts to use encoder-decoder models like BART or T5, it seemed like people ultimately decided against it. XLM-RoBERTa (XLM-R), MuRIL, and RemBERT showed up in all of the top team's solutions and for good reason - they are high-performing multi-lingual models that can be trained on English QA to get better scores in Hindi and Tamil.
+
+### The power of multilingual models 🔥
+
+Early on there was a discussion about whether it would be better to make two monolingual models, one for Hindi and one for Tamil, or just a single multilingual model.  Quick experiments showed that the multilingual models were superior, and nearly all of the top scores relied on only multilingual models*. It was somewhat mind-blowing for me to see first-hand how multilingual models were actually learning the similarities between related languages. This played a huge factor in getting good performance because there were Bengali and Telugu datasets that gave a slight boost to the model's performance in Hindi and Tamil.  My strategy, along with many others, was to use SQuAD and TyDiQA as "QA Pre-training." This is not typical pre-training using Masked Language Modeling (MLM), but rather pre-training in the sense that the model is gaining a general understanding of language and more fine-tuning will happen afterwards. It's just the name I'm using for the stage of training that occurs in between MLM pre-training and QA fine-tuning. This stage primes the model to have a sense of how the QA task works in general, and then the finetuning helps the model understand QA in the context of a specific language or two. Maybe midtuning or pre-fine-tuning would be better names.
+
+*There was at least one team in the gold medal range that used an [interesting technique to transfer model performance from English to Hindi and Tamil](https://www.kaggle.com/competitions/chaii-hindi-and-tamil-question-answering/discussion/288110).
+
+## The limit of multilingual models
+
+After seeing that SQuAD could serve as useful QA pre-training, I thought that I should find as many QA datasets in as many languages as possible. I found datasets in Korean, Thai, Bengali, German, French, Spanish, Arabic, Vietnamese, Japanese, Persian, English, and probably some more that I'm not remembering. This was quite a lot of data to churn through, but on TPU it only took an hour or so. 
+
+This was not a rigorous examination, but it did show the amazing cross-lingual learning ability of these models. Only after I did these tests did I notice that the XLM-R paper mentioned that training on too many languages resulted in reduced performance, making me choose only a small subset of QA datasets for QA pre-training.
+
+## Max Sequence Length and Doc Stride
+
+Two of the most important parameters when training a QA model are the max sequence length and doc stride. Ideally, the model would be able to look at all of the passage in one pass, but the passages are usually long enough that they need to be broken up to be able to be processed by the model. When breaking it up, the doc stride is how much overlap there is between one chunk and the next. There didn't seem to be any well-known methods for determining max sequence length and doc stride, so most people ended up empirically finding what worked best. The standard max sequence length/doc stride of 384/128 is a good default.
+
+## BigBird 🐥
+
+One trick that set me apart from other competitors was using BigBird to handle long sequence lengths. Because many of the texts had thousands of tokens, I thought it would be advantageous to use a model that does well on long contexts. BigBird is a model designed to have the attention mechanism scale linearly in memory rather than the quadratically like the default transformer. Unfortunately, the public pre-trained BigBird models are only in English, so I had to come up with a way to stick MuRIL into BigBird. This is much simpler than it seems because MuRIL is essentially RoBERTa trained on Indian languages, and since the authors of BigBird did a "warm start" by using the pre-trained weights from RoBERTa, nearly all of the parameters lined up one-to-one. 
+
+The only component of BigBird that could not directly come from RoBERTa/MuRIL was the position embeddings. These are trained parameters that are of the shape (max sequence length, hidden size) which means RoBERTa/MuRIL only has values up to a max sequence length of 512 and BigBird needs 4096. The solution to this is to tile the RoBERTa/MuRIL embeddings 8 times so it has the right dimensions.
+
+I used the [Google TPU Research Cloud (TRC)](https://sites.research.google/trc/about/) to do training on TPU v3-8 accelerators and ran a base and large model for a couple of days on mC4 data. I used the Flax scripts in the Hugging Face examples file and it could not have been simpler. 
+
+I couldn't train the large BigBird model at 4096 sequence length, nor even 2048, but rather a measly 1024 -- even at a batch size of 1 and using bfloat16. A 6-fold ensemble of these models got my highest score, earning me a silver medal in the competition.
+
+## Ensembling
+
+I had decent scoring models using XLMR, RemBERT, MuRIL, and BigBird MuRIL, but I ran out of time before I was able to ensemble them together. I took inspiration from [this notebook from a previous competition](https://www.kaggle.com/code/theoviel/character-level-model-magic) that turned token-level predictions into character-level predictions before combining. Different models have different tokenizers, so it isn't straightforward to combine token-level outputs from them. The one thing they do have in common is the original context, thus the need for character-level predictions. To go from token-level to character-level, the token-level predictions are duplicated for each character.  
+
+I submitted an ensemble for scoring after the competition end, but it did poorly. If I ever get around to seeing why my ensemble failed, I'd be curious to know how well it could have done. Many of the gold medalists did similar ensembling, but theirs got good results, unlike mine. 
+
+## What might be interesting to explore in the future
+
+Two approaches I tried that didn't yield any results were Splinter and SAM. 
+
+### Splinter 
+
+Splinter is a model designed for few-shot QA, which seems ideal for this competition. Just like BigBird, there was only an English model so I tried to replicate it in Hindi and Tamil to no avail. The [original work](https://github.com/oriram/splinter) was done using TensorFlow 1, and I attempted to replicate it using Hugging Face and PyTorch. I trained on TPU v3-8 but I was unable to get it to converge smoothly. Hugging Face even released [official support for Splinter](https://huggingface.co/docs/transformers/model_doc/splinter), but I was still unable to get it to work. I have trained [Splinter on SQuAD v2](https://huggingface.co/nbroad/splinter-base-squad2) so I can confirm that the English version works.
+
+### SAM
+
+Sharpness-Aware Minimization is an optimization technique that is supposed to lead to better generalization because it finds region of flatness in optimization space rather than sharp holes. There are papers that indicate it does well for [computer vision tasks](https://github.com/google-research/sam) and for [language tasks](https://arxiv.org/abs/2110.08529) as well. Having a model that generalizes well is useful to not have bad surprises on the final leaderboard. When I used SAM, I had no intuition on hyperparameters, so I found my trainings diverging or just scoring the same or worse as before.
+
+Both of these approaches seemed very useful, and I thought that Splinter would have been a gold-medal-worthy approach. I'm still curious to see how well it could have been for this competition.

blogs/2021/kaggle_coleridge.md

@@ -0,0 +1,67 @@
+---
+title: Reflections on Kaggle competition [Coleridge Initiative - Show US the Data]
+desc: Reflecting on what worked and what didn't
+published: true
+date_published: 2021-12-29
+tags: kaggle nlp
+---
+
+It's been several months since the [Coleridge Initiative - Show US the Data](https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data) competition has ended, but I recently got in the mood to write a quick reflection about my experience. This reflection is mainly a way for me to assess what I learned, but maybe you'll also find something worthwhile.
+
+
+## Competition Details 🏆
+
+The hosts wanted to find all of the mentions of public datasets hidden in published journals and articles. When I say hidden, I mean that the paper refers to a dataset but never officially cites anything in the references section.  In the hosts own words,  
+
+> Can natural language processing find the hidden-in-plain-sight data citations? [This will] show how public data are being used in science and help the government make wiser, more transparent public investments."  
+
+Participants competed to see who could find the most mentions of datasets in roughly 8,000 publications.
+
+As this was my first Kaggle competition, I quickly realized that it was much more nuanced and complicated than I expected. It was also a wake-up call for me to realize how little I knew about PyTorch, Tensorflow, GPUs, TPUs, and Machine Learning in general.  I was challenged and pushed to think in new ways, and I felt that I had grown significantly after the competition ended.  
+
+Moreover, the final results of the competition were very surprising because my team jumped over 1400 spots from the public to private leaderboard to end at 47th out of 1610 teams, earning us a silver medal (top 3%). If you would like to know more about why there are separate public and private leaderboards, [read this post here.](https://qr.ae/pG6Xc1) I've included a plot below of public ranking vs private ranking to show how much "shake-up" there was. My team had the 3rd highest positive delta, meaning that only 2 other teams jumped more positions from public to private. The person who probably had the worst day went from 176 (just outside the medal range) to 1438. To understand the figure, there are essentially 3 different categories. The first category would be any point on the line y=x. This means that the team had the exact same score on public and private leaderboards. The further the points get away from y=x, the bigger the difference between public and private leaderboards. The second category is for the teams who dropped from public to private -- they are in the region between the line y=x and the y-axis. The final category is for the teams that went up from public to private leaderboards: the region from the line y=x to the x-axis. My team fell into this third category and we were practically in the bottom right corner.
+
+{% include figure image_path="/assets/images/coleridge-shakeup.png" alt="Public vs private leaderboard rank for Coleridge competition"%}
+
+## Why the shake-up?
+
+ >*shake-up is the term used on Kaggle to describe the change in rank from public to private leaderboard*
+
+In general, shake-up is caused by overfitting to the public leaderboard, not having a good cross-validation method, and having a model that does not generalize well. 
+
+For this competition, the huge jump in score was because 
+1. string matching worked well on the public leaderboard but not the private one.
+   - the public leaderboard contained dataset names from the training data but the private leaderboard didn't have any.
+2. most people did not check to see if their approach could find datasets that weren't mentioned in the training dataset
+
+If I'm being honest, I didn't have great cross-validation, but I also refused to do string matching using dataset lists because it didn't seem like it was what the hosts wanted and I was more interested in applying a deep learning approach. 
+
+## Best solution 💡
+
+ 
+It was a bit annoying that my best submission actually didn't require any training whatsoever. I suppose I could consider this as being resourceful, but I still think it's annoying considering all the time I put into other methods that ultimately scored worse.
+
+My best submission used a combination of regular expressions and a pre-trained question answering model that I pulled off the 🤗 Hugging Face Model Hub. In terms of regular expressions, I came up with a few simple patterns that looked for the sentences that contained words indicative of a dataset, such as Study, Survey, Dataset, etc. These regex patterns were used to quickly narrow down the amount of text that I would run through the slow transformer model. I sampled a few different architectures, BERT, RoBERTa, ELECTRA, and ultimately chose an ELECTRA model that had been trained on SQuAD 2.0. The final step was to pass the sentences that matched the regex patterns into the question answering model as the context, alongside "What is the dataset?" as the question. Adding the question gives the model extra information which helps it extract the right span of text. This submission went from 0.335 to 0.339 from public to private leaderboard.
+
+ 
+## Setbacks 😖
+
+As mentioned earlier, my NER models were absolutely dreadful. I figured that identifying the dataset names from passages of text was essentially token classification so I focused 95% of my efforts on building an NER model. I think NER models struggled because the training set had over 14,000 different texts, but only 130 different dataset labels. If there were a wider variety of dataset labels, it might have done better but I think the model just memorized those 130 labels. Moreover, there were many spans that had dataset names that were not labeled, so I was implicitly training the model to ignore other dataset names. I ended up using my NER model for one of my two final submissions, and it went from 0.398 in the public leaderboard to 0.038 in the private leaderboard. Big oof.
+ 
+
+## Key Takeaways 🗝️
+
+ 
+
+I don't think this was a typical Kaggle competition, and it was a little surprising to see how some top teams relied much more on good rule-based approaches rather than on deep learning models. I think most Kagglers default to using the biggest, most complicated transformer-based approach because they've heard so much about BERT et al. While transformers have been able to accomplish remarkable scores on numerous benchmarks, they can't do everything! I fell into this trap too, and I think my final score was mainly due to luck. 🍀
+
+
+
+### Current stats
+
+![competition](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/competition)
+![dataset](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/dataset)
+![notebook](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/notebook)
+![discussion](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/discussion)
+
+If you want to add Kaggle badges to your GitHub profile or website, use this: <https://github.com/subinium/kaggle-badge>

blogs/2022/exotic_cheese.md

@@ -0,0 +1,27 @@
+---
+title: "Eccentric billionaire hobby #1 - exotic cheeses"
+desc: I like cheese
+published: true
+date_published: 2022-07-04
+tags: fun
+---
+
+# 🧀
+
+While I am nowhere close to becoming a billionaire, I do occasionally daydream about frivolous scenarios that, in all likelihood, will never happen but are entertaining nonetheless. One scenario I keep returning to is the one where I'm a billionaire. "Regular" billionaires like Bezos and Musk might take trips into outer space, but I've decided that my first eccentric billionaire hobby will be to open up an exotic cheese shop.  
+
+> You might be saying, "Exotic cheese? Aren't there better ways of spending your money?" Yes, and since this is a fictional scenario, let's just assume I've already funded the research that cured cancer, I've already ended child poverty and hunger, and I've already fixed whichever tragic cause is closest to your heart. Even after all of that heroic philanthropy, I will still have tons of cash to do dumb shit.
+
+Ok, back to exotic cheese. Have you ever wondered what jaguar cheese tastes like? What about elephant cheese? Dolphin cheese? It's questions like this that keep me up at night.
+
+{% include figure image_path="/assets/images/dolphin_cheese.jpeg" alt="Woman: He's probably thinking about other women. Man: I wonder what dolphin cheese tastes like."%}
+
+This cheese shop will only be possible if the milk from any mammal can be turned into cheese, which may or may not be true. For the sake of this fantasy, let's assume it is possible. 
+
+There are tons of different mammals like giraffes, whales, and wolves, and my shop will have hundreds of different types to try! I'd be willing to bet that a good portion of these cheeses will be absolutely disgusting, but it will certainly be a one-of-a-kind experience.
+
+Now, I have absolutely no idea how to actually procure the milk from endangered/rare/dangerous animals, but I'm sure that a billionaire would be able to make it happen. Who knows, maybe my exotic cheese shop will be so popular that it will spur a movement to protect more animals so that more cheese can be made.  
+
+Oh, and if a billionaire somehow reads this and wants to make a dream come true, please reach out 😄
+
+# 🐵 🐶 🦊 🐻 🐼 🧀

blogs/2022/kaggle_commonlit_readability.md

@@ -0,0 +1,63 @@
+---
+title: Reflections on Kaggle competition [CommonLit Readability Prize]
+desc: Reflecting on what worked and what didn't
+published: true
+date_published: 2022-01-05
+tags: kaggle nlp
+---
+
+The [CommonLit Readability Prize](https://www.kaggle.com/c/commonlitreadabilityprize) was my second Kaggle competition, as well as my worst leaderboard finish. In retrospect, it was a great learning opportunity and I'm still only a *little* disappointed that I didn't rank higher. Here are some details about what I learned, what I was impressed by, and what I would do differently in the future.
+
+{% include figure image_path="https://upload.wikimedia.org/wikipedia/commons/7/74/Open_books_stacked.jpg" alt="open books" caption="Hh1718, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons" %}
+
+## Competition Details 🏆
+
+The task for the competition was to rate the readability of a passage of text - basically assigning a grade level based on the difficulty of the passage. For 99% of the participants this task seemed like a straightforward regression problem, but some interesting approaches emerged after the competition ended. Transformers were, of course, necessary to do well. The competition had over 3000 teams which I think was due to the fact that there was a large amount of prize money and students who would normally be in school having more free time during the summer.
+
+## Defining Readability 📚
+
+Readability is pretty subjective, so the hosts got many teachers from grades 3-12 to do pairwise comparisons between book excerpts. That is, the task for each teacher was to decide, given two texts, which one was relatively harder/easier to read. After getting over 100k pairwise comparisons, they used a [Bradley-Terry (BT) Model](https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model) to get values for all excerpts.
+
+## Approaches
+
+Since the training data was basically a text and a number (the readability score), most participants turned it into a regression task. Give the model a bit of text and the model will output a score. Many of the winning solutions used this, but after the competition ended, a few people shared how they tried implementing pairwise models.
+
+In essence, the pairwise model takes two texts as input and then gives a score for how much harder/easier one text is over the other. If one of those texts has a known readability score, such as those that are in the training data, the unlabeled text can be assigned a score based on the known score and the model's prediction of the relative difference in readability.
+
+I'm probably not doing the explanation justice, so please refer to the following posts.
+
+- Chris Deotte ([@cdeotte](https://www.kaggle.com/cdeotte)) [explaining how he used BT to score passages](https://www.kaggle.com/c/commonlitreadabilityprize/discussion/257446),
+- Abhishek Thakur's ([@ahbishek](https://www.kaggle.com/ahbishek)) [notebook]() and post about his pairwise model.
+- User [@Allohvk](https://www.kaggle.com/allohvk) [giving more details about BT as a loss function](https://www.kaggle.com/c/commonlitreadabilityprize/discussion/276356).
+
+I don't think the pairwise approach was able to surpass the regression approach, but I think it could potentially be used again in future competitions, such as this one: [Jigsaw Rate Severity of Toxic Comments - Rank relative ratings of toxicity between comments](https://www.kaggle.com/c/jigsaw-toxic-severity-rating)
+
+To summarize the various techniques that the top solutions used, here is a quick list.
+
+- In-domain pre-training to adapt language models to this type of text.
+  - People used [Project Gutenberg](https://www.gutenberg.org/) and other freely available literature.
+- Pseudo-labeling on unlabeled text and then training on the pseudo-labels.
+  - The unlabeled text came from sources similar to what was used for in-domain pre-training.
+- Ensembling models
+  - Some of the top approaches used over 10 different models with different architectures, different training schemes, and different sizes.
+- Adjusting predictions to have the same mean score as the training data
+- Using SVR on top of model predictions
+- Not using dropout when fine-tuning
+
+## The magic of no dropout ✨
+
+One interesting discovery during this competition was the bizarre effect of dropout when using transformer models for regression. Typically dropout is essential to prevent overfitting, but if a transformer is being used for regression, dropout will actually ***hurt*** the performance.  It seems hard to believe but [it actually made a substantial difference](https://www.kaggle.com/competitions/commonlitreadabilityprize/discussion/260729).  Some users did some digging and found published articles claiming that dropout for classification is fine because the magnitude of the outputs does not really matter – as long as the relative values produce the right answer when taking the greatest value, it doesn’t matter how big or small that output is.  With regression, the magnitude of the output is precisely what you need fine control over. [Here is a discussion on it](https://www.kaggle.com/competitions/commonlitreadabilityprize/discussion/260729#1442448).
+
+## Lessons Learned 👨‍🏫
+
+Looking back, I can identify two main reasons why I struggled: working alone and refusing to use public notebooks. Due to a combination of factors I decided to fly solo, which meant I was able to learn a lot, but it also meant I went slowly and didn't have great support when running into problems. This was useful for learning purposes but it wasted a lot of valuable time when there were good public notebooks and models available. I realized how foolish my decision was when I read how the [person who got 1st place used the public notebooks to make even better models](https://www.kaggle.com/c/commonlitreadabilityprize/discussion/257844).  I'm still a little salty that I didn't do better, but I'm taking it as a learning opportunity and moving forward. 🤷‍♂️ Onto the next one!
+
+
+### Current stats
+
+![competition](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/competition)
+![dataset](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/dataset)
+![notebook](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/notebook)
+![discussion](https://road-to-kaggle-grandmaster.vercel.app/api/badges/nbroad/discussion)
+
+If you want to add Kaggle badges to your GitHub profile or website, use this: <https://github.com/subinium/kaggle-badge>

blogs/2024/ai_disconnect.md

@@ -0,0 +1,17 @@
+---
+title: The Great AI Disconnect
+desc: The dystopian AI future that is very close.
+date_published: 2024-11-09
+published: true
+---
+
+I have an uneasy feeling about the way the digital world is getting more and more cluttered with increasingly realistic AI-generated text, images, audio, and video. In the next few years, AI will become more convincingly real as well as cheaper to produce. Social media sites will become saturated with AI content, and there won't be any reliable ways of detecting AI content. Moreover, people are already primed to believe what they want to believe, so it shouldn't even be that hard to convince people of lies. To make matters worse, it is far easier to spread AI content than it is to correct people's misconceptions. Imagine a photo that gets seen by 10 million people in 1 hour - something that happens every single day on Twitter, Instagram, Tik Tok, facebook, etc. It might take a few hours, maybe a few days to figure out if that image was actually real. How many of the millions of people would see the update that the photo was fake? I'd imagine it would be a tiny fraction, thus showing the power of spreading AI content.
+
+I am anticipating a critical event where millions of people are misled by AI-generated content, resulting in people losing all trust in anything they can't see. There are already people who don't feel they can trust any news because of the "fake-news" rhetoric. How will people be able to trust anything that they don't see with their own eyes? How will information be spread when people are disconnected from the internet? Will there need to be a slow verification process by multiple third parties before anything can be posted? These are a few of the questions I have been thinking about, and it is very hard for me to see how this catastrophe can be avoided.
+
+The one possibility I see is that people will start to rebel and destroy data centers, leading to all GPUs behind consolidated within an international organization that has many layers of redundant oversight and transparency. Maybe the GPUs will even be air-gapped so that nothing can get out, meaning that all research and development would have to happen at this one organization. There would undoubtedly be a black market of GPUs that didn't get seized, or GPUs that get "lost" somewhere in the manufacturing process. Even a single DGX machine could be pretty useful for creating a large amount of AI content, and since it is about the size of a desktop PC, it would be pretty easy to conceal and smuggle.
+
+I suppose one positive aspect is that people may start developing more in-person relationships after disconnecting from the internet. This is one silver lining, but I think there are far more negative aspects than positive ones. I predict that the Great AI Disconnect will happen before 2030.
+
+
+Why don't I think that AI-detectors will be useful? There are already AI-detectors being used to check student's essays with alarming false-positive rates. I've worked on a project related to this task, and it seems like something that is extremely difficult to reliable detect. Moreover, just like cybersecurity, this is a cat and mouse game where the attacker is always trying to beat the latest defense, and vice versa. There is always a new way to beat a new technique, and I have a feeling that it is far easier to break the detection than it is to create a robust detector.

blogs/2024/beautiful_dog.md

@@ -0,0 +1,4 @@
+---
+title: Dogs are Beautiful
+---
+

main.py

@@ -0,0 +1,25 @@
+from fh_bootstrap import *
+import about, blog
+
+hdrs = (
+    Link(href='/assets/hl-styles.css', rel='stylesheet'),
+    Link(href='/assets/styles.css', rel='stylesheet'),
+    *Socials(title='Nicholas Broad', description='', site_name='',
+             twitter_site='@nbroad1881', image=f'/assets/og-sq.png', url='')
+)
+
+app,rt = fast_app(pico=False, hdrs=bst_hdrs+hdrs, live=False)
+
+app.get('/')(about.page)
+app.get('/blog')(blog.full_page)
+
+@rt("/blog/{blog_id}")
+def get(blog_id: str):
+    return blog.single_blog(blog_id)
+
+
+@rt("/{fname:path}.{ext:static}")
+def get(fname:str, ext:str): return FileResponse(f'{fname}.{ext}')
+
+
+serve()

requirements.txt

@@ -0,0 +1,6 @@
+uvicorn
+python-fasthtml
+python-dotenv
+fasthtml-hf
+markdown
+huggingface_hub