<story><title>Ask HN: Is Hackernews still using the file system as database?</title><text>I&amp;#x27;d love to learn a bit about what the current hosting&amp;#x2F;tech stack setup looks like. IIRC the posts were stored in files on the file system and scaled quite well vertically. Is that still the case?</text></story><parent_chain></parent_chain><comment><author>dchuk</author><text>While on the &amp;quot;how HN works&amp;quot; topic, I have a question:&lt;p&gt;I&amp;#x27;m working on an HN app reader idea that I think is unique (more to come) that requires regularly getting the front page posts and comments. I wrote a script to do that through the Firebase API but holy shit it ended up needing thousands of requests to get all the data for the current state of the front page.&lt;p&gt;So instead, I wrote a scraper script to produce the same thing, just with 30 requests (1 per front page post). (Trying to turn these damn &amp;lt;tr&amp;gt;&amp;#x27;s into a recursive comment tree was a quite the mindfuck btw)&lt;p&gt;...is it ok to scrape HN? I see nothing in the robots.txt to say it shouldn&amp;#x27;t be allowed, and I actually feel better making 30 scraping requests rather than XXXX API requests.&lt;p&gt;EDIT: Also happy to receive suggestions. I&amp;#x27;m coding in Ruby and don&amp;#x27;t see an obvious way to access the Firebase DB directly (quite frankly I don&amp;#x27;t even know if what I just said makes sense) so any help is appreciated.</text></comment>
<story><title>Ask HN: Is Hackernews still using the file system as database?</title><text>I&amp;#x27;d love to learn a bit about what the current hosting&amp;#x2F;tech stack setup looks like. IIRC the posts were stored in files on the file system and scaled quite well vertically. Is that still the case?</text></story><parent_chain></parent_chain><comment><author>Kaizyn</author><text>The source code used to be open, and it isn&amp;#x27;t clear what happened to it. Maybe ask over on the arclanguage.org forum as HN is written in arc.&lt;p&gt;The closest thing to a source repo is this from many years ago:
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;wting&amp;#x2F;hackernews&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;wting&amp;#x2F;hackernews&lt;/a&gt;</text></comment>