<story><title>Super-Structured Data: Rethinking the Schema</title><url>https://www.brimdata.io/blog/super-structured-data/</url></story><parent_chain></parent_chain><comment><author>simonw</author><text>&amp;gt; The idea here is that instead of manually creating schemas, what if the schemas were automatically created for you? When something doesn’t fit in a table, how about automatically adding columns for the missing fields?&lt;p&gt;I&amp;#x27;ve been experimenting with this approach against SQLite for a few years now, and I really like it.&lt;p&gt;My sqlite-utils package does exactly this. Try running this on the command line:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    brew install sqlite-utils
    echo &amp;#x27;[
      {&amp;quot;id&amp;quot;: 1, &amp;quot;name&amp;quot;: &amp;quot;Cleo&amp;quot;},
      {&amp;quot;id&amp;quot;: 2, &amp;quot;name&amp;quot;: &amp;quot;Azy&amp;quot;, &amp;quot;age&amp;quot;: 1.5}
    ]&amp;#x27; | sqlite-utils insert &amp;#x2F;tmp&amp;#x2F;demo.db creatures - --pk id
    sqlite-utils schema &amp;#x2F;tmp&amp;#x2F;demo.db
&lt;/code&gt;&lt;/pre&gt;
It outputs the generated schema:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    CREATE TABLE [creatures] (
       [id] INTEGER PRIMARY KEY,
       [name] TEXT,
       [age] FLOAT
    );
&lt;/code&gt;&lt;/pre&gt;
When you insert more data you can use the --alter flag to have it automatically create any missing columns.&lt;p&gt;Full documentation here: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;sqlite-utils.datasette.io&amp;#x2F;en&amp;#x2F;stable&amp;#x2F;cli.html#inserting-json-data&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;sqlite-utils.datasette.io&amp;#x2F;en&amp;#x2F;stable&amp;#x2F;cli.html#inserti...&lt;/a&gt;&lt;p&gt;It&amp;#x27;s also available as a Python library: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;sqlite-utils.datasette.io&amp;#x2F;en&amp;#x2F;stable&amp;#x2F;python-api.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;sqlite-utils.datasette.io&amp;#x2F;en&amp;#x2F;stable&amp;#x2F;python-api.html&lt;/a&gt;</text></comment>
<story><title>Super-Structured Data: Rethinking the Schema</title><url>https://www.brimdata.io/blog/super-structured-data/</url></story><parent_chain></parent_chain><comment><author>mamcx</author><text>Note: The relational model (even SQL) is THIS.&lt;p&gt;Despite the claims, SQL is NOT &amp;quot;schema-fixed&amp;quot;.&lt;p&gt;You can 100% create new schemas, alter them and modify them.&lt;p&gt;What actual happens is that if you have a CENTRAL repository of data (aka &amp;quot;source of truth&amp;quot;), then you bet you wanna &amp;quot;freeze&amp;quot; your schemas (because is like a API, where you need to fulfill contracts).&lt;p&gt;--&lt;p&gt;SQL have limitations in lack of composability, the biggest reason &amp;quot;NoSQL&amp;quot; work is this: A JSON is composable. A &amp;quot;stringy&amp;quot; SQL is not. If SQL were really around &amp;quot;relations, tupes&amp;quot; like (stealing from my project, TablaM):&lt;p&gt;&lt;pre&gt;&lt;code&gt;    [Customer id:i32, name:Str; 1, &amp;quot;Jhon&amp;quot;]
&lt;/code&gt;&lt;/pre&gt;
then developers will have less reason to go elsewhere.</text></comment>