<story><title>Dataset: Databases for lazy people</title><url>https://dataset.readthedocs.org/</url></story><parent_chain></parent_chain><comment><author>akrymski</author><text>This is excellent, I&amp;#x27;ve always wanted a simple db library that makes defining a schema upfront &lt;i&gt;optional&lt;/i&gt;.  After all a schema is a constraint that can be set in stone later on, just like adding indices for optimisation purposes.&lt;p&gt;Few questions:&lt;p&gt;- what&amp;#x27;s the performance like?  is there any overhead to alchemy, eg comparing schema every time you do an insert?&lt;p&gt;- no way to specify primary keys as an alternative to the auto-generated id column?&lt;p&gt;- no table.remove()?&lt;p&gt;- what about inserting more complicated data structures, eg a dict with a nested dict or a list?  would be great if those were serialized auto-magically into a blob type (or used to create another table with a foreign key?)&lt;p&gt;- would be nice to be able to freeze a schema with table.freeze() for example: from then on new columns don&amp;#x27;t get created automatically, or get stored in an extra blob column (this is a very common scenario for python devs where non-indexable columns just get stuck in a blob)&lt;p&gt;- let me optionally define a schema and specify defaults with table.schema(name=&amp;#x27;&amp;#x27;, price=0.0&amp;#x27;)&lt;p&gt;- would love to see table.ensureIndex(&amp;#x27;column&amp;#x27;, &amp;#x27;unique&amp;#x27;) similar to mongo for quickly creating indices&lt;p&gt;- db = Dataset() should do dataset.create(&amp;#x27;sqlite:&amp;#x2F;&amp;#x2F;&amp;#x2F;:memory:&amp;#x27;) for me - would be nice to have that as the default connector, so that Dataset() acts as LINQ for Python by default&lt;p&gt;- dataset.freeze is nice but I&amp;#x27;d rather have dataset.export() &amp;amp; dataset.import() letting me easily copy rows from one db to another (after inspection for example)&lt;p&gt;Thanks for creating this!</text></comment>
<story><title>Dataset: Databases for lazy people</title><url>https://dataset.readthedocs.org/</url></story><parent_chain></parent_chain><comment><author>alokv28</author><text>I&amp;#x27;m trying to figure out where something like this fits into the python data ecosystem.&lt;p&gt;For datasets that fit in memory, Pandas seems like the best bet. Good I&amp;#x2F;O functions (JSON, CSV), easy slicing (numpy array-like syntax), and some sql-like operations (groupby, join).&lt;p&gt;For large datasets, you&amp;#x27;d need a proper db.&lt;p&gt;So is Dataset then useful for datasets that cannot fit in memory but aren&amp;#x27;t &lt;i&gt;too&lt;/i&gt; large?</text></comment>