<story><title>A Year with MongoDB</title><url>http://blog.engineering.kiip.me/post/20988881092/a-year-with-mongodb</url></story><parent_chain><item><author>sirn</author><text>&amp;#62; We changed the structure of our heaviest used models a couple times in the past year, and instead of going back and updating millions of old documents, we simply added a “version” field to the document and the application handled the logic of reading both the old and new version. This flexibility was useful for both application developers and operations engineers.&lt;p&gt;Ugh, this sounds like a maintenance nightmare. How do you deal with adding extra field to the document? Do you ever feel the need of running on-the-fly migration of old versions? (But when you do, shouldn&apos;t running a migration for all documents a better idea?)&lt;p&gt;I&apos;ll admit I&apos;m a non-believer, but every time I see &quot;Schemaless&quot; in MongoDB, I think &quot;oh, so you&apos;re implementing schema in your application?&quot;</text></item></parent_chain><comment><author>mitchellh</author><text>&amp;#62; Ugh, this sounds like a maintenance nightmare. How do you deal with adding extra field to the document? Do you ever feel the need of running on-the-fly migration of old versions? (But when you do, shouldn&apos;t running a migration for all documents a better idea?)&lt;p&gt;Yes, we did on-the-fly migration as we loaded old data in.&lt;p&gt;Doing full data migration was not really an option because querying from MongoDB on un-indexed data is so slow, and paging in all that data would purge hot data, exacerbating the problem.&lt;p&gt;&amp;#62; I&apos;ll admit I&apos;m a non-believer, but every time I see &quot;Schemaless&quot; in MongoDB, I think &quot;oh, so you&apos;re implementing schema in your application?&quot;&lt;p&gt;That&apos;s exactly what happens.</text></comment>
<story><title>A Year with MongoDB</title><url>http://blog.engineering.kiip.me/post/20988881092/a-year-with-mongodb</url></story><parent_chain><item><author>sirn</author><text>&amp;#62; We changed the structure of our heaviest used models a couple times in the past year, and instead of going back and updating millions of old documents, we simply added a “version” field to the document and the application handled the logic of reading both the old and new version. This flexibility was useful for both application developers and operations engineers.&lt;p&gt;Ugh, this sounds like a maintenance nightmare. How do you deal with adding extra field to the document? Do you ever feel the need of running on-the-fly migration of old versions? (But when you do, shouldn&apos;t running a migration for all documents a better idea?)&lt;p&gt;I&apos;ll admit I&apos;m a non-believer, but every time I see &quot;Schemaless&quot; in MongoDB, I think &quot;oh, so you&apos;re implementing schema in your application?&quot;</text></item></parent_chain><comment><author>radicalbyte</author><text>&amp;#62; oh, so you&apos;re implementing schema in your application?&lt;p&gt;Isn&apos;t that where the schema belongs?  Each document represents a conceptual whole.  It doesn&apos;t contain fields which have to be NULL simply because they weren&apos;t in previous versions of the schema.&lt;p&gt;I&apos;ve been an rdbms guy (datawarehousing/ETL) for a long time now, I&apos;ve seen a lot of large databases which have been in production for considerable time.  They get messy.  Really messy.  They become basically unmaintainable.  Apples, oranges and pears all squashed into a schema the shape of a banana.&lt;p&gt;It&apos;s a pretty elegant solution, and is the problem XML/XSD were designed to solve&lt;i&gt;.&lt;p&gt;The cleanest solution that I&apos;ve seen in production used a relational database as a blob storage for XML-serialized entities.  Each table defined a basic interface for the models, but each model was free to use its own general schema.  After 10 years it contained a set of very clean individual entities which were conceptually correct.&lt;p&gt;&lt;/i&gt; As opposed to the usage as a serialization format for remoting, which has been largely replaced with JSON.</text></comment>