<commit_before># Waterline Changelog

### Edge

##### General
* [BREAKING] Waterline attribute names must now be [ECMAScript 5.1-compatible variable names](https://github.com/mikermcneil/machinepack-javascript/blob/3786c05388cf49220a6d3b6dbbc1d80312d247ec/machines/validate-varname.js#L41).
  + Custom column names can still be configured to anything, as long as it is supported by the underlying database.
* [BREAKING] Breaking changes to criteria usage:
  + For performance, criteria passed in to Waterline's model methods will now be mutated in-place in most situations (whereas in Sails/Waterline v0.12, this was not necessarily the case.)
  + Aggregation clauses (`sum`, `average`, `min`, `max`, and `groupBy`) are no longer supported in criteria.  Instead, see new model methods.
  + `limit: 0` **no longer does the same thing as `limit: undefined`**.  Instead of matching ∞ results, it now matches 0 results.
  + `skip: -20` **no longer does the same thing as `skip: undefined`**.  Instead of skipping zero results, it now refuses to run with an error.
  + Limit must be < Number.MAX_SAFE_INTEGER (...with one exception: for compatibility/convenience, `Infinity` is tolerated and normalized to `Number.MAX_SAFE_INTEGER` automatically.)
  + Skip must be < Number.MAX_SAFE_INTEGER
  + Criteria dictionaries with a mixed `where` clause are no longer supported.
    + e.g. instead of `{ username: 'santaclaus', limit: 4, select: ['beardLength', 'lat', 'long']}`,
    + use `{ where: { username: 'santaclaus' }, limit: 4, select: ['beardLength', 'lat', 'long'] }`.
    + Note that you can still do `{ username: 'santaclaus' }` as shorthand for `{ where: { username: 'santaclaus' } }` -- it's just that you can't mix other top-level criteria clauses (like `limit`) alongside constraints (e.g. `username`).
    + And as for anywhere you're building criteria using Waterline's chainable deferred object, then don't worry about this-- it's taken care of for you.
* [DEPRECATE] Deprecated criteria usage:
  + Avoid specifying a limit of < 0.  It is still ignored, and acts like `limit: undefined`, but it now logs a deprecation warning to the console.
* [BREAKING] With the major exception of `.populate()`, repeated use of any other one chainable query method like `.sort()`, `.where()`, `.set()`, `.meta()`, etc is no longer supported. For example, you should not do: `User.find().where({username: 'santaclaus'}).where({location: 'north pole'})`
* [BREAKING] Coercion of result records
  + Resulting records from calling model methods are no longer special instances-- they are just dictionaries (plain JavaScript objects)
  + There are now warning messages for some common problematic results from the adapter. This is designed to make it easier to catch schema migration issues, as well as to identify adapter bugs.
  

##### Automigrations
* [BREAKING] Automigrations now live outside of Waterline core (in waterline-util)
  + Remove `index` for automigrations
  + In core SQL adapters, `.create()` and `.createEach()` no longer deals with updating the current autoincrement sequence (the "next value to use") when a record with a greater value is explicitly created

##### Data types
* The data types in Waterline have changed to more closely reflect their purpose: validation and coercion of JavaScript values. This drastically reduced the number of types to just 5: string, number, boolean, json, and ref.
* To allow for flexibility in automigrations, attributes may also specify a new key, `columnType`. If specified, the `columnType` is sent to the appropriate adapter during automigration (in sails-hook-orm). This allows Sails/Waterline models to indicate how the values for individual attributes should be stored _at rest_ vs. how they are validated/coerced when your code calls `.create()` or `.update()`.
* All documented previously-supported types are checked for and adjusted if possible (in sails-hook-orm), but if you are using a custom type, you may need to choose an appropriate `type` and `columnType`.
* `defaultsTo` can no longer be specified as a function. In practice, this can lead to unintended consequences, and its implementation was adding considerable weight and complexity to Waterline (without a whole lot of tangible benefit).
* Optional attributes with no value specified are no longer necessarily stored as `null`. If they are set to `type: 'json'` or `type: 'ref'`, and there is no `defaultsTo`, then `null` is stored. But, if an attribute declares itself as `type: 'string'`, then when a record is created without specifying a value for that attribute, it is stored as `''` (empty string). Similarly, `type: 'number'` is stored as `0`, and `type: 'boolean'` as `false`. To represent an attribute which might be `null` or a string, use `type: 'json'` (combining it with the new `isString` validation rule, if you like).

##### Model methods
+ Revamped [.stream()](http://sailsjs.com/documentation/reference/waterline-orm/models/stream)
  + Simplify interface and remove reliance on emitters in favor of [adapter-agnostic batch processing](https://gitter.im/balderdashy/sails?at=58655edd9d4cc4fc53553d51).
  + Add support for `.populate()`
  + Now supports batch-at-a-time or record-at-a-time iteration.
  
  


<!--

  TODO: figure this out; see https://gist.github.com/mikermcneil/dfc6b033ea8a75cb467e8d50606c81cc

##### `required` & `allowNull`

* [BREAKING] Standardizing the definition of `required`
  + If an attribute specifies itself as `required`, it means that a value for the attribute must be _defined_ when using Waterline to do a `.create()`.
  + For example, if `foo` is a required attribute, then passing in `foo: undefined` or omitting `foo` on a `.create()` would fail the required check.
* [NEW] The introduction of `allowNull` (some details still TBD)
  + If an attribute specifies itself as `allowNull: true`, then if a value for that attr is explicitly provided as `null` in a `.create()` or `.update()`, it will always be allowed through-- even in cases where it wouldn't be normally (i.e. the RTTC type safety check is skipped.)  For example, this allows you to set `null` for `type: 'string'` attributes (or "number", or "boolean").
  + If you attempt to explicitly specify `allowNull: false`, then you're prevented from initializing Waterline.  (You'll see an error, suggesting that you should use `validations: { notNull: true }` instead.)  This behavior could change in future versions of Waterline to allow for more intuitive usage, but for now, since there are other changes to how type safety checks work, it's better to err on the side of strictness.
  + Other types (json and ref) allow `null` out of the box anyway, so `allowNull: true` is not necessary for them.  If you try to set `allowNull: true` on a type: 'json' or type: 'ref' attribute, Waterline will refuse to initialize (explaining that this configuration is redundant, and that you can remove `allowNull: true`, since `type: 'json'`/`type: 'ref'` implicitly allow it anyways).
    + This is completely separate from the `required` check, which works the same way regardless- an attribute can be both `required: true` AND `allowNull: true`...as long as it is allowed to be both of those things individually.
  + If `allowNull: true` is set, then `defaultsTo` is allowed to be set to `null` regardless of whether it would normally be allowed.
    + For type: 'string', the implicit default is `''`
    + For type: 'number', the implicit default is `0`
    + For type: 'boolean', the implicit default is `false`
    + For type: 'json', the implicit default is `null`
    + For type: 'ref', the implicit default is `null`
    + For `model: ...`, the implicit default is `null`
  + In waterline-schema: If a singular ("model") association is NOT `required`, and it does not specify a `allowNull` setting, then the association is implicitly set to `allowNull: true`.
  + In waterline-schema: If a singular ("model") association IS `required`, and it does not specify a `allowNull` setting, then `null` will not pass (because it is not a valid foreign key value)
  + `allowNull: true` is never allowed on plural ("collection") associations
  + `allowNull: true` is also never allowed to be explicitly set on singular ("model") associations: if `allowNull: true` is explicitly set, then Waterline fails to initialize w/ an error (telling you that the attribute definition is invalid because you should not ever need to explicitly set `allowNull` on a singular ("model") association)
  + **Best practice**
    + Most of the time, `allowNull` shouldn't need to be used.  (For most attributes, it tends to be better not to use `null`- since it's so easy to store `null` accidentally, and then cause hard-to-debug data type mismatch issues.)


-->
<commit_msg>Fix up TODO in changelog.<commit_after># Waterline Changelog

### Edge

##### General
* [BREAKING] Waterline attribute names must now be [ECMAScript 5.1-compatible variable names](https://github.com/mikermcneil/machinepack-javascript/blob/3786c05388cf49220a6d3b6dbbc1d80312d247ec/machines/validate-varname.js#L41).
  + Custom column names can still be configured to anything, as long as it is supported by the underlying database.
* [BREAKING] Breaking changes to criteria usage:
  + For performance, criteria passed in to Waterline's model methods will now be mutated in-place in most situations (whereas in Sails/Waterline v0.12, this was not necessarily the case.)
  + Aggregation clauses (`sum`, `average`, `min`, `max`, and `groupBy`) are no longer supported in criteria.  Instead, see new model methods.
  + `limit: 0` **no longer does the same thing as `limit: undefined`**.  Instead of matching ∞ results, it now matches 0 results.
  + `skip: -20` **no longer does the same thing as `skip: undefined`**.  Instead of skipping zero results, it now refuses to run with an error.
  + Limit must be < Number.MAX_SAFE_INTEGER (...with one exception: for compatibility/convenience, `Infinity` is tolerated and normalized to `Number.MAX_SAFE_INTEGER` automatically.)
  + Skip must be < Number.MAX_SAFE_INTEGER
  + Criteria dictionaries with a mixed `where` clause are no longer supported.
    + e.g. instead of `{ username: 'santaclaus', limit: 4, select: ['beardLength', 'lat', 'long']}`,
    + use `{ where: { username: 'santaclaus' }, limit: 4, select: ['beardLength', 'lat', 'long'] }`.
    + Note that you can still do `{ username: 'santaclaus' }` as shorthand for `{ where: { username: 'santaclaus' } }` -- it's just that you can't mix other top-level criteria clauses (like `limit`) alongside constraints (e.g. `username`).
    + And as for anywhere you're building criteria using Waterline's chainable deferred object, then don't worry about this-- it's taken care of for you.
* [DEPRECATE] Deprecated criteria usage:
  + Avoid specifying a limit of < 0.  It is still ignored, and acts like `limit: undefined`, but it now logs a deprecation warning to the console.
* [BREAKING] With the major exception of `.populate()`, repeated use of any other one chainable query method like `.sort()`, `.where()`, `.set()`, `.meta()`, etc is no longer supported. For example, you should not do: `User.find().where({username: 'santaclaus'}).where({location: 'north pole'})`
* [BREAKING] Coercion of result records
  + Resulting records from calling model methods are no longer special instances-- they are just dictionaries (plain JavaScript objects)
  + There are now warning messages for some common problematic results from the adapter. This is designed to make it easier to catch schema migration issues, as well as to identify adapter bugs.


##### Automigrations
* [BREAKING] Automigrations now live outside of Waterline core (in waterline-util)
  + Remove `index` for automigrations
  + In core SQL adapters, `.create()` and `.createEach()` no longer deals with updating the current autoincrement sequence (the "next value to use") when a record with a greater value is explicitly created

##### Data types
* The data types in Waterline have changed to more closely reflect their purpose: validation and coercion of JavaScript values. This drastically reduced the number of types to just 5: string, number, boolean, json, and ref.
* To allow for flexibility in automigrations, attributes may also specify a new key, `columnType`. If specified, the `columnType` is sent to the appropriate adapter during automigration (in sails-hook-orm). This allows Sails/Waterline models to indicate how the values for individual attributes should be stored _at rest_ vs. how they are validated/coerced when your code calls `.create()` or `.update()`.
* All documented previously-supported types are checked for and adjusted if possible (in sails-hook-orm), but if you are using a custom type, you may need to choose an appropriate `type` and `columnType`.
* `defaultsTo` can no longer be specified as a function. In practice, this can lead to unintended consequences, and its implementation was adding considerable weight and complexity to Waterline (without a whole lot of tangible benefit).
* Optional attributes with no value specified are no longer necessarily stored as `null`. If they are set to `type: 'json'` or `type: 'ref'`, and there is no `defaultsTo`, then `null` is stored. But, if an attribute declares itself as `type: 'string'`, then when a record is created without specifying a value for that attribute, it is stored as `''` (empty string). Similarly, `type: 'number'` is stored as `0`, and `type: 'boolean'` as `false`. To represent an attribute which might be `null` or a string, use `type: 'json'` (combining it with the new `isString` validation rule, if you like).

##### Model methods
+ Revamped [.stream()](http://sailsjs.com/documentation/reference/waterline-orm/models/stream)
  + Simplify interface and remove reliance on emitters in favor of [adapter-agnostic batch processing](https://gitter.im/balderdashy/sails?at=58655edd9d4cc4fc53553d51).
  + Add support for `.populate()`
  + Now supports batch-at-a-time or record-at-a-time iteration.


##### `required` & `allowNull`

* [BREAKING] Standardizing the definition of `required`
  + If an attribute specifies itself as `required`, it means that a value for the attribute must be _defined_ when using Waterline to do a `.create()`.
  + For example, if `foo` is a required attribute, then passing in `foo: undefined` or omitting `foo` on a `.create()` would fail the required check.
  + In addition, trying to .create() OR .update() the value as either `''` (empty string) or `null` would fail the required check.
+ If an attribute specifies itself as `type: 'string'`, then if a value for that attr is explicitly provided as `null` in a `.create()` or `.update()`, it will **no longer be allowed through**-- regardless of the attribute's `required` status.
+ Other types (json and ref) allow `null` out of the box.  To support a string attribute which might be `null`, you'll want to set the attribute to `type: 'json'`.  If you want to prevent numbers, booleans, arrays, and dictionaries, then you'll also want to add the `isString: true` validation rule.
+ For more information and a reference of edge cases, see https://docs.google.com/spreadsheets/d/1whV739iW6O9SxRZLCIe2lpvuAUqm-ie7j7tn_Pjir3s/edit#gid=1927470769

{'fraction_non_alphanumeric': 0.09746512084888977, 'fraction_numerical': 0.01002161524857536, 'mean_word_length': 4.598826549321599, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 4, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1617246', 'n_tokens_mistral': 4303, 'n_tokens_neox': 4084, 'n_words': 2079}