<story><title>Microsoft REST API Guidelines</title><url>https://github.com/Microsoft/api-guidelines/blob/master/Guidelines.md</url></story><parent_chain><item><author>deathanatos</author><text>Pagination is one of those things I feel like so many of these things get wrong. LIMIT&amp;#x2F;OFFSET (or as MS likes to call it, TOP&amp;#x2F;SEEK) style results in O(n²) operations; an automated tool trying to pull the entirety of a large collection in such a scenario is not good. I have to again recommend the excellent &amp;quot;Pagination done the Right Way&amp;quot; presentation[1] from Use the Index, Luke (an equally excellent site).&lt;p&gt;Just return a link to the &amp;quot;next page&amp;quot; (make the next page opaque); while this removes the ability to of the client to go to an arbitrary page on its own, in practice, I&amp;#x27;ve never seen that matter, and the generalized URL format allows you to seek instead of by LIMIT&amp;#x2F;OFFSET by something indexable by the DB; HTTP even includes a standardized header for just this purpose[2].&lt;p&gt;I also think the generalized URL is more in line with Fielding&amp;#x27;s definition of REST, in that the response links to another resource. (I don&amp;#x27;t know if being in the header invalidates this; to me, it does not.)&lt;p&gt;If you get the metadata out of the data section of your response, and move it to the headers, where it belongs, this usually then lets you keep the &amp;quot;collection&amp;quot; of items you are return as an array (because if you need to put metadata along side it, you need:&lt;p&gt;&lt;pre&gt;&lt;code&gt;  {
     &amp;quot;the_data_you_really_want&amp;quot;: [1, 2, 3],
     &amp;quot;metadata, e.g., pagination&amp;quot;: {}
  }
&lt;/code&gt;&lt;/pre&gt;
vs.&lt;p&gt;&lt;pre&gt;&lt;code&gt;  [&amp;quot;the&amp;quot;, &amp;quot;data&amp;quot;, &amp;quot;you&amp;quot;, &amp;quot;wanted&amp;quot;, &amp;quot;in a sensible format&amp;quot;]
&lt;/code&gt;&lt;/pre&gt;
)&lt;p&gt;(I&amp;#x27;ve seen metadata-pushed-into-the-body turn API endpoints that literally need to return no more than &amp;quot;true&amp;quot; or &amp;quot;false&amp;quot; into object that then force a client to know and look up the correct key in an object… &lt;i&gt;sigh&lt;/i&gt;.)&lt;p&gt;[1]: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;use-the-index-luke.com&amp;#x2F;no-offset&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;use-the-index-luke.com&amp;#x2F;no-offset&lt;/a&gt;&lt;p&gt;[2]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;tools.ietf.org&amp;#x2F;html&amp;#x2F;rfc5988&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;tools.ietf.org&amp;#x2F;html&amp;#x2F;rfc5988&lt;/a&gt;</text></item></parent_chain><comment><author>vmarsy</author><text>From the parent Microsoft guideline [1], they make the explicit differentiation between Server-driven and Client-driven paging, the Server-driven seems to be exactly what you describe in your first paragraph:&lt;p&gt;&amp;gt; &lt;i&gt;Paginated responses MUST indicate a partial result by including a continuation token in the response. The absence of a continuation token means that no additional pages are available.&lt;/i&gt;&lt;p&gt;&amp;gt; &lt;i&gt;Clients MUST treat the continuation URL as opaque, which means that query options may not be changed while iterating over a set of partial results.&lt;/i&gt;&lt;p&gt;&amp;gt; &lt;i&gt;Example:&lt;/i&gt;&lt;p&gt;&lt;pre&gt;&lt;code&gt;  {
    ...,
    &amp;quot;value&amp;quot;: [...],
    &amp;quot;@nextLink&amp;quot;: &amp;quot;{opaqueUrl}&amp;quot;
  }

&lt;/code&gt;&lt;/pre&gt;
They then explain Client-driven paging, if the client really wants to do it:&lt;p&gt;&amp;gt; &lt;i&gt;Clients MAY use $top and $skip query parameters to specify a number of results to return and an offset. The server SHOULD honor the values specified by the client; however, clients MUST be prepared to handle responses that contain a different page size or contain a continuation token.&lt;/i&gt;&lt;p&gt;But if you want to obtain the &lt;i&gt;entire&lt;/i&gt; collection, you would not use Client-driven paging, but only rely on Server-driven paging&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;Microsoft&amp;#x2F;api-guidelines&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;Guidelines.md#98-pagination&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;Microsoft&amp;#x2F;api-guidelines&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;Guid...&lt;/a&gt;</text></comment>
<story><title>Microsoft REST API Guidelines</title><url>https://github.com/Microsoft/api-guidelines/blob/master/Guidelines.md</url></story><parent_chain><item><author>deathanatos</author><text>Pagination is one of those things I feel like so many of these things get wrong. LIMIT&amp;#x2F;OFFSET (or as MS likes to call it, TOP&amp;#x2F;SEEK) style results in O(n²) operations; an automated tool trying to pull the entirety of a large collection in such a scenario is not good. I have to again recommend the excellent &amp;quot;Pagination done the Right Way&amp;quot; presentation[1] from Use the Index, Luke (an equally excellent site).&lt;p&gt;Just return a link to the &amp;quot;next page&amp;quot; (make the next page opaque); while this removes the ability to of the client to go to an arbitrary page on its own, in practice, I&amp;#x27;ve never seen that matter, and the generalized URL format allows you to seek instead of by LIMIT&amp;#x2F;OFFSET by something indexable by the DB; HTTP even includes a standardized header for just this purpose[2].&lt;p&gt;I also think the generalized URL is more in line with Fielding&amp;#x27;s definition of REST, in that the response links to another resource. (I don&amp;#x27;t know if being in the header invalidates this; to me, it does not.)&lt;p&gt;If you get the metadata out of the data section of your response, and move it to the headers, where it belongs, this usually then lets you keep the &amp;quot;collection&amp;quot; of items you are return as an array (because if you need to put metadata along side it, you need:&lt;p&gt;&lt;pre&gt;&lt;code&gt;  {
     &amp;quot;the_data_you_really_want&amp;quot;: [1, 2, 3],
     &amp;quot;metadata, e.g., pagination&amp;quot;: {}
  }
&lt;/code&gt;&lt;/pre&gt;
vs.&lt;p&gt;&lt;pre&gt;&lt;code&gt;  [&amp;quot;the&amp;quot;, &amp;quot;data&amp;quot;, &amp;quot;you&amp;quot;, &amp;quot;wanted&amp;quot;, &amp;quot;in a sensible format&amp;quot;]
&lt;/code&gt;&lt;/pre&gt;
)&lt;p&gt;(I&amp;#x27;ve seen metadata-pushed-into-the-body turn API endpoints that literally need to return no more than &amp;quot;true&amp;quot; or &amp;quot;false&amp;quot; into object that then force a client to know and look up the correct key in an object… &lt;i&gt;sigh&lt;/i&gt;.)&lt;p&gt;[1]: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;use-the-index-luke.com&amp;#x2F;no-offset&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;use-the-index-luke.com&amp;#x2F;no-offset&lt;/a&gt;&lt;p&gt;[2]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;tools.ietf.org&amp;#x2F;html&amp;#x2F;rfc5988&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;tools.ietf.org&amp;#x2F;html&amp;#x2F;rfc5988&lt;/a&gt;</text></item></parent_chain><comment><author>brandonbloom</author><text>Big +1 to killing offset in favor of order-by and an &amp;quot;after&amp;quot; or &amp;quot;before&amp;quot; criteria.&lt;p&gt;However, -1 to the idea of sticking any important &amp;quot;metadata&amp;quot; in headers. For one, it&amp;#x27;s dramatically easier to interact with response bodies than headers at the terminal or with simple programming libraries and tools. Having only one way of representing data requires fewer special cases. JSON is far superior to key&amp;#x2F;value pairs in terms of ability to represent structure. Lastly, if you ever decide to use a different &amp;quot;transport&amp;quot; protocol other than HTTP, you&amp;#x27;ll be glad you are already using a common representation without a side channel.</text></comment>