<commit_before>/*

Package minquery provides an efficient mgo-like Query type that supports
MongoDB query pagination (cursors to continue listing documents where
we left off).

Example using MinQuery

Let's say we have a users collection in MongoDB modeled with this Go struct:

    type User struct {
        ID      bson.ObjectID `bson:"_id"`
        Name    string        `bson:"name"`
        Country string        `bson:"country"`
    }

To query users having country=USA, sorted by Name and ID:

    q := minquery.New(session.DB(""), "users", bson.M{"country" : "USA"}).
        Sort("name", "_id").Limit(10)
    // If this is not the first page, set cursor:
    if cursor := getLastCursor(); cursor != "" {
        q = q.Cursor(cursor)
    }

    var users []*User
    newCursor, err := q.All(&users, "country", "name", "_id")

And that's all. newCursor is the cursor to be used to fetch the next batch.

Note that when calling MinQuery.All(), you have to provide the name
of the cursor fields, this will be used to build the cursor data
(and ultimately the cursor string) from.
<commit_msg>Added note about retrieving partial results and required cursor fields.<commit_after>/*

Package minquery provides an efficient mgo-like Query type that supports
MongoDB query pagination (cursors to continue listing documents where
we left off).

Example using MinQuery

Let's say we have a users collection in MongoDB modeled with this Go struct:

    type User struct {
        ID      bson.ObjectID `bson:"_id"`
        Name    string        `bson:"name"`
        Country string        `bson:"country"`
    }

To query users having country=USA, sorted by Name and ID:

    q := minquery.New(session.DB(""), "users", bson.M{"country" : "USA"}).
        Sort("name", "_id").Limit(10)
    // If this is not the first page, set cursor:
    if cursor := getLastCursor(); cursor != "" {
        q = q.Cursor(cursor)
    }

    var users []*User
    newCursor, err := q.All(&users, "country", "name", "_id")

And that's all. newCursor is the cursor to be used to fetch the next batch.

Note that when calling MinQuery.All(), you have to provide the name
of the cursor fields, this will be used to build the cursor data
(and ultimately the cursor string) from.

Note #2: if you're retrieving partial results (MinQuery.Select()), you have to include all the fields that are part of the cursor (the index), else MinQuery.All() will not be able to create the proper cursor value.

{'fraction_non_alphanumeric': 0.09919028340080972, 'fraction_numerical': 0.0020242914979757085, 'mean_word_length': 3.188135593220339, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1898012', 'n_tokens_mistral': 699, 'n_tokens_neox': 675, 'n_words': 320}