<commit_before>=====================
Querying the database
=====================
:class:`~mongoengine.Document` classes have an :attr:`objects` attribute, which
is used for accessing the objects in the database associated with the class.
The :attr:`objects` attribute is actually a
:class:`~mongoengine.queryset.QuerySetManager`, which creates and returns a new
:class:`~mongoengine.queryset.QuerySet` object on access. The
:class:`~mongoengine.queryset.QuerySet` object may be iterated over to
fetch documents from the database::

    # Prints out the names of all the users in the database
    for user in User.objects:
        print user.name

.. note::

    As of MongoEngine 0.8 the querysets utilise a local cache.  So iterating
    it multiple times will only cause a single query.  If this is not the
    desired behaviour you can call :class:`~mongoengine.QuerySet.no_cache`
    (version **0.8.3+**) to return a non-caching queryset.

Filtering queries
=================
The query may be filtered by calling the
:class:`~mongoengine.queryset.QuerySet` object with field lookup keyword
arguments. The keys in the keyword arguments correspond to fields on the
:class:`~mongoengine.Document` you are querying::

    # This will return a QuerySet that will only iterate over users whose
    # 'country' field is set to 'uk'
    uk_users = User.objects(country='uk')

Fields on embedded documents may also be referred to using field lookup syntax
by using a double-underscore in place of the dot in object attribute access
syntax::

    # This will return a QuerySet that will only iterate over pages that have
    # been written by a user whose 'country' field is set to 'uk'
    uk_pages = Page.objects(author__country='uk')

.. note::

   (version **0.9.1+**) if your field name is like mongodb operator name (for example
   type, lte, lt...) and you want to place it at the end of lookup keyword
   mongoengine automatically  prepend $ to it. To avoid this use  __ at the end of
   your lookup keyword. For example if your field name is ``type`` and you want to
   query by this field you must use ``.objects(user__type__="admin")`` instead of
   ``.objects(user__type="admin")``

Query operators
===============
Operators other than equality may also be used in queries --- just attach the
operator name to a key with a double-underscore::

    # Only find users whose age is 18 or less
    young_users = Users.objects(age__lte=18)

Available operators are as follows:

* ``ne`` -- not equal to
* ``lt`` -- less than
* ``lte`` -- less than or equal to
* ``gt`` -- greater than
* ``gte`` -- greater than or equal to
* ``not`` -- negate a standard check, may be used before other operators (e.g.
  ``Q(age__not__mod=5)``)
* ``in`` -- value is in list (a list of values should be provided)
* ``nin`` -- value is not in list (a list of values should be provided)
* ``mod`` -- ``value % x == y``, where ``x`` and ``y`` are two provided values
* ``all`` -- every item in list of values provided is in array
* ``size`` -- the size of the array is
* ``exists`` -- value for field exists

String queries
--------------

The following operators are available as shortcuts to querying with regular
expressions:

* ``exact`` -- string field exactly matches value
* ``iexact`` -- string field exactly matches value (case insensitive)
* ``contains`` -- string field contains value
* ``icontains`` -- string field contains value (case insensitive)
* ``startswith`` -- string field starts with value
* ``istartswith`` -- string field starts with value (case insensitive)
* ``endswith`` -- string field ends with value
* ``iendswith`` -- string field ends with value (case insensitive)
* ``match``  -- performs an $elemMatch so you can match an entire document within an array


Geo queries
-----------

There are a few special operators for performing geographical queries.
The following were added in MongoEngine 0.8 for
:class:`~mongoengine.fields.PointField`,
:class:`~mongoengine.fields.LineStringField` and
:class:`~mongoengine.fields.PolygonField`:

* ``geo_within`` -- check if a geometry is within a polygon. For ease of use
  it accepts either a geojson geometry or just the polygon coordinates eg::

        loc.objects(point__geo_within=[[[40, 5], [40, 6], [41, 6], [40, 5]]])
        loc.objects(point__geo_within={"type": "Polygon",
                                 "coordinates": [[[40, 5], [40, 6], [41, 6], [40, 5]]]})

* ``geo_within_box`` -- simplified geo_within searching with a box eg::

        loc.objects(point__geo_within_box=[(-125.0, 35.0), (-100.0, 40.0)])
        loc.objects(point__geo_within_box=[<bottom left coordinates>, <upper right coordinates>])

* ``geo_within_polygon`` -- simplified geo_within searching within a simple polygon eg::

        loc.objects(point__geo_within_polygon=[[40, 5], [40, 6], [41, 6], [40, 5]])
        loc.objects(point__geo_within_polygon=[ [ <x1> , <y1> ] ,
                                                [ <x2> , <y2> ] ,
                                                [ <x3> , <y3> ] ])

* ``geo_within_center`` -- simplified geo_within the flat circle radius of a point eg::

        loc.objects(point__geo_within_center=[(-125.0, 35.0), 1])
        loc.objects(point__geo_within_center=[ [ <x>, <y> ] , <radius> ])

* ``geo_within_sphere`` -- simplified geo_within the spherical circle radius of a point eg::

        loc.objects(point__geo_within_sphere=[(-125.0, 35.0), 1])
        loc.objects(point__geo_within_sphere=[ [ <x>, <y> ] , <radius> ])

* ``geo_intersects`` -- selects all locations that intersect with a geometry eg::

        # Inferred from provided points lists:
        loc.objects(poly__geo_intersects=[40, 6])
        loc.objects(poly__geo_intersects=[[40, 5], [40, 6]])
        loc.objects(poly__geo_intersects=[[[40, 5], [40, 6], [41, 6], [41, 5], [40, 5]]])

        # With geoJson style objects
        loc.objects(poly__geo_intersects={"type": "Point", "coordinates": [40, 6]})
        loc.objects(poly__geo_intersects={"type": "LineString",
                                          "coordinates": [[40, 5], [40, 6]]})
        loc.objects(poly__geo_intersects={"type": "Polygon",
                                          "coordinates": [[[40, 5], [40, 6], [41, 6], [41, 5], [40, 5]]]})

* ``near`` -- find all the locations near a given point::

        loc.objects(point__near=[40, 5])
        loc.objects(point__near={"type": "Point", "coordinates": [40, 5]})

  You can also set the maximum and/or the minimum distance in meters as well::

        loc.objects(point__near=[40, 5], point__max_distance=1000)
        loc.objects(point__near=[40, 5], point__min_distance=100)

The older 2D indexes are still supported with the
:class:`~mongoengine.fields.GeoPointField`:

* ``within_distance`` -- provide a list containing a point and a maximum
  distance (e.g. [(41.342, -87.653), 5])
* ``within_spherical_distance`` -- same as above but using the spherical geo model
  (e.g. [(41.342, -87.653), 5/earth_radius])
* ``near`` -- order the documents by how close they are to a given point
* ``near_sphere`` -- Same as above but using the spherical geo model
* ``within_box`` -- filter documents to those within a given bounding box (e.g.
  [(35.0, -125.0), (40.0, -100.0)])
* ``within_polygon`` -- filter documents to those within a given polygon (e.g.
  [(41.91,-87.69), (41.92,-87.68), (41.91,-87.65), (41.89,-87.65)]).

  .. note:: Requires Mongo Server 2.0

* ``max_distance`` -- can be added to your location queries to set a maximum
  distance.
* ``min_distance`` -- can be added to your location queries to set a minimum
  distance.

Querying lists
--------------
On most fields, this syntax will look up documents where the field specified
matches the given value exactly, but when the field refers to a
:class:`~mongoengine.fields.ListField`, a single item may be provided, in which case
lists that contain that item will be matched::

    class Page(Document):
        tags = ListField(StringField())

    # This will match all pages that have the word 'coding' as an item in the
    # 'tags' list
    Page.objects(tags='coding')

It is possible to query by position in a list by using a numerical value as a
query operator. So if you wanted to find all pages whose first tag was ``db``,
you could use the following query::

    Page.objects(tags__0='db')

If you only want to fetch part of a list eg: you want to paginate a list, then
the `slice` operator is required::

    # comments - skip 5, limit 10
    Page.objects.fields(slice__comments=[5, 10])

For updating documents, if you don't know the position in a list, you can use
the $ positional operator ::

    Post.objects(comments__by="joe").update(**{'inc__comments__$__votes': 1})

However, this doesn't map well to the syntax so you can also use a capital S instead ::

    Post.objects(comments__by="joe").update(inc__comments__S__votes=1)

.. note::
    Due to :program:`Mongo`, currently the $ operator only applies to the
    first matched item in the query.


Raw queries
-----------
It is possible to provide a raw :mod:`PyMongo` query as a query parameter, which will
be integrated directly into the query. This is done using the ``__raw__``
keyword argument::

    Page.objects(__raw__={'tags': 'coding'})

.. versionadded:: 0.4

Limiting and skipping results
=============================
Just as with traditional ORMs, you may limit the number of results returned or
skip a number or results in you query.
:meth:`~mongoengine.queryset.QuerySet.limit` and
:meth:`~mongoengine.queryset.QuerySet.skip` and methods are available on
:class:`~mongoengine.queryset.QuerySet` objects, but the `array-slicing` syntax
is preferred for achieving this::

    # Only the first 5 people
    users = User.objects[:5]

    # All except for the first 5 people
    users = User.objects[5:]

    # 5 users, starting from the 10th user found
    users = User.objects[10:15]

You may also index the query to retrieve a single result. If an item at that<commit_msg>Fix array-slicing documentation<commit_after>=====================
Querying the database
=====================
:class:`~mongoengine.Document` classes have an :attr:`objects` attribute, which
is used for accessing the objects in the database associated with the class.
The :attr:`objects` attribute is actually a
:class:`~mongoengine.queryset.QuerySetManager`, which creates and returns a new
:class:`~mongoengine.queryset.QuerySet` object on access. The
:class:`~mongoengine.queryset.QuerySet` object may be iterated over to
fetch documents from the database::

    # Prints out the names of all the users in the database
    for user in User.objects:
        print user.name

.. note::

    As of MongoEngine 0.8 the querysets utilise a local cache.  So iterating
    it multiple times will only cause a single query.  If this is not the
    desired behaviour you can call :class:`~mongoengine.QuerySet.no_cache`
    (version **0.8.3+**) to return a non-caching queryset.

Filtering queries
=================
The query may be filtered by calling the
:class:`~mongoengine.queryset.QuerySet` object with field lookup keyword
arguments. The keys in the keyword arguments correspond to fields on the
:class:`~mongoengine.Document` you are querying::

    # This will return a QuerySet that will only iterate over users whose
    # 'country' field is set to 'uk'
    uk_users = User.objects(country='uk')

Fields on embedded documents may also be referred to using field lookup syntax
by using a double-underscore in place of the dot in object attribute access
syntax::

    # This will return a QuerySet that will only iterate over pages that have
    # been written by a user whose 'country' field is set to 'uk'
    uk_pages = Page.objects(author__country='uk')

.. note::

   (version **0.9.1+**) if your field name is like mongodb operator name (for example
   type, lte, lt...) and you want to place it at the end of lookup keyword
   mongoengine automatically  prepend $ to it. To avoid this use  __ at the end of
   your lookup keyword. For example if your field name is ``type`` and you want to
   query by this field you must use ``.objects(user__type__="admin")`` instead of
   ``.objects(user__type="admin")``

Query operators
===============
Operators other than equality may also be used in queries --- just attach the
operator name to a key with a double-underscore::

    # Only find users whose age is 18 or less
    young_users = Users.objects(age__lte=18)

Available operators are as follows:

* ``ne`` -- not equal to
* ``lt`` -- less than
* ``lte`` -- less than or equal to
* ``gt`` -- greater than
* ``gte`` -- greater than or equal to
* ``not`` -- negate a standard check, may be used before other operators (e.g.
  ``Q(age__not__mod=5)``)
* ``in`` -- value is in list (a list of values should be provided)
* ``nin`` -- value is not in list (a list of values should be provided)
* ``mod`` -- ``value % x == y``, where ``x`` and ``y`` are two provided values
* ``all`` -- every item in list of values provided is in array
* ``size`` -- the size of the array is
* ``exists`` -- value for field exists

String queries
--------------

The following operators are available as shortcuts to querying with regular
expressions:

* ``exact`` -- string field exactly matches value
* ``iexact`` -- string field exactly matches value (case insensitive)
* ``contains`` -- string field contains value
* ``icontains`` -- string field contains value (case insensitive)
* ``startswith`` -- string field starts with value
* ``istartswith`` -- string field starts with value (case insensitive)
* ``endswith`` -- string field ends with value
* ``iendswith`` -- string field ends with value (case insensitive)
* ``match``  -- performs an $elemMatch so you can match an entire document within an array


Geo queries
-----------

There are a few special operators for performing geographical queries.
The following were added in MongoEngine 0.8 for
:class:`~mongoengine.fields.PointField`,
:class:`~mongoengine.fields.LineStringField` and
:class:`~mongoengine.fields.PolygonField`:

* ``geo_within`` -- check if a geometry is within a polygon. For ease of use
  it accepts either a geojson geometry or just the polygon coordinates eg::

        loc.objects(point__geo_within=[[[40, 5], [40, 6], [41, 6], [40, 5]]])
        loc.objects(point__geo_within={"type": "Polygon",
                                 "coordinates": [[[40, 5], [40, 6], [41, 6], [40, 5]]]})

* ``geo_within_box`` -- simplified geo_within searching with a box eg::

        loc.objects(point__geo_within_box=[(-125.0, 35.0), (-100.0, 40.0)])
        loc.objects(point__geo_within_box=[<bottom left coordinates>, <upper right coordinates>])

* ``geo_within_polygon`` -- simplified geo_within searching within a simple polygon eg::

        loc.objects(point__geo_within_polygon=[[40, 5], [40, 6], [41, 6], [40, 5]])
        loc.objects(point__geo_within_polygon=[ [ <x1> , <y1> ] ,
                                                [ <x2> , <y2> ] ,
                                                [ <x3> , <y3> ] ])

* ``geo_within_center`` -- simplified geo_within the flat circle radius of a point eg::

        loc.objects(point__geo_within_center=[(-125.0, 35.0), 1])
        loc.objects(point__geo_within_center=[ [ <x>, <y> ] , <radius> ])

* ``geo_within_sphere`` -- simplified geo_within the spherical circle radius of a point eg::

        loc.objects(point__geo_within_sphere=[(-125.0, 35.0), 1])
        loc.objects(point__geo_within_sphere=[ [ <x>, <y> ] , <radius> ])

* ``geo_intersects`` -- selects all locations that intersect with a geometry eg::

        # Inferred from provided points lists:
        loc.objects(poly__geo_intersects=[40, 6])
        loc.objects(poly__geo_intersects=[[40, 5], [40, 6]])
        loc.objects(poly__geo_intersects=[[[40, 5], [40, 6], [41, 6], [41, 5], [40, 5]]])

        # With geoJson style objects
        loc.objects(poly__geo_intersects={"type": "Point", "coordinates": [40, 6]})
        loc.objects(poly__geo_intersects={"type": "LineString",
                                          "coordinates": [[40, 5], [40, 6]]})
        loc.objects(poly__geo_intersects={"type": "Polygon",
                                          "coordinates": [[[40, 5], [40, 6], [41, 6], [41, 5], [40, 5]]]})

* ``near`` -- find all the locations near a given point::

        loc.objects(point__near=[40, 5])
        loc.objects(point__near={"type": "Point", "coordinates": [40, 5]})

  You can also set the maximum and/or the minimum distance in meters as well::

        loc.objects(point__near=[40, 5], point__max_distance=1000)
        loc.objects(point__near=[40, 5], point__min_distance=100)

The older 2D indexes are still supported with the
:class:`~mongoengine.fields.GeoPointField`:

* ``within_distance`` -- provide a list containing a point and a maximum
  distance (e.g. [(41.342, -87.653), 5])
* ``within_spherical_distance`` -- same as above but using the spherical geo model
  (e.g. [(41.342, -87.653), 5/earth_radius])
* ``near`` -- order the documents by how close they are to a given point
* ``near_sphere`` -- Same as above but using the spherical geo model
* ``within_box`` -- filter documents to those within a given bounding box (e.g.
  [(35.0, -125.0), (40.0, -100.0)])
* ``within_polygon`` -- filter documents to those within a given polygon (e.g.
  [(41.91,-87.69), (41.92,-87.68), (41.91,-87.65), (41.89,-87.65)]).

  .. note:: Requires Mongo Server 2.0

* ``max_distance`` -- can be added to your location queries to set a maximum
  distance.
* ``min_distance`` -- can be added to your location queries to set a minimum
  distance.

Querying lists
--------------
On most fields, this syntax will look up documents where the field specified
matches the given value exactly, but when the field refers to a
:class:`~mongoengine.fields.ListField`, a single item may be provided, in which case
lists that contain that item will be matched::

    class Page(Document):
        tags = ListField(StringField())

    # This will match all pages that have the word 'coding' as an item in the
    # 'tags' list
    Page.objects(tags='coding')

It is possible to query by position in a list by using a numerical value as a
query operator. So if you wanted to find all pages whose first tag was ``db``,
you could use the following query::

    Page.objects(tags__0='db')

If you only want to fetch part of a list eg: you want to paginate a list, then
the `slice` operator is required::

    # comments - skip 5, limit 10
    Page.objects.fields(slice__comments=[5, 10])

For updating documents, if you don't know the position in a list, you can use
the $ positional operator ::

    Post.objects(comments__by="joe").update(**{'inc__comments__$__votes': 1})

However, this doesn't map well to the syntax so you can also use a capital S instead ::

    Post.objects(comments__by="joe").update(inc__comments__S__votes=1)

.. note::
    Due to :program:`Mongo`, currently the $ operator only applies to the
    first matched item in the query.


Raw queries
-----------
It is possible to provide a raw :mod:`PyMongo` query as a query parameter, which will
be integrated directly into the query. This is done using the ``__raw__``
keyword argument::

    Page.objects(__raw__={'tags': 'coding'})

.. versionadded:: 0.4

Limiting and skipping results
=============================
Just as with traditional ORMs, you may limit the number of results returned or
skip a number or results in you query.
:meth:`~mongoengine.queryset.QuerySet.limit` and
:meth:`~mongoengine.queryset.QuerySet.skip` and methods are available on
:class:`~mongoengine.queryset.QuerySet` objects, but the `array-slicing` syntax
is preferred for achieving this::

    # Only the first 5 people
    users = User.objects[:5]

    # All except for the first 5 people
    users = User.objects[5:]

    # 5 users, starting from the 11th user found
    users = User.objects[10:15]

You may also index the query to retrieve a single result. If an item at that
{'fraction_non_alphanumeric': 0.1319201995012469, 'fraction_numerical': 0.024638403990024937, 'mean_word_length': 3.479669347631814, 'pattern_counts': {'":': 20, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3589736', 'n_tokens_mistral': 6275, 'n_tokens_neox': 5728, 'n_words': 2357}