<jupyter_start><jupyter_text>Filtered searchesThis tutorial walks through how to do filtered searches via the API. This is useful if you want to find specific parts of the data, especially if the dataset is large. Here, I will be testing on a small dataset for the purposes of this tutorial. This dataset is from [deleting entity sets tutorial](https://github.com/Lattice-Works/api-clients/blob/master/examples/pipelines/deleting_entity_sets.ipynb).To follow along, you will need to install [olpy](https://github.com/openlattice/olpy) and [openlattice](https://github.com/Lattice-Works/api-clients/tree/master/python). [This](https://help.openlattice.com/article/104-overview-and-installation-of-api-clients) is a detailed setup instructions for setting up the openlattice module. Configurations<jupyter_code>import openlattice
import olpy
baseurl = 'https://api.openlattice.com'

configuration = openlattice.Configuration()
configuration.host = baseurl
configuration.access_token = olpy.get_jwt(base_url = configuration.host)

edmAPI = openlattice.EdmApi(openlattice.ApiClient(configuration))
entitySetsAPI = openlattice.EntitySetsApi(openlattice.ApiClient(configuration))
permissionsAPI = openlattice.PermissionsApi(openlattice.ApiClient(configuration))
dataAPI = openlattice.DataApi(openlattice.ApiClient(configuration))
integrationAPI = openlattice.DataIntegrationsApi(openlattice.ApiClient(configuration))
orgAPI = openlattice.OrganizationsApi(openlattice.ApiClient(configuration))
searchAPI = openlattice.SearchApi(openlattice.ApiClient(configuration))
entitysetid = '0f898abd-edfe-4212-8642-74db49efa474'
data = dataAPI.load_entity_set_data(entitysetid)
data<jupyter_output><empty_output><jupyter_text>This loads a dataset that's a list of dictionaries. The field `openlattice.@id` is a unique entity key for each entity and is generated uniquely by OpenLattice during integration. The field `nc.SubjectIdentification` is a standardized field for the entity type `general.person`. To see what property types exist in an entity type, check out our [EDM page](https://openlattice.com/edm//propertyTypes). For a more thorough explanation of entity models, check out [this help page](https://help.openlattice.com/article/20-the-openlattice-standardized-entity-data-model). This dataset is small enough to pull everything all at once, but for the purposes of this exercise, let's filter the dataset on `Woman1`  _without_ first loading everything. To do so, you will use search constraints, as shown in the code below.Note: There is a way to do fuzzy searches, but unfortunately I haven't been able to figure it out yet. This search is mostly useful for searching a specific category of a property.<jupyter_code>propertytypeid = edmAPI.get_property_type_id(namespace = "nc", name = "SubjectIdentification")
condition = "Woman1"

search_str = f"entity.{propertytypeid}: '{condition}'"

constraint1 = openlattice.Constraint(
    type="simple",
    search_term=search_str,
    fuzzy=False
)

constraints = [constraint1]

constraintgroup = openlattice.ConstraintGroup(
    min = len(constraints),
    constraints = constraints
)


constraints = openlattice.SearchConstraints(
    entity_set_ids = [entitysetid],
    start = 0,
    max_hits = 10000,
    constraints = [constraintgroup]
)

results = searchAPI.search_entity_set_data(constraints)

results.hits<jupyter_output><empty_output><jupyter_text>You can also search between certain dates. For example, one useful metadata is to look for the last write time. (Note, the following code will return the whole dataset, since I wrote them all at once).<jupyter_code>import dateutil

starttime = dateutil.parser.parse("2021-01-15T16:00:00-08:00")
endtime = dateutil.parser.parse("2021-01-19T16:00:00-08:00")

constraint2 = openlattice.Constraint(
    type = 'writeDateTimeFilter',
    start = starttime.isoformat(),
    end = endtime.isoformat())

constraints = [constraint2]

constraintgroup = openlattice.ConstraintGroup(
    min = len(constraints),
    constraints = constraints
)


constraints = openlattice.SearchConstraints(
    entity_set_ids = [entitysetid],
    start = 0,
    max_hits = 10000,
    constraints = [constraintgroup]
)

results = searchAPI.search_entity_set_data(constraints)

results.hits<jupyter_output><empty_output><jupyter_text>You can search both at once like this:<jupyter_code>constraints = [constraint1, constraint2]

constraintgroup = openlattice.ConstraintGroup(
    min = len(constraints),
    constraints = constraints
)


constraints = openlattice.SearchConstraints(
    entity_set_ids = [entitysetid],
    start = 0,
    max_hits = 10000,
    constraints = [constraintgroup]
)

results = searchAPI.search_entity_set_data(constraints)

results.hits<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09008819823603528, 'fraction_numerical': 0.018059638807223857, 'mean_word_length': 5.804285714285714, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '449969', 'license': 'Apache-2.0', 'n_tokens_mistral': 1461, 'n_tokens_neox': 1397, 'n_words': 382, 'path': 'examples/pipelines/Python/filtered_searches.ipynb', 'repo_name': 'openlattice/api-clients', 'size': '4762'}