<jupyter_start><jupyter_text>Example of DOV search methods for lithologische beschrijvingen@pjhaestdate 20180908  Use cases:* Select records in a bbox* Select records in a bbox with selected properties* Select records in a municipality* Get records using info from wfs fields, not available in the standard output dataframe<jupyter_code>%matplotlib inline
import os, sys
import inspect
import warnings; warnings.simplefilter('ignore')
# check pydov path
import pydov
print(pydov.__path__)<jupyter_output>['c:\\projecten\\pydov\\pydov_git\\pydov']<jupyter_text>Get information about code base<jupyter_code>from pydov.search.interpretaties import LithologischeBeschrijvingenSearch
ip_litho = LithologischeBeschrijvingenSearch()

# information about the HydrogeologischeStratigrafie type (In Dutch):
print(ip_litho.get_description())

# information about the available fields for a HydrogeologischeStratigrafie object
fields = ip_litho.get_fields()
# print available fields
for f in fields.values():
    print(f['name'])
# print information for a certain field
fields['beschrijving']<jupyter_output><empty_output><jupyter_text>The cost is an arbitrary attribute to indicate if the information is retrieved from a wfs query (cost = 1), or from an xml (cost = 10)<jupyter_code># if an attribute can have several values, these are listed under 'values', e.g. for 'Type_proef':
fields['Type_proef']<jupyter_output><empty_output><jupyter_text>Try-out of use cases  Select interpretations in a bbox<jupyter_code>from pydov.util.location import Within, Box

# Get all lithological descriptions in a bounding box (llx, lly, ulx, uly)
# the pkey_boring link is not available below, but is in the df
df = ip_litho.search(location=Within(Box(152145, 204930, 153150, 206935)))
df = df[df.beschrijving.notnull()]
print(df.head())<jupyter_output>[000/017] .................
                                  pkey_interpretatie  \
0  https://www.dov.vlaanderen.be/data/interpretat...   
1  https://www.dov.vlaanderen.be/data/interpretat...   
2  https://www.dov.vlaanderen.be/data/interpretat...   
3  https://www.dov.vlaanderen.be/data/interpretat...   
4  https://www.dov.vlaanderen.be/data/interpretat...   

                                         pkey_boring  \
0  https://www.dov.vlaanderen.be/data/boring/1973...   
1  https://www.dov.vlaanderen.be/data/boring/1973...   
2  https://www.dov.vlaanderen.be/data/boring/1973...   
3  https://www.dov.vlaanderen.be/data/boring/1973...   
4  https://www.dov.vlaanderen.be/data/boring/1973...   

  betrouwbaarheid_interpretatie         x         y  diepte_laag_van  \
0                          goed  152634.0  205540.0              0.0   
1                          goed  152634.0  205540.0              1.0   
2                          goed  152634.0  205540.0              2.0   
3         [...]<jupyter_text>Select interpretations in a bbox with selected properties<jupyter_code># list available query methods
methods = [i for i,j in inspect.getmembers(sys.modules['owslib.fes'], 
                                           inspect.isclass) 
           if 'Property' in i]
print(methods)
from owslib.fes import PropertyIsGreaterThanOrEqualTo<jupyter_output>['PropertyIsBetween', 'PropertyIsEqualTo', 'PropertyIsGreaterThan', 'PropertyIsGreaterThanOrEqualTo', 'PropertyIsLessThan', 'PropertyIsLessThanOrEqualTo', 'PropertyIsLike', 'PropertyIsNotEqualTo', 'PropertyIsNull', 'SortProperty']<jupyter_text>The property feature methodes listed above are available from the owslib module. These were not adapted for use in pydov.<jupyter_code># Get deep boreholes in a bounding box 
from owslib.fes import PropertyIsEqualTo
# the propertyname can be any of the fields of the hydrogeological interpretations object that belong to the wfs source
# the literal is always a string, no matter what its definition is in the boring object (string, float...)
query = PropertyIsGreaterThanOrEqualTo(
        propertyname='betrouwbaarheid_interpretatie', literal='goed')
df = ip_litho.search(location=Within(Box(153145, 206930, 153150, 206935)),
                        query=query
                       )
print(df.head())<jupyter_output>[000/001] c
                                  pkey_interpretatie  \
0  https://www.dov.vlaanderen.be/data/interpretat...   
1  https://www.dov.vlaanderen.be/data/interpretat...   
2  https://www.dov.vlaanderen.be/data/interpretat...   
3  https://www.dov.vlaanderen.be/data/interpretat...   
4  https://www.dov.vlaanderen.be/data/interpretat...   

                                         pkey_boring  \
0  https://www.dov.vlaanderen.be/data/boring/1974...   
1  https://www.dov.vlaanderen.be/data/boring/1974...   
2  https://www.dov.vlaanderen.be/data/boring/1974...   
3  https://www.dov.vlaanderen.be/data/boring/1974...   
4  https://www.dov.vlaanderen.be/data/boring/1974...   

  betrouwbaarheid_interpretatie         x         y  diepte_laag_van  \
0                          goed  153147.0  206931.0              0.0   
1                          goed  153147.0  206931.0              0.5   
2                          goed  153147.0  206931.0              1.0   
3                         [...]<jupyter_text>Select interpretations in a municipality<jupyter_code>query = PropertyIsEqualTo(propertyname='gemeente',
                          literal='Aartselaar')
df = ip_litho.search(query=query)
print(df.head())<jupyter_output>[000/039] .......................................
                                  pkey_interpretatie  \
0  https://www.dov.vlaanderen.be/data/interpretat...   
1  https://www.dov.vlaanderen.be/data/interpretat...   
2  https://www.dov.vlaanderen.be/data/interpretat...   
3  https://www.dov.vlaanderen.be/data/interpretat...   
4  https://www.dov.vlaanderen.be/data/interpretat...   

                                         pkey_boring  \
0  https://www.dov.vlaanderen.be/data/boring/1879...   
1  https://www.dov.vlaanderen.be/data/boring/1879...   
2  https://www.dov.vlaanderen.be/data/boring/1879...   
3  https://www.dov.vlaanderen.be/data/boring/1879...   
4  https://www.dov.vlaanderen.be/data/boring/1879...   

  betrouwbaarheid_interpretatie         x         y  diepte_laag_van  \
0                          goed  151929.0  202628.0              0.0   
1                          goed  151929.0  202628.0              0.2   
2                          goed  151929.0  202628.0         [...]<jupyter_text>Visualize results Using Folium, we can display the results of our search on a map.<jupyter_code># import the necessary modules (not included in the requirements of pydov!)
import folium
from folium.plugins import MarkerCluster
from pyproj import Proj, transform
# convert the coordinates to lat/lon for folium
def convert_latlon(x1, y1):
    inProj = Proj(init='epsg:31370')
    outProj = Proj(init='epsg:4326')
    x2,y2 = transform(inProj, outProj, x1, y1)
    return x2, y2
df['lon'], df['lat'] = zip(*map(convert_latlon, df['x'], df['y'])) 
# convert to list
loclist = df[['lat', 'lon']].values.tolist()
# initialize the Folium map on the centre of the selected locations, play with the zoom until ok
fmap = folium.Map(location=[df['lat'].mean(), df['lon'].mean()], zoom_start=12)
marker_cluster = MarkerCluster().add_to(fmap)
for loc in range(0, len(loclist)):
    # limit marker size for folium (:10)
    folium.Marker(loclist[loc], popup=df['beschrijving'][loc][:10]).add_to(marker_cluster)
fmap<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1195812642598309, 'fraction_numerical': 0.046168299557106426, 'mean_word_length': 2.6298100340964443, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 30, 'lorem ipsum': 0, 'www.': 30, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '163295', 'license': 'MIT', 'n_tokens_mistral': 2643, 'n_tokens_neox': 2455, 'n_words': 542, 'path': 'docs/notebooks/search_lithologische_beschrijvingen.ipynb', 'repo_name': 'jorisvandenbossche/pydov', 'size': '7451'}