<commit_before>h1. Kvadrere

Apache Pig UDF for dividing GeoJSON geometries into square polygons at varying resolutions. 

Quadkeys were used as the basis for determining tile resolution (how big the tiles are).  A thorough introduction to quadkeys can be found "here":http://msdn.microsoft.com/en-us/library/bb259689.aspx 

Zoom level for quadkeys ranges from 1 to 23, with 1 being approx. 78km per pixel to 23 which is a fraction of a meter per pixel.  ZL 11 is often a good middle ground.  The full breakdown can be found at the link above.

h3. Usage

1. Running @mvn package@ should build the UDF JAR into @target/@

2. @test/test_tile_geometry.sh@ runs the above command and combines the @part-*@ files into a single GeoJSON feature collection for display in a program like QGIS.

@test/data/admin0_MX.geojson@ contains a single GeoJSON Feature MultiPolygon for Mexico.

3. You can inspect the file in QGIS: @qgis test/data/admin0_MX.geojson@

!kvadrere/raw/master/test/admin0_MX.png!

4. Run @./test/test_tile_geometry.sh@ which will produce @mexico_tiles_Z11.geojson@, a FeatureCollection of quadkey tiles at ZL 11.

This script runs @test/tile_geometries.pig@, simple Pig runner script which calls out to the TileGeometry UDF:

5. Inspect the output in QGIS with @qgis test/mexico_tiles_Z11.geojson@

!kvadrere/raw/master/test/admin0_MX_tiledZ11.png!

<pre><code>wc -l admin0_MX_tiled_Z11/*
6735 admin0_MX_tiled_Z11/part-m-00000
</code></pre>

The routine has produced 6735 tiles at zoom level 11. 

h4. Notes 

Multiple GeoJSON shapes do NOT need to be in a feature collection, just one on each line of the input file. See @test/data@

For a GeoJSON point, the UDF will compute the quadkey and return tuple (quadkey,point) where point is the GeoJSON Feature with a new attached property 'quadkey'.
<commit_msg>tweaks to README<commit_after>h1. Kvadrere

Apache Pig UDF TileGeometry for dividing GeoJSON geometries into square polygons at varying resolutions. 

Quadkeys were used as the basis for determining tile resolution (how big the tiles are).  A thorough introduction to quadkeys can be found "here":http://msdn.microsoft.com/en-us/library/bb259689.aspx 

Zoom level for quadkeys ranges from 1 to 23, with 1 being approx. 78km per pixel to 23 which is a fraction of a meter per pixel.  ZL 11 is often a good middle ground.  The full breakdown can be found at the link above.

h3. Usage

1. Running @mvn package@ should build the UDF JAR into @target/@

2. @test/test_tile_geometry.sh@ runs the @test/tile_geometries.pig@ runner script and combines the output @part-*@ files into a single GeoJSON feature collection for display in a program like QGIS.

The input is @test/data/admin0_MX.geojson@, which contains a single GeoJSON Feature MultiPolygon for Mexico.

3. You can inspect the file in QGIS: @qgis test/data/admin0_MX.geojson@

!kvadrere/raw/master/test/admin0_MX.png!

4. Run @./test/test_tile_geometry.sh@ which will produce @mexico_tiles_Z11.geojson@, a FeatureCollection of quadkey tiles at ZL 11.

This script runs @test/tile_geometries.pig@, simple Pig runner script which calls out to the TileGeometry UDF:

5. Inspect the output in QGIS with @qgis test/mexico_tiles_Z11.geojson@

!kvadrere/raw/master/test/admin0_MX_tiledZ11.png!

<pre><code>wc -l admin0_MX_tiled_Z11/*
6735 admin0_MX_tiled_Z11/part-m-00000
</code></pre>

The routine has produced 6735 tiles at zoom level 11. 

h4. Notes 

Multiple GeoJSON shapes do NOT need to be in a feature collection, just one on each line of the input file. See @test/data@

For a GeoJSON point, the UDF will compute the quadkey and return tuple (quadkey,point) where point is the GeoJSON Feature with a new attached property 'quadkey'.

{'fraction_non_alphanumeric': 0.06765498652291105, 'fraction_numerical': 0.030727762803234502, 'mean_word_length': 4.378260869565217, 'pattern_counts': {'":': 2, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3570932', 'n_tokens_mistral': 1279, 'n_tokens_neox': 1170, 'n_words': 505}