<jupyter_start><jupyter_text>Zhiqi: I renamed your file to zhiqi.tif:<jupyter_code>!ls imagery/zhiqi -l
!gdalinfo -stats imagery/zhiqi/zhiqi.tif<jupyter_output>Warning 1: TIFFReadDirectory:Sum of Photometric type-related color channels and ExtraSamples doesn't match SamplesPerPixel. Defining non-color channels as ExtraSamples.
Driver: GTiff/GeoTIFF
Files: imagery/zhiqi/zhiqi.tif
       imagery/zhiqi/zhiqi.tif.aux.xml
Size is 297, 1198
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 44N",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["UTM zone 44N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",81,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
    [...]<jupyter_text>Here is an RGB plot of bands 3,4,5:<jupyter_code>%run scripts/dispms -f imagery/zhiqi/zhiqi.tif -e 2 -p [3,4,5]<jupyter_output><empty_output><jupyter_text>The emnew.py code adds gaussian noise to the zero-valued edge pixels. They cause the algorithm interpret the edge pixels as a compact cluster:<jupyter_code>%run scripts/em  -K 4 -M 2 -n 0 imagery/zhiqi/5bands-fullarea.tif
%run scripts/dispms -f imagery/zhiqi/zhiqi_em.tif -c
%run scripts/em -h<jupyter_output>Usage: 
--------------------------------------

Perform Gaussian mixture clustering on multispectral imagery 

python scripts/em.py [OPTIONS] filename

Options:
  -h            this help
  -p  <list>    band positions e.g. -p [1,2,3,4,5,7]
  -d  <list>    spatial subset [x,y,width,height] 
                              e.g. -d [0,0,200,200]
  -K  <int>     number of clusters (default 6)
  -M  <int>     maximum scale (default 2)
  -m  <int>     minimum scale (default 0) 
  -n  <int>     add Gaussian noise to pixel value (default None)
  -t  <float>   initial annealing temperature (default 0.5)
  -s  <float>   spatial mixing factor (default 0.5)  
  -P            generate class probabilities image 
  
If the input file is named 

         path/filenbasename.ext then

The output classification file is named 

         path/filebasename_em.ext

and the class probabilities output file is named

         path/filebasename_emprobs.ext
  
  -------------------------------------
{'fraction_non_alphanumeric': 0.13208983631518842, 'fraction_numerical': 0.05519604111153407, 'mean_word_length': 2.5609756097560976, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '491212', 'license': 'MIT', 'n_tokens_mistral': 1023, 'n_tokens_neox': 877, 'n_words': 231, 'path': 'src/zhiqi.ipynb', 'repo_name': 'mortcanty/EESARDocker', 'size': '2627'}