<commit_before>-- Feature matching still needs improvements:
   the creation of corner regions to match is now very good,
   but the analysis of the comparisons needs to be improved.

   -- all but books have better matching now
      -- might need to consider keeping top 10 or so for each index1
         best in BlobCornersScaleFinder, then looking at most frequent
         parameters solution before evaluation.
      -- the books test images should be better matched by the next
         class for a matcher which will not use euclidean transformations
         for evaluation

start of rough outline of requirements of a non-euclidean matcher:

CornerMatcher
    -- given dither, and two lists of corner regions,
       finds the best and second best match for each point
       and discards the matches that have SSD_best/SSD_second_best < 0.8-ish.
       (Mikolajczyk and Schmid 2005))
       -- currently, the algorithm compares each corner in list 2 to 
          each corner in list 2, and performs 
          a set of alternate tries around the point in list 1,
          so the matching is ~O(N^2) ignoring the dithering pattern.
    -- follows that with greedy or optimal bipartite matching
       to ensure unique matches.
    methods:
      + CornerMatcher(int dither)
      + matchCorners(
          final IntensityFeatures features1, final IntensityFeatures features2,
          final List<T> corners1,final List<T> corners2,
          GreyscaleImage img1, GreyscaleImage img2, int binFactor1, int binFactor2) : boolean
      + getSolutionStats() : List<FeatureComparisonStat>
      + getSolutionCost() : double
      - bipartiteFilter(List<FeatureComparisonStat>)

NonEuclideanSegmentFeatureMatcher:
    -- uses atrous based segmentation
    -- extract blobs and blob corners
    -- the matching of blob to blob corners or all corners to all corners
       will be performed using the checkerboard test data to derive
       best practices. 
    -- regardless of curve membership uses CornerMatcher w/ a 
       dither of 2 or 3.
    -- two different kinds of analysis of the matched and remaining
       unmatched points.
         -- if can assume that the geometry is such that objects have
            not moved and the perspective has not changed much,
            can use an adaptation of the method suggested by Geiger et al. 2010.
            the well determined points just found become the control points.
            then delanauy triangulation for all points, then find unmatched by
            interpolating between control points, that is searching within
            the delanauy space in other image that is bounded by the cells
            of the control points.
    -- are the results fewer false positives than NonEuclideanInvHessianFeatureMatcher?
    methods:
      + NonEuclideanSegmentFeatureMatcher()
      - initialize()
      + match();
      + List<PairInt> getMatched1();
      + List<PairInt> getMatched2();
      + getSolutionStats() : List<FeatureComparisonStat>

NonEuclideanInvHessianFeatureMatcher:
    -- begins with finding the inverse hessian points
    -- uses CornerMatcher for all corners and a dither of 0 or 1
    -- are these results enough to provide good correspondence
       for the epipolar solver?  
       No need to bootstrap more points from these?
       -- are there too many false positives?
    methods:
      + NonEuclideanInvHessianFeatureMatcher()
      - initialize()
      + match();
      + List<PairInt> getMatched1();
      + List<PairInt> getMatched2();
      + getSolutionStats() : List<FeatureComparisonStat>

-- for calculation of orientation angle in IntensityFeatures,
   for the method that uses first derivative, might need to alter the
   method to find the minimum of the absolute values of thte differences
   (an approx of zero crossings).
   the current method is consistently determined, but might be improvable.
  
-- for the checkerboard test images:
   if not well solved by the non-euclidean trans solver,
   consider adding a flag at a higher setting in the finder settings
   to allow immediate change of segmentation to canny segmentation
   (see the middle of the blob corners scale finder for data w/ grids).

-- more ransac and epipolar solver tests.
   -- improving the ransac sampling method (and making it useful for an
      LMedS implementation too)
      -- instead of choosing randomly uniformly from all pairs of points,
         would like to choose points likely to be spatially distributed
         across the intersection of points to achieve higher spatial coverage.
         A form of stratified sampling with a probability function based upon 
         distance from spatially distributed centers of cells, a.k.a. seeds, 
         that span the intersection of points.
         A probability function based upon the value would for example
         choose uniformly from [2, 2, 3, 3, 3, 4, 4, 4, 4] when the
         original values to choose from were [2, 3, 4].
         That probability function pattern is the inverse of what is
         needed in order to preferentially sample points close to the
         seed centers, that is distances from seed0 would
         need to use a max distance and subtract from it before 
         creating the distribution by value (or could use a normalized inverse).
         -- need to consider how fine the grid of seeds should be in order
            to sample well (that must be dependent upon the fact that the
            sample size is only 7).
         -- distance transform or watershed labelling could be used to
            determine the distances from the seeds quickly.
         -- the statistical determination of the number of iterations
            needed possibly needs to change.

   X (1) given a set of points,
         determine the intersection and k seeds
         -- find min and max x's and y's
            -- bisect the data by k cells in y
               -- determine the min and max for each stripe
            -- bisect the data by k cells in x
               -- determine the min and max for each cell
            -- adjust cell sizes that are much smaller than the first 
               determined (if cell is < 3*first cell cize, make it same as 
               and remove the other cells)
         -- test with concave datasets such as the dbscan
design class to hold sampling

   X (2) calculate the distance transform in the points
         -- creating a point set of the seeds  
         -- edit the distance transform to use a perimeter of the points for each
            row in x and column in y instead of 0 and width or height as bounds.
         -- let the algorithm finish, but extract only the
            original points from the point set.
         the runtime complexity is O(n_pixels = width*height).
    
         alternatives, knowing the seeds already would be to use Vornoi Fortunes.
         That is O(n_points * lg2(n_points)).  (I currently only have voronoi half planes)

         The merton college dataset for example has n_points = 100, but
         n_pixels = 1024*768 so 
            dt has rt 768,000 and voronoi has rt 100*lg2(100)=664
         so for these datasets which are more sparse than dense,
         voronoi is a better way to find cells then distances are O(n_points).

    (2.5) consider implementing the bit vector random selector
          compare to max and min runtime complexities for new method choose7RandomPoints
             in PointSampling and choosing 7 points one at a time, both using
             value based probability distributions.

          create a PointValueDistr with no value dependence and give it to
              randomlyChooseKBitNumber for avalanche tests


       on subject of forward indexed gosper's hack:
          write out first few iterations of this to see if a formula is possible that
          just takes index, nValues, k subset size and can return the index-th output of this
          without using feedback:
              x0 = x0 + (x0 & -x0) + ((((x0 + (x0 & -x0)) ^ x0) / (x0 & -x0)) >> 2);

   X (3) try 2 ways, one with a value based on the distance and the other being
         the reverse of the distance (where the max distance is determined by
         the distance between seeds).
         (a) make the value and index distance arrays to chose randomly from.
         (b) make the reverse value and index distance arrays to choose randomly from.
     (4) calculate nIterations as before without modifications yet.
     (5) plot the points chosen using (3a).  plot the same using 3(b).
         do this many times to see trends, but for each sample
         (red points for chosen, and other color for all points).
         make a combined plot too.
     (6) compare plots of random sampling (standard manner) with results from (5)
     (7) if one of the proposed sampling methods works well,
         use that as the default in RANSAC.
     (8) complete the epipolar tests
         -- consider evaluation based upon not only sum of dist from epipolar lines
            but also on the number of points (a combined calculation).
         -- having chosen the preferred sampling, look in detail at the individual
            solutions to make sure that the evaluation and final solution are the
            best from among the solutions.
     (9) implement or consider implementing the LMedS

   -- what is gradient weighted epipolar errors?

   -- look again at my estimate for number of iterations.
      a sony site uses for ransac a k iterations =
          log(f)/( log(1 - a^4) ) where f = probability of failure, a = apriori inlier ratio
      after each iteration, can recompute the number of inliers.
              maxC = count of maximum consensus so far.
              c = count of current consensus
              a = c/(data set size)
              then the recomputed k is a smaller number

    -- instead of a grid of 9 cells to draw from as considered below, 
       consider a monte carlo draw from uniformly random points of square of sides of length 2*r.
       two 1D probability density functions for x and y:
           probability of drawing a point inside unit dist is
                  area_disk/area_square = (pi*r^2)/(2*r)^2 = pi/4 for r > 0
           can see probability is indep of radius.
           so then can assume r=1
       Buffon's needle is a faster convergence of the monte carlo model.
          -- http:// numbers.computation.free.fr/ Constants/ PiProgram/ pifast.html
          
   ** change the plotter of epipolar results to plot the
      points used for the solution as one color (green with white block behind)
         and the points not used as red
   -- consider for LMedS the first solution from feature matcher could be used
      as a starter set to assess errors for guided sampling.
         the merton college starter points are nearly 50 and about 7 errors
         so eps of 20%, for example.

   -- when revisiting the books epipolar test, see if evaluation needs to focus
      on dist from epipolar line as much as the number of points within tolerance.
      --> make the combined score

   -- http://coewww.rutgers.edu/riul/research/papers/pdf/pbm.pdf
      possibly can use the first solution from feature matcher before the 
         entire image corners are added as the starter needed.

      http://coewww.rutgers.edu/riul/research/code/GPBM/index.html

   update the open agile story and close it

-- for line drawings, should use an edge detector that is a a second
   derivative operator instead of the canny first deriv operator.
   see the canny edge detector for lines...

-- median absolute deviation
<NAME>, <NAME>. Alternatives to the Median Absolute Deviation. J. Am. Statistical Assoc. 1993;vol. 88(no. 424):1273–1283.

-- update the command line runners.

-- Parzen window method in pattern recognition

** -- consider a detailed look at the books test matching.
      the largest group of matches alone determines the solution, but
      it looks like other group solutions should contribute, but might need larger tolerance

** -- consider the extreme unlikely case of only circles in an image.  there wouldn't be stable
      corners nor stable inflection points (except in the backgrounds possibly).
      the sillhouette solution combined with object identification by shapes 
          (such as using hough transform for circles)?
** -- impl a circle hough transform
   -- and explore elliptical transforms for the rainbow fitting which
      currently has a long runtime.
      x2+Bxy+Cy2+Dx+Ey+F=0  and (B^2-4C)< 0
         consider replacing the ellipse w/ a parabola (review choice of ellipse for easier fit to projected parabolas)
      -- consider impl a skip list and fitting fewer points
         https://www.sonycsl.co.jp/person/nielsen/visualcomputing/visualcomputing-chapter7-colorlowres.pdf

** -- use the added half dome test images to assert that matching same
      from other cameras is working

-- change any debugging code to not run by default and change log levels where needed
-- finish implementing recursive gaussian and gaussian derivative filters

-- for cases in which there are no discernible blobs or the ones that are present
   might be a changing component (for example, see the test image of stonehenge in sillhouette)
   may need to use a corners only feature matching.
   (can adapt the code to perform same steps, but for a all corners being part of a fake repr blob).
   might need a way to limit the open curves being matched such as a look at convex and concave
   for any 3 or more points?  I thnk BL2003 chose large blocks of cells of pixels to compare between
   images.  edges and groups of corners should be faster...
   ... contours


-- update aspects if needed and considering replacing them
    -- consider replacing aspectj with something like nanning aop

-- improve skyline extraction code

-- similarity that uses an exponential base?  can transform the numbers and use a distance
   transform to make the calculations faster

-- low priority, changes for KMPP...
   -- use a watershed at line 133
      and use the labeled components throughout code

-- explore a little the biorthogonal wavelet transform 
    in the JPEG-2000 image storage and compression standard

-- might need to make a method for the latest segmentation that can
   correct for equal values of green and red and blue by making
   3 separate wavelet segmentations and combining them in a way
   to preserve differences, but only if flag is set.  this kind
   of correction could possibly make matching of different images with
   different illumination more difficult (sky conditions, sun altitude, room lighting etc).

-- text recognition
   -- would like a class to extract and match text
      -- can see from the books test image that a binary segmentation would
         work best.
         -- mean adaptive thresholding might be a good follow up.
         -- blobs can be found as contiguous pixels of same value
         -- for the separated letters, should apply skeletonization possibly
         -- match against fonts.
            -- note font databases such as
               OCR-A, OCR-B, or MICR fonts
    some papers:
      http://www.rfai.li.univ-tours.fr/fr/ressources/_dh/DOC/DocOCR/OCRbinarisation.pdf

-- control flow graph for method called resolve() in DistanceTransform
   and same for resolveIterative to simplify the later

-- considering implementing a max-tree (Meister and Wilkinson)
   -- for a shape filter for blobs for example
       http://www.cs.rug.nl/michael/mt-pres.pdf

-- impl wavelets using "lifted wavelets".  see Sweldens 1996 and 1997

-- begin outline for rectification code

-- make the method to combine 2 skyline masks to decide what is sky and what
   is land when there are differences and there are more than 2 images to decide.  
   This can happen when there are clouds or snow or varying occlusion, etc

-- move all of debugging code to aspects or test.  jfreegraph should not need to be included
   in the command line jar unless add an output that uses it.

-- improve my canny edge detector w/ some new entries here for computing
    gradient using 6 neighbors
    and the suggested derivations of the 2-layer thresh
    and the line thinner
    https://en.wikipedia.org/wiki/Canny_edge_detector#Finding_the_Intensity_Gradient_of_the_Image

    note that the gradient changes might help with the normalized gradient right away

-- for more feature matching tests,
    see
    http://web.stanford.edu/~bgirod/pdfs/Chandrasekhar_CVPR2009.pdf
    "For evaluating the performance of low-bit-rate descrip- tors, we use two data sets provided by Winder and Brown [10], Trevi Fountain and Notre Dame. Each data set has matching pairs of 64×64 pixel patches. For algorithms that require training, we use matching pairs from the Trevi Fountain. For testing, we randomly select 10,000 matching"
    <NAME> and <NAME>, “Learning Local Image Descriptors,”
in Computer Vision and Pattern Recognition, 2007. CVPR ’07. IEEE
Conference on, 2007, pp. 1–8. 1, 2

-- revisit areas where using CountingSort and compare 
     O(N_max_value) to O(N*lg2N) where the later is the number of points

-- make plans for a stereoscopic matcher, image rectification, and disparity map maker

-- http://proquest.safaribooksonline.com.ezproxy.spl.org:2048/book/electrical-engineering/communications-engineering/9780123965028/chapter-20dot-clustering/st010_html_17?query=((ransac))#snippet

-- for a random subset chooser, since
     the x only depends upon the subset size, not on n, the number to choose from,
     could think about a way to calculate the bitstring randomly.
     for k = 7, the lowest bitstring is 0111 1111 which is a decimal value of 127.
     n gives the largest number of bitstring positions with k at the highest value positions.
     To calculate it in this way probably needs to use BigInteger.
     It has some random methods in it, so one could probably work out how to draw
     a number randomly between those values.
     Then might find the next lowest number from that, that has k=7 '1' digits.
     For example, if the random number were 252, that is 1111 1100, the next lowest 
     bitstring with k=7 1's is 1111 1011.      (find lowest set bit, change it to 0 and set the 2 below it to 1)
     If that bitstring has already been selected randomly, then the SubsetChooser's
     Gosper's hack quickly returns the next larger k=7 bitstring.

     This should be compared to the way I'm currently selecting k=7 indexes randomly.
     currently:  not sure actually... I think I select 7 randomly with no repeating digits
     and then sort them and see if the combination has been selected before, and 
     repeat if they have.
        that mean 7 random operations * 7 * lg2(7) for sort operations = 7 random * 20 steps for single iteration.
     
        the random w/ biginteger is 1 or more random operations (see internal implementation)
        plus a find lowest bit operation, plus 3 set bits, plus gospers hack (a half dozen bit operations)
        so the result =  1 (or more internal to BigInteger) rand operation + a dozen or so bit operations.

        The advantage to the BigInteger method is that it is possibly faster, but the draw would be
        within highly uneven space (the universe drawn from is far larger than the number of possible
        permutations, and then one adjusts to find the next lowest matching number).
        The probability that a single number had the k=7 digits set can be estimated using
        the probability of drawing one unique set of all the possible subsets for k=7 and n (with one draw)
        so that's 1./((n!/(k!*(n-k)!)) and then would need to account for the dilution of
        that within a larger set of numbers (nbits with highest k bits set to '1' - kbits all set to '1').
        The adjustment to select the next lowest number with k bits set leads to an uneven
        distribution (not uniform), but that would not necessarily be a bad thing here.

    Following the line of reasoning I must have started with:
       Ideally, one would be randomly choosing between k bit positions whose maximum is n.
       That requires enumerating all combinations and choosing randomly from them.
       Can see a pattern easily within Gosper's hack already, so might be a clever way
       to store them or to recalculate with offsets or derive a new ordering
       from Gosper's Hack that would allow an O(1) access of a bitstring from a random number 
       chosen within the range of the number of possible permutations.
       This ideal implementation would be uniform random draws.

    For either, should be storing "selected" as a bitstring.
    
    For the current implementation, I am not checking for having selected the previous combination
    before because the chance of drawing the same combination should be pretty small.
       The probability of selecting the same 7 objects out of n objects is roughly calculated with
            the presence of unique set of k within n objects is 1./(n!/(k!*(n-k)!)) 
            but that is not considering 'k draws'.   Can't use Bernoulli principle because that
            gives an answer that is a number having any k digits rather than a uniquely ordered
            set of k digits.
            
    ** do a quick plot of the difference of k=7 among n=(some number such that nPermutations < 1<<63)
       drawn by 7 random draws from a max of n numbers
       versus the BigInteger random draw 

-- fix the contour matcher search to use a compare and binary search

-- when sun is found, such as in the NM image, might try to fit a
   radial profile (1/r^2) from it over the image.
   can wait until sky pixels are mostly found and then fit over
   that and extend that pattern to the rest of the image?
   that is, in the NM image, can see that the sky varies smoothly
   but there are clouds that are in the sky too. 
   ** what happens when one knows the sun is in the image,
   one fits a gradient to the least bright pixels and then divides
   the image by that gradient?  presumably, the skyline stands out
   even more.   cie XY are independent of illumination in any case,
   so this step probably isn't useful and not always possible...

-- include other atmospheric optically visible features:  sun dogs and moon dogs.
-- consider more images which are near evening or evening, but there is enough illumination
   to see a skyline.

-- consider simplest ways to find solar reflected light in the images:
     -- already have a method which finds visible sun's photosphere
     -- create a separate method for reflected from water (see earlier code... depending upon sun location,
        the light might be orange to white...)... can use a source function of the sun
        and mie and rayliegh scattering if more formal methods are needed and reflection off of optically
        thick H_2O.
     -- refracted and reflected from clouds that are thick enough to provide a surface of reflected light
        that has a pattern of brightness due to illumination by sun...
     -- consider whether the location of the sun when photosphere is not in image is learnable
        by gradient of brightness.

-- add more tests when have a working solution for skyline extraction
   -- test by resolution too.  the seed sky points probably have some resolution dependence at this point.
   -- find test images for:
         -- structured cloudy sky and smooth white foreground
         -- structured cloudy sky and smooth dark foreground
         Neither of those will be solved as well as the NM test image which had non-sky foreground colors.
         Need additional information to understand what is sky without external sensors
         or assumption of horizontal.  In ambiguous cases, might need to make the assumption of
         skyline being horizontal in the image and sky near the top of the image...would like to avoid
         that assumption if at all possible.

-- one day, put in my algorithms toolbox, deconvolution methods:
   weiner filter deconvolution
   Tikhonov regularization
   Richardson-Lucy

-- find image with naturally occuring skew for tests

-- for ways to reduce the corner list, consider reading:
   Shi, J., and <NAME>. "Good Features to Track." 
   Conference on Computer Vision and Pattern Recognition, 1994.

   journals to browse:

   CVGIP  Graphical Models /graphical Models and Image Processing /computer Vision, Graphics, and Image Processing
   CVIU  Computer Vision and Image Understanding
   IJCV  International Journal of Computer Vision
   IVC  Image and Vision Computing
   JMIV  Journal of Mathematical Imaging and Vision
   MVA  Machine Vision and Applications
   TMI - IEEE  Transactions on Medical Imaging

   http://pointclouds.org/documentation/

-- read:
    http://proquest.safaribooksonline.com.ezproxy.spl.org:2048/book/illustration-and-graphics/9780133373721

-- revisit the matrix math, especially where there's multiplication.
   did I replace dot operation with multiplication anywhere?
   needs tests...

-- reconsider later some properties.
   hue for blue skies.
   eucl dist of color = sqrt((r-r)^2 + (g-g)^2 + (b-b)^2)
   
another test image?
   http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/BSDS300/html/dataset/images/color/260058.html

-- read more on segmentation... lots of comparisons on the benchmark site.
   note the skyline extractor here is only partial segmentation and specific to a task.

   http://www.cs.berkeley.edu/~arbelaez/publications/amfm_pami2011.pdf

-- test datasets
http://sipi.usc.edu/database/database.php?volume=misc
http://homepages.inf.ed.ac.uk/rbf/CVonline/Imagedbase.htm#segmentation

-- finish the method converting from fundamental matrix to essential matrix.

 /*
    https://github.com/jesolem/PCV
    adapted from code licensed under  BSD license (2-clause "Simplified BSD License").
    private SimpleMatrix calculateCameraMatrixFromFundamentalMatrix(SimpleMatrix fm) {
        SimpleMatrix u = fm.svd().getU();
        SimpleMatrix leftE = u.extractVector(true, 2);
        leftE = leftE.divide(u.get(2, 2));

        double[][] skewSymmetric = new double[3][];
        skewSymmetric[0] = new double[]{0, -leftE.get(0, 2), leftE.get(0, 1)};
        skewSymmetric[1] = new double[]{leftE.get(0, 2), 0, -leftE.get(0, 0)};
        skewSymmetric[2] = new double[]{-leftE.get(0, 1), leftE.get(0, 0), 0};

        SimpleMatrix sMatrix = new SimpleMatrix(skewSymmetric);
        double v = sMatrix.dot( fm.transpose() );


        //essential matrix from F: E = transpose(K2)*F*K1
        SimpleMatrix k = DataForTests.getVenturiCameraIntrinsics()
            .mult(DataForTests.getVenturiRotationMatrixForImage001());
        SimpleMatrix k2 = DataForTests.getVenturiCameraIntrinsics()
            .mult(DataForTests.getVenturiRotationMatrixForImage010());

        SimpleMatrix essentialMatrix = k2.transpose().mult(fm).mult(k);

        int z = 1;
    }*/


-- consider following the implementation of disparity maps for stereo images
   and 3d modelling.  see notes in the docs directory.
   -- see http://vision.middlebury.edu/stereo/code/

-- consider implementing the LM-ICP from fitz-gibbons?
   did i look at this already and decide otherwise?
   does is handle projective transformations?

-- ** read: https://www.graphics.rwth-aachen.de/person/21/

-- for tests, make sure the projective point matcher can handle images like the
    ones in this tutorial:
    http://www.robots.ox.ac.uk/~az/tutorials/tutorialb.pdf

-- to the corner list makin',
   -- add a removal of redundant points

-- test for degenerate camera conditions:
   -- parallel camera motion w/o rotation 
-- test for degenerate scene structure configurations
   -- all points lying on a plane or nearly lying on a plane (?)
-- test for point sets containing noise
-- error estimate in fundamental matrix:
   6.5.3
   -- Gold Standard algorithm?  (need camera details...)
      summation over i( d(x_i, x_hat_i)^2 + d(x_prime_i, x_prime_hat_i)^2 ) 
   -- Sampson distance?

http://dev.ipol.im/~morel/Dossier_MVA_2011_Cours_Transparents_Documents/2011_Cours7_Document3_hartley.pdf
-- a hartley paper suggests rectifying images to evaluate point transformations.

<commit_msg>adding to todo notes methods for new classes.<commit_after>-- Feature matching still needs improvements:
   the creation of corner regions to match is now very good,
   but the analysis of the comparisons needs to be improved.

   -- all but books have better matching now
      -- might need to consider keeping top 10 or so for each index1
         best in BlobCornersScaleFinder, then looking at most frequent
         parameters solution before evaluation.
      -- the books test images should be better matched by the next
         class for a matcher which will not use euclidean transformations
         for evaluation

start of rough outline of requirements of a non-euclidean matcher:

CornerMatcher
    -- given dither, and two lists of corner regions,
       finds the best and second best match for each point
       and discards the matches that have SSD_best/SSD_second_best < 0.8-ish.
       (Mikolajczyk and Schmid 2005))
       -- currently, the algorithm compares each corner in list 2 to 
          each corner in list 2, and performs 
          a set of alternate tries around the point in list 1,
          so the matching is ~O(N^2) ignoring the dithering pattern.
    -- follows that with greedy or optimal bipartite matching
       to ensure unique matches.
    methods:
      + CornerMatcher(int dither)
      + matchCorners(
          final IntensityFeatures features1, final IntensityFeatures features2,
          final List<T> corners1,final List<T> corners2,
          GreyscaleImage img1, GreyscaleImage img2, int binFactor1, int binFactor2) : boolean
      + getSolutionStats() : List<FeatureComparisonStat>
      + getSolutionCost() : double
      - bipartiteFilter(List<FeatureComparisonStat>)

NonEuclideanSegmentFeatureMatcher:
    -- uses atrous or canny based segmentation
    -- extract blobs and blob corners
    -- the matching of blob to blob corners or all corners to all corners
       will be performed using the checkerboard test data to derive
       best practices. 
    -- regardless of curve membership uses CornerMatcher w/ a 
       dither of 2 or 3.
    -- two different kinds of analysis of the matched and remaining
       unmatched points.
         -- if can assume that the geometry is such that objects have
            not moved and the perspective has not changed much,
            can use an adaptation of the method suggested by Geiger et al. 2010.
            the well determined points just found become the control points.
            then delanauy triangulation for all points, then find unmatched by
            interpolating between control points, that is searching within
            the delanauy space in other image that is bounded by the cells
            of the control points.
    -- are the results fewer false positives than NonEuclideanInvHessianFeatureMatcher?
    methods:
      + NonEuclideanSegmentFeatureMatcher()
      + NonEuclideanSegmentFeatureMatcher(boolean useCannySegmentation)
      - initialize()
      + match();
      + List<PairInt> getMatched1();
      + List<PairInt> getMatched2();
      + getSolutionStats() : List<FeatureComparisonStat>

NonEuclideanInvHessianFeatureMatcher:
    -- begins with finding the inverse hessian points
    -- uses CornerMatcher for all corners and a dither of 0 or 1
    -- are these results enough to provide good correspondence
       for the epipolar solver?  
       No need to bootstrap more points from these?
       -- are there too many false positives?
    methods:
      + NonEuclideanInvHessianFeatureMatcher()
      - initialize()
      + match();
      + List<PairInt> getMatched1();
      + List<PairInt> getMatched2();
      + getSolutionStats() : List<FeatureComparisonStat>

-- for calculation of orientation angle in IntensityFeatures,
   for the method that uses first derivative, might need to alter the
   method to find the minimum of the absolute values of thte differences
   (an approx of zero crossings).
   the current method is consistently determined, but might be improvable.
  
-- for the checkerboard test images:
   if not well solved by the non-euclidean trans solver,
   consider adding a flag at a higher setting in the finder settings
   to allow immediate change of segmentation to canny segmentation
   (see the middle of the blob corners scale finder for data w/ grids).

-- more ransac and epipolar solver tests.
   -- improving the ransac sampling method (and making it useful for an
      LMedS implementation too)
      -- instead of choosing randomly uniformly from all pairs of points,
         would like to choose points likely to be spatially distributed
         across the intersection of points to achieve higher spatial coverage.
         A form of stratified sampling with a probability function based upon 
         distance from spatially distributed centers of cells, a.k.a. seeds, 
         that span the intersection of points.
         A probability function based upon the value would for example
         choose uniformly from [2, 2, 3, 3, 3, 4, 4, 4, 4] when the
         original values to choose from were [2, 3, 4].
         That probability function pattern is the inverse of what is
         needed in order to preferentially sample points close to the
         seed centers, that is distances from seed0 would
         need to use a max distance and subtract from it before 
         creating the distribution by value (or could use a normalized inverse).
         -- need to consider how fine the grid of seeds should be in order
            to sample well (that must be dependent upon the fact that the
            sample size is only 7).
         -- distance transform or watershed labelling could be used to
            determine the distances from the seeds quickly.
         -- the statistical determination of the number of iterations
            needed possibly needs to change.

   X (1) given a set of points,
         determine the intersection and k seeds
         -- find min and max x's and y's
            -- bisect the data by k cells in y
               -- determine the min and max for each stripe
            -- bisect the data by k cells in x
               -- determine the min and max for each cell
            -- adjust cell sizes that are much smaller than the first 
               determined (if cell is < 3*first cell cize, make it same as 
               and remove the other cells)
         -- test with concave datasets such as the dbscan
design class to hold sampling

   X (2) calculate the distance transform in the points
         -- creating a point set of the seeds  
         -- edit the distance transform to use a perimeter of the points for each
            row in x and column in y instead of 0 and width or height as bounds.
         -- let the algorithm finish, but extract only the
            original points from the point set.
         the runtime complexity is O(n_pixels = width*height).
    
         alternatives, knowing the seeds already would be to use Vornoi Fortunes.
         That is O(n_points * lg2(n_points)).  (I currently only have voronoi half planes)

         The merton college dataset for example has n_points = 100, but
         n_pixels = 1024*768 so 
            dt has rt 768,000 and voronoi has rt 100*lg2(100)=664
         so for these datasets which are more sparse than dense,
         voronoi is a better way to find cells then distances are O(n_points).

    (2.5) consider implementing the bit vector random selector
          compare to max and min runtime complexities for new method choose7RandomPoints
             in PointSampling and choosing 7 points one at a time, both using
             value based probability distributions.

          create a PointValueDistr with no value dependence and give it to
              randomlyChooseKBitNumber for avalanche tests


       on subject of forward indexed gosper's hack:
          write out first few iterations of this to see if a formula is possible that
          just takes index, nValues, k subset size and can return the index-th output of this
          without using feedback:
              x0 = x0 + (x0 & -x0) + ((((x0 + (x0 & -x0)) ^ x0) / (x0 & -x0)) >> 2);

   X (3) try 2 ways, one with a value based on the distance and the other being
         the reverse of the distance (where the max distance is determined by
         the distance between seeds).
         (a) make the value and index distance arrays to chose randomly from.
         (b) make the reverse value and index distance arrays to choose randomly from.
     (4) calculate nIterations as before without modifications yet.
     (5) plot the points chosen using (3a).  plot the same using 3(b).
         do this many times to see trends, but for each sample
         (red points for chosen, and other color for all points).
         make a combined plot too.
     (6) compare plots of random sampling (standard manner) with results from (5)
     (7) if one of the proposed sampling methods works well,
         use that as the default in RANSAC.
     (8) complete the epipolar tests
         -- consider evaluation based upon not only sum of dist from epipolar lines
            but also on the number of points (a combined calculation).
         -- having chosen the preferred sampling, look in detail at the individual
            solutions to make sure that the evaluation and final solution are the
            best from among the solutions.
     (9) implement or consider implementing the LMedS

   -- what is gradient weighted epipolar errors?

   -- look again at my estimate for number of iterations.
      a sony site uses for ransac a k iterations =
          log(f)/( log(1 - a^4) ) where f = probability of failure, a = apriori inlier ratio
      after each iteration, can recompute the number of inliers.
              maxC = count of maximum consensus so far.
              c = count of current consensus
              a = c/(data set size)
              then the recomputed k is a smaller number

    -- instead of a grid of 9 cells to draw from as considered below, 
       consider a monte carlo draw from uniformly random points of square of sides of length 2*r.
       two 1D probability density functions for x and y:
           probability of drawing a point inside unit dist is
                  area_disk/area_square = (pi*r^2)/(2*r)^2 = pi/4 for r > 0
           can see probability is indep of radius.
           so then can assume r=1
       Buffon's needle is a faster convergence of the monte carlo model.
          -- http:// numbers.computation.free.fr/ Constants/ PiProgram/ pifast.html
          
   ** change the plotter of epipolar results to plot the
      points used for the solution as one color (green with white block behind)
         and the points not used as red
   -- consider for LMedS the first solution from feature matcher could be used
      as a starter set to assess errors for guided sampling.
         the merton college starter points are nearly 50 and about 7 errors
         so eps of 20%, for example.

   -- when revisiting the books epipolar test, see if evaluation needs to focus
      on dist from epipolar line as much as the number of points within tolerance.
      --> make the combined score

   -- http://coewww.rutgers.edu/riul/research/papers/pdf/pbm.pdf
      possibly can use the first solution from feature matcher before the 
         entire image corners are added as the starter needed.

      http://coewww.rutgers.edu/riul/research/code/GPBM/index.html

   update the open agile story and close it

-- for line drawings, should use an edge detector that is a a second
   derivative operator instead of the canny first deriv operator.
   see the canny edge detector for lines...

-- median absolute deviation
6. <NAME>, <NAME>. Alternatives to the Median Absolute Deviation. J. Am. Statistical Assoc. 1993;vol. 88(no. 424):1273–1283.

-- update the command line runners.

-- Parzen window method in pattern recognition

** -- consider a detailed look at the books test matching.
      the largest group of matches alone determines the solution, but
      it looks like other group solutions should contribute, but might need larger tolerance

** -- consider the extreme unlikely case of only circles in an image.  there wouldn't be stable
      corners nor stable inflection points (except in the backgrounds possibly).
      the sillhouette solution combined with object identification by shapes 
          (such as using hough transform for circles)?
** -- impl a circle hough transform
   -- and explore elliptical transforms for the rainbow fitting which
      currently has a long runtime.
      x2+Bxy+Cy2+Dx+Ey+F=0  and (B^2-4C)< 0
         consider replacing the ellipse w/ a parabola (review choice of ellipse for easier fit to projected parabolas)
      -- consider impl a skip list and fitting fewer points
         https://www.sonycsl.co.jp/person/nielsen/visualcomputing/visualcomputing-chapter7-colorlowres.pdf

** -- use the added half dome test images to assert that matching same
      from other cameras is working

-- change any debugging code to not run by default and change log levels where needed
-- finish implementing recursive gaussian and gaussian derivative filters

-- for cases in which there are no discernible blobs or the ones that are present
   might be a changing component (for example, see the test image of stonehenge in sillhouette)
   may need to use a corners only feature matching.
   (can adapt the code to perform same steps, but for a all corners being part of a fake repr blob).
   might need a way to limit the open curves being matched such as a look at convex and concave
   for any 3 or more points?  I thnk BL2003 chose large blocks of cells of pixels to compare between
   images.  edges and groups of corners should be faster...
   ... contours


-- update aspects if needed and considering replacing them
    -- consider replacing aspectj with something like nanning aop

-- improve skyline extraction code

-- similarity that uses an exponential base?  can transform the numbers and use a distance
   transform to make the calculations faster

-- low priority, changes for KMPP...
   -- use a watershed at line 133
      and use the labeled components throughout code

-- explore a little the biorthogonal wavelet transform 
    in the JPEG-2000 image storage and compression standard

-- might need to make a method for the latest segmentation that can
   correct for equal values of green and red and blue by making
   3 separate wavelet segmentations and combining them in a way
   to preserve differences, but only if flag is set.  this kind
   of correction could possibly make matching of different images with
   different illumination more difficult (sky conditions, sun altitude, room lighting etc).

-- text recognition
   -- would like a class to extract and match text
      -- can see from the books test image that a binary segmentation would
         work best.
         -- mean adaptive thresholding might be a good follow up.
         -- blobs can be found as contiguous pixels of same value
         -- for the separated letters, should apply skeletonization possibly
         -- match against fonts.
            -- note font databases such as
               OCR-A, OCR-B, or MICR fonts
    some papers:
      http://www.rfai.li.univ-tours.fr/fr/ressources/_dh/DOC/DocOCR/OCRbinarisation.pdf

-- control flow graph for method called resolve() in DistanceTransform
   and same for resolveIterative to simplify the later

-- considering implementing a max-tree (Meister and Wilkinson)
   -- for a shape filter for blobs for example
       http://www.cs.rug.nl/michael/mt-pres.pdf

-- impl wavelets using "lifted wavelets".  see Sweldens 1996 and 1997

-- begin outline for rectification code

-- make the method to combine 2 skyline masks to decide what is sky and what
   is land when there are differences and there are more than 2 images to decide.  
   This can happen when there are clouds or snow or varying occlusion, etc

-- move all of debugging code to aspects or test.  jfreegraph should not need to be included
   in the command line jar unless add an output that uses it.

-- improve my canny edge detector w/ some new entries here for computing
    gradient using 6 neighbors
    and the suggested derivations of the 2-layer thresh
    and the line thinner
    https://en.wikipedia.org/wiki/Canny_edge_detector#Finding_the_Intensity_Gradient_of_the_Image

    note that the gradient changes might help with the normalized gradient right away

-- for more feature matching tests,
    see
    http://web.stanford.edu/~bgirod/pdfs/Chandrasekhar_CVPR2009.pdf
    "For evaluating the performance of low-bit-rate descrip- tors, we use two data sets provided by Winder and Brown [10], Trevi Fountain and Notre Dame. Each data set has matching pairs of 64×64 pixel patches. For algorithms that require training, we use matching pairs from the Trevi Fountain. For testing, we randomly select 10,000 matching"
    <NAME> and <NAME>, “Learning Local Image Descriptors,”
in Computer Vision and Pattern Recognition, 2007. CVPR ’07. IEEE
Conference on, 2007, pp. 1–8. 1, 2

-- revisit areas where using CountingSort and compare 
     O(N_max_value) to O(N*lg2N) where the later is the number of points

-- make plans for a stereoscopic matcher, image rectification, and disparity map maker

-- http://proquest.safaribooksonline.com.ezproxy.spl.org:2048/book/electrical-engineering/communications-engineering/9780123965028/chapter-20dot-clustering/st010_html_17?query=((ransac))#snippet

-- for a random subset chooser, since
     the x only depends upon the subset size, not on n, the number to choose from,
     could think about a way to calculate the bitstring randomly.
     for k = 7, the lowest bitstring is 0111 1111 which is a decimal value of 127.
     n gives the largest number of bitstring positions with k at the highest value positions.
     To calculate it in this way probably needs to use BigInteger.
     It has some random methods in it, so one could probably work out how to draw
     a number randomly between those values.
     Then might find the next lowest number from that, that has k=7 '1' digits.
     For example, if the random number were 252, that is 1111 1100, the next lowest 
     bitstring with k=7 1's is 1111 1011.      (find lowest set bit, change it to 0 and set the 2 below it to 1)
     If that bitstring has already been selected randomly, then the SubsetChooser's
     Gosper's hack quickly returns the next larger k=7 bitstring.

     This should be compared to the way I'm currently selecting k=7 indexes randomly.
     currently:  not sure actually... I think I select 7 randomly with no repeating digits
     and then sort them and see if the combination has been selected before, and 
     repeat if they have.
        that mean 7 random operations * 7 * lg2(7) for sort operations = 7 random * 20 steps for single iteration.
     
        the random w/ biginteger is 1 or more random operations (see internal implementation)
        plus a find lowest bit operation, plus 3 set bits, plus gospers hack (a half dozen bit operations)
        so the result =  1 (or more internal to BigInteger) rand operation + a dozen or so bit operations.

        The advantage to the BigInteger method is that it is possibly faster, but the draw would be
        within highly uneven space (the universe drawn from is far larger than the number of possible
        permutations, and then one adjusts to find the next lowest matching number).
        The probability that a single number had the k=7 digits set can be estimated using
        the probability of drawing one unique set of all the possible subsets for k=7 and n (with one draw)
        so that's 1./((n!/(k!*(n-k)!)) and then would need to account for the dilution of
        that within a larger set of numbers (nbits with highest k bits set to '1' - kbits all set to '1').
        The adjustment to select the next lowest number with k bits set leads to an uneven
        distribution (not uniform), but that would not necessarily be a bad thing here.

    Following the line of reasoning I must have started with:
       Ideally, one would be randomly choosing between k bit positions whose maximum is n.
       That requires enumerating all combinations and choosing randomly from them.
       Can see a pattern easily within Gosper's hack already, so might be a clever way
       to store them or to recalculate with offsets or derive a new ordering
       from Gosper's Hack that would allow an O(1) access of a bitstring from a random number 
       chosen within the range of the number of possible permutations.
       This ideal implementation would be uniform random draws.

    For either, should be storing "selected" as a bitstring.
    
    For the current implementation, I am not checking for having selected the previous combination
    before because the chance of drawing the same combination should be pretty small.
       The probability of selecting the same 7 objects out of n objects is roughly calculated with
            the presence of unique set of k within n objects is 1./(n!/(k!*(n-k)!)) 
            but that is not considering 'k draws'.   Can't use Bernoulli principle because that
            gives an answer that is a number having any k digits rather than a uniquely ordered
            set of k digits.
            
    ** do a quick plot of the difference of k=7 among n=(some number such that nPermutations < 1<<63)
       drawn by 7 random draws from a max of n numbers
       versus the BigInteger random draw 

-- fix the contour matcher search to use a compare and binary search

-- when sun is found, such as in the NM image, might try to fit a
   radial profile (1/r^2) from it over the image.
   can wait until sky pixels are mostly found and then fit over
   that and extend that pattern to the rest of the image?
   that is, in the NM image, can see that the sky varies smoothly
   but there are clouds that are in the sky too. 
   ** what happens when one knows the sun is in the image,
   one fits a gradient to the least bright pixels and then divides
   the image by that gradient?  presumably, the skyline stands out
   even more.   cie XY are independent of illumination in any case,
   so this step probably isn't useful and not always possible...

-- include other atmospheric optically visible features:  sun dogs and moon dogs.
-- consider more images which are near evening or evening, but there is enough illumination
   to see a skyline.

-- consider simplest ways to find solar reflected light in the images:
     -- already have a method which finds visible sun's photosphere
     -- create a separate method for reflected from water (see earlier code... depending upon sun location,
        the light might be orange to white...)... can use a source function of the sun
        and mie and rayliegh scattering if more formal methods are needed and reflection off of optically
        thick H_2O.
     -- refracted and reflected from clouds that are thick enough to provide a surface of reflected light
        that has a pattern of brightness due to illumination by sun...
     -- consider whether the location of the sun when photosphere is not in image is learnable
        by gradient of brightness.

-- add more tests when have a working solution for skyline extraction
   -- test by resolution too.  the seed sky points probably have some resolution dependence at this point.
   -- find test images for:
         -- structured cloudy sky and smooth white foreground
         -- structured cloudy sky and smooth dark foreground
         Neither of those will be solved as well as the NM test image which had non-sky foreground colors.
         Need additional information to understand what is sky without external sensors
         or assumption of horizontal.  In ambiguous cases, might need to make the assumption of
         skyline being horizontal in the image and sky near the top of the image...would like to avoid
         that assumption if at all possible.

-- one day, put in my algorithms toolbox, deconvolution methods:
   weiner filter deconvolution
   Tikhonov regularization
   Richardson-Lucy

-- find image with naturally occuring skew for tests

-- for ways to reduce the corner list, consider reading:
   <NAME>., and <NAME>. "Good Features to Track." 
   Conference on Computer Vision and Pattern Recognition, 1994.

   journals to browse:

   CVGIP  Graphical Models /graphical Models and Image Processing /computer Vision, Graphics, and Image Processing
   CVIU  Computer Vision and Image Understanding
   IJCV  International Journal of Computer Vision
   IVC  Image and Vision Computing
   JMIV  Journal of Mathematical Imaging and Vision
   MVA  Machine Vision and Applications
   TMI - IEEE  Transactions on Medical Imaging

   http://pointclouds.org/documentation/

-- read:
    http://proquest.safaribooksonline.com.ezproxy.spl.org:2048/book/illustration-and-graphics/9780133373721

-- revisit the matrix math, especially where there's multiplication.
   did I replace dot operation with multiplication anywhere?
   needs tests...

-- reconsider later some properties.
   hue for blue skies.
   eucl dist of color = sqrt((r-r)^2 + (g-g)^2 + (b-b)^2)
   
another test image?
   http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/BSDS300/html/dataset/images/color/260058.html

-- read more on segmentation... lots of comparisons on the benchmark site.
   note the skyline extractor here is only partial segmentation and specific to a task.

   http://www.cs.berkeley.edu/~arbelaez/publications/amfm_pami2011.pdf

-- test datasets
http://sipi.usc.edu/database/database.php?volume=misc
http://homepages.inf.ed.ac.uk/rbf/CVonline/Imagedbase.htm#segmentation

-- finish the method converting from fundamental matrix to essential matrix.

 /*
    https://github.com/jesolem/PCV
    adapted from code licensed under  BSD license (2-clause "Simplified BSD License").
    private SimpleMatrix calculateCameraMatrixFromFundamentalMatrix(SimpleMatrix fm) {
        SimpleMatrix u = fm.svd().getU();
        SimpleMatrix leftE = u.extractVector(true, 2);
        leftE = leftE.divide(u.get(2, 2));

        double[][] skewSymmetric = new double[3][];
        skewSymmetric[0] = new double[]{0, -leftE.get(0, 2), leftE.get(0, 1)};
        skewSymmetric[1] = new double[]{leftE.get(0, 2), 0, -leftE.get(0, 0)};
        skewSymmetric[2] = new double[]{-leftE.get(0, 1), leftE.get(0, 0), 0};

        SimpleMatrix sMatrix = new SimpleMatrix(skewSymmetric);
        double v = sMatrix.dot( fm.transpose() );


        //essential matrix from F: E = transpose(K2)*F*K1
        SimpleMatrix k = DataForTests.getVenturiCameraIntrinsics()
            .mult(DataForTests.getVenturiRotationMatrixForImage001());
        SimpleMatrix k2 = DataForTests.getVenturiCameraIntrinsics()
            .mult(DataForTests.getVenturiRotationMatrixForImage010());

        SimpleMatrix essentialMatrix = k2.transpose().mult(fm).mult(k);

        int z = 1;
    }*/


-- consider following the implementation of disparity maps for stereo images
   and 3d modelling.  see notes in the docs directory.
   -- see http://vision.middlebury.edu/stereo/code/

-- consider implementing the LM-ICP from fitz-gibbons?
   did i look at this already and decide otherwise?
   does is handle projective transformations?

-- ** read: https://www.graphics.rwth-aachen.de/person/21/

-- for tests, make sure the projective point matcher can handle images like the
    ones in this tutorial:
    http://www.robots.ox.ac.uk/~az/tutorials/tutorialb.pdf

-- to the corner list makin',
   -- add a removal of redundant points

-- test for degenerate camera conditions:
   -- parallel camera motion w/o rotation 
-- test for degenerate scene structure configurations
   -- all points lying on a plane or nearly lying on a plane (?)
-- test for point sets containing noise
-- error estimate in fundamental matrix:
   6.5.3
   -- Gold Standard algorithm?  (need camera details...)
      summation over i( d(x_i, x_hat_i)^2 + d(x_prime_i, x_prime_hat_i)^2 ) 
   -- Sampson distance?

http://dev.ipol.im/~morel/Dossier_MVA_2011_Cours_Transparents_Documents/2011_Cours7_Document3_hartley.pdf
-- a hartley paper suggests rectifying images to evaluate point transformations.


{'fraction_non_alphanumeric': 0.05175568221800014, 'fraction_numerical': 0.01335233270002111, 'mean_word_length': 2.8313001280582326, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 42, 'https://': 8, 'lorem ipsum': 0, 'www.': 18, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '579602', 'n_tokens_mistral': 15012, 'n_tokens_neox': 13843, 'n_words': 7535}