I'm writing deduplication program that implements content-defined chunking. Just now i know 3 algorithms to do it (hashing a fixed-size window and selecting hashes < maxuint/N, hashing an order-1 bounded window, and local maxima chunking). Are you know other algorithms or ideas to improve these ones? My goal is to get best compression for a given average chunk size.A few deduplication-related links:http://en.wikipedia.org/wiki/Data_deduplicationhttp://encode.ru/threads/1726-Deduplication-X-Files 
Deduplication: how to implement content-defined chunking?
algorithms;optimization