FIELD OF THE INVENTION

The present invention relates to hybrid blockchains and streamchains using non-cryptographic hashes for securing audio-, video-, image-, and speech-based transactions and contracts.

BACKGROUND OF THE INVENTION

Blockchain is a distributed and public ledger which maintains records of all the transactions. A blockchain network is a truly peer-to-peer network and it does not require a trusted central authority or intermediaries to authenticate or to settle the transactions or to control the network infrastructure. Users can interact and transact with the blockchain networks through Externally Owned Account (EOAs), which are owned and controlled by the users. Each EOA has a balance (in certain units of a Cryptocurrency associated with the Blockchain network) associated with it. EOAs do not have any associated code. All transactions on a blockchain network are initiated by EOAs. These accounts can send transactions to other EOAs or contract accounts. Another type of accounts support by second generation programmable Blockchain platforms are the Contract Accounts. A Contract Account is created and owned by an EOA and is controlled by the associated contract code which is stored with the account. The contract code execution is triggered by transactions sent by EOAs or messages sent by other contracts.

Blockchain networks can either be public or private. Public blockchain networks are free and open to all and any user can create an account and participate in the consensus mechanism on a public blockchain and view all the transactions on the network. Private blockchain networks are usually controlled and operated by a single organization and the transactions can be viewed only by the users within the organization. Public blockchain networks are usually unpermissioned or permissionless, as any node can participate in consensus process. Some public blockchain networks adopt a permissioned model where the consensus process is controlled by a pre-selected set of nodes. Private blockchain networks usually adopt the permissioned model. While public blockchain networks can be considered as fully decentralized, private blockchain networks are partially decentralized.

Audio fingerprinting is used to convert the an audio signal/recording into a series of short numerical values (or hashes) that aim to uniquely identify the audio signal/recording. Audio fingerprinting techniques use hashing algorithms that utilize the perceptual aspects of the audio contained in the file and not just the way that the file itself is encoded digitally. Audio fingerprinting systems can identify audio even if it has been altered in different ways such as by down-sampling, filtering, equalization, re-sampling, compression, adding noise or echo.

Audio fingerprints generally do not have the time of creation via real-time clock and timestamp, nor are audio fingerprints linked to other audio finger prints in terms of a streaming content. Nor are there support for different types of alternate fingerprints that may have very similar non-cryptographic hashes.

While existing Blockchain platforms such as Ethereum or databases such as Amazon Quantum Ledger Database (QLDB) can be used for storing audio/video fingerprint data (either within smart contracts or data records) and provide an immutable and cryptographically verifiable transaction log, however, these existing solutions use cryptographic hashes. Cryptographic hashes are unsuitable for audio fingerprinting because fingerprinting algorithms should generate similar hashes on audio that sounds the same to an ear but may be stored or encoded differently. The problem with using cryptographic hashes for transactions containing audio/video data is that even minute differences can result in mismatches. Transactions based on clean or noisy data can generate different cryptographic hash values even if both are authentic and valid.

Cryptographic hashes are not easily applied where transactions can contain audio, speech, image or video data that is affected by noise, or background, or context of the audio and video. For example, slurred speech or hazy video of the same scenes or same person may result in widely differing cryptographic hashes. The same audio recording when encoded differently and stored in different file formats can produce different cryptographic hashes. Audio processing techniques such as down-sampling, re-sampling, filtering, equalization can result in widely differing cryptographic hashes.

Some known and widely used audio fingerprinting techniques are:

1. EchoPrint: Echoprint is a freely available audio fingerprint algorithm (github.com/spotify/echoprint-codegen) and server (github.com/spotify/echoprint-server/). EchoPrint was created and released by music technology company The Echo Nest. Spotify acquired The Echo Nest in 2014. The Echoprint algorithm works by finding onsets points in time where musical notes occur. Features are created by calculating the difference in time between subsequent onsets and creating a hash of these time values. Matching recordings are found by looking for identical hashes in the reference database.

2. ChromaPrint: Chromaprint is an audio fingerprint library developed for the AcoustID project and available at github.com/acoustid/chromaprint. ChromaPrint is designed to identify near-identical audio and the fingerprints it generates are as compact as possible to achieve that. It trades precision and robustness for search performance. The target use cases are full audio file identification, duplicate audio file detection and long audio stream monitoring.

3. Landmark: Landmark audio fingerprinting technique is very well suited to identifying small, noisy excerpts from a large number of items. It is based on the ideas used in the Shazam music matching service, which can identify seemingly any commercial music tracks from short snippets recorded via cellphones even in very noisy conditions. (labrosa.ee.columbia.edu/matlab/fingerprint/).

Some known and widely used non-cryptographic hashing techniques are:

1. Murmurhash: Murmur hashing is a non-cryptographic hash function which is used for hash based look-ups. Murmurhash uses three basic operations—Multiply, Rotate and XOR. Within the hashing algorithm, multiple constants are used to make it good hash function. Murmurhash provides good distribution (passes Chi-Squared test) and good avalanche behavior (passes Avalanche test) and good collision resistance. (github.com/aappleby/smhasher)

2. CityHash: CityHash family of hash functions for strings. The functions mix the input bits thoroughly but are not suitable for cryptography. (github.com/google/cityhash)

3. Locality Sensitive Hashing (LSH): LSH is a hashing technique designed for reducing the dimensionality high-dimensional data by mapping similar items map to the same buckets with high probability. LSH is designed to maximize the probability of a collision for similar items. LSH is typically used to find similar entries in large databases.

This background information is provided to reveal information believed by the applicant to be of possible relevance to the present invention. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present invention.

SUMMARY OF THE INVENTION

With the above in mind, embodiments of the present invention are related to the use of audio/video fingerprints, non-cryptographic hashes, a new type of consensus architecture (StreamChain), and new mining process, including:a blockchain with cryptographic hashes & audio/video fingerprints in transactions;a blockchain with non-cryptographic hashes & audio/video fingerprints in transactions;a StreamChain with cryptographic hashes, audio/video fingerprints in transactions, multiple copies of blocks according to suitability & updates in mining process;a StreamChain with non-cryptographic hashes, audio/video fingerprints in transactions, updating of past blocks via quality & updates in mining process; anda StreamChain that uses a DB and a Blockchain.

Certain embodiments of the present invention may comprise the following features:1. Ability to securely identify and verify speech and audio content and transactions. StreamChain can verify if the audio/video data belongs to a certain user and time and corresponds to certain transaction or event—a musical composition, a sports event, or a business context.2. Ability to go backwards in time to “rewrite” or “substitute” an older block on a blockchain with higher quality data that has a similar fingerprint or hash also recorded at a contemporaneous real-time value. There is a difference between versions of transactions (that may represent modifications of same content over time), versus “alternates” or “alternatives” that use better audio (or less noisy or more relevant audio) with the same or similar real-time timestamp that can be used in current StreamChain.3. Ability to append (in a secure and verifiable manner) and select the “best” version of the information for the next or past blocks on a blockchain.4. Allows controlled rewriting that is verifiable as old blocks and new blocks that overwrite them coexist.5. The old blocks are also saved but they are “hidden” and can be put back into view if there is a problem. alternative versions of blocks can be ranked based on parameter such as quality of audio or video recording.6. Data blocks or transactions can be changed only if hashes are similar and have same or similar timestamp.7. In certain embodiments of StreamChain the hash for the entire block is a non-cryptographic hash. This can simplify the mining process. A fast mining approach can be used that does not consume extensive resources. Within StreamChain it is also possible to merge transactions from multiple chains that contain speech, audio and video data that can be checked and substituted or altered subject to certain constraints.8. Large amounts of audio and video data can be stored off the chain while their fingerprints are stored on the blockchain.9. Model View Controller (MVC) can be applied to StreamChain where different views are generated from same model where each view uses different alternatives chosen from those stored on the blockchain.10. The consensus mechanism is changed from the hard yes/no binary options available in existing blockchains to the “fuzzy” sliding match afforded by non-cryptographic hashes (such as LSH).

In certain embodiments, the present invention may provide the following advantages:1. Verification of transactions by miners can be simplified using non-cryptographic hashes when audio, speech or video transactions are part of the chain (not digital documents or transactions).2. This allows approximate matches to reference fingerprint information from similar transactions by the same user or other user to identify fraud and also to summarize or pool data.3. Also, can be used to provide verification of identity of the blockchain account holders in the party to the contract by having them include audio and speech or video evidence into the record in an automated manner that can be verified by the hybrid non-cryptographic/cryptographic blockchain.4. Approximate verification may result in the concept of fuzzy blockchains that trade-off accuracy for speed or wider inclusion of types of data that is frequency subjected to noise, interference (e.g., speech under noisy conditions from the same user can be different that in clear noise-free conditions, or smoky or fuzzy video can be the only available evidence from a scene).5. Different strategies for preventing attacks for non-cryptographic hashed blockchains or hybrid chains are to be used using variety of new techniques—such as having multi-factor verification or voting.6. Some blocks in the blockchain may remain cryptographic-based hashed and others could be non-cryptographic hashed or a hybrid with or without differentiating markers.7. Smart contracts may be used to operate on hybrid blocks using shared variables or bulletin boards or synchronizing transactions across multiple blockchains or Stream Chains.

Additionally, embodiments of the invention are directed to a method of securing media-based transactions and contracts on a distributed ledger network comprising receiving a first media file, obtaining a fingerprint of the first media file; generating a non-cryptographic hash value for the fingerprint of the first media file, generating a block for the distributed ledger network comprising a block header comprising previous hash, a state root hash, a receipts root hash, and a transaction list root hash, defining block header hashes, recording the non-cryptographic fingerprint hash value for the fingerprint of the first media file to the block, and recording the block to the distributed ledger network. In some embodiments, the block header hashes may be cryptographic hashes. In some embodiments, the block header hashes may be non-cryptographic hashes.

In some embodiments, the media file may be one of an image file, an audio file, and a video file. The media file may be extracted from a live recording, or from streaming multimedia content over a network.

In some embodiments, the fingerprint may be derived from temporal and frequency properties of the media file. The fingerprint may comprise a real-time clock timestamp corresponding to the media file.

In some embodiments, the distributed ledger network may be a blockchain network. In some embodiments, the distributed ledger network may be a streamchain network. In such embodiments, the method may further comprise receiving a second media file, obtaining a fingerprint of the second media file, generating a non-cryptographic hash value for the fingerprint of the second media file, generating a second version of the block for the distributed ledger network, recording the non-cryptographic hash value for the fingerprint of the second media file to the second version of the block, and recording the second version of the block to the streamchain network. Furthermore, the first and second media files may be derived from a common source media recording or event. In some embodiments, the method may further comprise determining whether the non-cryptographic hash value for the fingerprint of the second media file is within a threshold similarity of non-cryptographic hash value for the fingerprint of the first media file, upon determining the non-cryptographic fingerprint hash value for the fingerprint of the second media file is within a threshold similarity of non-cryptographic fingerprint hash value for the fingerprint of the first media file, accepting the second version of the non-cryptographic hash value in the ledger of the streamchain if a score value of the second media file is better than the score value of the first media file, and upon determining the non-cryptographic fingerprint hash value for the fingerprint of the second media file is not within a threshold similarity of non-cryptographic fingerprint hash value for the fingerprint of the first media file, rejecting the second version of the non-crypto hash value from the ledger of the streamchain. The first and second versions of the non-crypto hash values block may be ordered based upon a score value that is derived from at least one of match quality and relevance of the respective media files.

