<commit_before>Kaldi Gstreamer server
======================

This is an implementation of a real-time full-duplex speech recognition server, based on
the Kaldi toolkit and the GStreamer framwork.

Communication with the server is based on weksockets. Client sends speech to the server using
small chunks, while the server sends partial and full recognition hypotheses back to
the client via the same websocket, thus enabling full-duplex communication (as in Google's
voice typing in Android).

Server usage
------------

### Opening a session

To open a session, connect to the specified server websocket address (e.g. ws://server:8888/speech).
The server assumes by deafult that incoming audio is sent using 16 kHz, mono, 16bit little-endian format. This can be overriden
using the 'content-type' request parameter. The content type has to be specified using GStreamer 1.0 caps format,
e.g. to send 44100 Hz mono 16-bit data, use: "audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16LE, channels=(int)1".
This needs to be url-encoded of course, so the actual request is something like:

    ws://server:8888/speech?content-type=audio/x-raw,+layout=(string)interleaved,+rate=(int)44100,+format=(string)S16LE,+channels=(int)1

Audio can also be encoded using any codec recognized by GStreamer (assuming the needed packages are instealled on teh server).
E.g., to send audio encoded using the Speex codec in an Ogg container, use the following URL to open the session (server should
automatically recognize the codec):

    ws://server:8888/speech?content-type=audio/ogg

### Sending audio

Speech should be sent to the server in raw blocks of data, using the encoding specified when session was opened.
It is recommended that a new block is sent at least 4 times per second (more infrequent blocks would increase the recognition lag).

After the last block of speech data, a special string "EOS"  ("end-of-stream") needs to be sent to the server. This tells the
server that no more speech is coming and the recognition can be finalized.

### Reading results

Server sends recognition results and other information to the client using the JSON format.
The response can contain the following fields:

  * status -- response status, see codes below
  * message -- (optional) status message
  * result -- (optional) recognition result, containing the followig fields:
    - hypotheses - recognized words, a list of the following:
      + transcript -- recognized words
      + confidence -- (optional) confidence of the hypothesis (float, 0..1)
    - final -- true when the hypothesis is final, i.e., doesn't change any more

The following status codes are currently in use:

  * 0 -- Success. Usually used when recognition results are sent
  * 2 -- Aborted. Recognition was aborted for some reason.
  * 1 -- No speech. Sent when the incoming audio contains a large portion of silence or non-speech.
  * 9 -- Not avalailable. Used when all recognizer processes are currently in use and recognition cannot be performed.

Websocket is always closed by the server after sending a non-zero status update.
<commit_msg>updated docs<commit_after>Kaldi Gstreamer server
======================

This is an implementation of a real-time full-duplex speech recognition server, based on
the Kaldi toolkit and the GStreamer framwork.

Communication with the server is based on weksockets. Client sends speech to the server using
small chunks, while the server sends partial and full recognition hypotheses back to
the client via the same websocket, thus enabling full-duplex communication (as in Google's
voice typing in Android).

Server usage
------------

### Opening a session

To open a session, connect to the specified server websocket address (e.g. ws://server:8888/speech).
The server assumes by deafult that incoming audio is sent using 16 kHz, mono, 16bit little-endian format. This can be overriden
using the 'content-type' request parameter. The content type has to be specified using GStreamer 1.0 caps format,
e.g. to send 44100 Hz mono 16-bit data, use: "audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16LE, channels=(int)1".
This needs to be url-encoded of course, so the actual request is something like:

    ws://server:8888/speech?content-type=audio/x-raw,+layout=(string)interleaved,+rate=(int)44100,+format=(string)S16LE,+channels=(int)1

Audio can also be encoded using any codec recognized by GStreamer (assuming the needed packages are instealled on teh server).
E.g., to send audio encoded using the Speex codec in an Ogg container, use the following URL to open the session (server should
automatically recognize the codec):

    ws://server:8888/speech?content-type=audio/ogg

### Sending audio

Speech should be sent to the server in raw blocks of data, using the encoding specified when session was opened.
It is recommended that a new block is sent at least 4 times per second (more infrequent blocks would increase the recognition lag).

After the last block of speech data, a special string "EOS"  ("end-of-stream") needs to be sent to the server. This tells the
server that no more speech is coming and the recognition can be finalized.

### Reading results

Server sends recognition results and other information to the client using the JSON format.
The response can contain the following fields:

  * status -- response status, see codes below
  * message -- (optional) status message
  * result -- (optional) recognition result, containing the followig fields:
    - hypotheses - recognized words, a list of the following:
        + transcript -- recognized words
        + confidence -- (optional) confidence of the hypothesis (float, 0..1)
    - final -- true when the hypothesis is final, i.e., doesn't change any more

The following status codes are currently in use:

  * 0 -- Success. Usually used when recognition results are sent
  * 2 -- Aborted. Recognition was aborted for some reason.
  * 1 -- No speech. Sent when the incoming audio contains a large portion of silence or non-speech.
  * 9 -- Not avalailable. Used when all recognizer processes are currently in use and recognition cannot be performed.

Websocket is always closed by the server after sending a non-zero status update.

{'fraction_non_alphanumeric': 0.076223887737203, 'fraction_numerical': 0.015308563227555414, 'mean_word_length': 4.157894736842105, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5946872', 'n_tokens_mistral': 1689, 'n_tokens_neox': 1549, 'n_words': 851}