natural language toolkit timit corpus reader c 20012007 nltk project haejoong lee haejoongldc upenn edu steven bird stevenbird1gmail com jacob perkins japerkgmail com url https www nltk org for license information see license txt xx this docstring is outofdate read tokens phonemes and audio data from the nltk timit corpus this corpus contains selected portion of the timit corpus 16 speakers from 8 dialect regions 1 male and 1 female from each dialect region total 130 sentences 10 sentences per speaker note that some sentences are shared among other speakers especially sa1 and sa2 are spoken by all speakers total 160 recording of sentences 10 recordings per speaker audio format nist sphere single channel 16khz sampling 16 bit sample pcm encoding module contents the timit corpus reader provides 4 functions and 4 data items utterances list of utterances in the corpus there are total 160 utterances each of which corresponds to a unique utterance of a speaker here s an example of an utterance identifier in the list dr1fvmh0sx206 sentence number sentence type a all i shared x exclusive speaker id sex m male f female dialect region 1 8 speakers list of speaker ids an example of speaker id dr1fvmh0 note that if you split an item id with colon and take the first element of the result you will get a speaker id itemid dr1fvmh0sx206 spkrid sentid itemid split spkrid dr1fvmh0 the second element of the result is a sentence id dictionary phonetic dictionary of words contained in this corpus this is a python dictionary from words to phoneme lists spkrinfo speaker information table it s a python dictionary from speaker ids to records of 10 fields speaker ids the same as the ones in timie speakers each record is a dictionary from field names to values and the fields are as follows id speaker id as defined in the original timit speaker info table sex speaker gender m male f female dr speaker dialect region 1 new england 2 northern 3 north midland 4 south midland 5 southern 6 new york city 7 western 8 army brat moved around use corpus type trn training tst test in this sample corpus only trn is available recdate recording date birthdate speaker birth date ht speaker height race speaker race wht white blk black amr american indian spn spanishamerican orn oriental unknown edu speaker education level hs high school as associate degree bs bachelor s degree bs or ba ms master s degree ms or ma phd doctorate degree phd jd md unknown comments comments by the recorder the 4 functions are as follows tokenizedsentencesitems offsetfalse given a list of items returns an iterator of a list of word lists each of which corresponds to an item sentence if offset is set to true each element of the word list is a tuple of wordstring start offset and end offset where offset is represented as a number of 16khz samples phoneticsentencesitems offsetfalse given a list of items returns an iterator of a list of phoneme lists each of which corresponds to an item sentence if offset is set to true each element of the phoneme list is a tuple of wordstring start offset and end offset where offset is represented as a number of 16khz samples audiodataitem start0 endnone given an item returns a chunk of audio samples formatted into a string when the function is called if start and end are omitted the entire samples of the recording will be returned if only end is omitted samples from the start offset to the end of the recording will be returned playdata play the given audio samples the audio samples can be obtained from the timit audiodata function reader for the timit corpus or any other corpus with the same file layout and use of file formats the corpus root directory should contain the following files timitdic txt dictionary of standard transcriptions spkrinfo txt table of speaker information in addition the root directory should contain one subdirectory for each speaker containing three files for each utterance utteranceid txt text content of utterances utteranceid wrd tokenized text content of utterances utteranceid phn phonetic transcription of utterances utteranceid wav utterance sound file a regexp matching fileids that are used by this corpus reader utterancere rwww txt def initself root encodingutf8 ensure that wave files don t get treated as unicode data if isinstanceencoding str encoding r wav none encoding corpusreader init self root findcorpusfileidsroot self filere encodingencoding self utterances name 4 for name in findcorpusfileidsroot self utterancere return a list of file identifiers for the files that make up this corpus param filetype if specified then filetype indicates that only the files that have the given type should be returned accepted values are txt wrd phn wav or metadata return a list of the utterance identifiers for all utterances in this corpus or for the given speaker dialect region gender sentence type or sentence number if specified return a dictionary giving the standard transcription for each word return a list of all utterances associated with a given speaker return a dictionary mapping something offset is represented as a number of 16khz samples todo check this xx note this is currently broken we re assuming that the fileids are wav fileids aka riff but they re actually nist sphere fileids nltk chunk conflicts with the stdlib module chunk skip past frames before start then read the frames we want open a new temporary file the wave module requires an actual file and won t work w stringio write the parameters data to the new file read the data back from the file and return it the file will automatically be deleted when we return play the given audio sample param utterance the utterance id of the sample to play method 1 os audio dev method 2 pygame fixme this won t work under python 3 method 3 complain block reader for timit tagged sentences which are preceded by a sentence number that will be ignored natural language toolkit timit corpus reader c 2001 2007 nltk project haejoong lee haejoong ldc upenn edu steven bird stevenbird1 gmail com jacob perkins japerk gmail com url https www nltk org for license information see license txt xx this docstring is out of date read tokens phonemes and audio data from the nltk timit corpus this corpus contains selected portion of the timit corpus 16 speakers from 8 dialect regions 1 male and 1 female from each dialect region total 130 sentences 10 sentences per speaker note that some sentences are shared among other speakers especially sa1 and sa2 are spoken by all speakers total 160 recording of sentences 10 recordings per speaker audio format nist sphere single channel 16khz sampling 16 bit sample pcm encoding module contents the timit corpus reader provides 4 functions and 4 data items utterances list of utterances in the corpus there are total 160 utterances each of which corresponds to a unique utterance of a speaker here s an example of an utterance identifier in the list dr1 fvmh0 sx206 _ _ sentence number sentence type a all i shared x exclusive speaker id sex m male f female dialect region 1 8 speakers list of speaker ids an example of speaker id dr1 fvmh0 note that if you split an item id with colon and take the first element of the result you will get a speaker id itemid dr1 fvmh0 sx206 spkrid sentid itemid split spkrid dr1 fvmh0 the second element of the result is a sentence id dictionary phonetic dictionary of words contained in this corpus this is a python dictionary from words to phoneme lists spkrinfo speaker information table it s a python dictionary from speaker ids to records of 10 fields speaker ids the same as the ones in timie speakers each record is a dictionary from field names to values and the fields are as follows id speaker id as defined in the original timit speaker info table sex speaker gender m male f female dr speaker dialect region 1 new england 2 northern 3 north midland 4 south midland 5 southern 6 new york city 7 western 8 army brat moved around use corpus type trn training tst test in this sample corpus only trn is available recdate recording date birthdate speaker birth date ht speaker height race speaker race wht white blk black amr american indian spn spanish american orn oriental unknown edu speaker education level hs high school as associate degree bs bachelor s degree bs or ba ms master s degree ms or ma phd doctorate degree phd jd md unknown comments comments by the recorder the 4 functions are as follows tokenized sentences items offset false given a list of items returns an iterator of a list of word lists each of which corresponds to an item sentence if offset is set to true each element of the word list is a tuple of word string start offset and end offset where offset is represented as a number of 16khz samples phonetic sentences items offset false given a list of items returns an iterator of a list of phoneme lists each of which corresponds to an item sentence if offset is set to true each element of the phoneme list is a tuple of word string start offset and end offset where offset is represented as a number of 16khz samples audiodata item start 0 end none given an item returns a chunk of audio samples formatted into a string when the function is called if start and end are omitted the entire samples of the recording will be returned if only end is omitted samples from the start offset to the end of the recording will be returned play data play the given audio samples the audio samples can be obtained from the timit audiodata function reader for the timit corpus or any other corpus with the same file layout and use of file formats the corpus root directory should contain the following files timitdic txt dictionary of standard transcriptions spkrinfo txt table of speaker information in addition the root directory should contain one subdirectory for each speaker containing three files for each utterance utterance id txt text content of utterances utterance id wrd tokenized text content of utterances utterance id phn phonetic transcription of utterances utterance id wav utterance sound file a regexp matching fileids that are used by this corpus reader construct a new timit corpus reader in the given directory param root the root directory for this corpus ensure that wave files don t get treated as unicode data a list of the utterance identifiers for all utterances in this corpus return a list of file identifiers for the files that make up this corpus param filetype if specified then filetype indicates that only the files that have the given type should be returned accepted values are txt wrd phn wav or metadata return a list of the utterance identifiers for all utterances in this corpus or for the given speaker dialect region gender sentence type or sentence number if specified return a dictionary giving the standard transcription for each word return a list of all utterances associated with a given speaker return a dictionary mapping something offset is represented as a number of 16khz samples todo check this xx note this is currently broken we re assuming that the fileids are wav fileids aka riff but they re actually nist sphere fileids nltk chunk conflicts with the stdlib module chunk skip past frames before start then read the frames we want open a new temporary file the wave module requires an actual file and won t work w stringio write the parameters data to the new file read the data back from the file and return it the file will automatically be deleted when we return play the given audio sample param utterance the utterance id of the sample to play method 1 os audio dev method 2 pygame fixme this won t work under python 3 method 3 complain block reader for timit tagged sentences which are preceded by a sentence number that will be ignored
import sys
import time

from nltk.corpus.reader.api import *
from nltk.internals import import_from_stdlib
from nltk.tree import Tree


class TimitCorpusReader(CorpusReader):
    

    _FILE_RE = r"(\w+-\w+/\w+\.(phn|txt|wav|wrd))|" + r"timitdic\.txt|spkrinfo\.txt"
    
    _UTTERANCE_RE = r"\w+-\w+/\w+\.txt"

    def __init__(self, root, encoding="utf8"):
        
        
        if isinstance(encoding, str):
            encoding = [(r".*\.wav", None), (".*", encoding)]

        CorpusReader.__init__(
            self, root, find_corpus_fileids(root, self._FILE_RE), encoding=encoding
        )

        self._utterances = [
            name[:-4] for name in find_corpus_fileids(root, self._UTTERANCE_RE)
        ]
        

        self._speakerinfo = None
        self._root = root
        self.speakers = sorted({u.split("/")[0] for u in self._utterances})

    def fileids(self, filetype=None):
        
        if filetype is None:
            return CorpusReader.fileids(self)
        elif filetype in ("txt", "wrd", "phn", "wav"):
            return [f"{u}.{filetype}" for u in self._utterances]
        elif filetype == "metadata":
            return ["timitdic.txt", "spkrinfo.txt"]
        else:
            raise ValueError("Bad value for filetype: %r" % filetype)

    def utteranceids(
        self, dialect=None, sex=None, spkrid=None, sent_type=None, sentid=None
    ):
        
        if isinstance(dialect, str):
            dialect = [dialect]
        if isinstance(sex, str):
            sex = [sex]
        if isinstance(spkrid, str):
            spkrid = [spkrid]
        if isinstance(sent_type, str):
            sent_type = [sent_type]
        if isinstance(sentid, str):
            sentid = [sentid]

        utterances = self._utterances[:]
        if dialect is not None:
            utterances = [u for u in utterances if u[2] in dialect]
        if sex is not None:
            utterances = [u for u in utterances if u[4] in sex]
        if spkrid is not None:
            utterances = [u for u in utterances if u[:9] in spkrid]
        if sent_type is not None:
            utterances = [u for u in utterances if u[11] in sent_type]
        if sentid is not None:
            utterances = [u for u in utterances if u[10:] in spkrid]
        return utterances

    def transcription_dict(self):
        
        _transcriptions = {}
        with self.open("timitdic.txt") as fp:
            for line in fp:
                if not line.strip() or line[0] == ";":
                    continue
                m = re.match(r"\s*(\S+)\s+/(.*)/\s*$", line)
                if not m:
                    raise ValueError("Bad line: %r" % line)
                _transcriptions[m.group(1)] = m.group(2).split()
        return _transcriptions

    def spkrid(self, utterance):
        return utterance.split("/")[0]

    def sentid(self, utterance):
        return utterance.split("/")[1]

    def utterance(self, spkrid, sentid):
        return f"{spkrid}/{sentid}"

    def spkrutteranceids(self, speaker):
        
        return [
            utterance
            for utterance in self._utterances
            if utterance.startswith(speaker + "/")
        ]

    def spkrinfo(self, speaker):
        
        if speaker in self._utterances:
            speaker = self.spkrid(speaker)

        if self._speakerinfo is None:
            self._speakerinfo = {}
            with self.open("spkrinfo.txt") as fp:
                for line in fp:
                    if not line.strip() or line[0] == ";":
                        continue
                    rec = line.strip().split(None, 9)
                    key = f"dr{rec[2]}-{rec[1].lower()}{rec[0].lower()}"
                    self._speakerinfo[key] = SpeakerInfo(*rec)

        return self._speakerinfo[speaker]

    def phones(self, utterances=None):
        results = []
        for fileid in self._utterance_fileids(utterances, ".phn"):
            with self.open(fileid) as fp:
                for line in fp:
                    if line.strip():
                        results.append(line.split()[-1])
        return results

    def phone_times(self, utterances=None):
        
        results = []
        for fileid in self._utterance_fileids(utterances, ".phn"):
            with self.open(fileid) as fp:
                for line in fp:
                    if line.strip():
                        results.append(
                            (
                                line.split()[2],
                                int(line.split()[0]),
                                int(line.split()[1]),
                            )
                        )
        return results

    def words(self, utterances=None):
        results = []
        for fileid in self._utterance_fileids(utterances, ".wrd"):
            with self.open(fileid) as fp:
                for line in fp:
                    if line.strip():
                        results.append(line.split()[-1])
        return results

    def word_times(self, utterances=None):
        results = []
        for fileid in self._utterance_fileids(utterances, ".wrd"):
            with self.open(fileid) as fp:
                for line in fp:
                    if line.strip():
                        results.append(
                            (
                                line.split()[2],
                                int(line.split()[0]),
                                int(line.split()[1]),
                            )
                        )
        return results

    def sents(self, utterances=None):
        results = []
        for fileid in self._utterance_fileids(utterances, ".wrd"):
            with self.open(fileid) as fp:
                results.append([line.split()[-1] for line in fp if line.strip()])
        return results

    def sent_times(self, utterances=None):
        
        return [
            (
                line.split(None, 2)[-1].strip(),
                int(line.split()[0]),
                int(line.split()[1]),
            )
            for fileid in self._utterance_fileids(utterances, ".txt")
            for line in self.open(fileid)
            if line.strip()
        ]

    def phone_trees(self, utterances=None):
        if utterances is None:
            utterances = self._utterances
        if isinstance(utterances, str):
            utterances = [utterances]

        trees = []
        for utterance in utterances:
            word_times = self.word_times(utterance)
            phone_times = self.phone_times(utterance)
            sent_times = self.sent_times(utterance)

            while sent_times:
                (sent, sent_start, sent_end) = sent_times.pop(0)
                trees.append(Tree("S", []))
                while (
                    word_times and phone_times and phone_times[0][2] <= word_times[0][1]
                ):
                    trees[-1].append(phone_times.pop(0)[0])
                while word_times and word_times[0][2] <= sent_end:
                    (word, word_start, word_end) = word_times.pop(0)
                    trees[-1].append(Tree(word, []))
                    while phone_times and phone_times[0][2] <= word_end:
                        trees[-1][-1].append(phone_times.pop(0)[0])
                while phone_times and phone_times[0][2] <= sent_end:
                    trees[-1].append(phone_times.pop(0)[0])
        return trees

    
    
    
    def wav(self, utterance, start=0, end=None):
        
        wave = import_from_stdlib("wave")

        w = wave.open(self.open(utterance + ".wav"), "rb")

        if end is None:
            end = w.getnframes()

        
        w.readframes(start)
        frames = w.readframes(end - start)

        
        
        tf = tempfile.TemporaryFile()
        out = wave.open(tf, "w")

        
        out.setparams(w.getparams())
        out.writeframes(frames)
        out.close()

        
        
        tf.seek(0)
        return tf.read()

    def audiodata(self, utterance, start=0, end=None):
        assert end is None or end > start
        headersize = 44
        with self.open(utterance + ".wav") as fp:
            if end is None:
                data = fp.read()
            else:
                data = fp.read(headersize + end * 2)
        return data[headersize + start * 2 :]

    def _utterance_fileids(self, utterances, extension):
        if utterances is None:
            utterances = self._utterances
        if isinstance(utterances, str):
            utterances = [utterances]
        return [f"{u}{extension}" for u in utterances]

    def play(self, utterance, start=0, end=None):
        
        
        try:
            import ossaudiodev

            try:
                dsp = ossaudiodev.open("w")
                dsp.setfmt(ossaudiodev.AFMT_S16_LE)
                dsp.channels(1)
                dsp.speed(16000)
                dsp.write(self.audiodata(utterance, start, end))
                dsp.close()
            except OSError as e:
                print(
                    (
                        "can't acquire the audio device; please "
                        "activate your audio device."
                    ),
                    file=sys.stderr,
                )
                print("system error message:", str(e), file=sys.stderr)
            return
        except ImportError:
            pass

        
        try:
            
            import pygame.mixer
            import StringIO

            pygame.mixer.init(16000)
            f = StringIO.StringIO(self.wav(utterance, start, end))
            pygame.mixer.Sound(f).play()
            while pygame.mixer.get_busy():
                time.sleep(0.01)
            return
        except ImportError:
            pass

        
        print(
            ("you must install pygame or ossaudiodev " "for audio playback."),
            file=sys.stderr,
        )


class SpeakerInfo:
    def __init__(
        self, id, sex, dr, use, recdate, birthdate, ht, race, edu, comments=None
    ):
        self.id = id
        self.sex = sex
        self.dr = dr
        self.use = use
        self.recdate = recdate
        self.birthdate = birthdate
        self.ht = ht
        self.race = race
        self.edu = edu
        self.comments = comments

    def __repr__(self):
        attribs = "id sex dr use recdate birthdate ht race edu comments"
        args = [f"{attr}={getattr(self, attr)!r}" for attr in attribs.split()]
        return "SpeakerInfo(%s)" % (", ".join(args))


def read_timit_block(stream):
    
    line = stream.readline()
    if not line:
        return []
    n, sent = line.split(" ", 1)
    return [sent]