<commit_before>---
layout:     post
title:      "Video from subtitles or <NAME> to The Simpsons with TensorFlow"
date:       2018-08-30 00:40:00
keywords:   python, tensorflow, nlp, subtitles
image:      /assets/bobs_burgers_to_simpsons.jpg
---

![<NAME> to The Simpsons](/assets/bobs_burgers_to_simpsons.jpg)

Back in June I've [played a bit with subtitles and tried to generate a filmstrip](/2018/06/19/stock-film-strip/),
it wasn't that much successful, but it was fun. So I decided to try to go deeper and generate a video from
subtitles. The main idea is to get phrases from a part of some vide, get the most similar phrases from
another video and generate something.

As the "enemy" I've decided to use [a part from Bob's Burgers Tina-rannosaurus Wrecks episode](https://www.youtube.com/watch?v=hZ_EKHGgWJQ):
<iframe width="766" height="490" src="https://www.youtube.com/embed/hZ_EKHGgWJQ?enablejsapi=1&showinfo=0" frameborder="0" allowfullscreen></iframe>

As the source, I've decided to use The Simpsons, as they have a lot of episodes and *Simpsons Already Did It* whatever.
I somehow have 671 episode and managed to get perfectly matching subtitles for 452 of them.

**TLDR:** It was fun, but the result is meh at best:
<iframe width="766" height="490" src="https://www.youtube.com/embed/fBdfVw2oT7I?enablejsapi=1&showinfo=0" frameborder="0" allowfullscreen></iframe>

Initially, I was planning to use Friends and Seinfeld but the result was even worse.

As the first step I've parsed subtitles (boring, available in [the gist](https://gist.github.com/nvbn/f1365d2548f48fad449bb66d650ad95f)) and created a mapping from phrases and
"captions" (subtitles parts with timing and additional data) and a list of phrases from all available subtitles:

~~~python
data_text2captions = defaultdict(lambda: [])
for season in root.glob('*'):
    if season.is_dir():
        for subtitles in season.glob('*.srt'):
            for caption in read_subtitles(subtitles.as_posix()):
                data_text2captions[caption.text].append(caption)

data_texts = [*data_text2captions]
~~~
~~~python
>>> data_text2captions["That's just a dog in a spacesuit!"]
[Caption(path='The Simpsons S13E06 She of Little Faith.srt', start=127795000, length=2544000, text="That's just a dog in a spacesuit!")]<commit_msg>#N/A: Fix typo<commit_after>---
layout:     post
title:      "Video from subtitles or <NAME> to The Simpsons with TensorFlow"
date:       2018-08-30 00:40:00
keywords:   python, tensorflow, nlp, subtitles
image:      /assets/bobs_burgers_to_simpsons.jpg
---

![<NAME> to The Simpsons](/assets/bobs_burgers_to_simpsons.jpg)

Back in June I've [played a bit with subtitles and tried to generate a filmstrip](/2018/06/19/stock-film-strip/),
it wasn't that much successful, but it was fun. So I decided to try to go deeper and generate a video from
subtitles. The main idea is to get phrases from a part of some video, get the most similar phrases from
another video and generate something.

As the "enemy" I've decided to use [a part from Bob's Burgers Tina-rannosaurus Wrecks episode](https://www.youtube.com/watch?v=hZ_EKHGgWJQ):
<iframe width="766" height="490" src="https://www.youtube.com/embed/hZ_EKHGgWJQ?enablejsapi=1&showinfo=0" frameborder="0" allowfullscreen></iframe>

As the source, I've decided to use The Simpsons, as they have a lot of episodes and *Simpsons Already Did It* whatever.
I somehow have 671 episode and managed to get perfectly matching subtitles for 452 of them.

**TLDR:** It was fun, but the result is meh at best:
<iframe width="766" height="490" src="https://www.youtube.com/embed/fBdfVw2oT7I?enablejsapi=1&showinfo=0" frameborder="0" allowfullscreen></iframe>

Initially, I was planning to use Friends and Seinfeld but the result was even worse.

As the first step I've parsed subtitles (boring, available in [the gist](https://gist.github.com/nvbn/f1365d2548f48fad449bb66d650ad95f)) and created a mapping from phrases and
"captions" (subtitles parts with timing and additional data) and a list of phrases from all available subtitles:

~~~python
data_text2captions = defaultdict(lambda: [])
for season in root.glob('*'):
    if season.is_dir():
        for subtitles in season.glob('*.srt'):
            for caption in read_subtitles(subtitles.as_posix()):
                data_text2captions[caption.text].append(caption)

data_texts = [*data_text2captions]
~~~
~~~python
>>> data_text2captions["That's just a dog in a spacesuit!"]
[Caption(path='The Simpsons S13E06 She of Little Faith.srt', start=127795000, length=2544000, text="That's just a dog in a spacesuit!")]
{'fraction_non_alphanumeric': 0.10673913043478261, 'fraction_numerical': 0.04, 'mean_word_length': 4.75125, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 21, 'https://': 8, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3007366', 'n_tokens_mistral': 1654, 'n_tokens_neox': 1499, 'n_words': 523}