Q:

How to include model.predict into my input pipeline?

The problem:
I'm building a video captioning seq2seq model and I've a problem with the input pipeline:
I'm using a pre-trained InceptionV3 model to preprocess all my data
However, I've got a lot of data: 10.000 videos containing about several hundred frames each.
When I'm using Inceptionv3 to preprocess my data, it returns me a numpy array with a very huge size. 
It's then impossible for me to create a tf.data.Dataset from this numpy array because its size, 6GiB, is way bigger than 2GiB tensorflow's size limit.

My pipeline steps:
Step 1:
Extracting videos and storing frames on the disk. Creating a tf.data.Dataset from this files.
record_files, lowest_n_frames = build_tfrecord_dataset(videos_name, video_path_zip, record_file_path)
records_dataset = tf.data.TFRecordDataset(record_files)

Step 2:
Taking the frames from the TFRecord files created in step 1 and reformatting it for Inceptionv3.
video_dataset = records_dataset.map(lambda tfrecord: decode_tfrecord(tfrecord, n_frames))
video_dataset = video_dataset.map(format_video)

Step 3:
Preprocessing the video frames using Inceptionv3. This returns me a huge numpy array.
bottlenecks = image_features_extract_model.predict(video_dataset, verbose=1, steps=n_samples)

Step 4 (Error step):
I'm creating a tf.data.Dataset from bottlenecks. Of course, it works on a small number of data.
features_dataset = tf.data.Dataset.from_tensor_slices(bottlenecks)

The error is very simple and straightforward:
ValueError: Cannot create a tensor proto whose content is larger than 2GB.

What I would like:
I'm wondering how turning the model.predict operation as a part of my input pipeline !
I don't want to have a 6GiB numpy array in my RAM at a time, I want to do the preprocessing "step by step" over my data

A:

When your data gets big, its smarter to use tf.data.Dataset.from_generator
I don't know what your exact code and outputs look like but it should work if you try something like this
features_dataset = tf.data.Dataset.from_generator(iter(bottlenecks),
                                                  output_types=...,
                                                  output_shapes=...,)

