<jupyter_start><jupyter_text>Video compression According to the dataset paper, > However, for each target video in the test set, we randomly selected two clips out of three and applied augmentations thatapproximate actual degradations seen in real-life video distributions. Specifically, these augmentations were> 1. reduce the FPS of the video to 15> 2. reduce the resolution of the video to 1/4 of its original size> 3. reduce the overall encoding quality In this notebook we look at how to compress videos using `ffmpeg`.  Based on the above list, a quick search returns several useful examples:* General compression for the web 1: https://dev.to/benjaminblack/use-ffmpeg-to-compress-and-convert-videos-458l  * General compression for the web 2: https://trac.ffmpeg.org/wiki/Encode/YouTube   * Frame rate change: https://trac.ffmpeg.org/wiki/ChangingFrameRate  * Scaling: https://trac.ffmpeg.org/wiki/Scaling  The latest `ffmpeg` can be obtained at https://johnvansickle.com/ffmpeg/.  See the FAQ for how to install.<jupyter_code>#default_exp video_compress
#export
from IPython.display import HTML
import cv2
from fastai.core import *
from fastai.vision import *
from kgl_deepfake.data import *
SOURCE = Path('../data/dfdc_train_part_0/')
fnames = get_files(SOURCE, extensions=['.mp4'])
fn = fnames[159]
fn
HTML(html_vid(fn))<jupyter_output><empty_output><jupyter_text>Use `mediainfo` to show video properties<jupyter_code>! mediainfo {fn}<jupyter_output>General
Complete name                            : ../data/dfdc_train_part_0/bfktkdugru.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 15.2 MiB
Duration                                 : 10 s 24 ms
Overall bit rate                         : 12.7 Mb/s
Writing application                      : Lavf57.71.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration[...]<jupyter_text>File size To conveniently check the file size of compressed videos:<jupyter_code>#export
def get_file_size(fpath):
    assert fpath.is_file()
    p = subprocess.run(f"du -hs {fpath}".split(), stdout=subprocess.PIPE)
    return p.stdout.decode().split('\t')[0]
#export
Path.file_size = get_file_size<jupyter_output><empty_output><jupyter_text>Original file size<jupyter_code>fn.file_size()<jupyter_output><empty_output><jupyter_text>Decorator to run command in terminal<jupyter_code>#export
def runnit(f):
    def _func(*args, **kwargs):
        command = f(*args, **kwargs)
        p = subprocess.run(command.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        return p
    return _func<jupyter_output><empty_output><jupyter_text>Are these videos HDR? The following checks whether the video is HDR or not.<jupyter_code>#export
@runnit
def vid_colour_primaries(fpath):
    "Use `mediainfo` to get video's colour primaries."
    return f'''
    mediainfo f'{fpath}' --Inform="Video;%colour_primaries%"
    '''

def vid_is_HDR(fpath):
    "Is video HDR or not?"
    p = vid_colour_primaries(fpath)
    res = p.stdout.decode()
    return True if 'BT.2020' in res else False
vid_is_HDR(fn)<jupyter_output><empty_output><jupyter_text>Show video properties using `mediainfo`<jupyter_code>#export
def get_mediainfo(fpath=None): 
    @runnit
    def _func(fpath=None):
        return f'''mediainfo {fpath}'''
    p = _func(fpath=fpath)
    return p.stdout.decode()<jupyter_output><empty_output><jupyter_text>Encoding Each of the URLs above does something different: encoding, scaling, fps.  These are combined into a single function.  Note that probably not all combinations of input parameters will run, but the main parameters to adjust are:* **crf** "The range of the CRF scale is 0–51, where 0 is lossless, 23 is the default, and 51 is worst quality possible. A lower value generally leads to higher quality, and a subjectively sane range is 17–28. Consider 17 or 18 to be visually lossless or nearly so; it should look the same or nearly the same as the input but it isn't technically lossless." > 23 recommended.* **fps** "Changing frame rates requires the video to be re-encoded. Without setting appropriate output quality or bit rate, the video quality may be degraded. Please look at the respective encoding guides for the codec you've chosen."* **scale** Value between 0 and 1 to scale the original lengths. To display the video and its properties after it's been compressed<jupyter_code>#export
def show_vid_info(f):
    "Display video and its properties after it's been processed."
    def _f(*args, **kwargs):
        p = f(*args, **kwargs)
        fpath = Path(kwargs['fpath_to'])
        display(HTML(html_vid(fpath)))
        print(get_mediainfo(fpath))
    return _f<jupyter_output><empty_output><jupyter_text>Function for donig the encoding using `ffmpeg`, with default arguments defined.<jupyter_code>#export
def _ffmpeg_web_defaults():
    '''
    These are some reasonable values for uploading. i.e. YouTube, etc.
    '''
    return dict(video_encoder='libx264', video_bitrate='1.5M', fps=30, scale=.5, crf=23, #17-28
                audio_encoder='aac', audio_bitrate='128k')

def _ffmpeg_defaults():
    return dict(video_encoder=None, video_bitrate=None, fps=None, scale=None, crf=None,
                audio_encoder=None, audio_bitrate=None)

def _ffmpeg_fmts():
    "ffmpeg options syntax"
    return dict(video_encoder='-c:v {video_encoder:s}', 
                video_bitrate='-b:v {video_bitrate:s}', 
                fps='-r {fps:d}', 
                scale='-vf scale=iw*{scale:.2f}:ih*{scale:.2f}', 
                crf='-crf {crf:d}',
                audio_bitrate='-b:a {audio_bitrate}', 
                audio_encoder='-c:a {audio_encoder}')

#@show_vid_info
@runnit
def run_ffmpeg(fpath_from=None, fpath_to=None, **kwargs):
    '''
    Run ffmpeg
    '''
    ps = _ffmpeg_defaults()
    ps.update(kwargs)
    pstr = []
    for n, s in _ffmpeg_fmts().items():
        if ps[n] is None: pstr.append('')
        else: pstr.append(s.format(**ps))
    pstr = ' '.join(pstr)
    return f'ffmpeg -i {fpath_from} {pstr} {fpath_to}'<jupyter_output><empty_output><jupyter_text>Apply to the original (uncomment the `@show_vid_info` above `run_ffmpeg` to display legible output in the next cell):<jupyter_code>f = Path(f'enc_{fn.name}')
if f.exists(): os.remove(f)
run_ffmpeg(fpath_from=fn, fpath_to=f, crf=28)<jupyter_output><empty_output><jupyter_text>Double check the properties using `cv2`<jupyter_code>vcap = cv2.VideoCapture(str(f))
vlen = int(vcap.get(cv2.CAP_PROP_FRAME_COUNT))
ret, frame = vcap.read()
vcap.release()
vlen, frame.shape<jupyter_output><empty_output><jupyter_text>Generate compressed sets Use this section to generate sets of compressed videos.<jupyter_code>SOURCE = Path('../data')
DFDC = 'dfdc_train_part_35'
ENC_DICT = dict(fps15={'fps':15}, quartersize={'scale':.5}, crf28={'crf':28})
#export
def origfake_exist(row):
    fake_exists = row.fname.exists() 
    orig_exists = True if pd.isna(row.original) else row.original.exists()
    return (fake_exists and orig_exists)

def _cp_fname_original(r): 
    r.original = r.fname
    return r

def load_dfdc_json(path, dfdc, fpath=False, drop_missing=False, fill_real_na=False):
    a = get_files(path/dfdc, extensions=['.json'])[0]
    a = pd.read_json(a).T
    a.reset_index(inplace=True)
    a.rename({'index':'fname'}, axis=1, inplace=True)
    if fpath or drop_missing:
        a.fname = a.fname.apply(lambda o: path/dfdc/o)
        a.original = a.original.apply(lambda o: np.nan if pd.isna(o) else path/dfdc/o)
    if drop_missing: 
        exists = a.apply(origfake_exist, axis=1)
        a = a[exists].reset_index(drop=True)
    if fill_real_na:
        a[a.label=='REAL'] = a[a.label=='REAL'].apply(_cp_fname_original, axis=1)
    return a
vant = load_dfdc_json(SOURCE, DFDC, fpath=True, drop_missing=True, fill_real_na=True)
vant.shape<jupyter_output><empty_output><jupyter_text>Define how to compress the mp4s in `DFDC`.  The following functions define different ways of splitting the mp4s among differnt compression schemes.<jupyter_code>#export
def get_cvant_split(vant:pd.DataFrame, enc:str='fps15'):
    '''
    Split `vant` into three equal parts by its ORIGINALs. Then, return:
    1st third, if enc='fps15'
    2nd third, if enc='quartersize'
    3rd third, if enc='crf28'
    
    Parameter
    ---------
    vant: video annotation
    '''
    encs = ['fps15', 'quartersize', 'crf28']
    fnos = np.array(sorted(vant.original.unique(), key=lambda o: str(o)))
    idxss = torch.arange(len(fnos)).chunk(3)
    ichunk = encs.index(enc)
    fnos = fnos[idxss[ichunk]]
    vant = vant.set_index('original').loc[fnos].reset_index()
    vant['enc'] = enc
    return vant


def get_cvant_paper(vant:pd.DataFrame, fnos:list):
    '''
    Get compressed video annotation, `cvant`, according to 
    the DFDC paper.
    vant: video annotation
    fnos: iterable of original video names
    '''
    encs = list(ENC_DICT.keys()) + ['None']
    idxs = torch.randperm(len(fnos))
    chidxs = torch.chunk(idxs, len(encs))
    vantps = []
    for enc, idxs in zip(encs, chidxs):
        vantp = vant.set_index('original').loc[fnos[idxs]]
        vantp['enc'] = enc
        vantp = vantp.reset_index()
        vantps.append(vantp)
    cvant = pd.concat(vantps)
    return cvant
enc = 'quartersize' # 'fps15', 'quartersize', 'crf28'
cvant = get_cvant_split(vant, enc=enc)
# Because of a previous bug, ORIGINALs were not compressed.
# Let's compress them now
cvant = cvant[cvant.label=='REAL']<jupyter_output><empty_output><jupyter_text>Define directory path to save the compressed videos.<jupyter_code>dir_out = SOURCE/f'{DFDC}_{enc}'
#dir_out = SOURCE/f'{DFDC}_cv'
dir_out
cvant.shape
#export
def compress_vids(cvant, dir_out='compressed_videos'):
    '''
    Compress videos, save them and the annotation.
    '''
    dir_out = Path(dir_out)
    os.makedirs(dir_out, exist_ok=True)
    cvant['fname_cv'] = cvant.fname.apply(lambda o: dir_out/o.name)
    pb = progress_bar(list(cvant.iterrows()))
    for _, r in pb:
        if r.enc=='None': 
            shutil.copy(r.fname, r.fname_cv)
        else:
            run_ffmpeg(fpath_from=r.fname, fpath_to=r.fname_cv, **ENC_DICT[r.enc])
        pb.comment = f'{r.fname.name} {r.enc}'
    cvant.to_csv(dir_out/'metadata.csv', index=False)
    return cvant<jupyter_output><empty_output><jupyter_text>**Remember to comment out the `@show_vid_info` to avoid lots of output printed in the next cell.**<jupyter_code>cvant = compress_vids(cvant, dir_out)<jupyter_output><empty_output><jupyter_text>-fin<jupyter_code>from nbdev.export import *
notebook2script()<jupyter_output>Converted 00_data.ipynb.
Converted 00a_video_compression-Copy1.ipynb.
Converted 00a_video_compression.ipynb.
Converted 01_face_detection.ipynb.
Converted 01a_faces_probs_examples.ipynb.
Converted 01a_faces_probs_examples_hv.ipynb.
Converted 02_fix_luminosity.ipynb.
Converted 02a_create_faceimage_dataset.ipynb.
Converted 02bis_Create_Dataset.ipynb.
Converted 02c_faces_different_dfdc_zips.ipynb.
Converted 03_models.ipynb.
Converted 04_Baseline_Classification.ipynb.
Converted 04_Classification.ipynb.
Converted 04a_classification_videolist.ipynb.
Converted 04b_inference.ipynb.
Converted 05_Class_Imbalance.ipynb.
Converted 06_Focal_Loss.ipynb.
Converted 07_full_classification.ipynb.
This cell doesn't have an export destination and was ignored:
e
Converted 07a_classify_video_margin.ipynb.
Converted 07b_classify_resize.ipynb.
Converted 08_Validation.ipynb.
Converted 09_Data_Augmentation.ipynb.
Converted 09b_Albumentation.ipynb.
This cell doesn't have an export destination and was ignored:
e
T[...]
{'fraction_non_alphanumeric': 0.11104736928372398, 'fraction_numerical': 0.01466972627438125, 'mean_word_length': 3.645222687476209, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 68, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '578991', 'license': 'Apache-2.0', 'n_tokens_mistral': 4341, 'n_tokens_neox': 4117, 'n_words': 1026, 'path': 'nbs/00a_video_compression.ipynb', 'repo_name': 'qAp/kgl_deepfake', 'size': '12202'}