<commit_before>'''

1) Create a new collection in the mongo shell:

db.tracking_collection.aggregate([{$match :{$or : [{"event_type" : "seek_video"}, {"event_type" : "play_video"},{"event_type":"pause_video"}]}}, {$out : "video_watch_duration_collection"}])

2) Run: python video_watch_duration.py


desired final output.csv

username, video, chapter, vertical, sequential, watch duration

get the event_types : play_video, pause_video, seek_video

watch duration should include ONLY:

- time between play_video -> another video event (pause_video or seek_video)
- time between seek_video : {'new_time' : Time}  -> pause video (only with new_time > old_time, this is to avoid including rewinds)
- 



play_video(when user presses play on video console, may not be 0) -> pause video time
watch periods:
    event_type : pause_video - "event_type":"play_video" {"event":{"currentTime":TIME}} = add to watch duration
    if seek_video : {'old_time' : TIME} > seek_video : {'new_time' : TIME} = rewind (exclude from watch duration)<commit_msg>Update video_watch_duration.py<commit_after>'''

1) Create a new collection in the mongo shell:

db.tracking_collection.aggregate([{$match :{$or : [{"event_type" : "seek_video"}, {"event_type" : "play_video"},{"event_type":"pause_video"}]}}, {$out : "video_watch_duration_collection"}])

2) Run: python video_watch_duration.py


desired final output.csv

username, video, chapter, vertical, sequential, watch duration

get the event_types : play_video, pause_video, seek_video

watch duration should include ONLY:

- time between play_video -> another video event (pause_video or seek_video)
- time between seek_video : {'new_time' : Time}  -> pause video (only with new_time > old_time, this is to avoid including rewinds)

watch periods:
    event_type : pause_video - "event_type":"play_video" {"event":{"currentTime":TIME}} = add to watch duration
    if seek_video : {'old_time' : TIME} > seek_video : {'new_time' : TIME} = rewind (exclude from watch duration)
{'fraction_non_alphanumeric': 0.12325349301397205, 'fraction_numerical': 0.00249500998003992, 'mean_word_length': 5.639072847682119, 'pattern_counts': {'":': 8, '<': 3, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '400420', 'n_tokens_mistral': 621, 'n_tokens_neox': 601, 'n_words': 196}