This application relates to China Patent Application No. 202010836914.7, filed on Aug. 19, 2020, which is hereby incorporated by reference in its entirety.

FIELD OF THE INVENTION

The invention relates to computer vision generally and, more particularly, to a method and/or apparatus for implementing an event/object-of-interest centric timelapse video generation on camera device with the assistance of neural network input.

BACKGROUND

A timelapse video mode for conventional internet-connected/cloud-enabled cameras is usually implemented by software operated on a cloud server. The software relies on using the resources of distributed processing of the cloud server (i.e., scalable computing). Timelapse video clips are displayed at a fixed frame rate for a fast-forward effect. For example, video frames are selected at fixed intervals to create the timelapse video (i.e., every thirtieth frame is selected from a thirty frames per second video to create the timelapse video).

Internet-connected/cloud-enabled cameras encode video data and then communicate the encoded video streams to the cloud servers. In order to transcode the encoded video streams into a timelapse video, the cloud server has to first decode the encoded video stream. The cloud servers spend lots of CPU cycles to decode the conventional video streams (i.e., compressed video using AVC or HEVC encoding), extract the video frames at fixed frame intervals, then transcode these frames into a timelapse video. Meanwhile, even with a timelapse video, users have difficulty finding important details captured by the internet-connected/cloud-enabled cameras. Since a timelapse video always uses a fixed frame rate that does not use all the video data originally captured by the internet-connected/cloud-enabled, video frames at a normal display speed are not available for the entire duration of time that something of interest to the user is in the original video captured. For example, when a security camera observes a potential event of interest, such as a person on the premises, the timelapse will be the same as when the security camera observes nothing of particular interest.

It would be desirable to implement an event/object-of-interest centric timelapse video generation on camera device with the assistance of neural network input.

SUMMARY

The invention concerns an apparatus comprising an interface and a processor. The interface may be configured to receive pixel data generated by a capture device. The processor may be configured to receive the pixel data from the interface, generate video frames in response to the pixel data, perform computer vision operations on the video frames to detect objects, perform a classification of the objects detected based on characteristics of the objects, determine whether the classification of the objects corresponds to a user-defined event and generate encoded video frames from the video frames. The encoded video frames may be communicated to a cloud storage service. The encoded video frames may comprise a first sample of the video frames selected at a first rate when the user-defined event is not detected and a second sample of the video frames selected at a second rate while the user-defined event is detected. The second rate may be greater than the first rate.

