Home>

how to record the original video while processing heavy processing on the video in real time with opencv

Currently, we are creating a program that uses python + openCV + pytorch to perform real-time image processing on images from web cameras and save them. However, the image processing by pytorch is very heavy and uses a PC with specs, but FPS is at most around 10.

If possible, I would like to save the raw web camera video before processing, but also save the video after processing. Even if you insert, it takes time to process the image, so you can only save the video in a jumping state. There is no problem with skipping the processed video, but I want to save the original video before processing without any processing failure. What kind of process is possible?

I'm probably wondering if I can do something with multiprocess, but I wasn't very familiar with it and I thought there was a simpler way to solve it.

If i have any detailed information, thank you.

Applicable source code 
def video (output_path = ""):
    vid = cv2.VideoCapture (0, cv2.CAP_DSHOW)
    video_FourCC = cv2.VideoWriter_fourcc (* 'XVID')
    video_fps = vid.get (cv2.CAP_PROP_FPS)
    video_size = 1920, 1080
    isOutput = True if output_path! = "" else False
    if isOutput:
        out = cv2.VideoWriter (output_path, video_FourCC, video_fps, video_size)
    while True:
        return_value, frame = vid.read ()
        result_img = video_process (frame) #Process for the frame (process by pytorch)
        result_add = cv2.hconcat ([frame, result_img])
        cv2.namedWindow ("result", cv2.WINDOW_KEEPRATIO | cv2.WINDOW_NORMAL)
        cv2.imshow ("result", result_add)
        if isOutput:
            out.write (result_img)
        if cv2.waitKey (1)&0xFF == ord ('q'):
            break
if __name__ == '__main__':
    output_dir = "./movie.avi"
    video (output_dir)
Supplemental information (FW/tool version etc.)
  • Windows10 + Anaconda
  • Python 3.6.8
  • opencv-python 4.1.1.26
  • pytorch 1.2.0
python3.x opencv
  • Answer # 1

    It seems that the deep learning process performed by the GPU is heavy, and the elements that can be parallelized by the CPU process are not within the scope of the question code, so the speed will not change even if it is multi-processed.
    I think you should prepare one counter and do heavy processing only once every few frames.

    def video (output_path = ""):
    vid = cv2.VideoCapture (0, cv2.CAP_DSHOW)
    video_FourCC = cv2.VideoWriter_fourcc (* "XVID")
    video_fps = vid.get (cv2.CAP_PROP_FPS)
    video_size = 1920, 1080
    isOutput = True if output_path! = "" else False
    if isOutput:
        out = cv2.VideoWriter (output_path, video_FourCC, video_fps, video_size)
    i = 0
    while True:
        return_value, frame = vid.read ()
        if i% 10 == 0:
            result_img = video_process (frame)
            result_add = cv2.hconcat ([frame, result_img])
        cv2.namedWindow ("result", cv2.WINDOW_KEEPRATIO | cv2.WINDOW_NORMAL)
        cv2.imshow ("result", result_add)
        if isOutput:
            out.write (result_img)
        if cv2.waitKey (1)&0xFF == ord ("q"):
            break
        i + = 1
if __name__ == "__main__":
    output_dir = "./movie.avi"
    video (output_dir)

Related articles