catalogue
- catalogue
- Application platform
- Screen recording section
- Calculate the optimal fps of video and use numpy to calculate the intermediate frame array
- Using pynput to monitor keyboard keys
- How to save MP4 format video
- Source code
- summary
Recently, someone was using the screen recording software to record the desktop. He had a whim in the process of using it. Can he use python as a screen recording tool and exercise his hands-on ability. Next, I'm going to write a series of articles on how to use python as a screen recording tool:
- Recording screen making video
- Record audio
- Synthetic video, audio
- Making visual window based on pyqt5
About the above four parts, I hope I can improve them as soon as possible. Next, I begin to use python to make the screen recording part.
Application platform
- windows 10
- python 3.7
Screen recording section
Screen recording can be simply understood as playing the screen shot in the form of a moving picture. Here, I choose ImageGrab under PIL to capture the screen picture. First\
pip install Pillow\
After that, you need to combine the captured snapshots into video, and use cv2 module\
pip install opencv-python\
ImageGrab class cannot be directly stored as video. Use numpy module for array, and then pass cv2 COLOR_ Bgr2rgb is converted to cv2 color channel.
pip install numpy
Screen recording main code:
import numpy as np
from PIL import ImageGrab
import cv2
im = ImageGrab.grab()
width, high = im.size #Gets the width and height of the screen
fourcc = cv2.VideoWriter_fourcc(*'I420') #Set video encoding format
fps = 15 #Set frame rate
video = cv2.VideoWriter('test.avi', fourcc, fps, (width, high))
while True: #Start recording
im = ImageGrab.grab()
im_cv = cv2.cvtColor(np.array(im), cv2.COLOR_BGR2RGB)
#Image writing
video.write(im_cv)
if xx: #When certain conditions are met, interrupt the cycle
break
video.release() #Release cache and persist videoThe test run can save screen shots as videos, but the operation is not elegant and is not conducive to subsequent operations.
Encapsulated into a class and inheriting the thread parent class, it is convenient to use the keyboard to control the end of video recording.
from threading import Thread class ScreenshotVideo(Thread): def __init__(self): """Initialization parameters""" super().__init__()
The detailed code will be given at the end of the text.
Calculate the optimal fps of video and use numpy to calculate the intermediate frame array
In practice, video recording will have different frame rates in different computers, resulting in fast or slow video playback. It is necessary to calculate the corresponding optimal fps value according to different computers.
def video_best_fps(self, path): """Get the best frame rate for computer recorded video""" video = cv2.VideoCapture(path) #Read video fps = video.get(cv2.CAP_PROP_FPS) #Gets the frame rate of the current video count = video.get(cv2.CAP_PROP_FRAME_COUNT) #Get the number of video frames, that is, how many pictures the video has self.best_fps = int(fps * ((int(count) / fps) / self.spend_time)) #The optimal frame rate is obtained by comparing the playback time with the recording time video.release()
Then adjust the frame rate parameter to record the video, which will weaken the video playing too fast or too slow. You can also add frames to the video to prolong the playback time. Here I use a very simple method to add video frames for reference only.
from numba import jit #numpy is used to calculate the images of two adjacent frames and closer to the next frame #Call jit method to accelerate array calculation @jit(nopython=True) def average_n(x, y): """Numpy Calculate approach value""" return ((x + y + y) // 3).astype(x.dtype)
This method only aims at the video impression after processing when the set fps is higher than the optimal fps, but the detailed frames increase, so the playback time will be longer than that before processing, with slight residual shadow.
Using pynput to monitor keyboard keys
During video recording, we don't know when the video ends, so we wrap the recording code with a while loop, but it's impossible to let the code run endlessly. Here, we use the monitor keyboard module to interrupt the operation of the recording code.
from pynput import keyboard # pip install pynput def hotkey(self): """Hotkey monitor""" with keyboard.Listener(on_press=self.on_press) as listener: listener.join() def on_press(self, key): try: if key.char == 't': #After screen recording, save the video self.flag = True elif key.char == 'k': #Screen recording aborted, delete file self.flag = True self.kill = True except Exception as e: print(e)
When the "T" key on the keyboard is pressed, the recording ends and the video is saved. The "K" key stops recording and deletes the cache file.
How to save MP4 format video
The video encoding format should be ('a ',' V ',' C ',' 1 ') and the file suffix should be' mp4 ', go before recording https://github.com/cisco/open... Download the DLL of the corresponding platform Bz2 file, unzip the compressed package and put it under the project folder. Run the code again, and a line of coding description will appear after success:
OpenH264 Video Codec provided by Cisco Systems, Inc.
Source code
The source code of this paper is as follows:
import time
from PIL import ImageGrab
import cv2
from pathlib import Path
import numpy as np
from numba import jit
from pynput import keyboard
from threading import Thread
@jit(nopython=True)
def average_n(x, y):
"""Numpy Calculate approach value"""
return ((x + y + y) // 3).astype(x.dtype)
class ScreenshotVideo(Thread):
def __init__(self, width, high, path='', fps=15):
"""Initialization parameters"""
super().__init__()
self.save_file = path
self.best_fps = fps
self.fps = fps
self.width = width
self.high = high
self.spend_time = 1
self.flag = False
self.kill = False
self.video = None
def __call__(self, path):
"""Overload the video path to facilitate the secondary call of the class"""
self.save_file = Path(path)
self.video = self.init_videowriter(self.save_file)
@staticmethod
def screenshot():
"""Static method, screenshot, and convert to np.array array"""
return np.array(ImageGrab.grab())
@staticmethod
def get_fourcc(name):
"""Video coding dictionary"""
fourcc_maps = {'.avi': 'I420',
'.m4v': 'mp4v',
'.mp4': 'avc1',
'.ogv': 'THEO',
'.flv': 'FLV1',
}
return fourcc_maps.get(name)
def init_videowriter(self, path):
"""Get video encoding and create a new video file"""
if not path:
raise Exception('Video path is not set, please set\nvideo = ScreenshotVideo(fps,width,high)\nvideo = video(video_path)')
path = Path(path) if isinstance(path, str) else path
fourcc = cv2.VideoWriter_fourcc(*self.get_fourcc(path.suffix))
return cv2.VideoWriter(path.as_posix(), fourcc, self.fps, (self.width, self.high))
def video_record_doing(self, img):
"""take BGR Convert array to RGB array"""
im_cv = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
self.video.write(im_cv)
def video_record_end(self):
"""After recording, judge whether to save the file according to the conditions"""
self.video.release()
cv2.destroyAllWindows()
if self.save_file and self.kill:
Path(self.save_file).unlink()
def video_best_fps(self, path):
"""Get the best frame rate for computer recorded video"""
video = cv2.VideoCapture(path)
fps = video.get(cv2.CAP_PROP_FPS)
count = video.get(cv2.CAP_PROP_FRAME_COUNT)
self.best_fps = int(fps * ((int(count) / fps) / self.spend_time))
video.release()
def pre_video_record(self):
"""Pre record for best fps value"""
self.video = self.init_videowriter('test.mp4')
start_time = time.time()
for _ in range(10):
im = self.screenshot()
self.video_record_doing(im)
self.spend_time = round(time.time() - start_time, 4)
self.video_record_end()
time.sleep(2)
self.video_best_fps('test.mp4')
Path('test.mp4').unlink()
def insert_frame_array(self, frame_list):
"""Numpy Enhanced screenshot information"""
fps_n = round(self.fps / self.best_fps)
if fps_n <= 0:
return frame_list
times = int(np.log2(fps_n)) #Magnification
for _ in range(times):
frame_list2 = map(average_n, [frame_list[0]] + frame_list[:-1], frame_list)
frame_list = [[x, y] for x, y in zip(frame_list2, frame_list)]
frame_list = [j for i in frame_list for j in i]
return frame_list
def frame2video_run(self):
"""use opencv Convert continuous screenshots to video"""
self.video = self.init_videowriter(self.save_file)
start_time = time.time()
frame_list = []
while True:
frame_list.append(self.screenshot())
if self.flag:
break
self.spend_time = round(time.time() - start_time, 4)
if not self.kill: #Video recording is not terminated. Images are processed frame by frame
frame_list = self.insert_frame_array(frame_list)
for im in frame_list:
self.video_record_doing(im)
self.video_record_end()
def hotkey(self):
"""Hotkey monitor"""
with keyboard.Listener(on_press=self.on_press) as listener:
listener.join()
def on_press(self, key):
try:
if key.char == 't': #After screen recording, save the video
self.flag = True
elif key.char == 'k': #Screen recording aborted, delete file
self.flag = True
self.kill = True
except Exception as e:
print(e)
def run(self):
#Run function
#Set daemon thread
Thread(target=self.hotkey, daemon=True).start()
#Run screenshot function
self.frame2video_run()
screen = ImageGrab.grab()
width, high = screen.size
video = ScreenshotVideo(width, high, fps=60)
video.pre_video_record() #Pre recording for optimal fps
video('test1.mp4')
video.run()summary
In this paper, opencv and related modules are used to record the screen and convert it into video to save, and learn to package multiple functions into classes to facilitate subsequent function development. The road of learning is endless. Take bold steps!
Guys, practice it quickly! If you encounter any problems during the learning process, please pay attention to me. I'll pull you into the Python learning exchange group to discuss learning together.