Graphics and Media Lab

MSU Graphics & Media Lab (Video Group)

Algorithm allows to double frame rate (fps) of the video stream. This conversion is performed by calculating and inserting the new frames in the stream.

Task

Frame rate conversion is done as shown on the picture below:

Frame rate conversion scheme

The main advantages of our method are:

• high accuracy of motion tracking and compensation;
• constant temporal sharpness level of the frames in converted video stream.

This is achieved by construction of compensated frames with quarter pixel accuracy. Such high accuracy is obtained by using of bilinear and bicubic interpolation. Second advantage means that interpolated frames have the same sharpness as the original ones. Note that such the feature is not common for the majority of methods of this kind.

Examples

Here we present a results of comparison of described (MSU), Frame Repetition (FR) and Linear Interpolation (LI) methods on "akiyo" sequence. Results include PSNR metric visualization of interpolated frames in comparison to original ones on the right side. Following is the color semantics on these images: red color - high difference region, this difference is visually noticeable; yellow - little less than previous difference; green and blue - almost unnoticeable difference; black - no difference.

Visual comparison of FRC methods

Quality comparison of MSU, LI and FR methods is shown at the graphs below. In vertical direction there is PSNR quality level of interpolated frames in comparison to original ones (the higher values of metric means the higher quality of interpolated frames), in horizontal direction interpolated frame numbers are marked.

Quality graph for sequence BUS	Quality graph for sequence FOREMAN

We compared our FRC implementation with UCSD one. On the pictures below it is easy to see the sharpness difference on frames, which were calculated by increasing FPS four times:

Sharpness comparison of MSU and UCSD methods

Publications pending

D.Vatolin, S.Grishin, "Video Frame Rate Conversion Method Based on Compensated Frames Interpolation", "New Information Technologies in Automated Systems" Seminar at M.V.Keldysh Institute of Applied Mathematics (2006). (Russian)
D.Vatolin, S.Grishin, "Double Up-conversion of Video Frame Rate Based on Two-Dimensional Motion Compensation", "Programming" Magazine (2006). (Russian)

See also

Tool page on the Internet
Video Frame Rate N-Times Up-Conversion Algorithm
Sequence "Dogs" for comparing FR, MSU and UCSD methods, .avi
Sequence "Crowd" for comparing FR, MSU and UCSD methods, .avi

The project team

Project lead:
Dr. Dmitriy Vatolin

Researchers:
Sergey Grishin

Contacts<

e-mail:
video@graphics.cs.msu.ru