|
1 | | -# Introduction |
2 | | - |
3 | | -Official Pytorch implementation for [Deep Contextual Video Compression](https://proceedings.neurips.cc/paper/2021/file/96b250a90d3cf0868c83f8c965142d2a-Paper.pdf), NeurIPS 2021 |
4 | | - |
5 | | -# Prerequisites |
6 | | -* Python 3.8 and conda, get [Conda](https://www.anaconda.com/) |
7 | | -* CUDA 11.0 |
8 | | -* Environment |
9 | | - ``` |
10 | | - conda create -n $YOUR_PY38_ENV_NAME python=3.8 |
11 | | - conda activate $YOUR_PY38_ENV_NAME |
12 | | - |
13 | | - pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html |
14 | | - python -m pip install -r requirements.txt |
15 | | - ``` |
16 | | -
|
17 | | -
|
18 | | -
|
19 | | -# Test dataset |
20 | | -Currenlty the spatial resolution of video needs to be cropped into the integral times of 64. |
21 | | -
|
22 | | -The dataset format can be seen in dataset_config_example.json. |
23 | | -
|
24 | | -For example, one video of HEVC Class B can be prepared as: |
25 | | -* Crop the original YUV via ffmpeg: |
26 | | - ``` |
27 | | - ffmpeg -pix_fmt yuv420p -s 1920x1080 -i BasketballDrive_1920x1080_50.yuv -vf crop=1920:1024:0:0 BasketballDrive_1920x1024_50.yuv |
28 | | - ``` |
29 | | -* Make the video path: |
30 | | - ``` |
31 | | - mkdir BasketballDrive_1920x1024_50 |
32 | | - ``` |
33 | | -* Convert YUV to PNG: |
34 | | - ``` |
35 | | - ffmpeg -pix_fmt yuv420p -s 1920x1024 -i BasketballDrive_1920x1024_50.yuv -f image2 BasketballDrive_1920x1024_50/im%05d.png |
36 | | - ``` |
37 | | -At last, the folder structure of dataset is like: |
38 | | -
|
39 | | - /media/data/HEVC_B/ |
40 | | - * BQTerrace_1920x1024_60/ |
41 | | - - im00001.png |
42 | | - - im00002.png |
43 | | - - im00003.png |
44 | | - - ... |
45 | | - * BasketballDrive_1920x1024_50/ |
46 | | - - im00001.png |
47 | | - - im00002.png |
48 | | - - im00003.png |
49 | | - - ... |
50 | | - * ... |
51 | | - /media/data/HEVC_D |
52 | | - /media/data/HEVC_C/ |
53 | | - ... |
54 | | -
|
55 | | -# Pretrained models |
56 | | -
|
57 | | -* Download CompressAI models |
58 | | - ``` |
59 | | - cd ./checkpoints |
60 | | - python download_compressai_models.py |
61 | | - cd .. |
62 | | - ``` |
63 | | -
|
64 | | -* Download [DCVC models](https://1drv.ms/u/s!AozfVVwtWWYoiS5mcGX320bFXI0k?e=iMeykH) and put them into ./checkpoints folder. |
65 | | -
|
66 | | -# Test DCVC |
67 | | -
|
68 | | -Example of test the PSNR model: |
69 | | -```bash |
70 | | -python test_video.py --i_frame_model_name cheng2020-anchor --i_frame_model_path checkpoints/cheng2020-anchor-3-e49be189.pth.tar checkpoints/cheng2020-anchor-4-98b0b468.pth.tar checkpoints/cheng2020-anchor-5-23852949.pth.tar checkpoints/cheng2020-anchor-6-4c052b1a.pth.tar --test_config dataset_config_example.json --cuda true --cuda_device 0,1,2,3 --worker 4 --output_json_result_path DCVC_result_psnr.json --model_type psnr --recon_bin_path recon_bin_folder_psnr --model_path checkpoints/model_dcvc_quality_0_psnr.pth checkpoints/model_dcvc_quality_1_psnr.pth checkpoints/model_dcvc_quality_2_psnr.pth checkpoints/model_dcvc_quality_3_psnr.pth |
71 | | -``` |
72 | | - |
73 | | -Example of test the MSSSIM model: |
74 | | -```bash |
75 | | -python test_video.py --i_frame_model_name bmshj2018-hyperprior --i_frame_model_path checkpoints/bmshj2018-hyperprior-ms-ssim-3-92dd7878.pth.tar checkpoints/bmshj2018-hyperprior-ms-ssim-4-4377354e.pth.tar checkpoints/bmshj2018-hyperprior-ms-ssim-5-c34afc8d.pth.tar checkpoints/bmshj2018-hyperprior-ms-ssim-6-3a6d8229.pth.tar --test_config dataset_config_example.json --cuda true --cuda_device 0,1,2,3 --worker 4 --output_json_result_path DCVC_result_msssim.json --model_type msssim --recon_bin_path recon_bin_folder_msssim --model_path checkpoints/model_dcvc_quality_0_msssim.pth checkpoints/model_dcvc_quality_1_msssim.pth checkpoints/model_dcvc_quality_2_msssim.pth checkpoints/model_dcvc_quality_3_msssim.pth |
76 | | -``` |
77 | | -It is recommended that the ```--worker``` number is equal to your GPU number. |
78 | | - |
79 | | - |
80 | | -# Acknowledgement |
81 | | -The implementation is based on [CompressAI](https://github.com/InterDigitalInc/CompressAI) and [PyTorchVideoCompression](https://github.com/ZhihaoHu/PyTorchVideoCompression). The model weights of intra coding come from [CompressAI](https://github.com/InterDigitalInc/CompressAI). |
82 | | - |
83 | | -# Citation |
84 | | -If you find this work useful for your research, please cite: |
85 | | - |
86 | | -``` |
87 | | -@article{li2021deep, |
88 | | - title={Deep Contextual Video Compression}, |
89 | | - author={Li, Jiahao and Li, Bin and Lu, Yan}, |
90 | | - journal={Advances in Neural Information Processing Systems}, |
91 | | - volume={34}, |
92 | | - year={2021} |
93 | | -} |
94 | | -``` |
| 1 | +The code has been moved to [https://github.com/microsoft/DCVC](https://github.com/microsoft/DCVC). The new repository also includes our latest neural codec which outperforms H.266(VTM) using the highest compression ratio configuration and also supports smooth rate adjustment in single model. |
0 commit comments