18、ubutnu20.04使用RTX3060硬件进行视频编解码+推流
基本思想：先进行环境配置，系统环境如下
RTX3060
ubuntu 20.04
cuda 12.2
cudnn 11.3
NVIDIA-SMI 535.146.02
tensorRT 8.6
nv-codec-headers
Video_Codec_SDK_12.0.16
注意：发现一个很有意思的问题，如果我使用命令行安装的ffmpeg,似乎已经有了nvidia编码器和解码器，完全不需要下面的操作似乎，我用代码也可以调用使用，这是没安装之前的环境配置
ubuntu@ubuntu:~$ ffmpeg -codecs | grep cuvid
ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid ) (encoders: libx264 libx264rgb h264_nvenc h264_omx h264_v4l2m2m h264_vaapi nvenc nvenc_h264 )
DEV.L. hevc H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m hevc_cuvid ) (encoders: libx265 nvenc_hevc hevc_nvenc hevc_v4l2m2m hevc_vaapi )
DEVIL. mjpeg Motion JPEG (decoders: mjpeg mjpeg_cuvid ) (encoders: mjpeg mjpeg_vaapi )
DEV.L. mpeg1video MPEG-1 video (decoders: mpeg1video mpeg1_v4l2m2m mpeg1_cuvid )
DEV.L. mpeg2video MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m mpeg2_cuvid ) (encoders: mpeg2video mpeg2_vaapi )
DEV.L. mpeg4 MPEG-4 part 2 (decoders: mpeg4 mpeg4_v4l2m2m mpeg4_cuvid ) (encoders: mpeg4 libxvid mpeg4_v4l2m2m )
D.V.L. vc1 SMPTE VC-1 (decoders: vc1 vc1_v4l2m2m vc1_cuvid )
DEV.L. vp8 On2 VP8 (decoders: vp8 vp8_v4l2m2m libvpx vp8_cuvid ) (encoders: libvpx vp8_v4l2m2m vp8_vaapi )
DEV.L. vp9 Google VP9 (decoders: vp9 vp9_v4l2m2m libvpx-vp9 vp9_cuvid ) (encoders: libvpx-vp9 vp9_vaapi )
ubuntu@ubuntu:~$ ffmpeg -codecs | grep nvenc
ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid ) (encoders: libx264 libx264rgb h264_nvenc h264_omx h264_v4l2m2m h264_vaapi nvenc nvenc_h264 )
DEV.L. hevc H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m hevc_cuvid ) (encoders: libx265 nvenc_hevc hevc_nvenc hevc_v4l2m2m hevc_vaapi )这是配置之后的环境变化
ubuntu@ubuntu:~$ ffmpeg -codecs | grep cuvid
ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libfreetype --enable-libmp3lame --enable-libvpx --enable-libx264 --enable-libx265 --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-ffmpeg --enable-ffplay --enable-ffprobe --enable-libx264 --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
DEV.L. av1 Alliance for Open Media AV1 (decoders: av1 av1_cuvid) (encoders: av1_nvenc)
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid) (encoders: libx264 libx264rgb h264_nvenc h264_v4l2m2m)
DEV.L. hevc H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m hevc_cuvid) (encoders: libx265 hevc_nvenc hevc_v4l2m2m)
DEVIL. mjpeg Motion JPEG (decoders: mjpeg mjpeg_cuvid)
DEV.L. mpeg1video MPEG-1 video (decoders: mpeg1video mpeg1_v4l2m2m mpeg1_cuvid)
DEV.L. mpeg2video MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m mpeg2_cuvid)
DEV.L. mpeg4 MPEG-4 part 2 (decoders: mpeg4 mpeg4_v4l2m2m mpeg4_cuvid) (encoders: mpeg4 mpeg4_v4l2m2m)
D.V.L. vc1 SMPTE VC-1 (decoders: vc1 vc1_v4l2m2m vc1_cuvid)
DEV.L. vp8 On2 VP8 (decoders: vp8 vp8_v4l2m2m libvpx vp8_cuvid) (encoders: libvpx vp8_v4l2m2m)
DEV.L. vp9 Google VP9 (decoders: vp9 vp9_v4l2m2m libvpx-vp9 vp9_cuvid) (encoders: libvpx-vp9)
ubuntu@ubuntu:~$ ffmpeg -codecs | grep nvenc
ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libfreetype --enable-libmp3lame --enable-libvpx --enable-libx264 --enable-libx265 --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-ffmpeg --enable-ffplay --enable-ffprobe --enable-libx264 --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
DEV.L. av1 Alliance for Open Media AV1 (decoders: av1 av1_cuvid) (encoders: av1_nvenc)
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid) (encoders: libx264 libx264rgb h264_nvenc h264_v4l2m2m)
DEV.L. hevc H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m hevc_cuvid) (encoders: libx265 hevc_nvenc hevc_v4l2m2m)
所以搜索到类似的ubuntu 集成编译ffmpeg似乎没有必要，有点懵，且并不妨碍代码使用nvidia去解码和编码测试，所以下面编译方式可以直接忽视似乎
一、进行nv-codec-headers配置
ubuntu@ubuntu:~/nv-codec-headers$ cat README
FFmpeg version of headers required to interface with Nvidias codec APIs.
Corresponds to Video Codec SDK version 12.0.16.
Minimum required driver versions:
Linux: 530.41.03 or newer
Windows: 531.61 or newer
ubuntu@ubuntu:~/nv-codec-headers$ sudo make install
ubuntu@ubuntu:~/nv-codec-headers$ sudo vim ~/.bashrc 添加 export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
ubuntu@ubuntu:~/nv-codec-headers$ source ~/.bashrc二、进行video_codec_sdk配置
ubuntu@ubuntu:~/Downloads$ wget https://developer.nvidia.com/downloads/designworks/video-codec-sdk/secure/12.1/video_codec_sdk_12.1.14.zip
ubuntu@ubuntu:~/Video_Codec_SDK_12.0.16$ cat README #实际使用需要查看README对应的版本NVIDIA CUDA CUDNN
然后解压之后
ubuntu@ubuntu:~/Video_Codec_SDK_12.0.16$ sudo cp Interface/* /usr/local/cuda/include
ubuntu@ubuntu:~/Video_Codec_SDK_12.0.16$ sudo cp Lib/linux/stubs/x86_64/* /usr/local/cuda/lib64/stubs/三、进行ffmpeg编译
ubuntu@ubuntu:~$ sudo apt-get install libtool automake autoconf nasm yasm
ubuntu@ubuntu:~$ sudo apt-get install libx264-dev
ubuntu@ubuntu:~$ sudo apt-get install libx265-dev
ubuntu@ubuntu:~$ sudo apt-get install libmp3lame-dev
ubuntu@ubuntu:~$ sudo apt-get install libvpx-dev
ubuntu@ubuntu:~$ sudo apt-get install libfaac-dev
ubuntu@ubuntu:~$ git clone GitHub - FFmpeg/FFmpeg: Mirror of https://git.ffmpeg.org/ffmpeg.git
ubuntu@ubuntu:~/FFmpeg$ git checkout n6.0.1
Previous HEAD position was c9864adf34 release notes for 0.5.3
HEAD is now at e38092ef93 Changelog: update
在ffmpeg的configure中有这样的版本对应
if ! disabled ffnvcodec; then
ffnv_hdr_list="ffnvcodec/nvEncodeAPI.h ffnvcodec/dynlink_cuda.h ffnvcodec/dynlink_cuviddec.h ffnvcodec/dynlink_nvcuvid.h"
check_pkg_config ffnvcodec "ffnvcodec >= 12.0.16.0" "$ffnv_hdr_list" "" || \
check_pkg_config ffnvcodec "ffnvcodec >= 11.1.5.2 ffnvcodec < 12.0" "$ffnv_hdr_list" "" || \
check_pkg_config ffnvcodec "ffnvcodec >= 11.0.10.2 ffnvcodec < 11.1" "$ffnv_hdr_list" "" || \
check_pkg_config ffnvcodec "ffnvcodec >= 8.1.24.14 ffnvcodec < 8.2" "$ffnv_hdr_list" ""
fi
ubuntu@ubuntu:~/FFmpeg$./configure --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libfreetype --enable-libmp3lame --enable-libvpx --enable-libx264 --enable-libx265 --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-ffmpeg --enable-ffplay --enable-ffprobe --enable-libx264 --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
ubuntu@ubuntu:~/FFmpeg$ sudo vim /etc/ld.so.conf 添加: /usr/local/lib
ubuntu@ubuntu:~/FFmpeg$ sudo ldconfig
代码就放在这里了，使用显卡进行硬件编码和解码+yuv2bgr.cu+bgr2yuv.cu进行加速处理，然后进行视频推流，视频流畅且一点不卡顿，延迟只有1s之内，贴个地址https://github.com/sxj731533730/GPU_D_E_Rtserver_v2.git