## FFmpeg and GStreamer with OMX Plugin Building Guide ### Building Files Struct: ``` ├── build.sh # building script ├── ffmpeg-4.4.1.tar.xz # ffmpeg src from offical site ├── gst-omx-1.18.5.tar.xz ├── gst-plugins-bad-1.18.5.tar.xz ├── gst-plugins-base-1.18.5.tar.xz ├── gst-plugins-good-1.18.5.tar.xz ├── gst-plugins-ugly-1.18.5.tar.xz ├── gstreamer-1.18.5.tar.xz # gst offical source code ├── mm_libs-VF2_515_vx.x.x.tar.xz # tarball version include wave511, wave420l, codaj12, omx-il ├── patch │   ├── ffmpeg # ffmpeg patch of tag VF2_515_vx.x.x │ ├── gstreamer1 # gstreamer patch of tag VF2_515_vx.x.x │   └── mm_libs # wave511, wave420l, codaj12, omx-il patchs to fix the building issue on debian ├── README.md ├── run_env.sh # run env script after build completed ``` ### Prepare: Before build, the debian need to install the below packages: ``` # apt install build-essential flex bison git make cmake autoconf automake \ libtool pkg-config ninja-build meson bash-completion net-tools \ tmux vim tree strace gdb file htop nfs-common psmisc lm-sensors \ v4l-utils pciutils usbutils # apt install libchromaprint-dev libaom-dev liblilv-dev libiec61883-dev libass-dev \ libbluray-dev libbs2b-dev libcodec2-dev libdav1d-dev flite1-dev libgme-dev \ libgsm1-dev libmp3lame-dev libmysofa-dev libopenjp2-7-dev libopenmpt-dev \ libopus-dev librabbitmq-dev libssl-dev librubberband-dev libsamplerate0-dev \ libshine-dev libsnappy-dev libsoxr-dev libsoxr-lsr0 libspeex-dev libssh-4 \ libssh-dev libv4l-0 libv4l-dev libtheora-dev libtwolame-dev libvidstab-dev \ libvpx-dev libwebp-dev libwebpdemux2 libx264-dev libx265-dev libxvidcore-dev \ libzimg-dev libzvbi-dev libopenal-dev ocl-icd-opencl-dev opencl-c-headers \ opencl-clhpp-headers javascript-common libjs-jquery libpocketsphinx-dev \ libsphinxbase-dev libgnutls28-dev libsrt-openssl-dev libcaca-dev libdc1394-dev \ libjack-dev libcdio-dev libcdparanoia-dev libcdio-paranoia-dev librsvg2-dev libzmq3-dev \ libasound2-dev libflac-dev libudev-dev libgudev-1.0-dev libjpeg-dev libdrm-dev \ libx11-dev libxext-dev libxv-dev libxfixes-dev libxdamage-dev libpulse-dev liba52-0.7.4-dev Note: Need to remove the gstreamer installed from upstream repo # apt purge libgstreamer1.0-0 libgstreamer-plugins-base1.0-0 gstreamer1.0-plugins-base ``` The `mm_libs-VF2_515_vx.x.x.tar.xz` could be generated by the below, first git clone the usdk code ``` # git clone git@gitlab.starfivetech.com:sbc/visionfive.git # git checkout VF2_515_vx.x.x # checkout to the tag VF2_515_vx.x.x # cd visionfive/soft_3rdpart # tar cvJf mm_libs-VF2_515_vx.x.x.tar.xz --exclude=codaj12/yuv --exclude=wave420l/code/yuv --exclude=wave511/media --exclude=wave511/code/yuv codaj12 omx-il wave420l wave511 ``` ### How to Build: Run the below: ``` # ./build.sh ``` the ffmpeg and gstreamer will be built and installed to path `target` ``` # tree target/ -L 2 target/ ├── lib │ └── firmware ├── root │ ├── codaj12 │ ├── wave420l │ └── wave511 └── usr ├── bin ├── etc ├── include ├── lib ├── libexec └── share ``` The ffmpeg and gstreamer will be both built by default. If want to only build ffmpeg or only build gstreamer, can modify the below in the `build.sh` only ffmpeg: ``` Building_FFmpeg="enable" Building_GStreamer="disable" ``` only gstreamer: ``` Building_FFmpeg="disable" Building_GStreamer="enable" ``` ### How to Run: Now the wave511 driver vdec.ko, wave420l driver venc.ko, codaj12 driver jpu.ko have been loaded when booting up. So just run the below: Run ffmpeg: ``` # source run_env.sh # $CURDIR/target/usr/bin/ffmpeg -v verbose -vcodec h264_omx -i Sintel_720_10s_10MB.mp4 -pix_fmt yuv420p 720p_i420.yuv # $CURDIR/target/usr/bin/ffplay -v verbose -vcodec h264_omx -i Sintel_720_10s_10MB.mp4 # $CURDIR/target/usr/bin/ffplay -v verbose -vcodec hevc_omx -i /test1_720P.h265 ``` Run gstreamer: ``` # source run_env.sh # gst-launch-1.0 filesrc location=Audio_Video_Sync_Test_1_1920x1080.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! xvimagesink # gst-launch-1.0 filesrc location=4K_30FPS_AVC_MainL5_2.h265 ! h265parse ! omxh265dec ! videoconvert ! videoscale ! xvimagesink # gst-launch-1.0 filesrc location=youtube001_1080p_h264_aac.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! xvimagesink # gst-launch-1.0 filesrc location=2k.18fps.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! omxh265dec ! videoconvert ! videoscale ! xvimagesink # gst-launch-1.0 filesrc location=youtube002_1080p_h264_aac.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! xvimagesink # gst-launch-1.0 filesrc location=file_example_MP4_640_3MG.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! xvimagesink ``` Run omx unit test: ``` # source run_env.sh # $CURDIR/target/usr/bin/video_dec_test -i /1080P.h265 -o 1080P_nv12.yuv -f nv12 # $CURDIR/target/usr/bin/video_dec_test -i /test1_720P.h265 -o test1_720P_nv12.yuv -f nv12 # $CURDIR/target/usr/bin/video_enc_test -i /1080p_30fps_HEVC_nv12.yuv -o 1080p_HEVC_NV12_omx.h265 -s h265 -w 1920 -h 1080 -b 5120000 -c nv12 # $CURDIR/target/usr/bin/mjpeg_dec_test -i /test1_1080P_mjpeg.mp4 -o test1_1080P_mjpeg.yuv -f i420 ``` ### APPENDIX I: Build wave511 wave420l codaj12 vendor test example the package support to build wave511 wave420l codaj12 vendor test example ``` cd wave511/wave511/ ./build_for_vdec.sh # the vendor test example will be installed to vdec_driver/ cd wave420l/code/ ./build.sh # the vendor test example will be installed to venc_driver/ cd codaj12 ./build_for_riscv.sh # the vendor test example will be installed to jpu_driver/ ```