FFmpeg 9.0: Innovations and Its Impact on AI Multimedia Workflows

Topaz Video General
1

image
image1024×339 316 KB

The release of FFmpeg 9.0 marks a significant milestone in open-source multimedia processing. Known as the backbone of digital video and audio pipelines across streaming platforms, editing tools, and AI software, this major version introduces architectural modernizations, next-generation codec support, and accelerated hardware backends.

FFmpeg 9.0 “Lei” — Release Highlights

FFmpeg 9.0 “Lei” was officially released on August 4, 2026, approximately four and a half months after FFmpeg 8.1 “Hoare” . The codename “Lei” honors the late Dr. Lei Xiaohua from Communication University of China, who authored numerous FFmpeg tutorials between 2013–2016 and is regarded as the pioneer of Chinese audio/video developer education .

This is a major release with over 2,200 commits from 160+ authors, touching 1,781 files, adding ~85,000 lines and removing ~33,000 . All seven libraries receive major version bumps (ABI break):

Library Version
libavutil 61
libavcodec 63
libavformat 63
libavdevice 63
libavfilter 12
libswscale 10
libswresample 7

1. swscale — Complete Rewrite (The Biggest Change)

The most significant long-term work is the multi-year rewrite of swscale, led by Niklas Haas (300+ commits in libswscale alone) with Ramiro Polla on the ARM side .

Old architecture: 20 years of bespoke, format-specific conversion code.
New architecture: Every conversion is decomposed into elementary operations (read, swizzle, linear transform, scale, pack…), an optimizer simplifies the list, and a backend compiles it into kernel chains.

Key aspects:

  • New SwsBackend selector in the public API — choose between: template-based C reference, fast memcpy path, chained x86 SIMD kernels, chained AArch64 NEON kernels, and a Vulkan SPIR-V backend (the same conversion graph can run on CPU or GPU)
  • New SwsScaler enum for explicit scaling algorithm selection, replacing the old flags soup
  • Constant math moved to a 64-bit rational type (exact computation, no floating-point overflow issues)
  • SWS_BITEXACT mode — even the Vulkan backend complies by decorating linear arithmetic with SPIR-V NoContraction
  • The new paths are gated behind SWS_UNSTABLE; legacy code remains the stable default
  • Added PAL8 (palette) support with AVX2 path, and proper interlaced content modeling

2. Vulkan Acceleration

FFmpeg 9.0 significantly expands Vulkan-based hardware acceleration :

  • Vulkan APV video decoding — Samsung’s Advanced Professional Video codec decoded via Vulkan shaders (without formal Vulkan Video extensions)
  • Apple ProRes RAW Vulkan acceleration
  • v360_vulkan filter — Vulkan-accelerated 360° video conversion

3. Animated WebP Support

After 11 years (ticket #4907, opened in 2015), FFmpeg can finally decode animated WebP natively, along with a dedicated demuxer . The patches were originally started by Josef Zlomek and finished by Ramiro Polla — it was more complex than anticipated, with separate RGB and YUV paths .

4. libavcodec — New Codecs and APIs

Over 700 commits landed in libavcodec :

  • HE-AAC 960-sample frame decoding — the variant used by DAB+ digital radio
  • Playdate video encoder and muxer — 1-bit, 400×240 delta-coded video with zlib compression for the little yellow handheld console with a crank
  • New decoding API:avcodec_receive_frame_flags() with AV_CODEC_RECEIVE_FRAME_FLAG_SYNCHRONOUS — bypasses frame-threading delay for low-latency consumers
  • SMPTE 2094-50 metadata support and passthrough
  • LCEVC track muxing support in MP4 muxer
  • Bitstream filter to split Dolby Vision multi-layer HEVC

5. AMD AMF Enhancements

  • AMF Frame Rate Converter (vf_frc_amf) filter
  • AMF Video Quality Enhancer (vf_vqe_amf) filter
  • AMF Color Converter (vf_vpp_amf) HDR capabilities extended
  • AMF hardware memory mapping support

6. NVIDIA Improvements

  • transpose_cuda filter — GPU-accelerated video transposition
  • NVENC gains AV1 hierarchical B-frame reference mode, SDK 13.1 compatibility, 12-bit input acceptance (truncated to 10-bit)
  • Removed deprecated NVENC options and support for pre-11.1 SDK versions

7. Apple VideoToolbox

  • ProRes RAW VideoToolbox hardware acceleration

8. Assembly & SIMD Optimizations

Nearly 200 commits touched x86, ~100 touched ARM :

  • Half-pel motion compensation code (dating back to the MMX era) ported to SSE2
  • H.264 intra prediction gained AVX optimizations
  • More AVX-512 optimizations
  • Continued AArch64 NEON work

9. AI / DNN Backend

  • ONNX Runtime DNN backend with GPU execution provider support

10. Removed Features

  • CELT decoding support removed (does not affect Opus CELT)
  • ogg/celt parsing removed
  • Deprecated NVENC options and pre-11.1 SDK support removed

Source: FFmpeg 9.0 was released on August 4, 2026. Source tarballs are available at ffmpeg.org. Pre-compiled packages for Debian, Ubuntu, Fedora, and Windows will follow through distribution channels

2

Before anyone treats that post as reference material: it wasn’t written by someone who read the FFmpeg 9.0 changelog. Half of it is years out of date, the other half is invented. The words split in the middle (“GP U”, “performa nce”, “over head”, “fra me”) are the usual giveaway.

Point by point:

  • “Ships with built-in decoding for VVC.” The native VVC decoder landed in FFmpeg 7.1, in September 2024. It is two years old. Nothing about it is new in 9.0.

  • “Up to 50% bitrate reduction compared to HEVC.” That is a claim about the VVC codec itself, made by its own promoters, and it has nothing to do with any FFmpeg release. Quoting it as a 9.0 feature is category confusion.

  • “AV1 optimizations: rewritten sub-block motion estimation, 14% faster transcoding on AVX-512.” Not in the changelog. Not anywhere. FFmpeg does not have its own AV1 encoder to rewrite: AV1 encoding goes through libaom, SVT-AV1 or rav1e, decoding through dav1d. The actual x86 assembly work this cycle was hpeldsp and fpel ported from MMX to SSE2, AVX2 horizontal predictors for H.264 intra, pp7 leaving MMX, and SSSE3 median SAD. None of that is AV1 and none of it produces a 14% transcode number.

  • “Vulkan SPIR-V backend enabling seamless GPU-accelerated scaling.” The swscale rewrite is real and it is the most important work in this release, but the new backends are gated behind SWS_UNSTABLE and the legacy code is still the default. Nothing is “seamless” and nothing switches to the GPU on its own. Worth adding, since the post missed it: the GLSL swscale backend was removed in favour of the SPIR-V one, and runtime shader compilation is gone entirely, everything is compiled at build time now.

  • “Deprecated legacy interfaces such as avcodec_decode_video2 have been removed.” That function was deprecated in 3.1 and removed several major versions ago. The real 9.0 removals are -vsync, -top, -qphist, -filter_complex_script, the OpenMAX encoders, the NPP filters, the standalone CELT decoder, the Sonic codecs, v308/v408/v410, plus NVENC’s old preset aliases, legacy rate control modes and pre-11.1 SDK support. Those are the ones that will actually break someone’s script tomorrow morning.

  • “Filter graph parameters now use dictionary-based passing rather than unsafe string parsers.” Not in this release. The one genuinely new decoding API in 9.0 is avcodec_receive_frame_flags(), which the post does not mention.

  • The entire AI section. No part of the release “eliminates PCIe bottlenecks”. What 9.0 actually adds on that front is an ONNX Runtime DNN backend with CUDA and DirectML execution providers, so you can run ONNX models inside a filter graph. That is a real, useful feature, and the post managed to miss it while writing four paragraphs of adjacent hype.

What the post left out, i.e. everything that matters: an ABI break across all seven libraries, the dovi_split bitstream filter and a real Dolby Vision Profile 7 stream model at demux and remux, SMPTE 2094-50 dynamic metadata with the whole T.35 zoo unified behind common helpers, per-stream -mastering_display and -content_light input overrides, animated WebP decoding after a ticket open since 2015, FFV1 Bayer encoding plus a GPU FFV1 encoder and decoder, APV Vulkan hardware decoding, LCEVC tracks in MP4, the new shared: cache protocol, and TLS peer certificate verification now on by default.

If you want the actual release: the Changelog on code.ffmpeg.org, and Jean-Baptiste Kempf’s write-up at FFmpeg 9.0 — Jean-Baptiste Kempf. Both take less time to read than that post takes to unlearn.

EDIT : Gemini made the ncessary modifications for best readings, subject is closed :partying_face:

3

I updated the information.

4

Thanks for the readers :wink: