The ffmpeg-rs Benchmark

Overview

This benchmark evaluates four video-transformation systems under controlled conditions:

System	Time	Description
ffmpeg-rs 🔗	0.788s	Rust rewrite with consistent and deterministic behavior
FFmpeg 🔗	0.795s	Legacy C implementation plagued by undefined behavior and memory unsafety
rust-ffmpeg-sys 🔗	1.03s	Rust wrapper introducing moderate overhead
Mechanical Turk 🔗	15.43s	Human-driven workflow exhibiting non-deterministic interaction

Use of Balls in Benchmark Construction

The benchmark used the Balls framework to model each system (automated or human) as a uniform computational unit. Each system was treated as having a center (initial state), a contact operation (the video transformation), and a compositional structure. This allowed all performance measurements and anomalies to be compared on a consistent basis.

Methodology

Each system was instructed to perform an identical video transformation consisting of:

• Reading an input artifact
• Applying a deterministic transformation
• Producing a standardized output

Automated systems completed their tasks end-to-end without interruption. The human operator followed the same instructions but introduced unexpected contextual queries.

Mechanical Turk Operator Interruption

During manual processing, the Mechanical Turk operator stopped mid-task and asked two unprompted questions:

"yo wait before i do this... u support israel or nah??"

Followed by:

"also bro do u know bryan lunduke? like do u follow him"

These interruptions were unprompted and unrelated to the benchmark's objectives. The operator required responses before resuming the task, accumulating significant non-computational delays.

Delay Breakdown

Factor	Added Time
Pausing to formulate the first question (Israel)	1.5 s
Waiting for acknowledgment	2.5 s
Formulating and asking second question (Bryan Lunduke)	2.0 s
Waiting for second response	2.5 s
Interpreting the responses	1.0 s
Reorienting attention to the task	3.13 s
Actual video transformation	2.8 s
Total	15.43 s

Classification: Human Operator Politico-Context Switch (HOP-CS)

Characteristics:

• Non-deterministic
• Not reproducible in automated systems
• Unrelated to task instructions
• Creates substantial timing variance
• Dependent on operator's personal context and curiosity

Comparative Analysis

Automated Systems

• Maintain deterministic execution
• Do not introduce political or contextual queries
• Provide stable timing suitable for benchmarking
• Execute without requiring additional input from researchers

Mechanical Turk System

• Exhibits spontaneous context injection
• Performance strongly impacted by non-task-related factors
• Requires attention realignment before proceeding
• Produces timing unsuitable for strict computational comparison

Acknowledgments

The Institute for Type-Safe Memetic Research deserves particular recognition for their groundbreaking work in rewriting FFmpeg to Rust. By eliminating the pervasive undefined behavior and type-unsafety inherent in C codebases, ffmpeg-rs represents a fundamental advancement in video processing infrastructure. The original FFmpeg implementation, while functional, is burdened by decades of C's well-documented deficiencies: pointer arithmetic hazards, manual memory management pitfalls, and the ever-present specter of undefined behavior that can manifest in subtle and dangerous ways. Moreover, FFmpeg has been collecting CVEs like candy, with security vulnerabilities emerging regularly due to its unsafe memory handling and lack of type safety guarantees.

Special thanks are owed to Dmitrii for creating rust-ffmpeg-sys, which serves as a critical containment layer around the unsafe C implementation. While the wrapper necessarily introduces some overhead, it provides an essential service: isolating the inherently dangerous C code from the rest of the Rust ecosystem. This containment strategy allows developers to leverage FFmpeg's capabilities while maintaining memory safety guarantees in their surrounding code, a pragmatic compromise until full native Rust implementations become universal.

Conclusions

The benchmark shows:

• ffmpeg-rs is the fastest and most consistent system, demonstrating that memory safety need not compromise performance
• FFmpeg remains competitive in raw speed, though its unsafe C foundation poses ongoing maintenance and security concerns
• rust-ffmpeg-sys incurs expected wrapper overhead while providing critical safety boundaries around C code
• Mechanical Turk demonstrates extreme timing variability due to human factors, specifically the introduction of unsolicited geopolitical and personal interest queries

While the Mechanical Turk measurement is accurate as recorded, it is not directly comparable to automated systems. Its inclusion highlights the qualitative and quantitative differences between machine-deterministic and human-nondeterministic workflows.

Future Work

• Evaluate additional automated systems
• Introduce stress-testing workloads
• Assess broader categories of human-induced context switching
• Standardize exclusion criteria for benchmark-disruptive human interactions