[rust] Desktop recording with Selenium Manager and FFmpeg

[bonigarcia]

User description

Motivation and Context

This PR implements the first step for a new feature for Selenium users: desktop recording. The recording is done by Selenium Manager using FFmpeg. The idea is that bindings use this feature on demand. A possible way to enable this feature is as follows:

ChromeOptions options = new ChromeOptions();
options.enableRecording(); // or other name
WebDriver driver = new ChromeDriver(options);

With this, the bindings will call Selenium Manager as follows:

./selenium-manager --browser chrome --ffmpeg --debug

With that call, in addition to managing the driver/browser, Selenium Manager will manage the static build of FFmpeg (version 7.1) for the local operating system (Windows, Linux, or macOS). Like the drives and browsers, the FFmpeg binary is stored in the cache and used from there. Then, when the bindings receive the session start request, all Selenium Manager again start the desktop recording, as follows:

./selenium-manager --record --debug

An approach to this call is using a new thread in the bindings. The Selenium Manager process is called FFmpeg and records the desktop using different Windows, Linux, and macOS commands. Currently, the recording is stored in the cache folder as follows:

~/.cache/selenium/recordings/<timestamp>.avi

The created media files use MPEG-4 video codec since it balances size and quality well. The AVI container is suitable for creating playable media even when stopping the recording process abruptly.

When the Selenium session terminates (driver.quit()), the bindings will terminate the previous thread, ensuring the Selenium Manager process and its children process (i.e., ffmpeg) is terminated (like when executing Ctrl+C in the shell). This will stop the recording command, and the resulting AVI file should be in the cache.

I created an automated test to verify this feature in Rust. It is executed in CI, and the resulting recordings are attached to the job execution. For example:

https://github.com/SeleniumHQ/selenium/actions/runs/13423599803

• Linux recording: https://github.com/SeleniumHQ/selenium/actions/runs/13423599803/artifacts/2619814572
• Windows recording: https://github.com/SeleniumHQ/selenium/actions/runs/13423599803/artifacts/2619819319

I cannot get the macOS recordings since, in macOS, desktop recording is an operation that requires explicit permission (see here). I double-checked:

After granting permission to the terminal to record the desktop, Selenium Manager can also record the macOS desktop. Unfortunately, I believe we cannot do this with the GitHub Actions runners.

What do you think?

There are still improvements that could be made in Rust. For example, to customize the output folder and record file name through some parameter. Also, some FFmpeg parameters (e.g., video codec, recording size, etc.) can be parameterized. But before doing that, I would like to discuss if we want to implement this across all bindings. In my opinion, it will be an appealing feature for Selenium users. Perhaps for Selenium 5.

Context

I proposed this PR some time ago: #14452

The main aim of this PR was to allow users to record the Selenium sessions, which is a very appealing feature for troubleshooting. But discussing this PR, @shs96c said that using Docker to provide recording was not the way since Docker is a dependency we don't want. And that is right. Ideally, we don't want to impose the use of Docker just to get records. So, I started to look for an alternative. I first tried to use a pure Rust implementation for desktop recording to do the recording from Selenium Manager. I found a promising project: https://github.com/CapSoftware/scap

But that crate is not mature enough. Then, I got another idea: to manage the static builds of FFmpeg (like we already do with the drivers) and call it from Selenium Manager to record the desktop. It should not be that difficult. So, I implemented it, and it seems to work nicely.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• I have read the contributing document.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.

---

PR Type

Enhancement, Tests

---

Description

• Introduced desktop recording functionality using FFmpeg in Selenium Manager.

• Added support for downloading and managing FFmpeg binaries across platforms.

• Implemented CLI options for enabling FFmpeg and desktop recording.

• Added tests and CI workflows to validate recording functionality.

---

Changes walkthrough 📝

	Relevant files
Enhancement	5 files ffmpeg.rs Implement FFmpeg integration for desktop recording              +235/-0  files.rs Refactor file handling for FFmpeg support                                +4/-5      lib.rs Integrate FFmpeg recording into Selenium Manager                  +77/-14  main.rs Add CLI options for FFmpeg and recording                                  +17/-0    shell.rs Extend shell command execution for FFmpeg                                +33/-3
Tests	2 files common.rs Add helper for Selenium Manager binary in tests                    +6/-1      record_tests.rs Add tests for desktop recording functionality                        +48/-0
Configuration changes	2 files bazel.yml Update workflow to support recording artifacts                      +20/-1    ci-rust.yml Enable recording tests in CI workflow                                        +1/-0
Dependencies	1 files Cargo.toml Add dependencies for FFmpeg and recording functionality    +3/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 14452 - Partially compliant Compliant requirements: Allow browser recordings out of the box Enable session recording with MP4 output Non-compliant requirements: Implement ability to use dockerized browsers instead of local browsers Support browser version selection via Docker tags Provide VNC URL for debugging Requires further human verification: Verify that the recording functionality works properly across different OS platforms Check recording quality and performance impact

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Error Handling The recording functionality always returns an error when terminated. This may cause confusion for users as the error appears even when recording completes successfully. run_shell_command_with_stderr(log, os, command, true).unwrap(); return Err(anyhow!("Command for recording desktop terminated")); Resource Cleanup No explicit cleanup of temporary files and resources in case of errors during FFmpeg download and setup. pub fn download_ffmpeg( ffmpeg_version: String, http_client: &Client, os: &str, log: &Logger, cache_path: PathBuf, ) -> Result<(), Error> { let ffmpeg_url = get_ffmpeg_url(os, &ffmpeg_version)?; let ffmpeg_path_in_cache = get_ffmpeg_path_in_cache(&ffmpeg_version, os, cache_path)?; let ffmpeg_name_with_extension = get_filename_with_extension(FFMPEG_NAME, os); let mut lock = Lock::acquire( log, &ffmpeg_path_in_cache, Some(ffmpeg_name_with_extension.clone()), )?; if !lock.exists() && ffmpeg_path_in_cache.exists() { log.debug(format!( "{} is already in cache: {}", FFMPEG_NAME, ffmpeg_path_in_cache.display() )); return Ok(()); } log.debug(format!( "Downloading {} {} from {}", FFMPEG_NAME, ffmpeg_version, &ffmpeg_url )); let (_tmp_folder, driver_zip_file) = download_to_tmp_folder(http_client, ffmpeg_url, log)?; uncompress_ffmpeg( &driver_zip_file, &ffmpeg_path_in_cache, log, os, ffmpeg_name_with_extension, &ffmpeg_version, )?; lock.release(); Ok(())

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Fix incorrect error return The function always returns an error with "Command for recording desktop terminated". This should be changed to return Ok(()) when recording completes successfully. rust/src/ffmpeg.rs [232-234] -run_shell_command_with_stderr(log, os, command, true).unwrap(); +run_shell_command_with_stderr(log, os, command, true)?; +Ok(()) -return Err(anyhow!("Command for recording desktop terminated")); - Apply this suggestion Suggestion importance[1-10]: 9 __ Why: The function currently always returns an error regardless of success/failure, which is incorrect behavior. This fix ensures proper error handling and successful completion cases.	High
	Fix variable usage before declaration The output variable is used before being defined, causing a compilation error. Move the output declaration before the debug log. rust/src/shell.rs [82-84] +let output = run_shell_command_by_os_stderr(os, command, stderr)?; log.debug(format!("Output: {:?}", output)); -let output = run_shell_command_by_os_stderr(os, command, stderr)?; [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: The code attempts to use the 'output' variable in the debug log before it's declared, which would cause a compilation error. This fix ensures proper variable declaration order.	Medium
Learned best practice	Include command output and status in error messages when shell commands fail to aid debugging When running shell commands, include the command output in the error message when the command fails. This helps with debugging by showing what went wrong. Add error handling to check output.status() and include stderr in error messages. rust/src/shell.rs [124-129] let output = process.output()?; +if !output.status.success() { + return Err(anyhow!("Command failed with status {}: {}", + output.status, + String::from_utf8_lossy(&output.stderr))); +} let mut stdout = String::from_utf8_lossy(&output.stdout); if stderr { stdout = stdout + " " + String::from_utf8_lossy(&output.stderr); } Ok(strip_trailing_newline(&stdout).to_string()) Apply this suggestion Suggestion importance[1-10]: 6	Low
Update

[qodo-merge-pro]

CI Feedback 🧐

(Feedback updated until commit 1e804c7)

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: Rust / Build / Build

Failed stage: Run Bazel [❌]

Failure summary: The action failed because the Rust crates lockfile is out of date. Specifically: The current digest ("9c5df13665bbb4442be9c8674c176e8324d597425d8da1f20e952cea268d6168") does not match the expected digest ("7d75ecfab3b52d1d58721f87d2f3c5447aa0c2894f43756bb2671a0e7005a3cf") The error occurred during the fetch of repository 'crates' The fix is to re-run bazel with the environment variable CARGO_BAZEL_REPIN=true to update the lockfile

Relevant error logs: 1: ##[group]Operating System 2: Ubuntu ... 948: Package 'php-symfony-asset' is not installed, so not removed 949: Package 'php-symfony-asset-mapper' is not installed, so not removed 950: Package 'php-symfony-browser-kit' is not installed, so not removed 951: Package 'php-symfony-clock' is not installed, so not removed 952: Package 'php-symfony-debug-bundle' is not installed, so not removed 953: Package 'php-symfony-doctrine-bridge' is not installed, so not removed 954: Package 'php-symfony-dom-crawler' is not installed, so not removed 955: Package 'php-symfony-dotenv' is not installed, so not removed 956: Package 'php-symfony-error-handler' is not installed, so not removed ... 1142: Package 'php-uopz-all-dev' is not installed, so not removed 1143: Package 'php8.3-uploadprogress' is not installed, so not removed 1144: Package 'php-uploadprogress-all-dev' is not installed, so not removed 1145: Package 'php8.3-uuid' is not installed, so not removed 1146: Package 'php-uuid-all-dev' is not installed, so not removed 1147: Package 'php-validate' is not installed, so not removed 1148: Package 'php-vlucas-phpdotenv' is not installed, so not removed 1149: Package 'php-voku-portable-ascii' is not installed, so not removed 1150: Package 'php-wmerrors' is not installed, so not removed ... 1882: �[32mComputing main repo mapping:�[0m 1883: �[32mINFO: �[0mRepository crates instantiated at: 1884: /home/runner/work/selenium/selenium/WORKSPACE:40:18: in <toplevel> 1885: Repository rule crates_repository defined at: 1886: /home/runner/.bazel/external/rules_rust/crate_universe/private/crates_repository.bzl:124:36: in <toplevel> 1887: �[32mINFO: �[0mrepository @@crates' used the following cache hits instead of downloading the corresponding file. 1888: * Hash '6d6dbca50995c15bb01700522eeb583f0c08dfc48022add2c74a1ff75f0ffb4c' for https://github.com/bazelbuild/rules_rust/releases/download/0.55.5/cargo-bazel-x86_64-unknown-linux-gnu 1889: If the definition of 'repository @@crates' was updated, verify that the hashes were also updated. 1890: �[31m�[1mERROR: �[0m/home/runner/.bazel/external/rules_rust/crate_universe/private/generate_utils.bzl:422:13: An error occurred during the fetch of repository 'crates': 1891: Traceback (most recent call last): 1892: File "/home/runner/.bazel/external/rules_rust/crate_universe/private/crates_repository.bzl", line 45, column 28, in _crates_repository_impl 1893: repin = determine_repin( 1894: File "/home/runner/.bazel/external/rules_rust/crate_universe/private/generate_utils.bzl", line 422, column 13, in determine_repin 1895: fail(msg) 1896: Error in fail: Error: Digests do not match: Current Digest("9c5df13665bbb4442be9c8674c176e8324d597425d8da1f20e952cea268d6168") != Expected Digest("7d75ecfab3b52d1d58721f87d2f3c5447aa0c2894f43756bb2671a0e7005a3cf") 1897: Stack backtrace: 1898: 0: anyhow::error::<impl anyhow::Error>::msg ... 1902: 4: std::sys::backtrace::__rust_begin_short_backtrace 1903: 5: std::rt::lang_start::{{closure}} 1904: 6: std::rt::lang_start_internal 1905: 7: main 1906: 8: <unknown> 1907: 9: __libc_start_main 1908: 10: _start 1909: The current `lockfile` is out of date for 'crates'. Please re-run bazel using `CARGO_BAZEL_REPIN=true` if this is expected and the lockfile should be updated. 1910: �[31m�[1mERROR: �[0mError computing the main repository mapping: no such package '@@crates//': Error: Digests do not match: Current Digest("9c5df13665bbb4442be9c8674c176e8324d597425d8da1f20e952cea268d6168") != Expected Digest("7d75ecfab3b52d1d58721f87d2f3c5447aa0c2894f43756bb2671a0e7005a3cf") 1911: Stack backtrace: 1912: 0: anyhow::error::<impl anyhow::Error>::msg ... 1917: 5: std::rt::lang_start::{{closure}} 1918: 6: std::rt::lang_start_internal 1919: 7: main 1920: 8: <unknown> 1921: 9: __libc_start_main 1922: 10: _start 1923: The current `lockfile` is out of date for 'crates'. Please re-run bazel using `CARGO_BAZEL_REPIN=true` if this is expected and the lockfile should be updated. 1924: �[0m 1925: ##[error]Process completed with exit code 1.

[cgoldberg]

This doesn't seem to be getting any attention, but FWIW... I think it's a cool feature.

[nirtal85]

I think it's a great feature - very much missing - and I am waiting to use this.
can we record in MP4 and not in avi format?

[diemol]

We are still talking to see if this is the approach we want to follow.

[bonigarcia]

@cgoldberg There are some comments about this PR in #selenium-tlc channel in Slack: https://seleniumhq.slack.com/archives/CBH302726/p1740050766269619

@nirtal85 Yes, FFmpeg also supports screencasting to MP4 and other container formats (e.g., MOV, MKV, etc.) and different video codecs (e.g., MPEG-4, etc.). I have selected AVI since it is pretty "safe" in the sense that the media chunks are always playable once they are written in the file. For instance, MP4 needs a proper termination of the recording media, or otherwise, the resulting video is not playable. However, this choice (AVI+MPEG-4) could be changed (or perhaps to be configurable) if we finally decide to land this feature.