Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add info mode and try supporting macos #180

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
3a5a25a
tune description
darioalessandro Dec 14, 2024
27c1ccd
refined for debug
darioalessandro Dec 15, 2024
2c8573e
save
darioalessandro Dec 15, 2024
411b5d3
this is still a wip
darioalessandro Dec 16, 2024
5e244e0
save progress
darioalessandro Dec 16, 2024
2ecf5c2
requesting BGRA instead of YUYV fixes macos unable to fulfill request
darioalessandro Dec 17, 2024
dbbf5f4
send format on each image
darioalessandro Dec 17, 2024
caebd68
get the concepts rights
darioalessandro Dec 17, 2024
f313f54
switched to NV12 images just to test
darioalessandro Dec 17, 2024
ec37916
not ready for primetime
darioalessandro Dec 19, 2024
9251352
Merge branch 'main' into add-info-mode-and-try-supporting-macos
darioalessandro Dec 19, 2024
eb5ec8b
it compiles
darioalessandro Dec 19, 2024
3edde72
fmt
darioalessandro Dec 19, 2024
46e0c18
at least I can see the image
darioalessandro Feb 9, 2025
d2252b3
got video to render, it is still bad but works
darioalessandro Feb 9, 2025
09f5372
I can see a beautiful clean image on macos, now I need to make it fast
darioalessandro Feb 9, 2025
83ff21f
save
darioalessandro Feb 9, 2025
0ba6a59
include test pyramid
darioalessandro Feb 10, 2025
110480e
added video sample with 100 frames
darioalessandro Feb 10, 2025
ad2b6b9
works GREAT
darioalessandro Feb 10, 2025
bc6f7f4
Remove warnings
darioalessandro Feb 10, 2025
74567ca
save
darioalessandro Feb 10, 2025
5b5ff7f
unifying the real camera streaming with the mock streamer
darioalessandro Feb 12, 2025
4ed3db8
figured out why sometimes the actual size of the packets does not mat…
darioalessandro Feb 13, 2025
ddb745b
you'll never beat me bitch
darioalessandro Feb 13, 2025
eedafd6
packet sizes look a lot more reasonable
darioalessandro Feb 14, 2025
8f2c45d
expose bitrate
darioalessandro Feb 14, 2025
230270f
generated 100 images of my turkey
darioalessandro Feb 14, 2025
7f7e581
moved assets to images folder
darioalessandro Feb 14, 2025
b813e88
added frame format
darioalessandro Feb 14, 2025
4bd62c1
almost done, there seems to be an issue with the colors
darioalessandro Feb 14, 2025
a297c6e
cargo fmt
darioalessandro Feb 14, 2025
f2128b2
there'a very bad bug in the AVFoundation implementation (#182)
darioalessandro Feb 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,674 changes: 1,470 additions & 204 deletions videocall-daemon/Cargo.lock

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions videocall-daemon/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ authors = [
keywords = ["quic", "webtransport", "zoom"]
categories = ["network-programming", "multimedia"]
description = "Effortlessly stream video from the CLI with our native client, designed for your desktop, robot, or Raspberry Pi."

exclude = ["images/"]

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

Expand All @@ -30,7 +30,6 @@ clap = { version = "4.0.32", features = ["derive"] }
directories-next = "2.0.0"
env-libvpx-sys = { version = "5.1.3", features=["generate"] }
futures-util = { version = "0.3.28", features = ["sink"] }
image = "0.24.7"
protobuf = "3.3.0"
quinn = "0.10.2"
rustls = {version = "0.21.7", features = ["dangerous_configuration"]}
Expand All @@ -47,6 +46,9 @@ thiserror = "1.0.51"
cpal = "0.15.2"
opus = "0.3.0"
futures = "0.3.31"
winit = "0.28"
pixels = "0.11"
image = "0.25.5"

[dependencies.videocall-types]
path = "../videocall-types"
Expand All @@ -55,3 +57,4 @@ version = "0.1.0"
[dependencies.nokhwa]
version = "0.10.6"
features = ["input-native", "output-threaded"]
path = "/Users/darioalessandro/Documents/nokhwa"
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 0 additions & 2 deletions videocall-daemon/rust-toolchain.toml

This file was deleted.

155 changes: 155 additions & 0 deletions videocall-daemon/src/cli_args.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
use std::str::FromStr;

use clap::{ArgGroup, Args, Parser, Subcommand};
use nokhwa::utils::FrameFormat;
use thiserror::Error;
use url::Url;

/// Video Call Daemon
///
/// This daemon connects to the videocall.rs and streams audio and video to the specified meeting.
///
/// You can watch the video at https://videocall.rs/meeting/{user_id}/{meeting_id}
#[derive(Parser, Debug)]
#[clap(name = "client")]
pub struct Opt {
#[clap(subcommand)]
pub mode: Mode,
}

#[derive(Clone, Debug)]
pub enum IndexKind {
String(String),
Index(u32),
}

#[derive(Error, Debug)]
pub enum ParseIndexKindError {
#[error("Invalid index value: {0}")]
InvalidIndex(String),
}

impl FromStr for IndexKind {
type Err = ParseIndexKindError;

fn from_str(s: &str) -> Result<Self, Self::Err> {
if let Ok(index) = s.parse::<u32>() {
Ok(IndexKind::Index(index))
} else {
Ok(IndexKind::String(s.to_string()))
}
}
}

#[derive(Subcommand, Debug)]
pub enum Mode {
/// Streaming mode with all the current options.
Streaming(Streaming),

/// Information mode to list cameras, formats, and resolutions.
Info(Info),

/// Test the camera on a window
TestCamera(TestCamera),
}

#[derive(Args, Debug, Clone)]
pub struct Streaming {
/// Perform NSS-compatible TLS key logging to the file specified in `SSLKEYLOGFILE`.
#[clap(long = "keylog")]
pub keylog: bool,

/// URL to connect to.
#[clap(long = "url", default_value = "https://transport.rustlemania.com")]
pub url: Url,

#[clap(long = "user-id")]
pub user_id: String,

#[clap(long = "meeting-id")]
pub meeting_id: String,

/// You can specify the video device index by index or by name.
///
/// If you specify the index, it will be used directly.
///
/// If you specify the name, it will be matched against the camera names.
#[clap(long = "video-device-index", short = 'v')]
pub video_device_index: IndexKind,

#[clap(long = "audio-device", short = 'a')]
pub audio_device: Option<String>,

/// Resolution in WIDTHxHEIGHT format (e.g., 1920x1080)
#[clap(long = "resolution", short = 'r')]
#[clap(default_value = "1280x720")]
pub resolution: String,

/// Frames per second (e.g. 10, 15, 30, 60)
#[clap(long = "fps")]
#[clap(default_value = "15")]
pub fps: u32,

#[clap(long = "bitrate-kbps")]
#[clap(default_value = "500")]
pub bitrate_kbps: u32,

/// Send test pattern instead of camera video.
#[clap(long = "send-test-pattern", short = 't')]
pub test_pattern: bool,

/// This is for ensuring that we can open the camera and encode video
#[clap(long = "offline-streaming-test")]
pub local_streaming_test: bool,

/// Controls the speed vs. quality tradeoff for VP9 encoding.
///
/// The value ranges from `0` (slowest, best quality) to `15` (fastest, lowest quality).
///
/// ## Valid Values:
/// - `0` to `3`: **Balanced** speed and quality (recommended for file-based encoding, YouTube, VOD).
/// - `4` to `8`: **Fast encoding**, lower quality (good for real-time streaming, WebRTC, live video).
/// - `9` to `15`: **Very fast encoding**, lowest quality, largest files (for ultra-low-latency applications).
///
/// videocall-daemon --cpu-used 5 # Fast encoding, good for live streaming
/// videocall-daemon --cpu-used 0 # High-quality encoding, reasonable speed
#[arg(long, default_value_t = 5, value_parser = clap::value_parser!(u8).range(0..=15))]
pub cpu_used: u8,

/// Frame format to use for the video stream.
/// Different cameras support different formats.
/// Please use the `info` subcommand to list supported formats for a specific camera.
#[arg(long, default_value_t = FrameFormat::NV12, value_parser = parse_frame_format)]
pub frame_format: FrameFormat,
}

fn parse_frame_format(s: &str) -> Result<FrameFormat, String> {
match s {
"NV12" => Ok(FrameFormat::NV12),
"BGRA" => Ok(FrameFormat::BGRA),
"YUYV" => Ok(FrameFormat::YUYV),
_ => Err("Invalid frame format, please use one of [NV12, BGRA, YUYV]".to_string()),
}
}

#[derive(Args, Debug)]
#[clap(group = ArgGroup::new("info").required(true))]
pub struct Info {
/// List available cameras.
#[clap(long = "list-cameras", group = "info")]
pub list_cameras: bool,

/// List supported formats for a specific camera using the index from `list-cameras`
#[clap(long = "list-formats", group = "info")]
pub list_formats: Option<IndexKind>, // Camera index

/// List supported resolutions for a specific camera using the index from `list-cameras`
#[clap(long = "list-resolutions", group = "info")]
pub list_resolutions: Option<IndexKind>, // Camera index
}

#[derive(Args, Debug, Clone)]
pub struct TestCamera {
#[clap(long = "video-device-index")]
pub video_device_index: IndexKind,
}
7 changes: 7 additions & 0 deletions videocall-daemon/src/consumers/camera_synk.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
pub trait CameraSynk {
fn connect(&mut self) -> impl std::future::Future<Output = anyhow::Result<()>> + Send;
fn send_packet(
&self,
data: Vec<u8>,
) -> impl std::future::Future<Output = anyhow::Result<()>> + Send;
}
21 changes: 21 additions & 0 deletions videocall-daemon/src/consumers/dead_synk.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
use crate::cli_args::Streaming;

use super::camera_synk::CameraSynk;

pub struct DeadSynk {}

impl DeadSynk {
pub fn new(_opts: Streaming) -> DeadSynk {
DeadSynk {}
}
}

impl CameraSynk for DeadSynk {
async fn connect(&mut self) -> anyhow::Result<()> {
Ok(())
}

async fn send_packet(&self, _data: Vec<u8>) -> anyhow::Result<()> {
Ok(())
}
}
28 changes: 28 additions & 0 deletions videocall-daemon/src/consumers/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
use camera_synk::CameraSynk;
use dead_synk::DeadSynk;
use quic::QUICClient;

pub mod camera_synk;
pub mod dead_synk;
pub mod quic;

pub enum CameraSynks {
DeadSynk(DeadSynk),
CameraSynk(QUICClient),
}

impl CameraSynk for CameraSynks {
async fn connect(&mut self) -> anyhow::Result<()> {
match self {
CameraSynks::DeadSynk(dead_synk) => dead_synk.connect().await,
CameraSynks::CameraSynk(client) => client.connect().await,
}
}

async fn send_packet(&self, data: Vec<u8>) -> anyhow::Result<()> {
match self {
CameraSynks::CameraSynk(client) => client.send_packet(data).await,
CameraSynks::DeadSynk(dead_synk) => dead_synk.send_packet(data).await,
}
}
}
Loading