Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] Implement AV1 support #50

Open
cromefire opened this issue May 7, 2022 · 22 comments
Open

[feature request] Implement AV1 support #50

cromefire opened this issue May 7, 2022 · 22 comments

Comments

@cromefire
Copy link

cromefire commented May 7, 2022

First and foremost: Yes it is still a draft and you may of course reject this idea, but due to it already having player support (Kodi on FireTV Max), seeming relatively straight forward (is seems to be based on the already existing ITU-T T.35 Metadata/SMPTE ST 2094-40) and seeming relatively stable for a few months, I feel like this would be a great addition as it allows conversion of existing h265 content to AV1 (or back if needed, although currently it doesn't seem like there's a lot of content).

Specification (draft): https://aomediacodec.github.io/av1-hdr10plus/
Sample file I have verified working (HDR10+ metadata arriving at the TV) with Kodi (Nexus) on my FireTV Max: https://github.com/AOMediaCodec/av1-hdr10plus/blob/ecdafef3eaf104135a6b28aabd960552e8209c7d/HDR10_SIZZLE_PLUS_UP-FINAL_02_AV1.mp4
AV1 bitstream parser library: https://github.com/yohhoy/av1parser

The bitstream parser above seems to already support ITU-T T.35 metadata and your code already supports the parsing of the metadata from bytes from (hopefully) the same format. So in theory it should be as easy as "only" creating some conditions to support more than one codec and some verification that it works. One additional problem might arise from the seeming lack of a .av1 format or something similar, all the AV1 people seem to be using .ivf files (that also seem to be really low overhead over a raw bitstream), but luckily there seems to be a rust implementation from rav1e. I have considered trying to integrate it myself, but as I lack knowledge of the format of the metadata the chances of success would be pretty low.

@quietvoid
Copy link
Owner

quietvoid commented May 7, 2022

See yohhoy/av1parser#14

I don't have any plan to support AV1 until the spec is finalized.
Ideally we also need an encoder with support as reference.

@quietvoid
Copy link
Owner

xiph/rav1e#3000 should be working fine to generate test streams.
I'm not sure when I'll get around to this.

@cromefire
Copy link
Author

Happy to test stuff, with Kodi, the FireTV and Panasonic OLED, I can actually test whether HDR10+ metadata is arriving at the TV (and it already works with AV1 & HDR10+).

@quietvoid
Copy link
Owner

Good to know. I can also test on a Samsung TV.

@cromefire cromefire changed the title [feature request] Implement AV1 (draft) support [feature request] Implement AV1 support Jan 19, 2023
@sven-pke
Copy link

Any news about this?

@quietvoid
Copy link
Owner

Nope.

@thePublicer
Copy link

there is support to add hdr10+ to av1 see https://github.com/HandBrake/HandBrake/blob/master/NEWS.markdown

@thePublicer
Copy link

also any plans to extract hdr10plus from vp9?

@JaitinPrakash
Copy link

the official spec has reached a state where i believe you can now implement it.
https://aomediacodec.github.io/av1-hdr10plus/

@quietvoid
Copy link
Owner

The complexity isn't in the HDR10+ spec but rather parsing AV1 bitstreams.
There aren't stateful parsers that are usable so I'd probably have to implement one from scratch.

@sven-pke
Copy link

sven-pke commented Nov 9, 2023

https://gstreamer.freedesktop.org/documentation/gst-plugins-bad-codecparsers/gstav1parser.html?gi-language=c I think Gstreamer is also available in Rust. Could that be an option?

@quietvoid
Copy link
Owner

quietvoid commented Nov 9, 2023

GStreamer implies an additionnal dependency at runtime and I'd rather avoid that since it's more complicated for end users.
I also can't seem to find Rust bindings to the codec parsers plugin, anyways.

@JaitinPrakash
Copy link

handbrake seems to have a semi-functional system in place. my phone properly detects and decodes av1 hdr10+ videos, although mediainfo only reports hdr10.

@cromefire
Copy link
Author

my phone properly detects and decodes av1 hdr10+ videos

Do you have any good way to test that? In the past I've found you can't really tell well on the phone, except maybe with your eyes, although that's hard-ish to see.

@JaitinPrakash
Copy link

I took an hdr10+ video on my phone, encoded it with regular svt-av1 and with handbrake, and difference in color saturation, contrast, and overall vibrancy was very clear.

@cromefire
Copy link
Author

Do you have test files?

@JaitinPrakash
Copy link

not at the moment, and it's midnight right now, so no good bright footage. I'll get some stuff to you when the sun is up. in the mean time, HDR10+ technologies plus up video is on YouTube.

@cromefire
Copy link
Author

cromefire commented Dec 1, 2023

Okay cool.

YouTube BTW doesn't really have HDR10+ support (according to my docs and my tests), you can only upload HDR10+ content there and I think it will try and use that metadata to create a good approximation of the HDR10+ version. That's why I always test on my TV, it'll display whenever it receives HDR10+.

@JaitinPrakash
Copy link

here's a basic sample file from ff pictures GmbH, an hdr mastering company. have confirmed the hdr file is hdr10+ with mediainfo, video player, and hdr10plus_tool.

https://ff.wetransfer.com/downloads/0f596c6233476b3a74a7005f2912235320210201160114/257cf4

@cromefire
Copy link
Author

cromefire commented Dec 1, 2023

Yeah that one I know, but it's so incredibly hard to see the differences I'm always unsure if there are any or if I'm just imagining it.
(Seems to work though I think, but it's also HEVC it seems)

@JaitinPrakash
Copy link

Handbrake SVT-AV1 HDR10+: https://files.catbox.moe/qsc8wl.mkv

FFMPEG SVT-AV1 HDR10: https://files.catbox.moe/5e3qih.webm

Samsung Camera HDR10+ HEVC Source: https://files.catbox.moe/udbnuy.mp4

@JaitinPrakash
Copy link

They're not great, but with overcast skies i can't get sharp sunlight/shadow lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants