-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add KHR_texture_astc extension #2369
base: main
Are you sure you want to change the base?
Conversation
|
||
Regardless of the format used, these additional restrictions apply for compatibility reasons: | ||
|
||
- Swizzling metadata (`KTXswizzle`) MUST be `rgba` or omitted. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TBC, where we need this meta data or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think glTF uses swizzle anywhere so probably not needed.
|
||
This extension adds the ability to specify textures using KTX v2 images with (ASTC compression)[https://github.com/ARM-software/astc-encoder/blob/main/Docs/FormatOverview.md]. An implementation of this extension can use such images as an alternative to the PNG or JPEG images available in glTF 2.0 for more efficient asset transmission, reducing GPU memory footprint, higher texture throughput and faster execution. | ||
|
||
When this extension is used, it's allowed to use value `image/astc` for the `mimeType` property of images that are referenced by the `source` property of `KHR_texture_astc` texture extension object. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The example below uses mimeType image/ktx2
. Is that intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the extension will only allow image/ktx2
. image/astc
is a typo however.
> **Note:** Red textures from the core glTF 2.0 specification include: | ||
> - `occlusionTexture` (standalone) | ||
|
||
- **Luminance-Alpha:** A texture that uses only Luminance and Alpha channels. To sample these textures use `.ga` swizzle in your shaders. Such textures MUST NOT be encoded with sRGB transfer function. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need Luminance-Alpha types here?
(I don't know what other glTF extensions do, but if Red-Green was sufficient that'd be better.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is support for L+A in the encoder so it makes sense for gltf to have support too. Although my note below
**Note:** The core glTF 2.0 specification has no examples of luminance-alpha textures.
So maybe we remove it.
|
||
### KTX header fields for ASTC payloads | ||
- `supercompressionScheme` MUST be `0` (None). | ||
- ASTC HDR vs LDR as well as block sizes can be determined from the vkFormat of the image |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: vkFormat => VkFormat
(in a few other places)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The field in the KTX v2 header is called "vkFormat". So here "value of vkFormat" would be correct. I haven't looked at the other instances of its use.
"$schema": "http://json-schema.org/draft-04/schema", | ||
"title": "KHR_texture_astc glTF Texture Extension", | ||
"type": "object", | ||
"description": "glTF extension to specify textures using the KTX v2 images with astc compression.", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: astc => ASTC
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good but I do think you'll need to address the fragmentation of ASTC implementations into LDR-only, LDR+HDR and LDR+HDR+3D possibly by having multiple extensions. I'm no glTF expert but perhaps you can just have different extension names in this single extension document. That's what the GL extensions did IIRC.
|
||
## Overview | ||
|
||
This extension adds the ability to specify textures using KTX v2 images with (ASTC compression)[https://github.com/ARM-software/astc-encoder/blob/main/Docs/FormatOverview.md]. An implementation of this extension can use such images as an alternative to the PNG or JPEG images available in glTF 2.0 for more efficient asset transmission, reducing GPU memory footprint, higher texture throughput and faster execution. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"using ASTC compressed images in KTX v2 containers" would be more accurate.
|
||
When this extension is used, it's allowed to use value `image/astc` for the `mimeType` property of images that are referenced by the `source` property of `KHR_texture_astc` texture extension object. | ||
|
||
At runtime, engines can directly use the ktx images unlike EXT_texture_basisu which requires doing transcoding to other compressed formats. This also means you get support for all ASTC block formats and quality presets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/doing//
|
||
## glTF Schema Updates | ||
|
||
The `KHR_texture_astc` extension is added to the `textures` object and specifies a `source` property that points to the index of the `image` which defines a reference to the KTX v2 image with ASTC compression. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"KTX v2 file with the ASTC compressed image."
> **Note:** Red textures from the core glTF 2.0 specification include: | ||
> - `occlusionTexture` (standalone) | ||
|
||
- **Luminance-Alpha:** A texture that uses only Luminance and Alpha channels. To sample these textures use `.ga` swizzle in your shaders. Such textures MUST NOT be encoded with sRGB transfer function. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why must it not be encoded with sRGB? It is quite normal to encode luminance in sRGB?
|
||
### KTX header fields for ASTC payloads | ||
- `supercompressionScheme` MUST be `0` (None). | ||
- ASTC HDR vs LDR as well as block sizes can be determined from the vkFormat of the image |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The field in the KTX v2 header is called "vkFormat". So here "value of vkFormat" would be correct. I haven't looked at the other instances of its use.
|
||
Regardless of the format used, these additional restrictions apply for compatibility reasons: | ||
|
||
- Swizzling metadata (`KTXswizzle`) MUST be `rgba` or omitted. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think glTF uses swizzle anywhere so probably not needed.
- `colorPrimaries` MUST be `KHR_DF_PRIMARIES_UNSPECIFIED`; | ||
- `transferFunction` MUST be `KHR_DF_TRANSFER_LINEAR`. | ||
- When a texture refers to a sampler with mipmap minification, the KTX image MUST contain a full mip pyramid. | ||
- When a texture referencing a KTX v2 image with ASTC compression is used for glTF 2.0 material maps (both color and non-color), the KTX v2 image MUST be of **2D** type as defined in the KTX v2 Specification, Section 4.1. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"referencing a KTX v2 file with ASTC compressed image"
|
||
## KTX v2 Images with Astc compression | ||
|
||
To cover a broad range of use cases, this extension allows different ASTC block sizes as well as LDR, HDR and 3D images. These can be determined from the vkFormat of the KTX image. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the fragmented nature of ASTC implementations you may need to have different extensions for LDR only, LDR+HDR and 3D.
1d83f0e
to
6b6d872
Compare
6b6d872
to
94e4293
Compare
I am not sure why we need to make the distinction? If all of these can be specified from the vkFormat then why have multiple extensions? We probably need to add a bit more details of what other texture types could be supported and the formats like RGBM or RGB9E5 etc. |
Add an extension for native ASTC encoded ktx2 images. Extension supports LDR/HDR and 3D images.