GLTFLoader: Conflicting mesh/primitive/geometry mappings

[donmccurdy]

Description

In glTF's data model, we have:

- node: GLTF.Node
  - mesh: GLTF.Mesh
    - prim: GLTF.MeshPrimitive
      - attribute: Record<string, GLTF.Accessor>
      - material: GLTF.Material
    - prim: GLTF.MeshPrimitive
      - attribute: Record<string, GLTF.Accessor>
      - material: GLTF.Material
    ... 

Note that there is no distinct concept of a "geometry" here. Instead, we look for attributes (collections of named accessors) that happen to contain the same accessors, and cache them...

three.js/examples/jsm/loaders/GLTFLoader.js

Lines 2450 to 2456 in 09c38ab

	function createPrimitiveKey( primitiveDef ) {
	let geometryKey;
	const dracoExtension = primitiveDef.extensions && primitiveDef.extensions[ EXTENSIONS.KHR_DRACO_MESH_COMPRESSION ];
	if ( dracoExtension ) {

... so that if other primitives use the same attributes, they refer to the same BufferGeometry and we avoid a duplicate upload. If any attributes differ, the whole BufferGeometry must be duplicated (see #17089).

If (like the example above) there are multiple primitives in the mesh, we get this in three.js...

- node: THREE.Object3D
  - mesh: THREE.Group
    - prim: THREE.Mesh<BufferGeometry, Material>
    - prim: THREE.Mesh<BufferGeometry, Material>

... and if there were only one primitive in the mesh, we'd drop the THREE.Group and try to "merge" the mesh and primitive concepts, which inherently could lose names or .userData.

---

I noticed today that:

1. glTF mesh primitives may have .extras/userData
2. GLTFLoader assigns a primitive's .extras/userData to a BufferGeometry
3. If the geometry is cached, a primitive may get geometry with the wrong .extras/userData

The userData caching issue isn't urgent; I'm not aware that it's affecting users.

But relatedly (reported in #29753) if a glTF mesh has only one primitive, then GLTFLoader will collapse the primitive and the mesh into one THREE.Mesh object, and the mesh name appears nowhere in the resulting scene.

We could fix the .userData issue just by including .extras/userData in the cache key. May duplicate geometry and raise VRAM cost in rare cases.

To fix that and the missing mesh name issue, we would probably want to avoid 'flattening' the scene graph: when a mesh has only one primitive, still return a "Group>Mesh", not just a "Mesh", corresponding to the glTF "Mesh>Prim" pair. Then assign the primitive's .extras/userData to the Mesh, not the BufferGeometry. Arguably makes more sense than assigning .extras/userData to the Geometry, because a glTF primitive has a material and is uniquely mappable to a three.js Mesh, whereas we want to aggressively cache geometries for performance.

Reproduction steps

1. Load prim_extras_test.gltf (attached)

prim_extras_test.zip

2. Observe that .extras in the glTF file are unique per primitive

"meshes": [
{
    "name": "MeshA",
    "primitives": [
    {
        "attributes": {
          "POSITION": 0,
          "COLOR_0": 1
        },
        "mode": 0,
        "extras": { "data": "PrimA" }
    }
    ]
},
{
    "name": "MeshB",
    "primitives": [
    {
        "attributes": {
          "POSITION": 0,
          "COLOR_0": 1
        },
        "mode": 0,
        "extras": { "data": "PrimB" }
    }
    ]
}
],

3. Observe that geometry in the resulting scene is reused for both meshes, so the second .userData goes missing, and that the mesh names occur nowhere in the scene graph (only the parent node's name is found).

mesh.name: NodeA
mesh.userData: {"name":"NodeA"}
mesh.geometry.userData: {"data":"PrimA"}

mesh.name: NodeB
mesh.userData: {"name":"NodeB"}
mesh.geometry.userData: {"data":"PrimA"}

The mesh's name is lost because we've flattened the scene graph slightly: if a mesh has more than one primitive, the mesh corresponds to a Group, if the mesh has only one primitive, we skip the Group. I think this might be too complex.

Code

The model used to test this issue was generated with the glTF Transform script below.

script.js

import { NodeIO, Document, Primitive } from '@gltf-transform/core';

const document = new Document();
const buffer = document.createBuffer();

const primA = createPointsPrim(document, buffer).setExtras({ data: 'PrimA' });
const primB = primA.clone().setExtras({ data: 'PrimB' });

const meshA = document.createMesh('MeshA').addPrimitive(primA);
const meshB = document.createMesh('MeshB').addPrimitive(primB);

const nodeA = document.createNode('NodeA').setMesh(meshA).setTranslation([0, 0, 0]);
const nodeB = document.createNode('NodeB').setMesh(meshB).setTranslation([0, 0, 1]);
const scene = document.createScene().addChild(nodeA).addChild(nodeB);
document.getRoot().setDefaultScene(scene);

const io = new NodeIO();
await io.write('./prim_extras_test.gltf', document);

function createPointsPrim(document, buffer) {
	const position = document
		.createAccessor()
		.setType('VEC3')
		.setBuffer(buffer)
		.setArray(
			// prettier-ignore
			new Float32Array([
				0, 0, 0, // ax,ay,az
				0, 0, 1, // bx,by,bz
				0, 1, 0, // ...
				1, 0, 0,
			]),
		);

	const color = document
		.createAccessor()
		.setType('VEC4')
		.setBuffer(buffer)
		.setNormalized(true)
		.setArray(
			// prettier-ignore
			new Uint8Array([
				0, 0, 0,
				255, 0, 0,
				255, 255, 0,
				255, 0, 255,
				255, 0, 0, 255,
			]),
		);

	return document
		.createPrimitive()
		.setMode(Primitive.Mode.POINTS)
		.setAttribute('POSITION', position)
		.setAttribute('COLOR_0', color);
}

Live example

Open the model attached above in https://threejs.org/editor/.

Screenshots

No response

Version

r168

Device

Desktop, Mobile, Headset

Browser

Chrome, Firefox, Safari, Edge

OS

Windows, MacOS, Linux, ChromeOS, Android, iOS

[donmccurdy]

tl;dr — I think we should perhaps stop trying to automatically flatten the scene graph in GLTFLoader (reversing some changes from #11944) and instead return a consistent mapping...

# gltf
- node: GLTF.Node
  - mesh: GLTF.Mesh
    - prim: GLTF.MeshPrimitive
      - attribute: Record<string, GLTF.Accessor>
      - material: GLTF.Material
    ...

# three.js
- node: THREE.Object3D
  - mesh: THREE.Group
    - prim: THREE.Mesh<BufferGeometry, Material>
    ...

... even if there's only one THREE.Mesh in a THREE.Group.