Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Documentation on animation formats supported (e.g. VRM Expression animation?) #685

Open
alankent opened this issue Dec 19, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@alankent
Copy link

Is your feature request related to a problem? Please describe

I have a three.js application that animates VRM models. (Still early days, but small sample result https://youtu.be/9-00dQgjdUU?si=Y6TW55eOqUK985Fi). I have added additional expressions to VRM 1.0 models exported from VRoid Studio (blush, snear, etc). I had a few questions I could not work out from the documentation. I thought I would see if any docs before I got into this too deeply. (If not, a feature request for docs in this area, if you had the time.) I am using the default MToon shader. I am exploring the feasibility of exporting animation clips I create so they can be imported (and rendered) in Blender.

Questions I had:

  • Expression vs blend shape raw animation: I use standard expressions and add custom blend shapes and expressions. If I want to animate the "Snear" custom expression, can I animate VRMExpression_{name}.weight? Or should I animate the raw blend shape weights instead (bake them out?)
  • Expressions vs texture tiling animation: Expressions include texture offsets for some materials, so I can add a blush to the cheek or replace eyes with dizzy looks. Will they work with VRM Expression animation too? Will they work in Blender with animation playbacks? What property name should I animate to adjust texture offsets? (In three.js with the VRM extension from Pixiv I am using expression.addBind(new VRMExpressionTextureTransformBind({ material, scale, offset })); to add texture tiling into VRM Expressions.
  • Normalized bone names vs raw bone names: Should I export animation clips using the normalized VRM bone names or the raw bone names?
  • Springbone physics - I currently bake out animation clips for all the raw bones so springbone physics (for hair etc) occur in three.js, and I capture the result for exporting. I think this should be fine.

Describe the solution you'd like

I would like to know (via documentation or otherwise) the correct way to create GLB files containing animations that will work with VRM features like expressions and normalized bone names (based on the Pixiv extension for Three.js). Can I animate VRM Expression properties? Or should I bake out animations using the raw blend shape names, bone names, and texture tiling offsets (the latter seems the trickiest to be portable as it relates to material properties).

Describe alternatives you've considered

I bake expression animations to blend shape weights directly for OpenUSD export. (I do this for spring bones as well, so all the physics is baked into the animation clips.) I can do the same for blend shape weights - bake animation clips to animate blend shape weights instead of VRM Expressions. (I don't know yet how to bake texture tiling offset animations.) In three.js with the VRM extension, as soon as you start using Expressions, it overwrites any blend shape animations you do. So you use VRM expression animation OR blend shape animation - you cannot mix them.

Additional context

I saw the recent Goo Engine announcement that they are going to be merging their render engine support into Blender (as part of the Blender team), so thought exporting to Blender might be useful. Not sure how that will work with MToon materials.

@saturday06 saturday06 added the enhancement New feature or request label Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants