feat: PBR Preview Plugin #545

jasonjgardner · 2024-05-26T17:37:51Z

Adds PBR material support to Blockbench

Overrides scene materials with a THREE.MeshStandardMaterial compiled from the project textures and texture layers.
Allows adjusting the preview scene exposure and tone mapping
Adds normal map generation action
Adds .texture_set.json export
Adds support Bedrock's MER map format: Decodes and exports MERs

* feat: PBR Preview plugin - Initial plugin commit * upd: Add channel dialog - Created dialog for explicit channel assignment * feat: Add PBR preview controls - Added buttons to control PBR preview scene * upd: Use Preview class for PBR Mode Added PBR mode with a Preview class * feat: Material panel - Updated material definition UI with a custom panel component * fix: Toggle off - Fixed preview update when turn channels off * feat: Added more display settings controls - Added tonemapping select * upd: Render on display changes * fix: Updated localStorage logic - Fixed saving materials with projects * chore: Apply formatting / Remove unused code - Add basic styles for display settings panel * feat: Generate MER map - Created Action to join metal, emissive, and roughness layers into MER map * upd: Hide grid in PBR mode * feat: Plugin icon * feat: Generate texture set JSON * feat: Generate normal map from height map * feat: Add decode MER option - Split existing MER textures into PBR channels * feat: Get emissive color from albedo Infer the emissive color when decoding MERs if the albedo map is defined. * fix: Render nearest neighbor on textures - Made rendering pixelated - Added plugin info - Fixed normal map creation method reference * feat: PBR view in edit and paint modes - Modified plugin to allow PBR preview while in Edit and Paint mode * upd: Redraw canvas on update * upd: Optimized event subscription Added event listeners in a loop and centralized functions * upd: Avoid empty albedo in edit/paint mode * upd: Use layers for material channels * upd: Activate project elements / Update faces * upd: Toggle channels with layer * refactor: OOP material - Storing material channels within extended classes and properties * upd: BB UI / Action-based functions - Refactored to use more native Blockbench UI elements * fix: Corrected deactivation function * fix: Channel unassignment * fix: MER generation * upd: Unassign old channels - Unassign existing channel before assigning it to a new channel. No multi-use channels * upd: Correct texture set export * fix: Hide extended project settings * upd: Features outline in about.md * chore: Add changelog & descriptions * upd: Add project PBR mode - Updated doc blocks - Added safety checks - Added `Project.pbr_active` to replace relying on Settings.get(pbr_active) - Removed unused Actions * upd: Plugin about section - Plus minified latest build

- Temporarily disabled condition on PBR toggle - Removed leftover code from merge

- Added setting to control global metalness and roughness of a scene

- Created Action which bootstraps a texture into a PBR material with layers pre-assigned to channels - Updated about.md with info on new features

- Set global metalness/roughness to undefined when their maps are not present - Added normal map scale - Changed roughness settings icon

- Created tool which paints values across multiple layers/channels

- Remove extra UI elements - Fixed create material texture layer assignment

- Fixed logic to return color value for currently selected channel/layer

- Removed controls from settings panel to reduce jank between panel controls and settings values - Added display conditions for removing channel - Added channel assignment menu to menu group - Added channel assignment for textures that are not layered or with usage inferred - Updated tone mapping/exposure controls to affect all previews - Updated decode MER to split channels into layers - Updated texture set export to include depth map in output

* [Geckolib] Initial port to TypeScript and disable code that was removing hold on last frame loop mode when loading an animation * [Geckolib] Move animation_utils plugin to new blockbench 4.8.0 plugin structure * [Geckolib] Update about.md and fix updateManifest script to patch animation_utils section instead of overwriting format of entire file * [Geckolib] Add README to plugin src folder for developers * [Geckolib] WIP - working on support for "reverse keyframes" action * [Geckolib] Add easing reverse function with jest tests * [Geckolib] Finish implementing first pass of reverse keyframes function * [Geckolib] Integrate typescript-eslint and make some code updates to fix lint errors, update build scripts and docs on build tooling * [Geckolib] Move README.md to plugin root folder * [Geckolib] Add CHANGELOG linked from about.md and bump to version 3.1.0 * [Geckolib] Update CHANGELOG link in about.md to use production URL * [Geckolib] Build plugin for testing * [Geckolib] bump to blockbench-types 4.9.0 which contains my fixes and rebuild plugin * feat: Add texture baking Action - Added texture baking action and dialog * upd: Bake emissive lighting option * fix: Texture set export - Export all required textures in process - Fixed display condition * upd: Change export MER condition * fix: implement easings through compileBedrockKeyframe instead of monkeypatching getArray and breaking blockbench * fix: geckolib keyframes not saving properly in animation json * fix: use ES2017 to avoid weird typescript bug https://stackoverflow.com/questions/51860043/javascript-es6-typeerror-class-constructor-client-cannot-be-invoked-without-ne * Fixes issue with easing and keyframes * disable logging as not needed * Now include a license with plugin info, sourced MIT * Update Changelog for changes/fixes * whoops * Fixes about.md to use the same variables for the gradle info * Remove dead code * v1.0.8 * chore: Commit build output

* refactor: Split into multiple files - Added registry, setups, and teardowns to control plugin actions * feat: Reset exposure Action - Added messages to baking failures - Added reset exposure button - Added changelog build script * feat: labPBR output Action * feat: Decode labPBR channels * chore: Remove unused code - Revert normal map usage in PbrMaterial.getMaterial() * fix: labPBR specular map logic - Fixed bug in alpha channel - Added better specular and normal export names * feat: Material brush presets * feat: Channel assignment panel - Created panel to list current channel assignments * upd: Channel component cleanup * upd: Style channel list * feat: Material brush presets library - Created Dialog for editing, saving and loading material brush presets * upd: Change Action icon - Changed material brush Action icon * upd: Channel assignment condition

- Change create MER inference option

* fix: Channel assignment Action condition * upd: UI conditions / Mode select - PBR channel panel activates paint mode on texture selection * upd: Disable preview on unload - Set material render side - Disable PBR preview on plugin unload * fix: Assignment typo * upd: Channel assignment logic Plus chores * upd: Export MER try/catch * chore: Apply formatting * feat: Textures to material Action - Modified Create Material Action to use any selected textures as the channel sources * upd: Change material alpha test - Fix material texture channel object copying * upd: Channel panel methods - Updated process of finding selected textures in panel component - Added Vue methods to control template - Chore: Removed format npm script in favor of VS Code formatter * upd: Add material creation Undo - Disable buggy channels panel * fix: Include material layers in Undo

JannisX11 · 2024-06-09T22:40:55Z

Wow, really cool plugin!
I haven't had enough time yet to thoroughly test everything. But from what I have seen, I'm not expecting any major issues.

There are the issues that I've found:

You are using the ID "bedrock_entity" in two places, but the correct ID for the format is just "bedrock".
When using different textures as channels, undoing painting changes on them sometimes does not update the texture in the preview correctly
Is this a Minecraft-only plugin? To me it seems to have a wider range of applications
- If it is Minecraft only, let's add a tag "Minecraft", or "Minecraft: Bedrock Edition" while there is no Java support
- If it isn't, please update the about.md text to reflect this. Bedrock should be Minecraft Bedrock Edition or Minecraft Bedrock for short (at least once, to give context), and for bedrock-exclusive features, such as texture sets and MER maps (I believe those are Minecraft-only?) it should be explained that that's what they are for.
An entry to plugins.json is currently missing

Will continue to review tomorrow...

* feat: Channel select menu - Added Action menu to select the assigned channels in place of buggy channels panel component * feat: Generate AO * upd: Add debounce function * upd: Change exposure slider - Changed exposure settings to number input * chore: Update types * fix: Add brush controls * upd: labPBR Action condition * fix: Texture set condition - Fix formats value * upd: MER Action conditions * chore: Add build:watch script * chore: Rearrange material brush panel * upd: Remove material brush primitives * upd: Optimize listener function * upd: Skip disabled faces - Only apply PBR when face is enabled * chore: Commit build output * upd: AO generation height source * upd: Docs and plugin registration

jasonjgardner · 2024-06-10T18:06:55Z

Thanks for checking it out @JannisX11!

I've updated the about.md and plugins.json. Also tried to correct some of the display conditions for various actions. So now the labPBR actions should be present in Java projects, and MER/texture set actions should be present in Bedrock projects. I blame my Copilot for weird things like "bedrock_entity"

Please take a look at the applyPbrMaterial() function. It's used frequently throughout the plugin. Do you have any suggestions on optimizing it? Creating a new MeshStandardMaterial so often is probably incorrect; however, I haven't figured out a better way that works.

Feel free to catch me in Discord too.

JannisX11 · 2024-06-11T17:22:25Z

Do you have any suggestions on optimizing it?

Yes, definitely. It looks like at the moment it is recreating the material for each individual face in the scene. It would already be far more efficient, especially in larger scenes, to first get a list of all used textures, and then only update the material once per texture.
The function is also calling Canvas.updateAllFaces(texture); once per face. This function can be slow, and it should only be called once.
And then of course it would be more efficient to create the material once, and if it already exists to just update the respective values and texture maps.
Also, you can ignore the face.enabled value. It's specific to a certain format, and not what it sounds like.

Unrelated to this: It would be nice to have a way to see what channel a texture or later is currently assigned to.

* upd: Material Brush classification - No longer hijacking location of brush sliders - Refactored applyPbrMaterial to include generic meshes - Only auto-decoding MERs in Bedrock projects * upd: Material brush conditions * upd: Optimize face painting 🤡 * fix: Add Bedrock block format to texture set export

jasonjgardner · 2024-06-12T13:43:00Z

Unrelated to this: It would be nice to have a way to see what channel a texture or later is currently assigned to.

Indeed, it would be 😃

I created a panel component for this feature but I'm having problems with the Vue lifecycle it seems. I've got a thread going in Discord about it: https://discord.com/channels/314078526104141834/1247941848979214406

In the meantime, there is a "Select PBR Channel" menu in the PBR Controls panel which will at least help hunt down the assigned channels.

* refactor: Move normal map methods - Normal map and AO functions moved to own file - Further optimized applyPbrMaterial function - Optimized channel assignment setup/teardown * fix: Correct lighting button - Fixed physically correct lighting toggle - DRY code

* upd: Dispose material preview renderer - Moved material preview image generator to utils * upd: Add material texture property * upd: Dispose old materials * upd: Activate material brush with presets - Select material brush tool after using the presets dialog * upd: Add cleanup to util functions * upd: Use left-hand normals * upd: Material brush tool settings - Added tool_settings property - Updated display conditions of inputs * upd: Channel assignment menu conditions * feat: Material texture thumbnail preview - Texture thumbnail source using PBR material preview render * fix: Material texture size * fix: Material brush size * fix: Remove material texture button in teardown * upd: Brush softness / Better inference - Allow channels to provide individual inference regex - Reduce material brush effect at edges with softness slider

commit 6e44a32 Author: Jason Gardner <[email protected]> Date: Mon Jun 17 18:40:01 2024 -0500 fix: USD output right-side-up commit 442471b Merge: 2c5ba42 06106c8 Author: Jason Gardner <[email protected]> Date: Mon Jun 17 14:30:57 2024 -0500 Merge branch 'master' into feat/usdz commit 2c5ba42 Author: Jason Gardner <[email protected]> Date: Sat Jun 15 09:40:42 2024 -0500 feat: USDZ codec commit 19d657b Author: Jason Gardner <[email protected]> Date: Sat Jun 15 07:21:49 2024 -0500 feat: Material texture thumbnail preview - Texture thumbnail source using PBR material preview render commit 65a1824 Author: Jason Gardner <[email protected]> Date: Fri Jun 14 14:52:52 2024 -0500 upd: Channel assignment menu conditions commit ebf7b9e Author: Jason Gardner <[email protected]> Date: Fri Jun 14 14:51:13 2024 -0500 upd: Material brush tool settings - Added tool_settings property - Updated display conditions of inputs commit 17cce80 Author: Jason Gardner <[email protected]> Date: Fri Jun 14 14:46:13 2024 -0500 upd: Use left-hand normals commit 02a5874 Author: Jason Gardner <[email protected]> Date: Thu Jun 13 11:02:09 2024 -0500 upd: Add cleanup to util functions commit 5a175dc Author: Jason Gardner <[email protected]> Date: Thu Jun 13 10:19:33 2024 -0500 upd: Activate material brush with presets - Select material brush tool after using the presets dialog commit 02d81f3 Author: Jason Gardner <[email protected]> Date: Thu Jun 13 10:03:50 2024 -0500 upd: Dispose old materials commit 3a9bce0 Author: Jason Gardner <[email protected]> Date: Thu Jun 13 09:59:09 2024 -0500 upd: Add material texture property commit cea64f5 Author: Jason Gardner <[email protected]> Date: Thu Jun 13 09:58:09 2024 -0500 upd: Dispose material preview renderer - Moved material preview image generator to utils

* upd: Dispose material preview renderer - Moved material preview image generator to utils * upd: Add material texture property * upd: Dispose old materials * upd: Activate material brush with presets - Select material brush tool after using the presets dialog * upd: Add cleanup to util functions * upd: Use left-hand normals * upd: Material brush tool settings - Added tool_settings property - Updated display conditions of inputs * upd: Channel assignment menu conditions * feat: Material texture thumbnail preview - Texture thumbnail source using PBR material preview render * fix: Material texture size * fix: Material brush size * fix: Remove material texture button in teardown * upd: Brush softness / Better inference - Allow channels to provide individual inference regex - Reduce material brush effect at edges with softness slider * upd: Select paint mode on channel assignment - Fixed normal map creation in new material textures - Avoid creating material textures from existing ones - Optimized channel assignment display conditions * chore: Commit build output

- Updated Export USDZ icon - Fixed channels panel Vue component by updating it externally - Removed channel select menu - Updated docs

- Added custom format for importing/exporting material texture layers

* upd: Support normal map orientation - Added option to create normal map in either DirectX or OpenGL format * upd: USDZ normal map conversion - Added option to specify USDZ normal map type * upd: Disable .bbmat preview export - Disable empty normal map painting with material brush - Added changelog and about.md info on new features

jasonjgardner · 2024-06-20T17:32:17Z

Sorry @JannisX11! I couldn't help myself. I'll refrain from adding more features for now.

I've fixed the channels panel component issue, as well as some of the logic behind the material brush tool's size, and various other issues. #2379 is a blocker for getting the Undo option to work with the tool.

I've also added support for USDZ exports and created a .bbmat codec for importing/exporting PBR materials in Blockbench.

Here are some assets I created for testing. (The .bbmodel and .bbmat are saved as .json just for the sake of attaching them to this comment. The USDZ model is pretty cool in AR if you have an iPhone.)

example_materials.bbmodel
Tiles.bbmat

USDZ model: boombox_usdz.zip

And then of course it would be more efficient to create the material once, and if it already exists to just update the respective values and texture maps.

I haven't been able to pull this off yet. Using needsUpdate in THREE.js hasn't worked in my implementations. I think Blockbench may need updates to the Canvas class, and changes to some UI template strings, for the most efficient method of updating the materials.

JannisX11 · 2024-06-21T17:43:08Z

The PBR channels panel is nice!

There are currently a lot of unwarranted changes in plugins.json that are causing merge conflicts.

I've noticed a small error message when selecting the Neutral Tone Mapping option: three.min.js:6 THREE.WebGLProgram: Unsupported toneMapping: NaN

The applyPbrMaterial function still looks extremely inefficient. It is creating a new material for each face of each element in the scene. The Canvas.updateAllFaces function also seems to get called once for every material, so if there are 200 elements in a scene, each element will get its faces updated 200 times.
Creating a new material each time it updates is probably okay, but you can do it once per texture, not once per face.

- Corrected error in updateFace logic to reduce the number of times updateAllFaces is called - Disabled normal map channel in material brush until a color input is available - Disabled flipping normal map in preview. Normal map type should not be assumed

- Remove tone mapping which is not available in Blockbench's THREE.js

jasonjgardner · 2024-06-22T13:27:35Z

Aha, thank you! There was an error in my logic for updating the faces. They are no longer redundantly updated.

I've also fixed the merge conflict. The plugins.json file was formatted at some point during a merge. My mistake.

Creating a new material each time it updates is probably okay

Ideally, the PBR material would also be a ShaderMaterial. I think that would play nicer with Blockbench's existing Texture material. It'd also allow for previewing popular shader features, like POM for Java textures. I'm not well-versed enough in shaders to get it to work right now.

JannisX11 · 2024-07-12T11:21:39Z

Cool, the performance seems to be okay now for most models.
It looks like you didn't push the commit addressing the plugins.json merge issue. The file in this branch still contains changes to a lot of unrelated plugins in plugins.json.
JannisX11/blockbench#2379 is now fixed so you should be able to address the undo issue.

- Sync PR with upstream

- Fixed VS Code stomping on JSON formatting

jasonjgardner changed the title ~~feat: PBR Preview Plugin (#1)~~ feat: PBR Preview Plugin May 26, 2024

jasonjgardner added 17 commits May 26, 2024 13:10

fix: Toggle button condition

6927ebe

- Temporarily disabled condition on PBR toggle - Removed leftover code from merge

fix: Disable conditions for MER export

18fc13e

fix: Change texture set format version

390e80b

fix: Add metalness/roughness base values

0139629

upd: Global metal/roughness setting

05875de

- Added setting to control global metalness and roughness of a scene

feat: New material Action

d93d643

- Created Action which bootstraps a texture into a PBR material with layers pre-assigned to channels - Updated about.md with info on new features

upd: Add normal scale

c54e051

- Set global metalness/roughness to undefined when their maps are not present - Added normal map scale - Changed roughness settings icon

feat: Material brush

b6bf6db

- Created tool which paints values across multiple layers/channels

upd: Change display conditions

41f7cad

- Remove extra UI elements - Fixed create material texture layer assignment

fix: Material brush paint on selected layer

08d484c

- Fixed logic to return color value for currently selected channel/layer

Merge branch 'JannisX11:master' into master

f23bc5c

Merge remote-tracking branch 'upstream/master'

70822e1

fix: Create MER output

040eccc

- Change create MER inference option

jasonjgardner added 5 commits June 13, 2024 06:37

chore: Commit build output

ccc9e34

Merge remote-tracking branch 'upstream/master'

3916622

fix: MER decoding

06106c8

jasonjgardner added 9 commits June 17, 2024 18:46

revert: Remove material preview

a9e5d66

upd: Apply PBR on USDZ export

e792e53

upd: Change material texture icon

fd66faa

fix: Channels panel lifecycle

3d66662

- Updated Export USDZ icon - Fixed channels panel Vue component by updating it externally - Removed channel select menu - Updated docs

docs: Add USDZ section

60a655e

feat: .bbmat format (#10)

2723050

- Added custom format for importing/exporting material texture layers

jasonjgardner added 3 commits June 22, 2024 08:02

fix: Face update logic

3c47eb5

- Corrected error in updateFace logic to reduce the number of times updateAllFaces is called - Disabled normal map channel in material brush until a color input is available - Disabled flipping normal map in preview. Normal map type should not be assumed

Merge remote-tracking branch 'upstream/master'

b9896ed

fix: Remove neutral tone mapping

c56669d

- Remove tone mapping which is not available in Blockbench's THREE.js

Merge remote-tracking branch 'upstream/master'

21db458

jasonjgardner and others added 3 commits July 12, 2024 06:52

upd: Add Undo aspect

64f0fe8

- Sync PR with upstream

fix: Plugins conflict

bf53bf2

- Fixed VS Code stomping on JSON formatting

Fix plugins.json conflicts

e6de06b

JannisX11 merged commit ccd74b1 into JannisX11:master Aug 24, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: PBR Preview Plugin #545

feat: PBR Preview Plugin #545

jasonjgardner commented May 26, 2024

JannisX11 commented Jun 9, 2024

jasonjgardner commented Jun 10, 2024

JannisX11 commented Jun 11, 2024 •

edited

Loading

jasonjgardner commented Jun 12, 2024

jasonjgardner commented Jun 20, 2024

JannisX11 commented Jun 21, 2024

jasonjgardner commented Jun 22, 2024

JannisX11 commented Jul 12, 2024

feat: PBR Preview Plugin #545

feat: PBR Preview Plugin #545

Conversation

jasonjgardner commented May 26, 2024

JannisX11 commented Jun 9, 2024

jasonjgardner commented Jun 10, 2024

JannisX11 commented Jun 11, 2024 • edited Loading

jasonjgardner commented Jun 12, 2024

jasonjgardner commented Jun 20, 2024

JannisX11 commented Jun 21, 2024

jasonjgardner commented Jun 22, 2024

JannisX11 commented Jul 12, 2024

JannisX11 commented Jun 11, 2024 •

edited

Loading