feat: registers sd-transforms; includes refs in source-tokens-only output; resolves refs warnings #3219

adamstankiewicz · 2024-09-07T13:24:47Z

Description

Integrates style-dictionary with sd-transforms (GitHub) from Tokens Studio. Enables support for copying exported tokens from the Tokens Studio plugin in Figma and adding them to the Paragon tokens schema.
Registers a custom transform group paragon-css to abstract away our custom transforms, sd-transforms, and the default CSS transforms. The previously excluded size/rem filter for the default CSS transforms is added back in.
Resolves noisy warnings regarding missing output references in resulting token transforms, by now including the reference token(s) in the output. See screenshots below.
- In Paragon, this use case applies when transforming light theme variant tokens that reference core tokens. Now, these handful referenced core tokens are included in the light theme CSS variables, although duplicative of their definitions in core.css; this is an acceptable tradeoff to remove noisy, disruptive warnings as it only expands the contents of output files by the number of referenced tokens.
- In brand packages (e.g., @edx/elm-theme), this use case applies when either core and/or light tokens reference base, default Paragon tokens without redefining the base token within the brand package. Now, the referenced default Paragon token(s) are included in the brand package output.
Adds build-tokens:watch NPM script to automatically execute the build-tokens CLI on file changes, enabling more rapid development when changing tokens and verifying output without having to manually re-run build-tokens yourself.

Deploy Preview

Resolves output references warning in build output

Related to (3) above. Paragon tokens output on left; brand package tokens output on right (see related @edx/elm-theme PR).

Before

After

Merge Checklist

If your update includes visual changes, have they been reviewed by a designer? Send them a link to the Netlify deploy preview, if applicable.
Does your change adhere to the documented style conventions?
Do any prop types have missing descriptions in the Props API tables in the documentation site (check deploy preview)?
Were your changes tested using all available themes (see theme switcher in the header of the deploy preview, under the "Settings" icon)?
Were your changes tested in the example app?
Is there adequate test coverage for your changes?
Consider whether this change needs to reviewed/QA'ed for accessibility (a11y). If so, please request an a11y review for the PR in the #wg-paragon Open edX Slack channel.

Post-merge Checklist

Verify your changes were released to NPM at the expected version.
If you'd like, share your contribution in #show-and-tell.
🎉 🙌 Celebrate! Thanks for your contribution.

codecov · 2024-09-07T13:37:00Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.76%. Comparing base (029325a) to head (fbb084b).
Report is 1 commits behind head on Peter_Kulko/style-dictionary-v4.

Additional details and impacted files

@@                       Coverage Diff                        @@
##           Peter_Kulko/style-dictionary-v4    #3219   +/-   ##
================================================================
  Coverage                            93.76%   93.76%           
================================================================
  Files                                  229      229           
  Lines                                 3784     3784           
  Branches                               879      906   +27     
================================================================
  Hits                                  3548     3548           
+ Misses                                 232      229    -3     
- Partials                                 4        7    +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

adamstankiewicz · 2024-09-07T14:01:40Z

lib/build-tokens.js

@@ -82,7 +84,6 @@ async function buildTokensCommand(commandArgs) {
            },
          },
        ],
-        transforms: StyleDictionary.hooks.transformGroups.css.filter(item => item !== 'size/rem').concat('color/sass-color-functions', 'str-replace'),


[inform] abstracted into transformGroup: 'paragon-css', although the default CSS transform size/rem is now included, instead of filtered like here.

It seems the primary reason it was filtered out was due to tokens that defined a number while having a $type: "dimension" instead of $type: "number". Ensuring the number type keeps the number value with appending rem as expected, e.g.

--pgn-spacing-card-columns-count: 3;

adamstankiewicz · 2024-09-07T14:02:12Z

styles/css/core/variables.css

@@ -36,26 +36,26 @@
  --pgn-transition-base: all .2s ease-in-out; /* Generic transition for any property change */
  --pgn-transition-progress-bar-bar-transition: width .6s ease;
  --pgn-transition-progress-bar-bar-animation-timing: 1s linear infinite;
-  --pgn-transition-form-control: background-color .15s ease-in-out, border-color .15s ease-in-out, box-shadow .15s ease-in-out;
-  --pgn-transition-form-input: border-color .15s ease-in-out, box-shadow .15s ease-in-out;
+  --pgn-transition-form-control: background-color 0.15s ease-in-out, border-color 0.15s ease-in-out, box-shadow 0.15s ease-in-out;


[inform] sd-transforms includes a transform to add leading zeros where previously missing.

adamstankiewicz · 2024-09-07T14:02:31Z

styles/css/core/variables.css

  --pgn-typography-line-height-sm: 1.5; /* Small line height. */
  --pgn-typography-line-height-lg: 1.5; /* Large line height. */
  --pgn-typography-line-height-base: 1.5556; /* Basic line height. */
  --pgn-typography-font-weight-display-4: var(--pgn-typography-font-weight-bold); /* Font weight of display level 4. */
  --pgn-typography-font-weight-display-3: var(--pgn-typography-font-weight-bold); /* Font weight of display level 3. */
  --pgn-typography-font-weight-display-2: var(--pgn-typography-font-weight-bold); /* Font weight of display level 2. */
  --pgn-typography-font-weight-display-1: var(--pgn-typography-font-weight-bold); /* Font weight of display level 1. */
-  --pgn-typography-font-weight-table-th: bold; /* Table th font weight. */
+  --pgn-typography-font-weight-table-th: 700; /* Table th font weight. */


[inform] sd-transforms changes bold to the corresponding, explicit font weight.

adamstankiewicz · 2024-09-07T14:16:48Z

lib/build-tokens.js

@@ -56,9 +57,14 @@ async function buildTokensCommand(commandArgs) {
    source: tokensSource
      ? [`${tokensSource}/core/**/*.json`, `${tokensSource}/core/**/*.toml`]
      : [],
+    preprocessors: ['tokens-studio'],
+    expand: {
+      typesMap: (await getTokensStudioTransforms()).expandTypesMap,


[inform] See sd-transforms README for more information on expandTypesMap for expand.

Relates to #3222

adamstankiewicz · 2024-09-07T14:27:24Z

tokens/src/core/components/Card.json

@@ -15,7 +15,7 @@
      },
      "columns": {
        "margin": { "source": "$card-columns-margin", "$value": "{spacing.card.spacer.y}" },
-        "count": { "source": "$card-columns-count", "$value": "3" },
+        "count": { "source": "$card-columns-count", "$value": "3", "$type": "number" },


[inform] Bug fix around $type for a number value, as this token does not define a dimension type currently defined above.

adamstankiewicz · 2024-09-07T22:01:48Z

tokens/style-dictionary.js

-  themes.forEach((themeVariant) => {
-    StyleDictionary.registerFilter({
-      name: `isThemeVariant.${themeVariant}`,
-      filter: token => token.filePath.includes(themeVariant),
-    });
-  });
-


[inform] Duplicate. Already defined in paragonFilters above.

adamstankiewicz · 2024-09-08T12:57:53Z

lib/build-tokens.js

@@ -56,9 +57,14 @@ async function buildTokensCommand(commandArgs) {
    source: tokensSource
      ? [`${tokensSource}/core/**/*.json`, `${tokensSource}/core/**/*.toml`]
      : [],
+    preprocessors: ['pgn-annotate-token-extensions-with-references', 'tokens-studio'],


[inform] See docs on preprocessors. pgn-annotate-tokens-with-references is custom.

See sd-transforms README for more information on the tokens-studio preprocessor.

adamstankiewicz · 2024-09-08T13:03:41Z

tokens/src/core/global/breakpoints.json

@@ -3,7 +3,7 @@
    "$type": "dimension",
    "breakpoint": {
      "xs": {
-        "$value": "0",
+        "$value": "0px",


[inform] Ensures consistency in unit with other breakpoints.

…tput; resolves refs warnings

adamstankiewicz · 2024-09-08T19:21:49Z

tokens/utils.js

+          'org.openedx.paragon': {
+            ...referencedToken.$extensions?.['org.openedx.paragon'],
+            [propertyName]: true,
+          },


See https://tr.designtokens.org/format/#extensions for more details about $extensions. It's being used in this implementation to help include references tokens in build output, even if the original filtering doesn't apply.

E.g.,

{ // token metadata like $value, etc. $extensions: { 'org.openedx.paragon': { isReferencedBySourceToken: bool, // useful to include referenced non-source tokens alongside source-only tokens isReferencedByThemeVariant: bool, // useful to include referenced non-theme variant tokens alongside theme variant tokens }, }, }

This is the "magic" behind suppressing the (intentional) previous output reference warnings during build.

brian-smith-tcril

Overall this looks great! I'd be a bit more confident if there were some tests for the new functions in utils.js, but I'm on the fence about that being a blocking requirement. There might be other things that have already landed in the design tokens branch that might be missing tests, and in that case I think making an issue to add tests and just adding this new functions to the list of tests to add could work.

tl;dr

Looks good!
Tests?
Should probably comb through all of the new design tokens functionality and see if there are untested places we should add tests (out of scope for this PR)

adamstankiewicz · 2024-09-09T15:45:35Z

@brian-smith-tcril Thanks for the review! Agreed the lack of tests isn't ideal to ensure confidence in changes. However, as far as I can tell, there are no current tests for any of the logic defined in the tokens folder (inclusive of but not limited to the utils.js file), e.g. the base style-dictionary config, custom transforms/formatters, etc. Given this, I agree with the sentiment to tackle tests as a standalone issue/task separate from this PR.

brian-smith-tcril · 2024-09-09T16:06:58Z

I made an issue for adding tests #3229, we can add to it as we find places where tests are missing.

brian-smith-tcril

adamstankiewicz force-pushed the ags/sd-transforms branch from a68f6df to e8d310f Compare September 7, 2024 13:26

adamstankiewicz force-pushed the ags/sd-transforms branch 5 times, most recently from 5289111 to 97c585b Compare September 7, 2024 14:10

adamstankiewicz commented Sep 7, 2024

View reviewed changes

adamstankiewicz marked this pull request as ready for review September 7, 2024 14:11

adamstankiewicz force-pushed the ags/sd-transforms branch from 97c585b to 2667256 Compare September 7, 2024 14:16

adamstankiewicz commented Sep 7, 2024

View reviewed changes

adamstankiewicz force-pushed the ags/sd-transforms branch from 2667256 to 9e458f0 Compare September 7, 2024 14:24

adamstankiewicz commented Sep 7, 2024

View reviewed changes

adamstankiewicz force-pushed the ags/sd-transforms branch 2 times, most recently from 4edbef8 to 644bb59 Compare September 7, 2024 21:56

adamstankiewicz commented Sep 7, 2024

View reviewed changes

adamstankiewicz force-pushed the ags/sd-transforms branch 3 times, most recently from 27191be to e07b067 Compare September 7, 2024 23:05

adamstankiewicz marked this pull request as draft September 7, 2024 23:33

adamstankiewicz force-pushed the ags/sd-transforms branch from e07b067 to 338109f Compare September 8, 2024 12:27

adamstankiewicz marked this pull request as ready for review September 8, 2024 12:42

adamstankiewicz changed the title ~~feat: registers Tokens Studio transforms; build-tokens:watch NPM script~~ feat: registers sd-transforms; includes refs in source-tokens-only output; resolves refs warnings Sep 8, 2024

adamstankiewicz force-pushed the ags/sd-transforms branch from 338109f to f4c2332 Compare September 8, 2024 12:57

adamstankiewicz commented Sep 8, 2024

View reviewed changes

adamstankiewicz force-pushed the ags/sd-transforms branch from f4c2332 to 75f88ef Compare September 8, 2024 13:40

feat: registers sd-transforms; includes refs in source-tokens-only ou…

fbb084b

…tput; resolves refs warnings

adamstankiewicz force-pushed the ags/sd-transforms branch from 75f88ef to fbb084b Compare September 8, 2024 16:26

adamstankiewicz mentioned this pull request Sep 8, 2024

feat: added expanded tokens #3222

Merged

10 tasks

adamstankiewicz commented Sep 8, 2024

View reviewed changes

brian-smith-tcril reviewed Sep 9, 2024

View reviewed changes

brian-smith-tcril approved these changes Sep 9, 2024

View reviewed changes

PKulkoRaccoonGang approved these changes Sep 9, 2024

View reviewed changes

adamstankiewicz merged commit daffd74 into Peter_Kulko/style-dictionary-v4 Sep 9, 2024
6 checks passed

adamstankiewicz deleted the ags/sd-transforms branch September 9, 2024 18:58

PKulkoRaccoonGang mentioned this pull request Sep 10, 2024

feat: updating to Style Dictionary v4 #3186

Open

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: registers sd-transforms; includes refs in source-tokens-only output; resolves refs warnings #3219

feat: registers sd-transforms; includes refs in source-tokens-only output; resolves refs warnings #3219

adamstankiewicz commented Sep 7, 2024 •

edited

Loading

codecov bot commented Sep 7, 2024 •

edited

Loading

adamstankiewicz Sep 7, 2024 •

edited

Loading

adamstankiewicz Sep 7, 2024

adamstankiewicz Sep 7, 2024

adamstankiewicz Sep 7, 2024

adamstankiewicz Sep 8, 2024 •

edited

Loading

adamstankiewicz Sep 7, 2024 •

edited

Loading

adamstankiewicz Sep 7, 2024

adamstankiewicz Sep 8, 2024

adamstankiewicz Sep 8, 2024

adamstankiewicz Sep 8, 2024 •

edited

Loading

brian-smith-tcril left a comment •

edited

Loading

adamstankiewicz commented Sep 9, 2024

brian-smith-tcril commented Sep 9, 2024

brian-smith-tcril left a comment

feat: registers sd-transforms; includes refs in source-tokens-only output; resolves refs warnings #3219

feat: registers sd-transforms; includes refs in source-tokens-only output; resolves refs warnings #3219

Conversation

adamstankiewicz commented Sep 7, 2024 • edited Loading

Description

Deploy Preview

Resolves output references warning in build output

Before

After

Merge Checklist

Post-merge Checklist

codecov bot commented Sep 7, 2024 • edited Loading

Codecov Report

adamstankiewicz Sep 7, 2024 • edited Loading

Choose a reason for hiding this comment

adamstankiewicz Sep 7, 2024

Choose a reason for hiding this comment

adamstankiewicz Sep 7, 2024

Choose a reason for hiding this comment

adamstankiewicz Sep 7, 2024

Choose a reason for hiding this comment

adamstankiewicz Sep 8, 2024 • edited Loading

Choose a reason for hiding this comment

adamstankiewicz Sep 7, 2024 • edited Loading

Choose a reason for hiding this comment

adamstankiewicz Sep 7, 2024

Choose a reason for hiding this comment

adamstankiewicz Sep 8, 2024

Choose a reason for hiding this comment

adamstankiewicz Sep 8, 2024

Choose a reason for hiding this comment

adamstankiewicz Sep 8, 2024 • edited Loading

Choose a reason for hiding this comment

brian-smith-tcril left a comment • edited Loading

Choose a reason for hiding this comment

tl;dr

adamstankiewicz commented Sep 9, 2024

brian-smith-tcril commented Sep 9, 2024

brian-smith-tcril left a comment

Choose a reason for hiding this comment

adamstankiewicz commented Sep 7, 2024 •

edited

Loading

codecov bot commented Sep 7, 2024 •

edited

Loading

adamstankiewicz Sep 7, 2024 •

edited

Loading

adamstankiewicz Sep 8, 2024 •

edited

Loading

adamstankiewicz Sep 7, 2024 •

edited

Loading

adamstankiewicz Sep 8, 2024 •

edited

Loading

brian-smith-tcril left a comment •

edited

Loading