Initial migration of site to 11ty

[parlough]

I'm sorry that this ended up as such a large pull request, but that's mostly because it essentially touched every file with a small set of changes. See the different sections below for considerations for review or future reference. Please note that we'd like this to land despite it not being perfect, as then pain points can be identified and improved. Then I can also follow up with proper documentation as the implementation and syntax is stabilized. I'll be staying on top of any reported issue. If anything goes majorly wrong, we can always revert the change as well. Thanks for your time 💙

Resolves #5177
Resolves #3846
Resolves #5323
Fixes #4297
Fixes #4919
Closes #4631
Contributes to #4163

Tip

Reviewing this in GitHub's UI might be slow. Consider cloning the branch locally.

Changes potentially worth reviewing from a technical perspective are:

• Implemented 11ty in eleventy.config.js with customizations in /src/_11ty.
  • eleventy.config.js configures the site build, enables used plugins, and connects the custom logic added in /src/_11ty. This will be greatly simplified and better documented in follow-up work.
• Replaced prettify, rouge, and custom code excerpter logic with custom markdown plugin.
  • You can find this at /src/_11ty/plugins/highlight.js (super messy for now).
  • New custom theme is at /src/_11ty/syntax/dash-light.js.
  • The implementation of this needs a lot of cleanup and will be simplified greatly in follow-up work to rely on recent improvements in underlying highlighting package.
• Added a build and serve command to dart run dart_site (found in /tool/dart_site/lib).
• The deploy configs in /cloud_build have been migrated from using Docker to use Node to build the site. I reused the Docker image we built for firebase_tools, as that already has Node. Let me know if I shouldn't though :)
• Removed remnants of previous infra (Jekyll files, gem files, docker files, makefile, etc).
• All written docs and assets have been moved into /content to enable a separation from site tooling and configuration.

Changes potentially worth reviewing or knowing about from a writer/contributor perspective:

• Major: All doc content has been moved into /content. This allows site implementation to be separate from content, potentially enabling easier downstream localization as well.
• The pre-existing global variables are now defined in /src/_data/site.yml.
• Alerts/asides use a new syntax and have a design closer to other doc sites, search for ::: for examples.
• Prettify doesn't exist anymore. Always use Markdown code blocks. They now always support highlighting with [! !].
• Terminal code blocks now use console instead of terminal. Just textual ones use plaintext instead of nocode.
• Markdown code blocks now support specifying a title/filename with syntax like title="main.dart". Various code blocks have been updated to use it instead of one-off solutions.
• Code blocks have a new design, with slightly bigger text, new colors, reduced padding, a more defined border, and a tag with the code block's language.
• The README is updated with the core setup requirements. Note that more comprehensive docs will come.
• To change the anchor of a header, a slightly modified {:#new-id} syntax is now used.
• The include syntax is slightly different, put quotes around the include filename and define arguments with colons: {% include 'linter-rule-mention.md', rule:'unnecessary_getters_setters' %}.

Current issues:

The following are some issues or regressions that I plan to accept in the short term to get this landed.

• Terminal/console code blocks do not have their formatted black background yet. They appear as normal code blocks.
• Incremental rebuild is slower than it should be due to a new bug in 11ty.

Follow-up work:

The following are some remaining tasks and improvements that I'd like to complete in follow-up so this can land and stretch its feet. These improvements can then be reviewed in isolation and incorporate what I've learned from deploying.

• Incorporate fix for 11ty's current incremental rebuild issue.
• Add back external link icons where it makes sense during static build.
• Reintroduce terminal code block's having a custom style.
• Standardize old and new liquid functions and filters to modern 11ty suggestions for consistent experience.
• Document the 11ty, liquid, and custom features we use that are relevant to site writers and contributors.
• Speed up sidenav and TOC generation. Potentially reduce needed dependencies while doing so.
• Clean up implementation of custom 11ty and Markdown plugins. Rely on new features in underlying syntax highlighting plugin rather than implementing our own form.
• Automate image optimization in production builds.
• Use TypeScript for custom 11ty functionality to avoid painfully adding JSDocs everywhere.
• Add back a devcontainer setup for users who want to a functioning workflow in the browser.
• Add skippable setup checks to dart_site tool
• Move the Cloud Build commands to dart_site tool

[dart-github-bot]

Visit the preview URL for this PR (updated for commit 30ff913):

https://dart-dev--pr5483-feat-eleventy-migration-ph0qb4b2.web.app

[khanhnwin]

You're awesome, @parlough! I'm slowly but surely making my way through this PR! <3 Thank you for doing this!!

[chalin]

Oh my goodness! I'm floored 😲. This is a phenomenal upgrade! Well done @parlough! 🙌🏻

No joke that "GitHub's UI might be slow" 🙃

I've locally tested the setup, serve, and check instructions. Not only do they work like a charm, but the entire process was so quick! 🐎

I've included a few tweaks to the README as comments, and noted a few differences in the rendered site. You might know about these differences already (and I understand that you'rre wanting to tackle the upgrade incrementally), but I thought I'd mention them just in case.

I happened to stumble upon this work because I was wondering about the status of the code-excerpt tools. In the opening comment you state "Replaced ... custom code excerpter logic with custom markdown plugin". But I get the impression that the code-excerpt tools are still being invoked via tool/dart_site/lib/src/commands/refresh_excerpts.dart, is that correct?

Again, bravo for such an amazing job!

[chalin]

Not "first-timer friendly" anymore? :)

[parlough]

We are! But when I was updating the README I realized the label isn't there anymore, so I removed this link for now to avoid any confusion.

I have follow-up work to introduce more contributing docs for the site infra, syntax, and writing practices. Once that's done and this setup is finalized, I'll bring something back here so it's actually easy to get going as a "first-timer".

[chalin]

Consider adding a .nvmrc file containing lts/*.

[parlough]

I've never used nvm, but this seems like an easy win for those that do. Added! Is it popular enough that I should add a mention to the README?

Thanks :)

[atsansone]

@parlough : Indeed it is.

[chalin]

FYI, external links like these aren't displayed with an external-link icon anymore.

[parlough]

Yeah I left this out on purpose to figure out as follow up.

The Google documentation style guide recently updated its guidelines to suggest not adding external link icons and instead improve surrounding context. They explained that "Readers and writers have expressed confusion about the meaning and usage of this icon".

My initial thinking is that in most cases it's sufficient to explain the destination in the surrounding text/context. However, in UI elements like the sidenav, I feel external icons might still necessary.

[chalin]

The Google documentation style guide recently updated its guidelines to suggest not adding external link icons and instead improve surrounding context. They explained that "Readers and writers have expressed confusion about the meaning and usage of this icon".

Oh! That is very good to know. Thanks for providing those links to the style guide.

[chalin]

I'm curious, is the {:#...} syntax required by 11ty?

[parlough]

Short explanation: No, I configured it that way.

Longer explanation:

Compared to Jekyll, 11ty comes with a lot less in the box, but that also allows it to be more configurable.

The default Markdown package is markdown-it and this support comes from a plugin for that called markdown-it-attrs. It defaults to just { as the starting delimiter, but I configured it to start with {:. I had a reason at the time, but I am struggling to recall why now... If I remember, I'll document it, otherwise I might go back to the default. Thanks for asking!

[chalin]

The linter rule lines are styled differently (maybe you know already?):

• Before:

  

• After:

  

[parlough]

Good catch, I didn't notice this one!

I went ahead and updated the include to add the styling back and accept rules rather than rule, rule1, and rule2.

[chalin]

Markdown code blocks now support specifying a title/filename with syntax like title="main.dart". Various code blocks have been updated to use it instead of one-off solutions.

@parlough - I wanted to see some examples of the use of title=... but didn't find any. Can you point me to one? Thanks!

[parlough]

👋 Hey Patrice @chalin! I really appreciate you taking a look! I hope you are doing well :)

This is a phenomenal upgrade! Well done @parlough! 🙌🏻

I'm not sure I would say phenomenal with some of my 11ty and highlighting implementation 🙈, but I'm glad you think so! JS is at least a lot more familiar to all of us than Ruby.

For more context: The code block functionality ended up being the bulk of my mess and change here. Since I implemented it though, the syntax highlighting package I was using added a bunch of new functionality and configurability. Once this lands I'll try to use as much of that as I can. It looks like they consolidated with their upstream package and will release 1.0.0 soon (shiki).

I've locally tested the setup, serve, and check instructions. Not only do they work like a charm, but the entire process was so quick! 🐎

That's the goal!! Ruby, Jekyll, and Docker or some combination of them have been causing issues often and I think Googlers can't easily use Docker on their macs anymore. I would love all of this to be implemented in Dart, but for now, I think I can safely assume developers interested in contributing are more likely to have Node setup than Ruby.

I would also like to add a mechanism for using the dart_site tooling with a simple command from any nested directory. I haven't decided the best way though. Perhaps globally activating it with pub or a shell and ps script that you can add to your path?

I've included a few tweaks to the README as comments, and noted a few differences in the rendered site. You might know about these differences already (and I understand that you'r wanting to tackle the upgrade incrementally), but I thought I'd mention them just in case.

I didn't realize all of these, so thank you! Some I'm planning on leaving for follow-up work to keep this PR from ballooning further.

I happened to stumble upon this work because I was wondering about the status of the code-excerpt tools. But I get the impression that the code-excerpt tools are still ... is that correct?

They're mostly the same as you left them, outside of a few fixes and being invoked through that Dart script you mentioned. The changes in this PR are about not using the instructions in code block generation, so handling all Markdown code blocks the same.

Based on how we've ended up using the tools though, I started working on a version of the tooling with less functionality that doesn't use package:build_runner in dart-lang/site-shared#201. It's not done and needs to be tested, but I'd like to get back to it soon :D. If you're interested, I'd appreciate any feedback you have on that!

@parlough - I wanted to see some examples of the use of title=... but didn't find any. Can you point me to one? Thanks!

There aren't too many usages yet, but there's a few in /effective-dart/usage#prefer-relative-import-paths. Each linter rule page uses them as well: /tools/linter-rules/avoid_relative_lib_imports#usage.

For line numbers, I think /tools/analysis might be the only usage so far.

---

A side note, thanks for your documentation and examples on updating Docsy and its downstream sites to Bootstrap 5! I really need to work on that migration soon and your work will be a super useful reference.

[khanhnwin]

I went through all of the infra changes, mostly glossed over the .md changes as those are just syntax migrations.

Most of my comments are about documentation the code and making sure we have sufficient institutional knowledge about the code and why you opted to build things a certain way. (Highlighting code comes to mind!)

We might also want to flesh out the dependency instructions just a tad bit more because my biggest road block was updating node/npm (I think nvm would address this. I thinks contributors should check what they have first as opposed to running into an error and then having to correct it/upgrade. Also enabling corepack (permissions issue, needed sudo).

Otherwise, I was up and running super fast! ⚡ This PR looks WONDERFUL @parlough ! I can't wait to have the new setup in place. Thank you thank you thank you!!

[khanhnwin]

CC @johnpryan you've touched the dart.dev codebase more than I have, check out what Parker's done!!

[parlough]

I went through all of the infra changes, mostly glossed over the .md changes as those are just syntax migrations.

Thank you @khanhnwin!! I really appreciate you taking the time and all your feedback. It was super helpful to me. I addressed most of your comments, mentioned them as follow-up, and will work on some final improvements now.

(Highlighting code comes to mind!)

Don't worry about this too much as it's the main area I expect to change and be reduced significantly. I'll make sure the final version is easier to read and super well documented.

We might also want to flesh out the dependency instructions just a tad bit more because my biggest road block was updating node/npm (I think nvm would address this. I thinks contributors should check what they have first as opposed to running into an error and then having to correct it/upgrade. Also enabling corepack (permissions issue, needed sudo).

This is an amazing point. The main goal is to be easy and straightforward to set up so any improvements here are huge. I'm hesitant to suggest nvm as that's another dependency on top of what's already needed and it's incompatible with some workflows. In the end, I want the instructions to be agnostic of the installation mechanism, but I'll try some more adjustments to make it clearer, including mentioning a version check.

Could you expand on the issues you had updating node/npm? Or what would have helped you? I'd like to see how I could make it clearer for those that hit similar issues.

Otherwise, I was up and running super fast! ⚡

Together we can keep making it faster =]

Thank you thank you thank you!!

Thanks for your help getting to this point. It ended up taking a while to get here as I had to work on other things, so I appreciate your patience!

[khanhnwin]

Fair point on nvm! I hadn't thought of that, but agreed. Standardizing the process should be the goal.

Here's the steps that I went through:

$ pnpm install
ERROR: This version of pnpm requires at least Node.js v16.14
The current version of Node.js is v16.13.0
Visit https://r.pnpm.io/comp to see the list of past pnpm versions with respective Node.js version support.

$ sudo npm cache clean -f
npm WARN using --force Recommended protections disabled.

 $ sudo npm install -g n
changed 1 package, and audited 2 packages in 630ms
found 0 vulnerabilities

$ sudo n stable
  installing : node-v20.11.0
       mkdir : /usr/local/n/versions/node/20.11.0
       fetch : https://nodejs.org/dist/v20.11.0/node-v20.11.0-darwin-x64.tar.xz
     copying : node/20.11.0
   installed : v20.11.0 (with npm 10.2.4)

$ pnpm install
Lockfile is up to date, resolution step is skipped
Packages: +755
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

   ╭──────────────────────────────────────────────────────────────────╮
   │                                                                  │
   │                Update available! 8.14.3 → 8.15.1.                │
   │   Changelog: https://github.com/pnpm/pnpm/releases/tag/v8.15.1   │
   │     Run "corepack prepare [email protected] --activate" to update.     │
   │                                                                  │
   │      Follow @pnpmjs for updates: https://twitter.com/pnpmjs      │
   │                                                                  │
   ╰──────────────────────────────────────────────────────────────────╯

Progress: resolved 755, reused 0, downloaded 754, added 755, done
node_modules/.pnpm/[email protected]/node_modules/protobufjs: Running postinstall script, done in 628ms
node_modules/.pnpm/[email protected]/node_modules/re2: Running install script, done in 2.4s

dependencies:
+ bootstrap-scss 4.6.2

devDependencies:
+ @11ty/eleventy 3.0.0-alpha.4
+ firebase-tools 13.0.3
+ hast-util-from-html 2.0.1
+ hast-util-select 6.0.2
+ hast-util-to-html 9.0.0
+ hast-util-to-text 4.0.0
+ html-minifier-terser 7.2.0
+ js-yaml 4.1.0
+ markdown-it 14.0.0
+ markdown-it-anchor 8.6.7
+ markdown-it-attrs 4.1.6
+ markdown-it-container 4.0.0
+ markdown-it-deflist 3.0.0
+ markdown-it-table 4.1.1
+ sass 1.70.0
+ shikiji 0.10.0

Done in 15.2s

I'd probably make the first step in the instructions run node --version and make sure it is up to date with X version that pnpm requires - this makes sure that the result of the very first command contributors are running to contribute to the site isn't an error.

Admittedly it's been a bit since I've worked with node and npm, so it took a google search or two to remember how to upgrade node from the command line instead of downloading it from the website since I was already in the command line, didn't want to go to the browser, download, install, etc.

[parlough]

Thanks for expanding @khanhnwin!

It's a bit of challenge to resolve all of these potential issues since it really depends on your OS and the mechanism you used to install Node, as well as how you configured it during installation. I've however added some more details, including version checks like you suggested, a note about nvm, and for next steps if you do need to update. Thanks for thinking of this!

I hope to add some extra checks to dart_site tool to verify the setup as well. That will point out to users if there are any issues and what steps to consider to resolve them.

[atsansone]

@parlough : None of the code blocks have the copy button anymore. Will this be redressed in a subsequent PR? If so, no problem. If not, can you create an issue to handle this? I think this functionality is definitely needed.

[parlough]

@atsansone The Dart website never had a copy button actually, but the changes here will enable me adding one in a way that's consistent and works for here and the Flutter site. It's definitely part of my earlier follow-up work :) - I'll make sure to ping you on that.

[chalin]

👋 Hey Patrice @chalin! I really appreciate you taking a look! I hope you are doing well :)

Hi! 👋🏻 @parlough

I'm not sure I would say phenomenal with some of my 11ty and highlighting implementation 🙈, but I'm glad you think so! JS is at least a lot more familiar to all of us than Ruby.

Yes, +100 for JS.

For more context: The code block functionality ended up being the bulk of my mess and change here.

👍🏻

That's the goal!! Ruby, Jekyll, and Docker or some combination of them have been causing issues often and I think Googlers can't easily use Docker on their macs anymore.

The setup proposed here is sooo much better.

I understand your hesitation to bring in another dependency. But for those who can use it, nvm makes installing and periodically updating Node a simple single-command affair :). Here's how I phrased the tool-use suggestion elsewhere:

Install or upgrade to the active LTS release of Node.js. We recommend using nvm to manage your Node installation. Under Linux, run the following command (which will install/upgrade to the version specified in .nvmrc):

nvm install

That's it, whether I'm installing afresh or wanting to upgrade Node to the latest active LTS release.

They're mostly the same as you left them, outside of a few fixes and being invoked through that Dart script you mentioned. The changes in this PR are about not using the instructions in code block generation, so handling all Markdown code blocks the same.

👍🏻

Based on how we've ended up using the tools though, I started working on a version of the tooling with less functionality that doesn't use package:build_runner in dart-lang/site-shared#201. It's not done and needs to be tested, but I'd like to get back to it soon :D. If you're interested, I'd appreciate any feedback you have on that!

I took a quick look. I think that it's a great idea to avoid build_runner. Your proposed simplifications seem quite reasonable to me; the original tool had features inherited from the Angular Dart and Angular JS days.

I'd love to use such a "lighter-weight" tool. Looking forward to see dart-lang/site-shared#201 land as well. (I've subscribed so that I can keep up with your work.)

There aren't too many usages yet, but there's a few in /effective-dart/usage#prefer-relative-import-paths. Each linter rule page uses them as well: /tools/linter-rules/avoid_relative_lib_imports#usage.

For line numbers, I think /tools/analysis might be the only usage so far.

Thanks for pointing out those examples.

A side note, thanks for your documentation and examples on updating Docsy and its downstream sites to Bootstrap 5! I really need to work on that migration soon and your work will be a super useful reference.

I was wondering about that (when I saw the BS v4) dependency still there. I'll be very glad if any of that is helpful to others. Let me know if you have any questions. (Interestingly, I might be tackling the Docsy upgrade of BS from 5.2 to 5.3 next week. It isn't a major version upgrade, but there are significant changes in the way colors are handled.)

Again, great work. I'm glad to see this merged ✨