Upstream CF docs should be discoverable via cloud.gov

[mogul]

In order to ensure our users can find their way to all the relevant documentation for our service, we want to ensure upstream CF documentation is incorporated into cloud.gov navigation and search results.

Acceptance Criteria

• GIVEN I am looking at cloud.gov in my browser
  WHEN I look at the left-side navigation hierarchy
  THEN I see the titles of customer-relevant upstream CF documentation (see below)
  AND when I click the titles then I see the corresponding documentation in the cloud.gov domain.
• GIVEN I am looking at cloud.gov in my browser
  WHEN I search for a term that only appears in upstream CF docs
  THEN I see more than 0 search results.

Documentation we should be including:

• https://docs.cloudfoundry.org/concepts/index.html
• https://docs.cloudfoundry.org/cf-cli/index.html
• https://docs.cloudfoundry.org/devguide/index.html
• https://docs.cloudfoundry.org/buildpacks/

---

Security considerations

No concerns. All of the information to be published is already public at docs.cloudfoundry.org.

Implementation sketch

The upstream "book" repository uses an obsolete set of Gems and Ruby versions to process the files in the various repositories with a Ruby-centric static site generator, Middleman. The repository processes the ERBs into Markdown, and then the Markdown into a full site, served by Ruby, using template HTML/JS/CSS layouts to generate the content.

cg-site is a Hugo repository for generating the cloud.gov static site. Hugo only needs Markdown for input, since it already has its own layout for the cloud.gov branding and style. So rather than using Middleman, bookbinder, specific Gem and Ruby versions, etc. we can just process the ERB content to generate Markdown, and stop there.

• Make a copy of the variable template YAML file and customize it for cloud.gov
• Rename each of the .html.md.erb files in the referenced repositories to .md.erb to make it clear we don't intend to process them all the way to HTML
• Process each of the .md.erb files in the referenced repositories to produce cloud.gov-specific Markdown
  • A Stack Overflow post shows how to process a single ERB file from the CLI
  • For example, a loop like this is a good starting point for processing all the files in all the repositories:

    vars = YAML.load_file(configpath)
    folders.each do |folder|
      folder.files.each do |file|
        ERB.new(file).result_with_hash(vars)
      end
    end
    

• Make other transformations on the files as necessary for inclusion in Hugo (eg edit front-matter in the YAML header)
• Include the generated Markdown files in the cg-site repository at build time
• Add a table-of-contents file for the additional documentation, and include it in the left-hand site documentation
• s/https://docs.cloudfoundry.org// on all the existing links to the upstream docs site

[mogul]

This is an important shortcoming of cg-site to address, but maybe not the most important. We are removing it from the board with the anticipation that it will be considered alongside other website improvements in a dedicated backlog, potentially by a team that is prepared to own that dedicated backlog.

[mogul]

Arsalan and I got together with some Rubyists to discuss the problem. I edited the top post to include some of what we learned about how to process the upstream docs for inclusion in cg-site.