generated with markdown-toc
These instructions detail how to create a new manuscript based off of the manubot/rootstock
repository.
The process can be a bit challenging, because it requires a few steps that are difficult to automate.
However, you will only have to perform these steps once for each manuscript.
These steps should be performed in a command-line shell (terminal), starting in the directory where you want the manuscript folder be created. Setup is supported on Linux, macOS, and Windows. Windows setup requires Git Bash or Windows Subsystem for Linux.
Creating a new manuscript using GitHub actions, the recommended default CI service (see below), can be achieved easily using the setup script. This simply runs the steps detailed below in the manual configuration.
Use the command below to copy setup.bash
and run it.
You can check the code that will be executed here.
bash <( curl --location https://github.com/manubot/rootstock/raw/main/setup.bash )
The script will then take you through the process of cloning the rootstock repo, make the changes required to use GitHub actions, edit the README to point to your repo and commit the changes. Your new manuscript repo is then ready for you to start adding your own content.
This script does not not create the remote repo for you, so you will be prompted to manually create an empty GitHub repository at https://github.com/new. Do not initialize the repository, other than optionally adding a description.
There is also a command line interface for users who want to create manuscripts at scale and in an automated way. See the help for details.
bash setup.bash --help
If you do not wish to use the above setup script to configure your new manuscript repository, you can instead execute the steps manually.
First, you must configure two environment variables (OWNER
and REPO
).
These variables specify the GitHub repository for the manuscript (i.e. https://github.com/OWNER/REPO
).
Make sure that the case of OWNER
matches how your username is displayed on GitHub.
In general, assume that all commands in this setup are case-sensitive.
Edit the following commands with your manuscript's information:
# GitHub username or organization name (change from manubot)
OWNER=manubot
# Repository name (change from rootstock)
REPO=rootstock
Execute the remaining commands verbatim. They do not need to be edited (if the setup works as intended).
Next you must clone manubot/rootstock
and reconfigure the remote repositories:
# Clone manubot/rootstock
git clone --single-branch https://github.com/manubot/rootstock.git $REPO
cd $REPO
# Configure remotes
git remote add rootstock https://github.com/manubot/rootstock.git
# Option A: Set origin URL using its web address
git remote set-url origin https://github.com/$OWNER/$REPO.git
# Option B: If GitHub SSH key access is enabled for OWNER, run the following command instead
git remote set-url origin [email protected]:$OWNER/$REPO.git
Then create an empty repository on GitHub.
You can do this at https://github.com/new or via the GitHub command line interface (if installed) with gh repo create
.
Make sure to use the same "Owner" and "Repository name" specified above.
Do not initialize the repository, other than optionally adding a Description.
Next, push your cloned manuscript:
git push --set-upstream origin main
Manubot integrates with cloud services to perform continuous integration (CI). For Manubot that means automatically building and deploying your manuscript. Manubot supports the following CI services:
Service | Default | Artifacts | Deployment | Config | Private Repos |
---|---|---|---|---|---|
GitHub Actions | ✔️ | ✔️ | ✔️ | manubot.yaml |
2,000 minutes per month |
AppVeyor | ❌ | ✔️ with PR comments | ❌ | .appveyor.yml |
14 day trial |
Notes on table fields:
- Default: Whether the following uncollapsed setup instructions enable the service by default.
- Artifacts: Manuscript outputs that are saved alongside the CI build logs. This is especially helpful for previewing changes that are under development in a pull request. Both GitHub Actions and AppVeyor upload the rendered manuscript as an artifact for pull request builds. However, only AppVeyor comments on pull requests with a download link to the artifacts (example).
- Deployment: Whether the CI service can write outputs back to the GitHub repository (to the
output
andgh-pages
branches). Deployment provides GitHub Pages with the latest manuscript version to serve to the manuscript's URL. GitHub Actions will deploy by default without any additional setup. - Config: File configuring what operations CI will perform. Removing this file is one method to disable the CI service.
- Private Repos: Quota for private repos. Only GitHub Actions supports cost-free builds of private repositories beyond a trial period. All services are cost-free for public repos.
Manubot was originally designed to use Travis CI, but later switched to primarily use GitHub Actions. Support for Travis was removed in 2021.
GitHub Actions is the recommended default CI service because it requires no additional setup. To use GitHub Actions only, remove configuration files for other CI services:
# remove AppVeyor config
git rm .appveyor.yml
# remove ci/install.sh (only used by AppVeyor)
git rm ci/install.sh
GitHub Actions is usually able to deploy without any additional setup using the GITHUB_TOKEN
for authentication.
GitHub Pages deployment using GITHUB_TOKEN
recently started working on GitHub without an official announcement.
If it does not work for you after completing this setup, try reselecting "gh-pages branch" as the Source for GitHub Pages in the repository Settings.
GitHub Pages should now trigger on the next commit.
If not, let us know.
For an alternative deployment method on GitHub, you can use an SSH Deploy Key instead. However, the setup is more complex. The following sections, collapsed by default, detail how to generate an SSH Deploy Key.
Expand for SSH Deploy Key setup
Previously, GitHub Actions required an SSH Deploy Key,
but now GitHub can deploy using the GITHUB_TOKEN
secret.
Therefore, users following the default configuration can skip these steps.
Otherwise, generate a deploy key so CI can write to the repository.
# Generate deploy.key.pub (public) and deploy.key (private)
ssh-keygen \
-t rsa -b 4096 -N "" \
-C "[email protected]" \
-f ci/deploy.key
# Encode deploy.key to remove newlines, writing encoded text to deploy.key.txt.
# This was required for entry into the Travis settings.
openssl base64 -A -in ci/deploy.key > ci/deploy.key.txt
# Print the URL for adding the public key to GitHub
echo "https://github.com/$OWNER/$REPO/settings/keys/new"
# Print the public key for copy-pasting to GitHub
cat ci/deploy.key.pub
Go to the GitHub settings URL echoed above in a browser, and click "Add deploy key".
For "Title", add a description like "Manubot Deploy Key".
Copy-paste the contents of the ci/deploy.key.pub
text file (printed above by cat
) into the "Key" text box.
Check the "Allow write access" box below.
Finally, click "Add key".
If you would like GitHub Actions to use SSH for deployment, rather than via HTTPS using GITHUB_TOKEN
, perform the steps in this section.
# Print the URL for adding the private key to GitHub
echo "https://github.com/$OWNER/$REPO/settings/secrets"
# Print the encoded private key for copy-pasting to GitHub
cat ci/deploy.key.txt && echo
Next, go to the GitHub repository settings page (URL echoed above).
Click "Add a new secret".
For "Name", enter MANUBOT_SSH_PRIVATE_KEY
.
Next, copy-paste the content of ci/deploy.key.txt
into "Value"
(printed above by cat
, including any trailing =
characters if present).
Expand for AppVeyor setup
You can optionally enable AppVeyor continuous integration to view pull request builds. AppVeyor supports storing manuscripts generated during pull request builds as artifacts. These can be previewed to facilitate pull request review and ensure formatting and reference changes render as expected. When a pull request build runs successfully, @AppVeyorBot will comment on the pull request with a download link to the manuscript PDF.
To enable AppVeyor, follow steps 1 and 2 of the AppVeyor welcome to sign in to AppVeyor and add your manuscript repository as an AppVeyor project.
The repository already contains an .appveyor.yml
build configuration file, so no other setup is required.
AppVeyor only runs when it detects changes that are likely to affect the manuscript.
The continuous integration configuration should now be complete.
Now update README.md
files to reference your new repository:
# Perform substitutions
sed "s/manubot\/rootstock/$OWNER\/$REPO/g" README.md > tmp && mv -f tmp README.md
sed "s/manubot\.github\.io\/rootstock/$OWNER\.github\.io\/$REPO/g" README.md > tmp && mv -f tmp README.md
The content/02.delete-me.md
file details the Markdown syntax and formatting options available with Manubot.
Remove it to reduce the content to a blank manuscript:
# Remove deletable content file
git rm content/02.delete-me.md
Run git status
or git diff --color-words
to double check the changes thus far.
If the changes look okay, commit and push:
git add --update
git commit --message "Brand repo to $OWNER/$REPO"
git push origin main
You should be good to go now.
A good first step is to modify content/metadata.yaml
with the relevant information for your manuscript.
This section will describe how to incorporate changes to rootstock that occurred since initializing your manuscript. You will want to do this if there are new enhancements or bugfixes that you want to incorporate. This process can be difficult, especially if conflicts have arisen, and is recommended only for advanced git users.
It is recommended to do rootstock upgrades via a pull request to help you view the proposed changes and to ensure the build uses the updated environment. First, checkout a new branch to use as the pull request head branch:
# checkout a new branch, named using the current date, i.e. rootstock-2018-11-16
git checkout -b rootstock-$(date '+%Y-%m-%d')
Second, pull the new commits from rootstock, but do not automerge:
# if rootstock remote is not set, add it
git config remote.rootstock.url || git remote add rootstock https://github.com/manubot/rootstock.git
# pull the new commits from rootstock
git pull --no-ff --no-rebase --no-commit rootstock main
If all goes well, there won't be any conflicts. However, if there are conflicts, follow the suggested commands to resolve them.
You can add the changes incrementally using git add --patch
.
This is helpful to see each upstream change.
You may notice changes that affect how items in content
are processed.
If so, you should edit and stage content
files as needed.
When there are no longer any unstaged changes, then do git commit
.
If updating your default branch (i.e. main
or master
) via a pull request, proceed to push the commit to GitHub and open a pull request.
Once the pull request is ready to merge, use GitHub's "Create a merge commit" option rather than "Squash and merge" or "Rebase and merge" to preserve the rootstock commit hashes.
The environment for local builds does not automatically update when build/environment.yml
changes.
To update your local conda manubot
environment with new changes, run:
# update a local conda environment
conda env update --file build/environment.yml
On 2020-10-01, GitHub changed the default branch name for new repositories from master
to main
.
More information on GitHub's migration is available at github/renaming.
On 2020-12-10, Manubot updated the Rootstock default branch to main
.
For existing manuscripts, the default branch will remain master
,
unless manually switched to main
.
Rootstock has been configured to run continuous integration on both main
and master
,
so existing manuscripts can, but are not required, to switch their default branch to main
.
Upgrading to the latest Rootstock will change several READMEs links to main
.
For manuscripts that do not plan to switch their default branch,
do not include these changes in the upgrade merge commit.