🚧 new landing page, CI build, simplified structure and getting starte…

…d guide - add landing page #2 - enable GitHub pages #3 - flatten out the docs/docs structure #4 - add getting started guide as the first tutorial #5
wtsi-hgi · Feb 22, 2023 · 0ad9726 · 0ad9726
1 parent ff88f5a
commit 0ad9726
Show file tree

Hide file tree

Showing 38 changed files with 1,566 additions and 422 deletions.
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
@@ -0,0 +1,28 @@
+name: Deploy to GitHub Pages
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  deploy:
+    name: Deploy to GitHub Pages
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+        with:
+          node-version: 18
+          cache: yarn
+
+      - name: Install dependencies
+        run: yarn install --frozen-lockfile
+      - name: Build website
+        run: yarn build
+
+      - name: Deploy to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./build
diff --git a/.github/workflows/test-deploy.yml b/.github/workflows/test-deploy.yml
@@ -0,0 +1,22 @@
+name: Test deployment
+
+on:
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  test-deploy:
+    name: Test deployment
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+        with:
+          node-version: 18
+          cache: yarn
+
+      - name: Install dependencies
+        run: yarn install --frozen-lockfile
+      - name: Test build website
+        run: yarn build
diff --git a/README.md b/README.md
@@ -1,41 +1,64 @@
 # SoftPack
 
-This website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator.
+**SoftPack** is an open-source software packaging platform for any operating
+system where Docker or Singularity are supported.
 
-### Installation
+SoftPack is primarily aimed at scientific research community where running a
+stable yet flexible research software environment is a necessity. It is not
+uncommon for researchers to require their environments to be shared with other
+members of their team, either locally or across organizations.
 
-```
-$ yarn
-```
+A number of software packaging solutions already exist and generally work well
+when the number of dependencies remains manageable and the research environment
+either doesn't change very often or doesn't need to be shared with others,
+or both.
 
-### Local Development
+- [Conda](https://docs.conda.io/en/latest/) (and its reimplementation
+  [Mamba](https://mamba.readthedocs.io/en/latest)) are the de-facto standard for
+  python based scientific computing in single user environments.
 
-```
-$ yarn start
-```
+- [venv](https://docs.python.org/3/library/venv.html) combined with
+  [Pip](https://pip.pypa.io/en/stable) or especially [Poetry](https://python-poetry.org)
+  offer an excellent option for python package development but are limited elsewhere.
 
-This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
+- For R users, [renv](https://rstudio.github.io/renv/articles/renv.html) has
+  superceded the now out-dated [packrat](https://rstudio.github.io/packrat/) and
+  is a great option for managing different versions of a package or even different
+  versions of R, but its utility is limited outside of R.
 
-### Build
+- Other solutions like [Nix](https://nixos.org), [Guix](https://guix.gnu.org) or
+  [Spack](https://spack.io) offer some promise but generally require higher
+  levels of maintenance and administration for the end-user to tackle on their
+  own.
 
-```
-$ yarn build
-```
+- [Docker](https://www.docker.com), [Singularity](https://docs.sylabs.io/), and
+  [Podman](https://podman.io) offer unmatched stability and reproducibility by
+- using containerization but with the added cost of steeper learning curve.
 
-This command generates static content into the `build` directory and can be served using any static contents hosting service.
+- A number of home-grown tools have surfaced over the years but no solution
+  offers stability, flexibility and ease of use in a multi-user shareable
+  environment.
 
-### Deployment
+SoftPack attempts to overcome the shortcomings of previous efforts, not by
+reinventing the wheel, but by leveraging the strengths of some of the packaging
+solutions listed above and offering flexible multi-user environments with the
+stability of containers necessary for reproducible research.
 
-Using SSH:
+SoftPack is more than just a package management tool, as it offers managed
+multi-user software environments and administration tools for creating,
+managing, and discovering shareable environments. Under the hood, SoftPack uses
+[Spack](https://spack.io),
+[renv](https://rstudio.github.io/renv/articles/renv.html),
+[Docker](https://www.docker.com), and [Singularity](https://docs.sylabs.io/)
+but without exposing the complexities of the underlying technology to the
+end-user.
 
-```
-$ USE_SSH=true yarn deploy
-```
 
-Not using SSH:
+## Requirements
 
-```
-$ GIT_USER=<Your GitHub username> yarn deploy
-```
-
-If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
+SoftPack runs on any POSIX-compliant operating system including any
+distribution of Linux, FreeBSD, and macOS. SoftPack also supports virtual
+machine environments including VirtualBox, Vagrant and VMware, provided the
+operating system running on the virtual machine supports Docker or Singularity.
+SoftPack has been tested on OpenStack and will  support Google Cloud
+Platform (GCP) and Amazon Web Services (AWS).
diff --git a/docs/docs/_category_.yml → docs/_category_.yml b/docs/docs/_category_.yml → docs/_category_.yml
diff --git a/docs/docs/reference/cli.md b/docs/docs/reference/cli.md
diff --git a/docs/docs/tutorials/getting_started.md b/docs/docs/tutorials/getting_started.md
diff --git a/docs/docs/how-to-guides/_category_.yml → docs/how-to-guides/_category_.yml b/docs/docs/how-to-guides/_category_.yml → docs/how-to-guides/_category_.yml
diff --git a/...ocs/how-to-guides/creating_environment.md → docs/how-to-guides/creating_environment.md b/...ocs/how-to-guides/creating_environment.md → docs/how-to-guides/creating_environment.md
@@ -4,4 +4,8 @@ sidebar_position: 1
 
 # Creating an environment
 
+:::caution
+This documentation is under development and may be incomplete.
+:::
+
 How-to guides for environment management will be available in [Milestone 2](/docs/reference/development_roadmap/#milestone-2)
diff --git a/...ocs/how-to-guides/updating_environment.md → docs/how-to-guides/updating_environment.md b/...ocs/how-to-guides/updating_environment.md → docs/how-to-guides/updating_environment.md
@@ -4,4 +4,8 @@ sidebar_position: 2
 
 # Updating an environment
 
+:::caution
+This documentation is under development and may be incomplete.
+:::
+
 How-to guides for environment management will be available in [Milestone 2](/docs/reference/development_roadmap/#milestone-2)
diff --git a/docs/docs/overview/_category_.yml → docs/overview/_category_.yml b/docs/docs/overview/_category_.yml → docs/overview/_category_.yml
diff --git a/docs/docs/overview/architecture.md → docs/overview/architecture.md b/docs/docs/overview/architecture.md → docs/overview/architecture.md
@@ -4,6 +4,10 @@ sidebar_position: 3
 
 # System Architecture 
 
+:::caution
+This documentation is under development and may be incomplete.
+:::
+
 SoftPack is built using a modern [microservices architecture](https://letmegooglethat.com/?q=what+is+microservices+architecture). Unlike monolithic
 monstrosities of the past, modern software systems are almost always built
 with smaller lightweight services that interoperate together and communicate 
@@ -77,7 +81,7 @@ flowchart TB
     softpack_clients -->|GraphQL / HTTP | softpack_core
     
     subgraph softpack_artifacts["SoftPack Artifacts (GitLab)"]
-      softpack_env(fa:fa-file-lines Envmoironment Definitions <br/> Spack environment files)
+      softpack_env(fa:fa-file-lines Environment Definitions <br/> Spack environment files)
       softpack_modules(fa:fa-box-archive Module Files <br/> For use with module command)
       softpack_images(fa:fa-box-open Container Images <br/> Docker/Singularity images)
       softpack_env --- softpack_images
@@ -166,7 +170,7 @@ flowchart TB
     class softpack_core dashed_container;
         
     subgraph softpack_artifacts["SoftPack Artifacts (GitLab)"]
-      softpack_env(fa:fa-file-lines Envmoironment Definitions <br/> Spack environment files)
+      softpack_env(fa:fa-file-lines Environment Definitions <br/> Spack environment files)
       softpack_modules(fa:fa-box-archive Module Files <br/> For use with module command)
       softpack_images(fa:fa-box-open Container Images <br/> Docker/Singularity images)
       softpack_env --- softpack_images
@@ -186,7 +190,7 @@ SoftPack Artifacts repository.
 
 ### SoftPack GraphQL
 
-SoftPack GrahQL implements the [GrahQL](https://graphql.org) service in 
+SoftPack GraphQL implements the [GraphQL](https://graphql.org) service in 
 SoftPack. GraphQL is used because of the flexibility it offers when performing 
 complex queries. Unlike REST APIs where multiple queries from multiple 
 endpoints often require complex logic to be embedded in client interfaces, 
@@ -214,7 +218,7 @@ flowchart TB
     classDef dashed_container fill:#FFFFFF, stroke-dasharray:4;
 
     subgraph softpack_artifacts["SoftPack Artifacts (GitLab)"]
-      softpack_env(fa:fa-file-lines Envmoironment Definitions <br/> Spack environment files)
+      softpack_env(fa:fa-file-lines Environment Definitions <br/> Spack environment files)
       softpack_modules(fa:fa-box-archive Module Files <br/> For use with module command)
       softpack_images(fa:fa-box-open Container Images <br/> Docker/Singularity images)
       softpack_env --- softpack_images
@@ -290,7 +294,7 @@ flowchart TB
     classDef dashed_container fill:#FFFFFF, stroke-dasharray:4;
 
     subgraph softpack_artifacts["SoftPack Artifacts (GitLab)"]
-      softpack_env(fa:fa-file-lines Envmoironment Definitions <br/> Spack environment files)
+      softpack_env(fa:fa-file-lines Environment Definitions <br/> Spack environment files)
       softpack_modules(fa:fa-box-archive Module Files <br/> For use with module command)
       softpack_images(fa:fa-box-open Container Images <br/> Docker/Singularity images)
       softpack_env --- softpack_images
@@ -406,11 +410,11 @@ sequenceDiagram
 | #   | Event                                                              | Description                                                                                                                                                |
 |-----|--------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | 1   | Create Environment                                                 | End-user sends a request to create a new environment using either the Web Client or the Command Line Interface.                                            |
-| 2   | GraphQL Mutation                                                   | SoftPack client sends the request to create a new environment using GrapQL Mutation over HTTP                                                              |
+| 2   | GraphQL Mutation                                                   | SoftPack client sends the request to create a new environment using GraphQL Mutation over HTTP                                                             |
 | 3   | Create Environment in SoftPack Artifacts repository                | SoftPack Core creates a new environment in SoftPack Artifacts repository                                                                                   | 
-| 4   | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Artifacts repository returns ```OK``` to SoftPack Core                                                                                            |
-| 5   | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Core returns ```OK``` to the client                                                                                                               |
-| 6   | WebSocket (WS) <br/> Status = In Progress                          | SoftPack Core sends ```In Progress``` status to the client in a WebSocket notification                                                                     | 
+| 4   | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Artifacts repository returns `OK` to SoftPack Core                                                                                                |
+| 5   | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Core returns `OK` to the client                                                                                                                   |
+| 6   | WebSocket (WS) <br/> Status = In Progress                          | SoftPack Core sends `In Progress` status to the client in a WebSocket notification                                                                         | 
 | 7   | Environment Created Webhook from SoftPack Artifacts repository     | SoftPack Artifacts repository sends a Webhook notification to the Softpack Builder that a new environment has been created                                 |
 | 8   | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Builder returns OK to SoftPack Artifacts repository                                                                                               |
 | 9   | Push Module Definition from SoftPack Builder                       | SoftPack Builder pushes a new module definition file to SoftPack Artifacts repository                                                                      |
@@ -419,6 +423,6 @@ sequenceDiagram
 | 12  | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Artifacts repository returns OK to SoftPack Builder                                                                                               |
 | 13  | Module & UImage Created Webhook from SoftPack Artifacts repository | SoftPack Artifacts repository sends a Webhook notification to the SoftPack Deploy Agent that a new module definition and container image have been created |
 | 14  | HTTP RESPONSE<br/>Status = OK                                      | SoftPack Deploy Agent returns OK to SoftPack Artifacts repository                                                                                          |                                              
-| 15  | WebSocket (WS) <br/> Status = OK                                   | SoftPack Deploy Agent sends ```OK``` to SoftPack Core once the module file and container images are deployed                                               |
-| 16  | WebSocket (WS) <br/> Status = OK                                   | SoftPack Core sends ```OK``` to SoftPack client that the new environment has been created                                                                  |
+| 15  | WebSocket (WS) <br/> Status = OK                                   | SoftPack Deploy Agent sends `OK` to SoftPack Core once the module file and container images are deployed                                                   |
+| 16  | WebSocket (WS) <br/> Status = OK                                   | SoftPack Core sends `OK` to SoftPack client that the new environment has been created                                                                      |
 | 17  | Environment Created                                                | SoftPack client updates the status and notifies that user that their new environment has been created                                                      |
diff --git a/docs/docs/overview/deployment.md → docs/overview/deployment.md b/docs/docs/overview/deployment.md → docs/overview/deployment.md
@@ -4,4 +4,8 @@ sidebar_position: 6
 
 # Deployment
 
+:::caution
+This documentation is under development and may be incomplete.
+:::
+
 Deployment details will be available once [Milestone 2](/docs/reference/development_roadmap/#milestone-2) is scheduled. 
diff --git a/docs/docs/overview/features.md → docs/overview/features.md b/docs/docs/overview/features.md → docs/overview/features.md
@@ -4,6 +4,10 @@ sidebar_position: 2
 
 # Features
 
+:::caution
+This documentation is under development and may be incomplete.
+:::
+
 SoftPack supports the following list of features.
 
 ## Environment management