Skip to content

4. Installation

Jump to bottom
Clara Köhne edited this page Nov 7, 2022 · 33 revisions

You can install Patchwork in three different ways:

  1. Run Patchwork without compiling (recommended)
  2. Compile Patchwork from source
  3. Run Patchwork using Docker

Building Patchwork from Source

If you want to download Patchwork and build the program from source, you will need to ensure you have the following dependencies installed on your computer for the program to build and run correctly.

Dependencies

Patchwork requires the sequence aligner DIAMOND and the programming language Julia to be installed. Both have installation instructions on their respective websites. Here is how you would go about installing them on Linux:

DIAMOND

First, change to the directory that you want to download DIAMOND to, for example your home folder (~). Then run the following commands for installing DIAMOND:

wget http://github.com/bbuchfink/diamond/releases/download/v2.0.13/diamond-linux64.tar.gz
tar xzf diamond-linux64.tar.gz

(installs DIAMOND version 2.0.13)

The executable is called diamond and can be found in your current directory. Make sure that it is also included in your $PATH if you want to avoid having to call the program like /path/to/diamond on the command line. You can add DIAMOND to your path by editing the ~/.bashrc file in your home directory (ensure that the correct /path/to/diamonddirectory is specified, where diamonddirectory is the folder that contains your DIAMOND installation):

cat >> "~/.bashrc"
# include DIAMOND in PATH
export PATH="$PATH:/path/to/diamonddirectory"

Press Ctrl-D to close the interactive prompt again. After editing the file, you have to restart the terminal for the changes to become active.

Julia

For installing Julia, you can go about much the same way as you did to install DIAMOND. First, change to the directory that you want to download Julia to, for example your home folder. Then run the following commands for installing Julia:

wget https://julialang-s3.julialang.org/bin/linux/x64/1.8/julia-1.8.2-linux-x86_64.tar.gz
tar -xzf julia-1.8.2-linux-x86_64.tar.gz

(installs Julia version 1.8.2)

The Julia executable will be in the julia-1.8.2/bin folder in your current directory. You can add it to your $PATH by appending the following line to your ~/.bashrc file, making sure that the correct /path/to/julia-1.8.2 is specified:

cat >> "~/.bashrc"
# include julia in PATH
export PATH="$PATH:/path/to/julia-1.8.2/bin"

Press Ctrl-D to close the interactive prompt again. After editing the file, you have to restart the terminal for the changes to become active.

Julia packages

The Julia packages that the program depends on are typically handled by Julia itself. If this is not the case, please notify us by opening an issue. Before doing so, however, please check to see if the issue already exists.

In the meantime, you can try installing the missing package yourself. Type julia followed by Enter to open up a Julia REPL. It should look something like this:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.2 (2021-07-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>

Type ] to enter the Pkg mode (see this page for a description of the various modes in the Julia REPL). Once in the Pkg mode, type add NAME to add the package of your choice. For example, if Patchwork was complaining that ArgParse is missing, then type add ArgParse. You can use tab completion in this mode to make sure that your spelling is correct.

NOTE

Both DIAMOND and Julia can also be installed in other ways, for example using Conda. However, the Julia 1.8.2 installation then seems to be missing some certificate files, and might not work. Therefore, this approach is currently not recommended. However, this would be how you would install DIAMOND and Julia with Conda:

conda create -n patchwork -c bioconda -c conda-forge julia diamond

This will produce a Conda environment called patchwork which contains installations of DIAMOND and Julia. The option -c specifies which installation channels to use; in this case, DIAMOND is available on the bioconda channel and Julia on the conda-forge channel. The command should always automatically install the latest stable versions of both. You can also specify the versions by typing e.g. julia=1.8.2 to obtain version 1.8.2. Both programs will only be accessible after activating the environment, which you can do by typing the following:

conda activate patchwork

When the environment is active, you can run Patchwork. Note that, if you forget to activate, you will not be able to call DIAMOND or Julia, unless you have installed them before in some other way. To avoid the potential mess of multiple installations next to one another, we recommend you set up the separate environment for Patchwork as shown to encapsulate all dependencies in it.

After running Patchwork, you can deactivate the environment again by running:

conda deactivate

Obtaining the source code

Move into the directory where you want to install Patchwork and type this into your terminal:

$ git clone https://github.com/fethalen/patchwork

Go into the newly created directory by typing cd Patchwork, then follow the instructions below to compile the program.

Run Patchwork without compiling

With Julia, programs can be compiled during runtime (so called, just-in-time compilation). In practice, this is similar to launching a script with an interpreter, just like you would do with a Python script. The only downside is that there is a slight delay during the program's upstart. If you're just trying out Patchwork, we highly recommend this way as a quick and easy way to start using it.

After installing Julia and DIAMOND as described above, and downloading the source code for Patchwork, activate the patchwork environment that you created following the instructions above:

conda activate patchwork

Before running Patchwork for the first time, you will need to instantiate the project to ensure all required Julia packages are added to your environment. The Julia flag --project requires the path to the project you want to use; more specifically to the directory that contains the Project.toml and Manifest.toml files of the program you are planning to run. These list the Julia packages your program depends on. To install all of the dependencies in your project environment, from inside the Patchwork directory type:

julia --project=. -e "import Pkg; Pkg.instantiate()"

Ater this you can launch Patchwork like so (again from within the Patchwork folder):

julia --project=. src/Patchwork.jl --help

src/Patchwork.jl is the program file that you need to call to run Patchwork. The flag --help will bring up the help menu and simultaneously let you know if you are missing any dependencies.

Compiling Patchwork

Note: At the moment of writing, the compiled binary is approximately 1 GB in size. Please ensure that you have enough space left on your device. This is because Julia and all the dependencies are installed alongside the program itself.

Please make sure to follow the instructions above for downloading the source code and moving into the source code's base directory. Once there, type the following to compile Patchwork:

$ julia src/compile.jl . src/precompiled.jl OUTPUT_DIR

Where OUTPUT_DIR is the path to the output directory where you wish to install Patchwork. The compilation process takes around 10 minutes, so please be patient.

The program's binary will be installed to OUTPUT_DIR/bin/patchwork. You may want to create a symbolic link between this file to a file that is in your path, so that you may use Patchwork from anywhere:

$ ln -s OUTPUT_DIR/bin/patchwork /usr/local/bin

Installing Patchwork with Docker

Patchwork is available in the Biocontainers repository on Docker Hub. To pull the Patchwork Image to your PC and run it inside a Docker Container, you must have Docker installed.

Installing Docker

Go to the Docker Docs, select your platform and follow the instructions to set up the Docker repositories and install Docker Engine or Docker Desktop. On Ubuntu, you can use the following commands to install Docker:

Install packages to allow apt access to repositories over HTTPS:

$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release

Add Docker's official GPG key:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Set up the stable repository:

$ echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Then you can install Docker Engine using the following command:

$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io

Verify the installation by running the hello-world image:

$ sudo docker run hello-world

Pulling the Patchwork Image from Docker Hub

To obtain the image from the Biocontainers repository, run:

$ sudo docker pull biocontainers/patchwork:0.5.0_cv1

This image contains everything you need to run Patchwork inside a Docker Container, i.e. the Patchwork binary itself as well as its runtime dependency DIAMOND. You can have a look at the currently available version tags if you want to pull a specific version of the Patchwork Image. View your available images by running the following command:

$ sudo docker images

Running Patchwork inside a Docker Container

Now that you have the Patchwork Image on your local computer, you can run the program inside a Docker Container. To create a container from the image, modify the following command according to your preferences. The options and flags used in the command will be explained in more detail underneath.

$ sudo docker run -it --name NAME --user $(id -u) \
  --mount type=bind,src=SOURCE,dst=DESTINATION REPO/IMAGE:TAG COMMAND
  • -it: The container will start in interactive mode with an open terminal window.
  • --name NAME: The name you want to give your container, e.g. patchwork.
  • --user $(id -u): This option is necessary if you want to run Patchwork inside the container and e.g. create an output directory that you can access from your local computer afterwards. On your local computer, the files and directories belong to you, and that allows you to modify them. Inside the Docker Container however, the pre- configured user might have a different ID that would prevent them from messing with your files on the host computer. The --user option overrides the pre-configured user; you enter the container with your local user ID. That means you are allowed to change files from your host system that you have access to from inside the container. The files you can access are determined by the next option --mount.
  • --mount type=bind,src=SOURCE,dst=DESTINATION: Patchwork requires at least one contigs fasta file and one reference fasta file or database to run. These data are normally not available inside the Docker Container but you can make them accessible with a bind mount. src requires the absolute path to the directory on the host computer that contains your data. dst is the path inside the container that you want to access the data from. So, say your contigs and reference are in src=/home/yourname/data and you set dst=/home/patchwork/data. Then you can access all the files and subdirectories in /home/yourname/data via /home/patchwork/data. To learn more about on bind mounts, see the Docker Docs.Thus, you would call the program like
$ patchwork --contigs /home/patchwork/data/contigs.fa --reference /home/patchwork/data/reference.fa \
  --output-dir /home/patchwork/data/out
  • REPO/IMAGE:TAG: The image you would like to use, e.g. biocontainers/patchwork:v0.1.2_cv1. The tag is optional, when omitted the default tag latest will be used.
  • COMMAND: The command you want to run whenever the container starts. After executing this command, the container will stop automatically. For example, using bash as a command will keep the container's terminal open and the container running until you type exit. While the container is running, you can enter other commands, e.g. call Patchwork, in the terminal.

The docker run command above will create and start the new Docker Container. If you want to restart the container called patchwork, run

$ sudo docker start patchwork
$ sudo docker attach patchwork

More options for using Docker can be viewed in the help menu:

$ docker --help

© Dept. for Animal Evolution and Biodiversity 2020

Clone this wiki locally