Galaxy inside Kubernetes for tool developers

Galaxy inside Kubernetes for tool developers

Here we explain the rationale happening behind the curtains when using the helm deployment for testing/developing Galaxy tool wrappers that call containerised software. You are not expected to follow this steps manually anymore, but they explain what in essence is happening when using the Helm deployment with the development_folder variable set.

The following setup allows developers to run Galaxy inside Kubernetes, and inject into it key files/folders that a developer might want to change:

• Galaxy config/config.ini: general Galaxy settings configurations.
• Galaxy config/job_conf.xml: where tools need to be associated to the Kubernetes runner.
• Galaxy config/tool_conf.xml: where tools order in the sidebar of galaxy is decided. Paths need to be coherent to what is used in the following option.
• Galaxy tools/phenomenal: folder where we will add tools (the developer can populate this as desired, but it needs to be coherent with config/job_conf.xml).

Provided that the Persistent Volume, Persistent Volume Claim and the service have been created (this is done by the Helm deployment for you), consider the following .yaml template file for the replication controller, where the key section is what happens under volumeMounts and volumes:

apiVersion: v1
kind: ReplicationController
metadata:
  name: galaxy-k8s 
spec:
  replicas: 1
  template:
    metadata:
      labels:
         app: galaxy-k8s
    spec:
      containers:
      - name: galaxy-k8s
        image: container-registry.phenomenal-h2020.eu/phnmnl/galaxy-k8s-runtime
        command: ["/bin/bash","-c","mkdir -p /opt/galaxy_data/database-sqlite && ./run.sh --daemon && tail -f paster.log"]
        lifecycle: 
          preStop:
            exec:
               command:
                   - "./run.sh"
                   - "--stop"
        ports:
          - containerPort: 8080
        volumeMounts:
          - mountPath: "/opt/galaxy_data"
            name: galaxy-pvc
          - mountPath: "/galaxy/config/galaxy.ini"
            name: galaxy-ini 
          - mountPath: "/galaxy/config/job_conf.xml"
            name: galaxy-job-conf 
          - mountPath: "/galaxy/config/tool_conf.xml"
            name: galaxy-tool-conf
          - mountPath: "/galaxy/tools/phenomenal"
            name: phenomenal-tools 
      volumes:
        - name: galaxy-pvc
          persistentVolumeClaim:
              claimName: galaxy-pvc
        - name: galaxy-ini
          hostPath:
            path: [PATHTODEVFILES]/config/galaxy.ini
        - name: galaxy-job-conf
          hostPath:
            path: [PATHTODEVFILES]/config/job_conf.xml
        - name: galaxy-tool-conf
          hostPath:
            path: [PATHTODEVFILES]/config/tool_conf.xml
        - name: phenomenal-tools
          hostPath:
            path: [PATHTODEVFILES]/tools/phenomenal

on deployment with the development_folder variable set, Helm will replace [PATHTODEVFILES] with the path which would allow us to reach the config/galaxy.ini, config/job_conf.xml, config/tool_conf.xml and the tools/phenomenal folder. As starting point, you can use the config/job_conf.xml, config/tool_comf.xml and tools folder available in this repo.

Adding a tool

We would suggest having a single folder where you leave your config and tools folders, which could well be part of a complete Galaxy installation (Kubernetes will only mount the specified files to the Galaxy pod). For the sake of this documentation, we will suppose that this folder's path is /Users/jdoe/galaxy_central_dev.

Supposing that you are developing a Galaxy wrapper for a tool called MyTool, these are the steps that you need to execute to make it visible inside Galaxy in Kubernetes:

• Copy (don't symlink) the wrapper to the tools/phenomenal folder. You can as well create a subsequent folder if your tools can be grouped somehow (for instance, NMR). Lets suppose for this example that MyTool is used for NMR data analysis:

GALAXY_ROOT=/Users/jdoe/galaxy_central_dev
mkdir -p $GALAXY_ROOT/tools/phenomenal/nmr
cp MyTool.xml $GALAXY_ROOT/tools/phenomenal/nmr/

• Add a container destination for the tool on the job_conf.xml file on $GALAXY_ROOT/config/job_conf.xml:

...
<destinations default="local">
        ...
        <destination id="MyTool-container" runner="k8s">
            <param id="docker_repo_override">container-registry.phenomenal-h2020.eu</param>
            <param id="docker_owner_override">phnmnl</param>
            <param id="docker_image_override">mytool-container</param>
            <param id="docker_tag_override">latest</param>
            <param id="max_pod_retrials">1</param>
            <param id="docker_enabled">true</param>
        </destination>
        ...
</destinations>
...

Alternatively, if you want to rely on the docker image set at the tool level, you just need to write:

...
<destinations default="local">
        ...
        <destination id="MyTool-container" runner="k8s">
            <param id="max_pod_retrials">1</param>
            <param id="docker_enabled">true</param>
        </destination>
        ...
</destinations>
...

• Assign the tool to the container destination just created on the same job_conf.xml file, inside the existing <tools> section:

...
<tools>
       ...
       <tool id="MyTool" destination="MyTool-container"/>
       ...
</tools>

• Adds path to tool in the tool_conf.xml file, under the PhenoMeNal label, NMR section in this case (create if doesn't exist):

<label id="phenomenal" text="PhenoMeNal H2020 Tools" />
<section name="NMR" id="pheno-nmr">
  <tool file="phenomenal/nmr/MyTool.xml"/>
</section>

Restarting Galaxy to pick up changes

To restart Galaxy inside Kubernetes you can either:

• Delete the pod that is running:

# Find out the pods name
$ kubectl get pods
NAME               READY     STATUS    RESTARTS   AGE
galaxy-k8s-8izfr   1/1       Running   0          11h
# then delete it
$ kubectl delete pods/galaxy-k8s-8izfr
# This will force the replication controller to lift a new pod from the same definition, loading your files again.
$ kubectl get pods
NAME               READY     STATUS        RESTARTS   AGE
galaxy-k8s-x4ed1   1/1       Running       0          41s
galaxy-k8s-8izfr   1/1       Terminating   0          15m
# galaxy-k8s-x4ed1 is lifted by the RC

• Get into the running pod and restart the Galaxy daemon

$ kubectl exec -i -t galaxy-k8s-x4ed1 bash
root@galaxy-k8s-x4ed1:/galaxy# ./run.sh --restart

Bring your tool to the public PhenoMeNal Galaxy instance

Once you have tested your tool intensively, it is ready to be used in the running public PhenoMeNal Galaxy instance and other PhenoMeNal deployments.

For that, you should clone the container-galaxy-k8s-runtime repo, checkout the development branch, and either open a new feature through git flow or create a new branch (and switch to it).

$ git clone -b develop https://github.com/phnmnl/container-galaxy-k8s-runtime
$ # create a new feature through git flow.
$ git flow feature start myToolPhenoNMR
$ # alternatively, simple create a new branch
$ git checkout -b myToolPhenoNMRBranch

Next. You make the following changes:

• First, add your tool xml to the appropriate Galaxy tool folder.
• Add your tool to the config/job_conf.xml, including setting destination container as explained.
• Add your tool to the config/tool_conf.xml.

Finally, publish your changes by commiting and pushing to our Galaxy repo, or by pushing to your own fork. Once these changed are pushed, open a pull request to merge these changes into the development branch of our Galaxy repo.