aiida-tutorials/high-throughput-screening.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_VQOcUDx26EH"
   },
   "source": [
    "# Run high throughput screening calculation: single point calculation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial the aim is to submit singlepoint calculations for all the structures in a folder. \n",
    "Initially, we want to create a group to store all the calculations. While this step is not mandatory, it does make it much easier to handle big amounts of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! verdi  group create \"hts_calc\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! verdi group show \"hts_calc\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import all the needed modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "oXvCvITp5I5X"
   },
   "outputs": [],
   "source": [
    "import click\n",
    "from aiida.engine import run_get_node, submit, run, run_get_pk\n",
    "from aiida.orm import load_code, load_node, load_group\n",
    "from aiida.plugins import CalculationFactory\n",
    "from pathlib import Path\n",
    "from aiida_mlip.data.model import ModelData\n",
    "from aiida_mlip.data.config import JanusConfigfile\n",
    "from aiida_mlip.helpers.help_load import load_structure\n",
    "import csv\n",
    "import sys\n",
    "from aiida.common import NotExistent\n",
    "import time\n",
    "from aiida import load_profile\n",
    "load_profile()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define inputs for the calculation. All the inputs are in common for all the structures, except the StructureData inputs which is gonna be defined in a loop cycle through the folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Calculation = CalculationFactory(\"mlip.sp\")\n",
    "model = ModelData.local_file(\"mlips/mace_mp/mace_mp_small.model\", architecture=\"mace_mp\")\n",
    "metadata = {\"options\": {\"resources\": {\"num_machines\": 1}}}\n",
    "code = load_code(\"janus@localhost\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use a config file to pass the other parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conf = JanusConfigfile(\"/home/jovyan/config_sp.yaml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the group as a variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "group = load_group(pk=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define folder where to find the structure files "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = Path(\"structures\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define for each file in the folder the `StructureData` and run the calculation. In this case we use the command `run_get_pk` to run, but when submitting large amounts of calculations, especially if they are more time consuming than the single point calculation, it is better to submit to the queue with the command `submit`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list_of_nodes = []\n",
    "for child in p.glob('**/*'):\n",
    "    if child.name.endswith(\"cif\"):   \n",
    "        print(child.name)\n",
    "        metadata['label']=f\"{child.name}\"\n",
    "        # This structure will overwrite the one in the config file if present\n",
    "        structure = load_structure(child.absolute())\n",
    "        # Run calculation\n",
    "        result,pk = run_get_pk(\n",
    "        Calculation,\n",
    "        code=code,\n",
    "        struct=structure,\n",
    "        metadata=metadata,\n",
    "        config=conf,\n",
    "        model=model\n",
    "    )\n",
    "        list_of_nodes.append(pk)\n",
    "\n",
    "        group.add_nodes(load_node(pk))\n",
    "        time.sleep(1)\n",
    "        print(f\"Printing results from calculation: {result}\")\n",
    "\n",
    "print(f\"FINISHED calculations, printing dictionary with all nodes {list_of_nodes}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we look at the group now, it will contain the calculations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! verdi group show \"hts_calc\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check if the calculations finished correctly. Exit status should be 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for node in group.nodes:\n",
    "    print(f\"Calculation {node.label, node.pk} finished with exit status {node.exit_status}\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, if we generate the provenance graph of a calculation we will see that it shows that calculation only, not the others. That is because the calculations are disconnected to each other, even if we submitted together using the same inputs (insert pk of a calculations in the code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! verdi node graph generate pk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the `geomopt.py` tutorial I show a way to \"connect\" calculations to make them appear in the provenance graph together. Otherwise, the high-throughput screening can be set up as a AiiDA `WorkChain` for this purpose. This WorkChain is currently a work in progress and will be added to the plugin."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can interact with the nodes by playing around with the functions of the node.\n",
    "Here I am deleting the node to empty the group (which is useful for the tutorial but not something you want to do normally, especially if the data is good)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aiida.tools import delete_nodes\n",
    "\n",
    "for node in group.nodes:\n",
    "    pks_to_be_deleted = delete_nodes(\n",
    "        [node.pk], dry_run=False, create_forward=True, call_calc_forward=True, call_work_forward=True\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! verdi group show \"hts_calc\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! verdi process list -a"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "private_outputs": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}