From 1ffb456261572cc94d7211e9a8e1257c8710b0c0 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 19 Jul 2023 19:15:43 -0500 Subject: [PATCH 01/70] Extended introduction, explanations and definitions added. Restructure. Miniscope tutorial added. --- README.md | 67 ++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 42 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index f020f81..51e922a 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,19 @@ -# DataJoint interactive tutorial with GitHub Codespace +# Welcome to DataJoint tutorials! -Interactive tutorials on the DataJoint framework, in Python. Throughout this set of tutorials, you will learn +DataJoint is an open-source software and cloud platform to design, build and automate data analysis, pipelines and data sharing for neuroscience experiments. -- DataJoint basics (~1 hour) +This document will guide you as a new DataJoint user through interactive tutorials organized in notebooks and written in Python. + +*Please note that these hands-on tutorials are friendly to non-expert users and no prior programming knowledge is required.* + + +## Key learnings from the tutorials + +After completing this set of tutorials, you will gain real experience in the basics of the DataJoint framework. These skills will allow you to design, implement and manage data workflows effectively applied to your scientific research. + +Here is a summary of the content that you can expect to have learned: + +- Understanding DataJoint basics: concepts, design and structure (~1 hour) - Create schemas/tables - Table tiers (`Lookup`, `Manual`, `Imported`, `Computed`) - Insert entries and view entries in tables @@ -18,49 +29,55 @@ Interactive tutorials on the DataJoint framework, in Python. Throughout this set - Retrieve selective attributes - Delete operations -- DataJoint advanced topics (~1 hour) +- DataJoint advanced topics: workflow automation (~1 hour) - `Imported` and `Computed` tables - `make()` function - `.populate()` for automated computation - `.populate(reserve_jobs=True)` for parallelization + -## Interactive Tutorial +## Quick start -The easiest way to learn DataJoint is to use the tutorial notebooks within the included interactive environment configured using [DevContainer](https://containers.dev/). +The DataJoint tutorials are easily accessible using an **interactive environment** that contains all the software required to run the experiments. The environment is configured by [DevContainer] (https://containers.dev/). Here are two options to launch the interactive environment: -### Launch Environment +- **Cloud-based IDE: GitHub Codespaces**: (*recommended*) + - This is the easiest option for **tutorial users**. You will immediately start coding using DataJoint and Python, with no installation of software or environments, using built-in tools in the cloud. + + - Instructions: + - A codespace is a development environment hosted in the cloud. To use the DataJoint tutorials with [GitHub Codespaces](https://github.com/features/codespaces), you need a [GitHub](https://github.com/) account. Fork the [datajoint-tutorials](https://github.com/datajoint/datajoint-tutorials) repository into your repository. + - From your `datajoint-tutorials` repository, click on `Code`, then click on `Codespaces` tab, and `+` option will `Create codespace on main` on your fork with default options. For more control, see the `...` where you may create `New with options...`. + - The building time for a codespace is **~5m**. This is done infrequently and cached for convenience. + - The start time for a codespace is **~30s**. This will pull the built codespace from the cache when you need it. + - *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of computing and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete Codespaces when not actively in use and recreate them when needed. We'll soon be creating prebuilds to avoid larger build times. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. + - *Tip*: GitHub auto names the codespace but you can rename the codespace so that it is easier to identify later. -Here are some options that provide a great experience: - -- **Cloud-based IDE**: (*recommended*) - - Launch using [GitHub Codespaces](https://github.com/features/codespaces) using the `+` option which will `Create codespace on main` in the codebase repository on your fork with default options. For more control, see the `...` where you may create `New with options...`. - - Build time for a codespace is **~5m**. This is done infrequently and cached for convenience. - - Start time for a codespace is **~30s**. This will pull the built codespace from cache when you need it. - - *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of compute and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete Codespaces when not actively in use and recreate when needed. We'll soon be creating prebuilds to avoid larger build times. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. - - *Tip*: GitHub auto names the codespace but you can rename the codespace so that it is easier to identify later. - **Local IDE**: - - Ensure you have [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - - Ensure you have [Docker](https://docs.docker.com/get-docker/) - - Ensure you have [VSCode](https://code.visualstudio.com/) - - Install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) - - `git clone` the codebase repository and open it in VSCode - - Use the `Dev Containers extension` to `Reopen in Container` (More info in the `Getting started` included with the extension) + - We highly recommend this option for users seeking to apply DataJoint for **their own neuroscience experiments** and lab research. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). + + - Instructions: + - Ensure you have [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) + - Ensure you have [Docker](https://docs.docker.com/get-docker/) + - Ensure you have [VSCode](https://code.visualstudio.com/) + - Install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) + - `git clone` the codebase repository and open it in VSCode + - Use the `Dev Containers extension` to `Reopen in Container` (More info in the `Getting started` included with the extension) -You will know your environment has finished loading once you either see a terminal open related to `Running postStartCommand` with a final message: `Done` or the `README.md` is opened in `Preview`. +You will know your environment has finished loading once you see a terminal open related to `Running postStartCommand` with a final message: `Done` or the `README.md` is opened in `Preview`. ### Instructions -1. We recommend you start by navigating to the `notebooks` directory on the left panel and go through the `00-Getting_Started/01-DataJoint Basics - Interactive.ipynb` Jupyter notebook. Execute the cells in the notebooks to begin your walk through of the tutorial. +1. We recommend you start by navigating to the `notebooks` directory on the left panel and go through the `00-Getting_Started/01-DataJoint Basics - Interactive.ipynb` Jupyter notebook. Execute the cells in the notebooks to begin your walkthrough of the tutorial. -1. Once you are done, see the options available to you in the menu in the bottom-left corner. For example, in Codespace you will have an option to `Stop Current Codespace` but when running DevContainer on your own machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespace after 30 minutes of inactivity. +2. Once you are done, see the options available to you in the menu in the bottom-left corner. For example, in Codespace you will have the option to `Stop Current Codespace`, but when running DevContainer on your own machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespace after 30 minutes of inactivity. -If you are new to GitHub and run into any errors, please contact us via email at support@datajoint.com. If you are experienced with GitHub, please create an issue on the upstream repository or if you'd like help contribute, feel free to create a pull request. Please include a thorough explanantion of the error and/or proposed solution. +If you are new to GitHub and run into any errors, please contact us via email at support@datajoint.com. If you are experienced with GitHub, please create an issue on the upstream repository or if you'd like help contribute, feel free to create a pull request. Please include a thorough explanation of the error and/or proposed solution. ## Additional DataJoint Tutorials - DataJoint Elements are a collection of curated modules for assembling data pipelines for several modalities of neurophysiology experiments. - [Element Calcium Imaging Tutorial](https://github.com/datajoint/element-calcium-imaging#interactive-tutorial) - [Element Array Electrophysiology Tutorial](https://github.com/datajoint/workflow-array-ephys#interactive-tutorial) + - [Element Miniscope Calcium Imaging Tutorial](https://github.com/datajoint/workflow-miniscope#interactive-tutorial) - [Machine Intelligence from Cortical Networks (MICrONS) program](https://www.microns-explorer.org/) - [MICrONS Tutorial](https://github.com/datajoint/microns_phase3_nda#interactive-environment) \ No newline at end of file From 6c96c3b2bc935c31e623f118f4b42a4ca178e4fa Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 20 Jul 2023 20:33:17 -0500 Subject: [PATCH 02/70] Update README: Installation extended, Miniscope tutorial reference deleted. --- README.md | 43 ++++++++++++++++++------------------------- 1 file changed, 18 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 51e922a..1a95e24 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,14 @@ # Welcome to DataJoint tutorials! -DataJoint is an open-source software and cloud platform to design, build and automate data analysis, pipelines and data sharing for neuroscience experiments. +DataJoint is an open-source software to design, build and automate data analysis, pipelines and data sharing for neuroscience experiments and research labs. -This document will guide you as a new DataJoint user through interactive tutorials organized in notebooks and written in Python. - -*Please note that these hands-on tutorials are friendly to non-expert users and no prior programming knowledge is required.* +This document will guide you as a new DataJoint user through interactive tutorials organized in [Jupyter notebooks](https://jupyter-notebook.readthedocs.io/en/stable/) and written in [Python](https://www.python.org/). +*Please note that these hands-on DataJoint tutorials are friendly to non-expert users and no prior programming knowledge is required.* ## Key learnings from the tutorials -After completing this set of tutorials, you will gain real experience in the basics of the DataJoint framework. These skills will allow you to design, implement and manage data workflows effectively applied to your scientific research. +After completing this set of tutorials, you will gain real experience in the basics of the DataJoint framework. These skills will allow you to design, implement and manage data pipelines effectively applied to your scientific research. Here is a summary of the content that you can expect to have learned: @@ -38,46 +37,40 @@ Here is a summary of the content that you can expect to have learned: ## Quick start -The DataJoint tutorials are easily accessible using an **interactive environment** that contains all the software required to run the experiments. The environment is configured by [DevContainer] (https://containers.dev/). Here are two options to launch the interactive environment: +### User installation +DataJoint tutorials are easily accessible using an **interactive environment** that contains all the software required to run the experiments. The environment is configured by [DevContainer] (https://containers.dev/). Here are two options to launch the interactive environment. + +*Please note that to use the DataJoint Python package with an interactive environment you need a [GitHub](https://github.com/) account.* -- **Cloud-based IDE: GitHub Codespaces**: (*recommended*) - - This is the easiest option for **tutorial users**. You will immediately start coding using DataJoint and Python, with no installation of software or environments, using built-in tools in the cloud. +- **Cloud-based environment: GitHub Codespaces**: (*recommended*) + - This is the easiest option for **tutorial users**. You will immediately start coding using DataJoint and Python, with no installation of software or local environments. Cloud-based environments (IDEs), such as [GitHub Codespaces](https://github.com/features/codespaces), use built-in tools directly connected to the cloud and work on the browser. - Instructions: - - A codespace is a development environment hosted in the cloud. To use the DataJoint tutorials with [GitHub Codespaces](https://github.com/features/codespaces), you need a [GitHub](https://github.com/) account. Fork the [datajoint-tutorials](https://github.com/datajoint/datajoint-tutorials) repository into your repository. + - Fork the [datajoint-tutorials](https://github.com/datajoint/datajoint-tutorials) repository into your repository. - From your `datajoint-tutorials` repository, click on `Code`, then click on `Codespaces` tab, and `+` option will `Create codespace on main` on your fork with default options. For more control, see the `...` where you may create `New with options...`. - The building time for a codespace is **~5m**. This is done infrequently and cached for convenience. - The start time for a codespace is **~30s**. This will pull the built codespace from the cache when you need it. - *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of computing and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete Codespaces when not actively in use and recreate them when needed. We'll soon be creating prebuilds to avoid larger build times. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. - *Tip*: GitHub auto names the codespace but you can rename the codespace so that it is easier to identify later. -- **Local IDE**: - - We highly recommend this option for users seeking to apply DataJoint for **their own neuroscience experiments** and lab research. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). +- **Local environment**: + - We highly recommend this option for users that after exploring the tutorials want to apply DataJoint for **their own neuroscience experiments** and lab research. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git), [Docker](https://docs.docker.com/get-docker/), [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) and [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). For more detailed instructions, please check out the [User Guide in DataJoint Documentation](https://datajoint.com/docs/elements/user-guide/). - - Instructions: - - Ensure you have [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - - Ensure you have [Docker](https://docs.docker.com/get-docker/) - - Ensure you have [VSCode](https://code.visualstudio.com/) - - Install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) - - `git clone` the codebase repository and open it in VSCode - - Use the `Dev Containers extension` to `Reopen in Container` (More info in the `Getting started` included with the extension) - -You will know your environment has finished loading once you see a terminal open related to `Running postStartCommand` with a final message: `Done` or the `README.md` is opened in `Preview`. +Before we start, remember that all of the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that you are interested in keeping the changes. ### Instructions +- To begin, we suggest navigating to the notebooks directory located in the left panel and proceeding through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. -1. We recommend you start by navigating to the `notebooks` directory on the left panel and go through the `00-Getting_Started/01-DataJoint Basics - Interactive.ipynb` Jupyter notebook. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - -2. Once you are done, see the options available to you in the menu in the bottom-left corner. For example, in Codespace you will have the option to `Stop Current Codespace`, but when running DevContainer on your own machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespace after 30 minutes of inactivity. +- Once you are done, see the options available to you in the menu in the bottom-left corner. For example, in Codespace you will have the option to `Stop Current Codespace`, but when running DevContainer on your machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespace after 30 minutes of inactivity. -If you are new to GitHub and run into any errors, please contact us via email at support@datajoint.com. If you are experienced with GitHub, please create an issue on the upstream repository or if you'd like help contribute, feel free to create a pull request. Please include a thorough explanation of the error and/or proposed solution. +## Support +If you need help getting started or run into any errors, please open a GitHub Issue or contact our team by email at support@datajoint.com. ## Additional DataJoint Tutorials - DataJoint Elements are a collection of curated modules for assembling data pipelines for several modalities of neurophysiology experiments. - [Element Calcium Imaging Tutorial](https://github.com/datajoint/element-calcium-imaging#interactive-tutorial) - [Element Array Electrophysiology Tutorial](https://github.com/datajoint/workflow-array-ephys#interactive-tutorial) - - [Element Miniscope Calcium Imaging Tutorial](https://github.com/datajoint/workflow-miniscope#interactive-tutorial) - [Machine Intelligence from Cortical Networks (MICrONS) program](https://www.microns-explorer.org/) - [MICrONS Tutorial](https://github.com/datajoint/microns_phase3_nda#interactive-environment) \ No newline at end of file From 5c55ac19c96a48423a7b339fdd27637790432f9f Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Fri, 21 Jul 2023 10:52:40 -0500 Subject: [PATCH 03/70] Updated README to add Table of contents --- README.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 1a95e24..5bd0bcd 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,25 @@ # Welcome to DataJoint tutorials! -DataJoint is an open-source software to design, build and automate data analysis, pipelines and data sharing for neuroscience experiments and research labs. +DataJoint is an open-source tool library to design, build and automate data analysis, pipelines and data sharing for neuroscience experiments and research labs. This document will guide you as a new DataJoint user through interactive tutorials organized in [Jupyter notebooks](https://jupyter-notebook.readthedocs.io/en/stable/) and written in [Python](https://www.python.org/). -*Please note that these hands-on DataJoint tutorials are friendly to non-expert users and no prior programming knowledge is required.* +*Please note that these hands-on DataJoint tutorials are friendly to non-expert users and advanced programming skills are not required.* + +## Table of contents +- Notebooks: interactive notebooks to be a master of DataJoint. The Calcium Imaging and Electrophysiology tutorials serve as common examples of data structure and data analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! + - 01-DataJoint Basics + - 02-Calcium Imaging Imported Tables + - 03-Calcium Imaging Computed Tables + - 04-Electrophysiology Imported Tables + - 05-Electrophysiology Computed Tables + +- Completed exercises: notebooks with the code sections completed and solved. + +- Pipeline + - ephys + - imaging + - mouse ## Key learnings from the tutorials From 1223ebf3a36330d3d86cd67b881c11ee0cff8f97 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 15:09:48 +0000 Subject: [PATCH 04/70] Added gcc install to Dockerfile --- .devcontainer/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 5a4a2fe..0ef41ce 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -19,7 +19,7 @@ RUN \ RUN \ # dev setup apt update && \ - apt-get install sudo git bash-completion graphviz default-mysql-client -y && \ + apt-get install sudo git gcc bash-completion graphviz default-mysql-client -y && \ usermod -aG sudo vscode && \ echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \ pip install --no-cache-dir --upgrade black pip && \ From 114702495b5cfce356542c6c5940a0937969e264 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 15:14:55 +0000 Subject: [PATCH 05/70] Added configuration DJ instructions to tutorial 01 --- .../01-DataJoint Basics - Interactive.ipynb | 81 ++++++++++++++++--- 1 file changed, 71 insertions(+), 10 deletions(-) diff --git a/00-Getting_Started/01-DataJoint Basics - Interactive.ipynb b/00-Getting_Started/01-DataJoint Basics - Interactive.ipynb index f2f152a..f8f78dc 100644 --- a/00-Getting_Started/01-DataJoint Basics - Interactive.ipynb +++ b/00-Getting_Started/01-DataJoint Basics - Interactive.ipynb @@ -11,21 +11,72 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that you have successfully connected to DataJoint (if not, please visit [Connecting to DataBase](00-ConnectingToDatabase.ipynb) first), let's dive into using DataJoint! In this notebook, we will:\n", + "Congratulations! If you are reading this, then you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Configurating DataJoint package\n", "\n", - "1. learn what a data pipeline is\n", - "2. create our first simple data pipeline in DataJoint\n", - "3. insert some data into the pipeline\n", - "4. basic queries to flexibly explore the data pipeline\n", - "5. fetch the data from the pipeline\n", - "6. delete entries from tables" + "### Import package\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular Python packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Just like any other packages, to start using [DataJoint](https://datajoint.io/), you must first import the package `datajoint`. Convention is to alias the package to `dj`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*Run code cells by clicking on the left-bottom corner of the cell, or using Ctrl Enter shortcut.*" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "ename": "ModuleNotFoundError", + "evalue": "No module named 'datajoint'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[1], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mdatajoint\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mdj\u001b[39;00m\n", + "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'datajoint'" + ] + } + ], + "source": [ + "import datajoint as dj" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Connect to the DataJoint database server" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "As always, let's start by importing the `datajoint` library." + "Before you can get connected to the database server with DataJoint, you need to make sure that `datajoint` is configured properly. All `datajoint` configs can be found under `dj.config`. Let's take a look at what's inside the configuration." ] }, { @@ -34,7 +85,17 @@ "metadata": {}, "outputs": [], "source": [ - "import datajoint as dj" + "dj.config" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In particular, take a look at the `database.host`, `database.user`, and `database.password` fields - these fields tell DataJoint:\n", + "* which database to connect to (`database.host`)\n", + "* what user name to use (`database.user`), and\n", + "* the password for the user (`database.password`)" ] }, { @@ -1574,7 +1635,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.16" + "version": "3.9.6" }, "vscode": { "interpreter": { From 30f4b611c0867cb6e9ed20831a4d6cd44821b6d6 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 15:54:40 +0000 Subject: [PATCH 06/70] Tutorials organized in two folders and renamed --- .../01-DataJoint Basics.ipynb | 0 .../02-Calcium Imaging Imported Tables.ipynb | 0 .../03-Calcium ImagingComputed Tables.ipynb | 0 .../04-Electrophysiology Imported Tables.ipynb | 0 .../05-Electrophysiology Computed Tables.ipynb | 0 .../01-DataJoint Basics.ipynb | 0 .../02-Calcium Imaging Imported Tables.ipynb | 0 .../03-Calcium Imaging Computed Tables.ipynb | 0 .../04-Electrophysiology Imported Tables.ipynb | 0 .../05-Electrophysiology Computed Tables.ipynb | 0 10 files changed, 0 insertions(+), 0 deletions(-) rename 03-Completed_Exercises/01-DataJoint Basics - Completed.ipynb => completed_exercises/01-DataJoint Basics.ipynb (100%) rename 03-Completed_Exercises/02-calcium_imaging - Imported Tables - Completed.ipynb => completed_exercises/02-Calcium Imaging Imported Tables.ipynb (100%) rename 03-Completed_Exercises/03-calcium_imaging - Computed Table, Lookup Table, and Part Table - Completed.ipynb => completed_exercises/03-Calcium ImagingComputed Tables.ipynb (100%) rename 03-Completed_Exercises/02-electrophysiology - Imported Tables - Completed.ipynb => completed_exercises/04-Electrophysiology Imported Tables.ipynb (100%) rename 03-Completed_Exercises/03-electrophysiology - Computed Table, Lookup Table, and Part Table - Completed.ipynb => completed_exercises/05-Electrophysiology Computed Tables.ipynb (100%) rename 00-Getting_Started/01-DataJoint Basics - Interactive.ipynb => notebooks/01-DataJoint Basics.ipynb (100%) rename 01-Calcium_Imaging/02-Imported Tables - Interactive.ipynb => notebooks/02-Calcium Imaging Imported Tables.ipynb (100%) rename 01-Calcium_Imaging/03-Computed Table, Lookup Table, and Part Table - Interactive.ipynb => notebooks/03-Calcium Imaging Computed Tables.ipynb (100%) rename 02-Electrophysiology/02-Imported Tables - Interactive.ipynb => notebooks/04-Electrophysiology Imported Tables.ipynb (100%) rename 02-Electrophysiology/03-Computed Table, Lookup Table, and Part Table - Interactive.ipynb => notebooks/05-Electrophysiology Computed Tables.ipynb (100%) diff --git a/03-Completed_Exercises/01-DataJoint Basics - Completed.ipynb b/completed_exercises/01-DataJoint Basics.ipynb similarity index 100% rename from 03-Completed_Exercises/01-DataJoint Basics - Completed.ipynb rename to completed_exercises/01-DataJoint Basics.ipynb diff --git a/03-Completed_Exercises/02-calcium_imaging - Imported Tables - Completed.ipynb b/completed_exercises/02-Calcium Imaging Imported Tables.ipynb similarity index 100% rename from 03-Completed_Exercises/02-calcium_imaging - Imported Tables - Completed.ipynb rename to completed_exercises/02-Calcium Imaging Imported Tables.ipynb diff --git a/03-Completed_Exercises/03-calcium_imaging - Computed Table, Lookup Table, and Part Table - Completed.ipynb b/completed_exercises/03-Calcium ImagingComputed Tables.ipynb similarity index 100% rename from 03-Completed_Exercises/03-calcium_imaging - Computed Table, Lookup Table, and Part Table - Completed.ipynb rename to completed_exercises/03-Calcium ImagingComputed Tables.ipynb diff --git a/03-Completed_Exercises/02-electrophysiology - Imported Tables - Completed.ipynb b/completed_exercises/04-Electrophysiology Imported Tables.ipynb similarity index 100% rename from 03-Completed_Exercises/02-electrophysiology - Imported Tables - Completed.ipynb rename to completed_exercises/04-Electrophysiology Imported Tables.ipynb diff --git a/03-Completed_Exercises/03-electrophysiology - Computed Table, Lookup Table, and Part Table - Completed.ipynb b/completed_exercises/05-Electrophysiology Computed Tables.ipynb similarity index 100% rename from 03-Completed_Exercises/03-electrophysiology - Computed Table, Lookup Table, and Part Table - Completed.ipynb rename to completed_exercises/05-Electrophysiology Computed Tables.ipynb diff --git a/00-Getting_Started/01-DataJoint Basics - Interactive.ipynb b/notebooks/01-DataJoint Basics.ipynb similarity index 100% rename from 00-Getting_Started/01-DataJoint Basics - Interactive.ipynb rename to notebooks/01-DataJoint Basics.ipynb diff --git a/01-Calcium_Imaging/02-Imported Tables - Interactive.ipynb b/notebooks/02-Calcium Imaging Imported Tables.ipynb similarity index 100% rename from 01-Calcium_Imaging/02-Imported Tables - Interactive.ipynb rename to notebooks/02-Calcium Imaging Imported Tables.ipynb diff --git a/01-Calcium_Imaging/03-Computed Table, Lookup Table, and Part Table - Interactive.ipynb b/notebooks/03-Calcium Imaging Computed Tables.ipynb similarity index 100% rename from 01-Calcium_Imaging/03-Computed Table, Lookup Table, and Part Table - Interactive.ipynb rename to notebooks/03-Calcium Imaging Computed Tables.ipynb diff --git a/02-Electrophysiology/02-Imported Tables - Interactive.ipynb b/notebooks/04-Electrophysiology Imported Tables.ipynb similarity index 100% rename from 02-Electrophysiology/02-Imported Tables - Interactive.ipynb rename to notebooks/04-Electrophysiology Imported Tables.ipynb diff --git a/02-Electrophysiology/03-Computed Table, Lookup Table, and Part Table - Interactive.ipynb b/notebooks/05-Electrophysiology Computed Tables.ipynb similarity index 100% rename from 02-Electrophysiology/03-Computed Table, Lookup Table, and Part Table - Interactive.ipynb rename to notebooks/05-Electrophysiology Computed Tables.ipynb From 9860c63aacd981b04e8461030a8b7f4c78390458 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 15:55:03 +0000 Subject: [PATCH 07/70] Deleted Tutorial 00-Getting Started.ipynb --- 00-Getting_Started/00-Getting Started.ipynb | 363 -------------------- 1 file changed, 363 deletions(-) delete mode 100644 00-Getting_Started/00-Getting Started.ipynb diff --git a/00-Getting_Started/00-Getting Started.ipynb b/00-Getting_Started/00-Getting Started.ipynb deleted file mode 100644 index 4a1c1a5..0000000 --- a/00-Getting_Started/00-Getting Started.ipynb +++ /dev/null @@ -1,363 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Welcome to DataJoint Tutorial!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Congratulations! If you are reading this, then you have successfully connected to the workshop JupyterHub and opened up your very first workshop notebook! These notebooks are oranized with numbers to highlight what should come next. First, we'll walk through the startup process. Next, you'll see an introduction to DataJoint, and how to structure a data pipeline. The Calcium Imaging and Electrophysiology folders show example data structures and analysis techniques from common experimental techniques. We've included some fill-in-the-blank sections for you to code yourself. If you want to see completed notebooks, see the folder of completed exercises.\n", - "\n", - "Before we start, remember that all of the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content everytime you restart the server. However, you can easily download the notebooks that you're interested in keeping the changes." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this notebook, we will:\n", - "1. learn to import DataJoint package\n", - "2. connect DataJoint to our workshop database server\n", - "3. learn how to save your connection configuration\n", - "4. change your database password to something more secure/memorable and save it into your configuration" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# First thing first - Importing DataJoint package" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This JupyterHub environment comes with the latest DataJoint Python package pre-installed, along with many other popular scientific computation Python packages such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Just like any other packages, to start using [DataJoint](https://datajoint.io/), you must first import the package - `datajoint`! Convention is to alias the package to `dj`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import datajoint as dj" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You have now successfully imported `datajoint` package. However, `datajoint` is still not connected to a database. We need to **configure the connection information**." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Configuring connection to the DataJoint database server" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Before you can get connected to the database server with DataJoint, you need to make sure that `datajoint` is configured properly. All `datajoint` configs can be found under `dj.config`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's take a look at what's inside the configuration." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.config" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In particular, take a look at the `database.host`, `database.user`, and `database.password` fields - these fields tell DataJoint:\n", - "* which database to connect to (`database.host`)\n", - "* what user name to use (`database.user`), and\n", - "* the password for the user (`database.password`)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For this workshop, some of the configuration values are pre-filled (e.g. `database.host` already points to the workshop database or the local database if you are using Github Codespace). Let's complete the configuration by specifying the username and password.\n", - "\n", - "Go ahead and enter the username and password of your [datajoint.io](https://datajoint.io) account below.\n", - "\n", - "***NOTE***: If you are running this tutorial with GitHub Codespace, you can skip this credentials setting step, as they are already preconfigured correctly for you." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.config['database.user'] = 'ENTER USERNAME HERE'\n", - "dj.config['database.password'] = 'ENTER PASSWORD HERE'" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Check that the config now contains your username and password" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.config" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Testing your connection" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can now test your connection configuration by trying to explicitly connect to the database with `dj.conn()` function call." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.conn() # establish the connection" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If the above call returned without an error, then you have successfully established a connection with the database server!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Saving DataJoint configuration across sessions" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "By default, all changes made to the `dj.config` are reset when you reset your Python session, and thus you would have to configure the connection every time you start a new Python kernel.\n", - "\n", - "To save yourself the hassle, you can save the current configuration to **a local configuration file**, by default called `dj_local_config.json`. DataJoint will automatically load the configuration file when you import DataJoint the next time." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To save the current configuration, call the `save_local` method on the `dj.config` object." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# save to local config file\n", - "dj.config.save_local()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now your configuration is successfully saved into the local configuration file." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you would like to save this config globally, in which case you automatically load this configuration in any directory, you could do `save_global()`" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.config.save_global()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The local configuration overwrites the global one." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Changing the password" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The username and password used to connect to this tutorial database is that of your ***datajoint.io*** account, you can change this password on the [datajoint.io](https://datajoint.io) website. \n", - "\n", - "You can also directly change this database connection password using the `dj.set_password()` method. This is particularly useful if you are using DataJoint outside of this tutorial, where your ***datajoint.io*** credentials are not used for database connection." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# To change your password, run: dj.set_password()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Congratulations! You have successfully updated your database password via DataJoint! Now be sure to update and save the configuration with the new password." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# if you did change your password, update the config\n", - "# dj.config['database.password'] = 'ENTER YOUR NEW PASSWORD HERE'\n", - "\n", - "# and save it globally within this environment\n", - "dj.config.save_global()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Final check of your DataJoint configuration" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To verify that everything is working, go ahead and restart the Jupyter notebook kernel (hit restart icon or go Kernel > Restart). Once restarted, execute the following to verify your connection works. The connection information should now be loaded automatically." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import datajoint as dj\n", - "dj.conn() # connect using saved configuration" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Continue with the tutorial" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If all of the above worked, then you are now ready to continue on with the tutorial!\n", - "The next step in the tutorial is to:\n", - "1. learn the basic of the DataJoint framework with [01-DataJoint Basics](./01-DataJoint%20Basics%20-%20Interactive.ipynb)\n", - "2. data ingestion in a pipeline, with practical examples in 2 domains:\n", - " + [02-electrophysiology](/02-Electrophysiology/02-Imported%20Tables%20-%20Interactive.ipynb)\n", - " + [02-calcium imaging](./01-Calcium_Imaging/02-Imported%20Tables%20-%20Interactive.ipynb)\n", - "3. more advanced topics on built-in automatic computation of the pipeline with DataJoint, also with practical examples from:\n", - " + [03-electrophysiology](/02-Electrophysiology/03-Computed%20Table%2C%20Lookup%20Table%2C%20and%20Part%20Table%20-%20Completed.ipynb)\n", - " + [03-calcium imaging](./01-Calcium_Imaging/03-Computed%20Table%2C%20Lookup%20Table%2C%20and%20Part%20Table%20-%20Completed.ipynb)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As you will find out that these tutorial notebooks are meant to be interactive. There will be empty cells meant for you to write the code yourself, as mini exercises. To access the notebooks with complete solutions, visit this [folder](../03-Completed_Exercises)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.16" - }, - "vscode": { - "interpreter": { - "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" - } - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} From 5ff0746500d0e640bf474ca4ba52619c30931cd9 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 16:24:53 +0000 Subject: [PATCH 08/70] Tutorial 01 Deleted Configuration and Pipeline def update --- notebooks/01-DataJoint Basics.ipynb | 77 +++++------------------------ 1 file changed, 13 insertions(+), 64 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index f8f78dc..0dad82b 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -18,101 +18,50 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Configurating DataJoint package\n", - "\n", - "### Import package\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular Python packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Just like any other packages, to start using [DataJoint](https://datajoint.io/), you must first import the package `datajoint`. Convention is to alias the package to `dj`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "*Run code cells by clicking on the left-bottom corner of the cell, or using Ctrl Enter shortcut.*" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "ename": "ModuleNotFoundError", - "evalue": "No module named 'datajoint'", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[1], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[39mimport\u001b[39;00m \u001b[39mdatajoint\u001b[39;00m \u001b[39mas\u001b[39;00m \u001b[39mdj\u001b[39;00m\n", - "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'datajoint'" - ] - } - ], - "source": [ - "import datajoint as dj" + "## Essential concepts and steps\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Connect to the DataJoint database server" + "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular Python packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). Just like any other packages, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. Convention is to alias the package to `dj`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Before you can get connected to the database server with DataJoint, you need to make sure that `datajoint` is configured properly. All `datajoint` configs can be found under `dj.config`. Let's take a look at what's inside the configuration." + "*NOTE: Run code cells by clicking on the left-top corner bottom of the cell, or using Ctrl+Enter shortcut.*" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ - "dj.config" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In particular, take a look at the `database.host`, `database.user`, and `database.password` fields - these fields tell DataJoint:\n", - "* which database to connect to (`database.host`)\n", - "* what user name to use (`database.user`), and\n", - "* the password for the user (`database.password`)" + "import datajoint as dj" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# So... What is a data pipeline?" + "### Data pipeline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "If you visit the [documentation for DataJoint](https://docs.datajoint.io/introduction/Data-pipelines.html), we define a data pipeline as follows:\n", - "> A data pipeline is a sequence of steps (more generally a directed acyclic graph) with integrated storage at each step. These steps may be thought of as nodes in a graph.\n", "\n", - "While this is an accurate description, it may not be the most intuitive definition. Put succinctly, a data pipeline is a listing or a \"map\" of various \"things\" that you work with in a project, with line connecting things to each other to indicate their dependecies. The \"things\" in a data pipeline tends to be the *nouns* you find when describing a project. The \"things\" may include anything from mouse, experimenter, equipment, to experiment session, trial, two-photon scans, electric activities, to receptive fields, neuronal spikes, to figures for a publication! A data pipeline gives you a framework to:\n", + "A data pipeline is a sequence of steps (more generally a directed acyclic graph) with integrated storage at each step. These steps may be thought of as nodes in a graph. In other words, a data pipeline is a \n", + "listing or a \"map\" of various \"things\" that you work with in a project, with line connecting things to each other to indicate their dependecies. The \"things\" in a data pipeline tends to be the *nouns* you find when describing a project. The \"things\" may include anything from mouse, experimenter, equipment, to experiment session, trial, two-photon scans, electric activities, to receptive fields, neuronal spikes, to figures for a publication! \n", + "\n", + "![pipeline](https://raw.githubusercontent.com/datajoint/datajoint-python/master/images/pipeline.png)\n", + "\n", + "A data pipeline gives you a framework to:\n", "\n", "1. define these \"things\" as tables in which you can store the information about them\n", "2. define the relationships (in particular the dependencies) between the \"things\"\n", @@ -1635,7 +1584,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.6" + "version": "3.9.17" }, "vscode": { "interpreter": { From d1d368c04a4b6a03f1774a9b4a6915100252e8c7 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 17:58:52 +0000 Subject: [PATCH 09/70] Updated Table of Contents in README --- README.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 5bd0bcd..9c96e36 100644 --- a/README.md +++ b/README.md @@ -16,10 +16,14 @@ This document will guide you as a new DataJoint user through interactive tutoria - Completed exercises: notebooks with the code sections completed and solved. -- Pipeline - - ephys - - imaging - - mouse +- Short tutorials: + - DataJoint in 30min + - University + +- Tutorial pipeline + - Ephys cell activity + - Imaging + - Mouse session ## Key learnings from the tutorials From 73fee2ea55cd6b677b82116e1e52d3af1bd4bd50 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 20:05:45 +0000 Subject: [PATCH 10/70] Update bullet points in Installation --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 9c96e36..15ff392 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,13 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - *Tip*: GitHub auto names the codespace but you can rename the codespace so that it is easier to identify later. - **Local environment**: - - We highly recommend this option for users that after exploring the tutorials want to apply DataJoint for **their own neuroscience experiments** and lab research. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git), [Docker](https://docs.docker.com/get-docker/), [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) and [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). For more detailed instructions, please check out the [User Guide in DataJoint Documentation](https://datajoint.com/docs/elements/user-guide/). + - We highly recommend this option for users that after exploring the tutorials want to apply DataJoint for **their own neuroscience experiments** and lab research. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have: + - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) + - [Docker](https://docs.docker.com/get-docker/) + - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) + - [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). + + For more detailed instructions, please check out the [User Guide in DataJoint Documentation](https://datajoint.com/docs/elements/user-guide/). Before we start, remember that all of the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that you are interested in keeping the changes. From f7422270f2d6a71bc2131227ca3c4f432339a20c Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 20:25:14 +0000 Subject: [PATCH 11/70] Minor typos in README.md --- README.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 15ff392..c644192 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,13 @@ # Welcome to DataJoint tutorials! -DataJoint is an open-source tool library to design, build and automate data analysis, pipelines and data sharing for neuroscience experiments and research labs. +DataJoint is an open-source tool library to design, build and automate data analysis, pipelines, and data sharing for neuroscience experiments and research labs. This document will guide you as a new DataJoint user through interactive tutorials organized in [Jupyter notebooks](https://jupyter-notebook.readthedocs.io/en/stable/) and written in [Python](https://www.python.org/). -*Please note that these hands-on DataJoint tutorials are friendly to non-expert users and advanced programming skills are not required.* +*Please note that these hands-on DataJoint tutorials are friendly to non-expert users, and advanced programming skills are not required.* ## Table of contents -- Notebooks: interactive notebooks to be a master of DataJoint. The Calcium Imaging and Electrophysiology tutorials serve as common examples of data structure and data analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! +- Notebooks: interactive notebooks to be a master of DataJoint. The Calcium Imaging and Electrophysiology tutorials are common examples of data structure and analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! - 01-DataJoint Basics - 02-Calcium Imaging Imported Tables - 03-Calcium Imaging Computed Tables @@ -31,7 +31,7 @@ After completing this set of tutorials, you will gain real experience in the bas Here is a summary of the content that you can expect to have learned: -- Understanding DataJoint basics: concepts, design and structure (~1 hour) +- Understanding DataJoint basics: concepts, design, and structure (~1 hour) - Create schemas/tables - Table tiers (`Lookup`, `Manual`, `Imported`, `Computed`) - Insert entries and view entries in tables @@ -57,23 +57,23 @@ Here is a summary of the content that you can expect to have learned: ## Quick start ### User installation -DataJoint tutorials are easily accessible using an **interactive environment** that contains all the software required to run the experiments. The environment is configured by [DevContainer] (https://containers.dev/). Here are two options to launch the interactive environment. +DataJoint tutorials are easily accessible using an **interactive environment** that contains all the software required to run the experiments. The setting is configured by [DevContainer] (https://containers.dev/). Here are two options to launch the interactive environment. -*Please note that to use the DataJoint Python package with an interactive environment you need a [GitHub](https://github.com/) account.* +*Please note that to use the DataJoint Python package with an interactive environment, you need a [GitHub](https://github.com/) account.* - **Cloud-based environment: GitHub Codespaces**: (*recommended*) - - This is the easiest option for **tutorial users**. You will immediately start coding using DataJoint and Python, with no installation of software or local environments. Cloud-based environments (IDEs), such as [GitHub Codespaces](https://github.com/features/codespaces), use built-in tools directly connected to the cloud and work on the browser. + - This is the easiest option for **tutorial users**. You will immediately start coding using DataJoint and Python, without installing software or local environments. Cloud-based environments (IDEs), such as [GitHub Codespaces](https://github.com/features/codespaces), use built-in tools directly connected to the cloud and work on the browser. - Instructions: - Fork the [datajoint-tutorials](https://github.com/datajoint/datajoint-tutorials) repository into your repository. - From your `datajoint-tutorials` repository, click on `Code`, then click on `Codespaces` tab, and `+` option will `Create codespace on main` on your fork with default options. For more control, see the `...` where you may create `New with options...`. - - The building time for a codespace is **~5m**. This is done infrequently and cached for convenience. - - The start time for a codespace is **~30s**. This will pull the built codespace from the cache when you need it. + - The building time for Codespaces is **~5m**. This is done infrequently and cached for convenience. + - The start time for Codespaces is **~30s**. This will pull the built codespace from the cache when you need it. - *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of computing and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete Codespaces when not actively in use and recreate them when needed. We'll soon be creating prebuilds to avoid larger build times. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. - - *Tip*: GitHub auto names the codespace but you can rename the codespace so that it is easier to identify later. + - *Tip*: GitHub auto names the Codespaces, but you can rename the Codespaces so that it is easier to identify later. - **Local environment**: - - We highly recommend this option for users that after exploring the tutorials want to apply DataJoint for **their own neuroscience experiments** and lab research. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have: + - We highly recommend this option for users that who want to apply DataJoint to **their own neuroscience experiments** and lab research after exploring the tutorials. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have the following: - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - [Docker](https://docs.docker.com/get-docker/) - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) @@ -81,19 +81,19 @@ DataJoint tutorials are easily accessible using an **interactive environment** t For more detailed instructions, please check out the [User Guide in DataJoint Documentation](https://datajoint.com/docs/elements/user-guide/). -Before we start, remember that all of the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that you are interested in keeping the changes. +Before we start, remember that all the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that interest you in keeping the changes. ### Instructions -- To begin, we suggest navigating to the notebooks directory located in the left panel and proceeding through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. +- To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. -- Once you are done, see the options available to you in the menu in the bottom-left corner. For example, in Codespace you will have the option to `Stop Current Codespace`, but when running DevContainer on your machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespace after 30 minutes of inactivity. +- Once you are done, see the options in the menu in the bottom-left corner. For example, in Codespaces you can `Stop Current Codespace`, but when running DevContainer on your machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. ## Support If you need help getting started or run into any errors, please open a GitHub Issue or contact our team by email at support@datajoint.com. ## Additional DataJoint Tutorials -- DataJoint Elements are a collection of curated modules for assembling data pipelines for several modalities of neurophysiology experiments. +- DataJoint Elements is a collection of curated modules for assembling data pipelines for several modalities of neurophysiology experiments. - [Element Calcium Imaging Tutorial](https://github.com/datajoint/element-calcium-imaging#interactive-tutorial) - [Element Array Electrophysiology Tutorial](https://github.com/datajoint/workflow-array-ephys#interactive-tutorial) From f4cbd5deb19a17b5c69017f1438b6a3e870b7bca Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 21:31:13 +0000 Subject: [PATCH 12/70] Update Data pipelines Concept in tutorial 01. --- notebooks/01-DataJoint Basics.ipynb | 36 ++++++++++++++++++----------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 0dad82b..2e18c45 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -18,14 +18,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Essential concepts and steps\n" + "## Essential concepts and setup\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular Python packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). Just like any other packages, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. Convention is to alias the package to `dj`." + "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular [Python](https://www.python.org/) packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). Just like any other packages, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. Convention is to alias the package to `dj`." ] }, { @@ -37,7 +37,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -48,7 +48,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Data pipeline" + "### Data pipelines\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### Concept" ] }, { @@ -56,33 +63,34 @@ "metadata": {}, "source": [ "\n", - "A data pipeline is a sequence of steps (more generally a directed acyclic graph) with integrated storage at each step. These steps may be thought of as nodes in a graph. In other words, a data pipeline is a \n", - "listing or a \"map\" of various \"things\" that you work with in a project, with line connecting things to each other to indicate their dependecies. The \"things\" in a data pipeline tends to be the *nouns* you find when describing a project. The \"things\" may include anything from mouse, experimenter, equipment, to experiment session, trial, two-photon scans, electric activities, to receptive fields, neuronal spikes, to figures for a publication! \n", + "A data pipeline is a collection of data processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex sequences of data acquisition, processing and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", "\n", - "![pipeline](https://raw.githubusercontent.com/datajoint/datajoint-python/master/images/pipeline.png)\n", + "In other words, a data pipeline can be seen as a network where each node or *noun* represents a **table**. The information involved in a research project, or in a whole research lab, can be organized and stored in these tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", "\n", - "A data pipeline gives you a framework to:\n", + "The data pipeline is formed by making these tables interdependent (as the nodes connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms a full cohesive data pipeline.\n", "\n", - "1. define these \"things\" as tables in which you can store the information about them\n", - "2. define the relationships (in particular the dependencies) between the \"things\"\n", + "This is an example of a data pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", "\n", - "A data pipeline can then serve as a map that describes everything that goes on in your experiment, capturing what is collected, what is processed, and what is analyzed/computed. A well designed data pipeline not only let's you organize your data well, but can bring out logical clarity to your experiment, and may even bring about new insights by making how everything in your experiment relates together obvious.\n", + "![pipeline](https://raw.githubusercontent.com/datajoint/element-optogenetics/main/images/pipeline.svg)\n", "\n", - "Let's go ahead and build together a pipeline from scratch to better understand what a data pipeline is all about." + "A **well-designed data pipeline**: \n", + "- 1. Collects, organizes and stores **every relevant piece of information during the scientific research**\n", + "- 2. Integrates, processes and connects these pieces of information through **several steps**\n", + "- 3. Analyses and transforms the input data into **valuable insights for the research**, bringing together logical clarity to the experiments\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Building our first pipeline: " + "##### Practical example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's build a pipeline to collect, store and process data and analysis for our hypothetical single electrode recording or calcium imaging recording in mice. To help us understand the project better, here is a brief description:" + "The practical example of these tutorials consists of building a pipeline for a scientific project based on two experiments on rodents: single electrode recording and calcium imaging recording. Here is a brief description:" ] }, { From b05310ea7cc5dcac7acd091be514cc1645a8912d Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 22:01:02 +0000 Subject: [PATCH 13/70] Update Data pipelines-Practical Example-Tutorial01 --- notebooks/01-DataJoint Basics.ipynb | 90 ++++++++++++++++++----------- 1 file changed, 55 insertions(+), 35 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 2e18c45..ef0e779 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -65,11 +65,13 @@ "\n", "A data pipeline is a collection of data processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex sequences of data acquisition, processing and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", "\n", - "In other words, a data pipeline can be seen as a network where each node or *noun* represents a **table**. The information involved in a research project, or in a whole research lab, can be organized and stored in these tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", + "In other words, a data pipeline can be seen as a network where each node or entity set represents a **table**. The information involved in a research project, or in a whole research lab, can be organized and stored in these entity sets or tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", "\n", - "The data pipeline is formed by making these tables interdependent (as the nodes connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms a full cohesive data pipeline.\n", + "The data pipeline is formed by making these tables interdependent (as the nodes connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms a full cohesive data pipeline. \n", "\n", - "This is an example of a data pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", + "A [DataJoint pipeline](https://datajoint.com/docs/core/datajoint-python/0.14/concepts/terminology/) contains database table definitions, dependencies, and associated coomputations, together with the transformations underlying a DataJoint workflow. \n", + "\n", + "This is an example of a DataJoint pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", "\n", "![pipeline](https://raw.githubusercontent.com/datajoint/element-optogenetics/main/images/pipeline.svg)\n", "\n", @@ -90,7 +92,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The practical example of these tutorials consists of building a pipeline for a scientific project based on two experiments on rodents: single electrode recording and calcium imaging recording. Here is a brief description:" + "The practical example that is used during these tutorials will be to design and compute a data pipeline for a scientific project of two experiments on rodents: \n", + "- Single-electrode recording\n", + "- Calcium imaging recording\n", + "\n", + "Here is a brief description:" ] }, { @@ -98,31 +104,31 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> * Your lab houses many mice, and each mouse is identified by a unique ID. You also want to keep track of information about each mouse such as their date of birth, and gender.\n", - "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day! However, on any given day, a mouse undergoes at most one recording session.\n", - "> * For each experimental session, you would like to record what mouse you worked with and when you performed the experiment. You would also like to keep track of other helpful information such as the experimental setup you worked on. \n", + "> * Your lab houses many mice, and each mouse is identified by a unique ID. You also want to keep track of other information about each mouse such as their date of birth, and gender.\n", + "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day. However, on any given day, a mouse undergoes at most one recording session.\n", + "> * For each experimental session, you want to record what mouse you worked with and when you performed the experiment. You also want to keep track of other helpful information such as the experimental setup you worked on. \n", "\n", - "> * In a session of electrophysiology\n", - ">> * you record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you recorded.\n", + "> * In a session of electrophysiology:\n", + ">> * You record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you recorded.\n", ">> * Neuron's activities are recorded as raw traces. Neuron's spikes needs to be detected for further analysis to be performed.\n", - "> * In a session of calcium imaging\n", - ">> * you scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", - ">> * you would like to segment the frames and get the regions of interest (ROIs), and save a mask for each ROI\n", - ">> * finally you would like to extract the trace from each segmented ROI" + "> * In a session of calcium imaging:\n", + ">> * You scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", + ">> * You need to segment the frames and get the regions of interest (ROIs), and save a mask for each ROI\n", + ">> * In addition, you need to extract the trace from each segmented ROI" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Pipeline design starts by identifying **things** or **entities** in your project. Common entities includes experimental subjects (e.g. mouse), recording sessions, and two-photon scans." + "The design of a data pipeline starts by identifying the **entities** or **tables** in your research project. Common entities include experimental subjects (e.g. mouse), recording sessions, and two-photon scans." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's revisit the project description, this time paying special attention to **what** (e.g. nouns) about your experiment. Here I have highlighted some nouns in particular." + "Let's revisit the project description, this time paying special attention to **what** (e.g. nouns or entities) about your experiment. Here some particular entities are highlighted." ] }, { @@ -130,45 +136,59 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> * Your lab houses many **mice**, and each mouse is identified by a unique ID. You also want to keep track of information about each mouse such as their date of birth, and gender.\n", - "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day! However, on an any given day, a mouse undergoes at most one recording session.\n", - "> * For each **experimental session**, you would like to record what mouse you worked with and when you performed the experiment. You would also like to keep track of other helpful information such as the experimental setup you worked on. \n", + "> * Your lab houses many **mice**, and each mouse is identified by a unique ID. You also want to keep track of other information about each mouse such as their date of birth, and gender.\n", + "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day. However, on any given day, a mouse undergoes at most one recording session.\n", + "> * For each **experimental session**, you want to record what mouse you worked with and when you performed the experiment. You also want to keep track of other helpful information such as the experimental setup you worked on. \n", "\n", - "> * In a session of electrophysiology\n", - ">> * you record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you recorded.\n", + "> * In a session of electrophysiology:\n", + ">> * You record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you recorded.\n", ">> * Neuron's activities are recorded as raw traces. **Neuron's spikes** needs to be detected for further analysis to be performed.\n", - "> * In a session of calcium imaging\n", - ">> * you scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each **scan** you performed.\n", - ">> * you would like to segment the frames and get the **regions of interest (ROIs)**, and save a mask for each ROI\n", - ">> * finally you would like to extract the **trace** from each segmented ROI" + "> * In a session of calcium imaging:\n", + ">> * You **scan** a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", + ">> * You need to segment the frames and get the **regions of interest (ROIs)**, and save a mask for each ROI\n", + ">> * In addition, you need to extract the **trace** from each segmented ROI" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Just by going though the description, we can start to identify **things** or **entities** that we might want to store and represent in our data pipeline:\n", + "Just by going though the description, we can start to identify **entities** that need to be stored and represented in our data pipeline:\n", "\n", - "* mouse\n", - "* experimental session\n", + ">* Mouse\n", + ">* Experimental session\n", "\n", - "For ephys:\n", + "For Ephys:\n", "\n", - ">* neuron\n", - ">* spikes\n", + ">* Neuron\n", + ">* Spikes\n", "\n", - "For calcium imaging:\n", + "For Calcium imaging:\n", "\n", - ">* scan\n", - ">* regions of interest\n", - ">* trace" + ">* Scan\n", + ">* Regions of interest (ROI)\n", + ">* Trace" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the next section you will learn to design the tables and manipulate the data for `Mouse` and `Experimental sessions`. The rest of the pipeline will be addressed in the subsequent tutorials." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In the current notebook, we will design the tables for mouse and experimental sessions, the rest of the pipeline will be designed in the subdirectory `electrophysiology` and `calcium_imaging`" + "### Schemas, tables and basic data manipulations" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### Concept" ] }, { From 1d57fa211969196c7b5d2c649d1016dcdbe34d68 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 22:54:12 +0000 Subject: [PATCH 14/70] Update Practical example-Data pipelines-Tutorial01 --- notebooks/01-DataJoint Basics.ipynb | 1065 ++++++++++++++++++++++++--- 1 file changed, 978 insertions(+), 87 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index ef0e779..a1b46cb 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -163,7 +163,7 @@ ">* Neuron\n", ">* Spikes\n", "\n", - "For Calcium imaging:\n", + "For Calcium Imaging:\n", "\n", ">* Scan\n", ">* Regions of interest (ROI)\n", @@ -174,94 +174,109 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In the next section you will learn to design the tables and manipulate the data for `Mouse` and `Experimental sessions`. The rest of the pipeline will be addressed in the subsequent tutorials." + "In the next section you will learn to design the tables and manipulate the data for `Mouse` and `Experimental sessions`. The rest of the pipeline (`Ephys` and `Calcium Imaging` will be addressed in the subsequent tutorials." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Schemas, tables and basic data manipulations" + "### Schemas, tables and basic relational operators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "##### Concept" + "##### Concepts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In DataJoint data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each row of the table is a single example (instance) of the category of entity. \n", + "In a data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each row of the table is a single example (instance) of the entity's category. \n", + "\n", + "For example, if we have a `Mouse` table, then each row in the mouse table represents a single mouse. \n", + "\n", + "It is important to think what information will **uniquely identify** each entry. \n", + "\n", + "In this case, the information that uniquely identifies the `Mouse` table is their **mouse IDs** - a unique ID number assigned to each animal in the lab. This attribute is named the **primary key** of the table.\n", "\n", - "For example, if we have a `Mouse` table, then each row in the mouse table represents a single mouse!" + "| Mouse_ID (*Primary key attribute*)|\n", + "|:--------: | \n", + "| 11234 |\n", + "| 11432 |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "When constructing such table, we need to figure out what it would take to **uniquely identify** each entry. Let's take the example of the **mouse** and think about what it would take to uniquely identify a mouse." + "After some thought, we might conclude that each mouse can be uniquely identified by knowing its **mouse ID** - a unique ID number assigned to each mouse in the lab. \n", + "\n", + "The mouse ID is then a column in the table or an **attribute** that can be used to **uniquely identify** each mouse. \n", + "\n", + "Such attribute is called the **primary key** of the table: the subset of table attributes that uniquely identify each entity in the table. The **secondary attribute** refers to any field in a table not in the primary key.\n", + "\n", + "| Mouse_ID (*Primary key attribute*) \n", + "|:--------:| \n", + "| 11234 (*Secondary attribute*)\n", + "| 11432 (*Secondary attribute*)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "After some thought, we might conclude that each mouse can be uniquely identified by knowing its **mouse ID** - a unique ID number assigned to each mouse in the lab. The mouse ID is then a column in the table or an **attribute** that can be used to **uniquely identify** each mouse. Such attribute is called the **primary key** of the table.\n", - "\n", - "| mouse_id* |\n", - "|:--------:|\n", - "| 11234 |\n", - "| 11432 |" + "Once we have successfully identified the primary key of the table, we can now think about what other columns, or **non-primary key attributes** - additional information **about each entry in the table that need to be stored as well**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Once we have successfully identified the primary key of the table, we can now think about what other columns, or **non-primary key attributes** that we would want to include in the table. These are additional information **about each entry in the table that we want to store**." + "For the case of `Mouse`, what other information about the mouse you might want to store? \n", + "\n", + "Based on the project description, we would probably want to store information such as the mouse's **date of birth** (DOB) and **sex**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "For the case of mouse, what other information about the mouse you might want to store? Based on the project description, we would probably want to store information such as the mouse's **date of birth** and **gender**." + "| Mouse_ID | DOB | sex |\n", + "|:--------:|------------|--------|\n", + "| 11234 | 2017-11-17 | M |\n", + "| 11432 | 2018-03-04 | F |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "| mouse_id* | dob | sex |\n", - "|:--------:|------------|--------|\n", - "| 11234 | 2017-11-17 | M |\n", - "| 11432 | 2018-03-04 | F |" + "Now we have an idea on how to represent information about mouse, let's create the table using **DataJoint**!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now we have an idea on how to represent information about mouse, let's create the table using **DataJoint**!" + "##### Practical example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Create a schema - house for your tables" + "##### Schema" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Every table lives inside a schema - a logical collection of one or more tables in your pipeline. Your final pipeline may consists of many tables spread across one or more schemas. Let's go ahead and create the first schema to house our table." + "Every table lives inside a schema - a logical collection of one or more tables in your pipeline. Your final pipeline may consists of many tables spread across one or more schemas. Let's go ahead and create the first schema to house our `Mouse` table using DataJoint." ] }, { @@ -269,14 +284,23 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We create the schema using `dj.schema()` function, passing in the name of the schema. For this workshop, you are given the database privilege to use any schema name. Let's create a schema called `tutorial`." + "We create the schema using `dj.schema()` function, passing in the name of the schema. For this tutorial, we create a schema called `tutorial`." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "[2023-07-24 22:36:18,016][INFO]: Connecting root@fakeservices.datajoint.io:3306\n", + "[2023-07-24 22:36:18,073][INFO]: Connected root@fakeservices.datajoint.io:3306\n" + ] + } + ], "source": [ "schema = dj.schema('tutorial')" ] @@ -285,26 +309,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have a schema to place our table into, let's go ahead and define our first table!" + "Now that we have a schema to place our table into, let's go ahead and define our first table. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Creating your first table" + "##### Table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In DataJoint, you define each table as a class, and provide the table definition (e.g. attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)." + "In DataJoint, you define each table as a `class`, and provide the table definition (e.g. attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -328,9 +352,95 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +-----+ +-----+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse()" ] @@ -339,21 +449,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Insert entries with `insert1` and `insert` methods" + "##### Insert operators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The table was successfully defined, but without any content, the table is not too interesting. Let's go ahead and insert some **mouse** into the table, one at a time using the `insert1` method." + "The table was successfully defined, but without any content, the table is not too interesting. Let's go ahead and **insert some mouse information** into the table, one at a time using the `insert1` method." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert a mouse with the following information:\n", + "Let's `insert1` a mouse with the following information:\n", "* mouse_id: 0\n", "* date of birth: 2017-03-01\n", "* sex: male" @@ -361,7 +471,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ @@ -370,9 +480,97 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + " (Total: 1)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse()" ] @@ -381,12 +579,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You could also insert1 as a dictionary" + "You can also `insert1` as a dictionary" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -399,7 +597,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": {}, "outputs": [], "source": [ @@ -408,9 +606,100 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
1002017-05-12F
\n", + " \n", + "

Total: 2

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + "100 2017-05-12 F \n", + " (Total: 2)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse()" ] @@ -424,7 +713,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -437,7 +726,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -448,12 +737,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Of course, you can insert a list of dictionaries" + "Of course, you can `insert` a list of dictionaries" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 12, "metadata": {}, "outputs": [], "source": [ @@ -468,9 +757,115 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
1002017-05-12F
\n", + " \n", + "

Total: 7

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +---------+\n", + "0 2017-03-01 M \n", + "1 2016-11-19 M \n", + "2 2016-11-20 unknown \n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "100 2017-05-12 F \n", + " (Total: 7)" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse()" ] @@ -479,7 +874,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Data integrity" + "##### Data integrity" ] }, { @@ -491,9 +886,23 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 14, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "DuplicateError", + "evalue": "(\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1(\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m,\n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdob\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2018-01-01\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 4\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39msex\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 5\u001b[0m })\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", + "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" + ] + } + ], "source": [ "Mouse.insert1(\n", "{'mouse_id': 0,\n", @@ -511,7 +920,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -529,16 +938,137 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", + " \n", + "

Total: 12

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +---------+\n", + "0 2017-03-01 M \n", + "1 2016-11-19 M \n", + "2 2016-11-20 unknown \n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "19 2018-07-21 M \n", + "22 2019-12-15 F \n", + "34 2018-09-22 M \n", + "100 2017-05-12 F \n", + " (Total: 12)" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse()" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 17, "metadata": {}, "outputs": [], "source": [ @@ -549,7 +1079,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Create tables with dependencies" + "### Create tables with dependencies" ] }, { @@ -558,24 +1088,24 @@ "source": [ "Congratulations! We have successfully created your first table! We are now ready to tackle and include other **entities** in the project into our data pipeline. \n", "\n", - "Let's now take a look at representing an **experimental session**." + "Let's now have a look at representing an `experimental session`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "As with mouse, we should think about **what information (i.e. attributes) is needed to uniquely identify an experimental session**. Here is the relevant section of the project description:\n", + "As with `mouse`, we should think about **what information (i.e. attributes) is needed to uniquely identify an `experimental session`**. Here is the relevant section of the project description:\n", "\n", - "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with **more than one mouse in a day**! However, on an any given day, **a mouse undergoes at most one recording session**.\n", - "> * For each experimental session, you would like to record **what mouse you worked with** and **when you performed the experiment**. You would also like to keep track of other helpful information such as the **experimental setup** you worked on." + "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with **more than one mouse in a day**. However, on any given day, **a mouse undergoes at most one recording session**.\n", + "> * For each **experimental session**, you want to record **what mouse you worked with** and **when you performed the experiment**. You also want to keep track of other helpful information such as the **experimental setup** you worked on. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Based on the above, it appears that you need to know:\n", + "Based on the above, it seems that you need to know:\n", "\n", "* the date of the session\n", "* the mouse you recorded from in that session\n", @@ -596,14 +1126,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Thus we will need both **mouse** and a new attribute **session_date** to uniquely identify a single session. \n", + "Thus, we will need both **mouse** and a new attribute **session_date** to uniquely identify a single `session`. \n", "\n", "Remember that a **mouse** is already uniquely identified by its primary key - **mouse_id**. In DataJoint, you can declare that **session** depends on the mouse, and DataJoint will automatically include the mouse's primary key (`mouse_id`) as part of the session's primary key, along side any additional attribute(s) you specificy." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 18, "metadata": {}, "outputs": [], "source": [ @@ -629,9 +1159,51 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "dj.Diagram(schema)" ] @@ -645,7 +1217,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [ @@ -661,9 +1233,105 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experiment session\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-150Edgar Y. Walker
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +------------+ +-----------+\n", + "0 2017-05-15 0 Edgar Y. Walke \n", + " (Total: 1)" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Session()" ] @@ -677,9 +1345,110 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experiment session\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-150Edgar Y. Walker
02018-01-15100Jacob Reimer
\n", + " \n", + "

Total: 2

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +------------+ +-----------+\n", + "0 2017-05-15 0 Edgar Y. Walke \n", + "0 2018-01-15 100 Jacob Reimer \n", + " (Total: 2)" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "data = {\n", " 'mouse_id': 0,\n", @@ -702,7 +1471,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 23, "metadata": {}, "outputs": [], "source": [ @@ -719,9 +1488,115 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experiment session\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-150Edgar Y. Walker
02018-01-15100Jacob Reimer
182018-01-15101Jacob Reimer
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +------------+ +-----------+\n", + "0 2017-05-15 0 Edgar Y. Walke \n", + "0 2018-01-15 100 Jacob Reimer \n", + "18 2018-01-15 101 Jacob Reimer \n", + " (Total: 3)" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Session()" ] @@ -735,7 +1610,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 25, "metadata": {}, "outputs": [], "source": [ @@ -749,11 +1624,27 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 26, "metadata": { "scrolled": true }, - "outputs": [], + "outputs": [ + { + "ename": "IntegrityError", + "evalue": "Cannot add or update a child row: a foreign key constraint fails (`tutorial`.`session`, CONSTRAINT `session_ibfk_1` FOREIGN KEY (`mouse_id`) REFERENCES `mouse` (`mouse_id`) ON DELETE RESTRICT ON UPDATE CASCADE)", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mIntegrityError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[26], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Session\u001b[39m.\u001b[39;49minsert1(bad_data)\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:440\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 424\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m 425\u001b[0m query \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39m{command}\u001b[39;00m\u001b[39m INTO \u001b[39m\u001b[39m{destination}\u001b[39;00m\u001b[39m(`\u001b[39m\u001b[39m{fields}\u001b[39;00m\u001b[39m`) VALUES \u001b[39m\u001b[39m{placeholders}\u001b[39;00m\u001b[39m{duplicate}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m 426\u001b[0m command\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mREPLACE\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mif\u001b[39;00m replace \u001b[39melse\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mINSERT\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m 427\u001b[0m destination\u001b[39m=\u001b[39m\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mfrom_clause(),\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 438\u001b[0m ),\n\u001b[1;32m 439\u001b[0m )\n\u001b[0;32m--> 440\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mconnection\u001b[39m.\u001b[39;49mquery(\n\u001b[1;32m 441\u001b[0m query,\n\u001b[1;32m 442\u001b[0m args\u001b[39m=\u001b[39;49m\u001b[39mlist\u001b[39;49m(\n\u001b[1;32m 443\u001b[0m itertools\u001b[39m.\u001b[39;49mchain\u001b[39m.\u001b[39;49mfrom_iterable(\n\u001b[1;32m 444\u001b[0m (v \u001b[39mfor\u001b[39;49;00m v \u001b[39min\u001b[39;49;00m r[\u001b[39m\"\u001b[39;49m\u001b[39mvalues\u001b[39;49m\u001b[39m\"\u001b[39;49m] \u001b[39mif\u001b[39;49;00m v \u001b[39mis\u001b[39;49;00m \u001b[39mnot\u001b[39;49;00m \u001b[39mNone\u001b[39;49;00m) \u001b[39mfor\u001b[39;49;00m r \u001b[39min\u001b[39;49;00m rows\n\u001b[1;32m 445\u001b[0m )\n\u001b[1;32m 446\u001b[0m ),\n\u001b[1;32m 447\u001b[0m )\n\u001b[1;32m 448\u001b[0m \u001b[39mexcept\u001b[39;00m UnknownAttributeError \u001b[39mas\u001b[39;00m err:\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/connection.py:340\u001b[0m, in \u001b[0;36mConnection.query\u001b[0;34m(self, query, args, as_dict, suppress_warnings, reconnect)\u001b[0m\n\u001b[1;32m 338\u001b[0m cursor \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_conn\u001b[39m.\u001b[39mcursor(cursor\u001b[39m=\u001b[39mcursor_class)\n\u001b[1;32m 339\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 340\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_execute_query(cursor, query, args, suppress_warnings)\n\u001b[1;32m 341\u001b[0m \u001b[39mexcept\u001b[39;00m errors\u001b[39m.\u001b[39mLostConnectionError:\n\u001b[1;32m 342\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m reconnect:\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/connection.py:296\u001b[0m, in \u001b[0;36mConnection._execute_query\u001b[0;34m(cursor, query, args, suppress_warnings)\u001b[0m\n\u001b[1;32m 294\u001b[0m cursor\u001b[39m.\u001b[39mexecute(query, args)\n\u001b[1;32m 295\u001b[0m \u001b[39mexcept\u001b[39;00m client\u001b[39m.\u001b[39merr\u001b[39m.\u001b[39mError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 296\u001b[0m \u001b[39mraise\u001b[39;00m translate_query_error(err, query)\n", + "\u001b[0;31mIntegrityError\u001b[0m: Cannot add or update a child row: a foreign key constraint fails (`tutorial`.`session`, CONSTRAINT `session_ibfk_1` FOREIGN KEY (`mouse_id`) REFERENCES `mouse` (`mouse_id`) ON DELETE RESTRICT ON UPDATE CASCADE)" + ] + } + ], "source": [ "Session.insert1(bad_data)" ] @@ -762,14 +1653,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Querying data" + "### Querying data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Often times, you don't want all data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your own parser, it is far more efficient to narrow your data to the subset before fetching.\n", + "Often times, you don't want all the data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your own parser, it is far more efficient to narrow your data to the subset before fetching.\n", "\n", "For this, DataJoint offers very powerful yet intuitive **querying** syntax that let's you select exactly the data you want before you fetch it.\n", "\n", From 0be639c6995986d64cbf7c2deedf790d781c48d1 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 24 Jul 2023 23:31:36 +0000 Subject: [PATCH 15/70] Update of Practical Example -final part-Tutorial01 --- notebooks/01-DataJoint Basics.ipynb | 2697 +++++++++++++++++++++++++-- 1 file changed, 2570 insertions(+), 127 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index a1b46cb..98d14a0 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -11,7 +11,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! If you are reading this, then you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint. " + "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint. " ] }, { @@ -25,14 +25,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular [Python](https://www.python.org/) packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). Just like any other packages, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. Convention is to alias the package to `dj`." + "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular [Python](https://www.python.org/) packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). Like any other package, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. The convention is to alias the package to `dj`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "*NOTE: Run code cells by clicking on the left-top corner bottom of the cell, or using Ctrl+Enter shortcut.*" + "*NOTE: Run code cells by clicking on the left-top corner bottom of the cell or using Ctrl+Enter shortcut.*" ] }, { @@ -63,13 +63,13 @@ "metadata": {}, "source": [ "\n", - "A data pipeline is a collection of data processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex sequences of data acquisition, processing and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", + "A data pipeline is a collection of processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex data acquisition sequences, processing, and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", "\n", - "In other words, a data pipeline can be seen as a network where each node or entity set represents a **table**. The information involved in a research project, or in a whole research lab, can be organized and stored in these entity sets or tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", + "In other words, a data pipeline can be seen as a network where each node or entity set represents a **table**. The information involved in a research project or a whole research lab can be organized and stored in these entity sets or tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", "\n", - "The data pipeline is formed by making these tables interdependent (as the nodes connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms a full cohesive data pipeline. \n", + "The data pipeline is formed by making these tables interdependent (as the nodes are connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms an entire cohesive data pipeline. \n", "\n", - "A [DataJoint pipeline](https://datajoint.com/docs/core/datajoint-python/0.14/concepts/terminology/) contains database table definitions, dependencies, and associated coomputations, together with the transformations underlying a DataJoint workflow. \n", + "A [DataJoint pipeline](https://datajoint.com/docs/core/datajoint-python/0.14/concepts/terminology/) contains database table definitions, dependencies, and associated computations, together with the transformations underlying a DataJoint workflow. \n", "\n", "This is an example of a DataJoint pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", "\n", @@ -1662,7 +1662,7 @@ "source": [ "Often times, you don't want all the data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your own parser, it is far more efficient to narrow your data to the subset before fetching.\n", "\n", - "For this, DataJoint offers very powerful yet intuitive **querying** syntax that let's you select exactly the data you want before you fetch it.\n", + "For this, DataJoint offers a very powerful yet intuitive **querying syntax** that will let you select exactly the data you want before you fetch it.\n", "\n", "It is also critical to note that the result of any DataJoint query represents a valid entity." ] @@ -1672,31 +1672,31 @@ "metadata": {}, "source": [ "We will introduce four major types of queries used in DataJoint:\n", - "* restriction (`&`) and negative restriction (`-`): filter data\n", - "* join (`*`): bring fields from different tables together\n", - "* projection (`.proj()`): focus on a subset of attributes\n", - "* aggregation (`.aggr()`): simple computation of one table against another table" + "* 1. Restriction (`&`) and negative restriction (`-`): filter data\n", + "* 2. Join (`*`): bring fields from different tables together\n", + "* 3. Projection (`.proj()`): focus on a subset of attributes\n", + "* 4. Aggregation (`.aggr()`): simple computation of one table against another table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Restrictions (`&`) - filter data with certain conditions" + "### Restrictions (`&`) - filter data with certain conditions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The **restriction** operation, `&`, let's you specify the criteria to narrow down the table on the left." + "The **restriction** operation, `&`, allows you to specify the criteria to narrow down the table on the left." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Exact match" + "##### Exact match" ] }, { @@ -1708,9 +1708,97 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 27, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + " (Total: 1)" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'mouse_id = 0'" ] @@ -1724,9 +1812,106 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 28, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
12016-11-19M
192018-07-21M
342018-09-22M
\n", + " \n", + "

Total: 4

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + "1 2016-11-19 M \n", + "19 2018-07-21 M \n", + "34 2018-09-22 M \n", + " (Total: 4)" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'sex = \"M\"'" ] @@ -1740,9 +1925,115 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 29, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", + " \n", + "

Total: 7

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "22 2019-12-15 F \n", + "100 2017-05-12 F \n", + " (Total: 7)" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'sex = \"F\"'" ] @@ -1756,9 +2047,97 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 30, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
52016-12-25F
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "5 2016-12-25 F \n", + " (Total: 1)" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & dict(mouse_id=5)" ] @@ -1786,9 +2165,118 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 31, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
112017-01-03F
122017-03-21F
182017-05-01F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", + " \n", + "

Total: 8

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "19 2018-07-21 M \n", + "22 2019-12-15 F \n", + "34 2018-09-22 M \n", + "100 2017-05-12 F \n", + " (Total: 8)" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'dob > \"2017-01-01\"'" ] @@ -1802,9 +2290,106 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 32, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
122017-03-21F
182017-05-01F
1002017-05-12F
\n", + " \n", + "

Total: 4

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "100 2017-05-12 F \n", + " (Total: 4)" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'dob between \"2017-03-01\" and \"2017-08-23\"'" ] @@ -1818,9 +2403,118 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 33, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", + " \n", + "

Total: 8

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +---------+\n", + "2 2016-11-20 unknown \n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "22 2019-12-15 F \n", + "100 2017-05-12 F \n", + " (Total: 8)" + ] + }, + "execution_count": 33, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'sex != \"M\"'" ] @@ -1841,18 +2535,218 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 34, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", + " \n", + "

Total: 5

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "22 2019-12-15 F \n", + "100 2017-05-12 F \n", + " (Total: 5)" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'sex != \"M\"' & 'dob > \"2017-01-01\"'" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 35, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", + " \n", + "

Total: 5

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "22 2019-12-15 F \n", + "100 2017-05-12 F \n", + " (Total: 5)" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & 'sex != \"M\" and dob > \"2017-01-01\"'" ] @@ -1861,14 +2755,120 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Result of one query can be used in another query! Let's first find **all female mice** and store the result." + "The result of one query can be used in another query! Let's first find **all the female mice** and store the result." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 36, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", + " \n", + "

Total: 7

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "22 2019-12-15 F \n", + "100 2017-05-12 F \n", + " (Total: 7)" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "female_mice = Mouse & 'sex = \"F\"'\n", "female_mice" @@ -1878,12 +2878,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "and among these mice, find ones with **mouse_id > 10**" + "and among these mice, find the ones with **mouse_id > 10**" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 37, "metadata": {}, "outputs": [], "source": [ @@ -1894,28 +2894,119 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In computer science/math lingo, DataJoint operations are said to **satisfy closure property**. Practically speaking, this means that the result of a query can immediately be used in another query, allowing you to build more complex queries from simpler ones. " + "In Computer Science and Math lingo, DataJoint operations are said to **satisfy closure property**. Practically speaking, this means that the result of a query can immediately be used in another query, allowing you to build more complex queries from simpler ones. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Restriction one table with another" + "### Restriction of one table with another" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All mice that have a session" + "All the mice that have a session:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 38, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
182017-05-01F
\n", + " \n", + "

Total: 2

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + "18 2017-05-01 F \n", + " (Total: 2)" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & Session " ] @@ -1931,7 +3022,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All the above queries could be combined " + "All the above queries could be combined:" ] }, { @@ -1943,9 +3034,97 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 39, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + " (Total: 1)" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & Session & 'sex = \"M\"'" ] @@ -1954,14 +3133,102 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Give me all mice that have had an experimental session done on or before 2017-05-19" + "Selec the mice that participated in an experimental session done on or before 2017-05-19" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 40, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + " (Total: 1)" + ] + }, + "execution_count": 40, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse & (Session & 'session_date <= \"2017-05-19\"')" ] @@ -1982,9 +3249,124 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 41, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", + " \n", + "

Total: 10

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +---------+\n", + "1 2016-11-19 M \n", + "2 2016-11-20 unknown \n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "19 2018-07-21 M \n", + "22 2019-12-15 F \n", + "34 2018-09-22 M \n", + "100 2017-05-12 F \n", + " (Total: 10)" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse - Session" ] @@ -1998,7 +3380,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 42, "metadata": {}, "outputs": [], "source": [ @@ -2009,7 +3391,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Joining (*) - bring fields from different tables together" + "### Joining (*) - bring fields from different tables together" ] }, { @@ -2023,19 +3405,137 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Behavior of join:\n", + "This is how the Join operator works:\n", "\n", - "1. match the common field(s) of the primary keys in the two tables\n", - "2. do a combination of the non-matched part of the primary key\n", - "3. listing out the secondary attributes for each combination\n", - "4. if two tables have secondary attributes that share a same name, it will throw an error. To join, we need to rename that attribute for at least one of the tables." + "1. Match the common field(s) of the primary keys in the two tables\n", + "2. Do a combination of the non-matched part of the primary key\n", + "3. List out the secondary attributes for each combination\n", + "4. If two tables have secondary attributes that share a same name, it will throw an error. To join, we need to rename that attribute for at least one of the tables." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 43, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-152017-03-01M0Edgar Y. Walker
02018-01-152017-03-01M100Jacob Reimer
182018-01-152017-05-01F101Jacob Reimer
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date dob sex experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+\n", + "0 2017-05-15 2017-03-01 M 0 Edgar Y. Walke \n", + "0 2018-01-15 2017-03-01 M 100 Jacob Reimer \n", + "18 2018-01-15 2017-05-01 F 101 Jacob Reimer \n", + " (Total: 3)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# looking at the combination of mouse and session\n", "Mouse * Session" @@ -2057,9 +3557,113 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 44, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02018-01-152017-03-01M100Jacob Reimer
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date dob sex experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+\n", + "0 2018-01-15 2017-03-01 M 100 Jacob Reimer \n", + " (Total: 1)" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Find 'experimenter = \"Jacob Reimer\"' and 'sex = \"M\"'\n", "Mouse * Session & 'experimenter = \"Jacob Reimer\"' & 'sex = \"M\"'" @@ -2067,9 +3671,120 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 45, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02018-01-152017-03-01M100Jacob Reimer
182018-01-152017-05-01F101Jacob Reimer
\n", + " \n", + "

Total: 2

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date dob sex experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+\n", + "0 2018-01-15 2017-03-01 M 100 Jacob Reimer \n", + "18 2018-01-15 2017-05-01 F 101 Jacob Reimer \n", + " (Total: 2)" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse * Session & 'session_date > \"2017-05-19\"'" ] @@ -2078,7 +3793,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Projection .proj(): focus on attributes of interest\n", + "### Projection .proj(): focus on attributes of interest\n", "Beside restriction (`&`) and join (`*`) operations, DataJoint offers another type of operation: projection (`.proj()`). Projection is used to select attributes (columns) from a table, to rename them, or to create new calculated attributes. " ] }, @@ -2091,9 +3806,115 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 46, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

sex

\n", + " sex\n", + "
0M
1M
2unknown
5F
10F
11F
12F
18F
19M
22F
34M
100F
\n", + " \n", + "

Total: 12

\n", + " " + ], + "text/plain": [ + "*mouse_id sex \n", + "+----------+ +---------+\n", + "0 M \n", + "1 M \n", + "2 unknown \n", + "5 F \n", + "10 F \n", + "11 F \n", + "12 F \n", + "18 F \n", + "19 M \n", + "22 F \n", + "34 M \n", + "100 F \n", + " (Total: 12)" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse.proj('sex')" ] @@ -2116,9 +3937,115 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 47, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

date_of_birth

\n", + " date of birth\n", + "
02017-03-01
12016-11-19
22016-11-20
52016-12-25
102017-01-01
112017-01-03
122017-03-21
182017-05-01
192018-07-21
222019-12-15
342018-09-22
1002017-05-12
\n", + " \n", + "

Total: 12

\n", + " " + ], + "text/plain": [ + "*mouse_id date_of_birth \n", + "+----------+ +------------+\n", + "0 2017-03-01 \n", + "1 2016-11-19 \n", + "2 2016-11-20 \n", + "5 2016-12-25 \n", + "10 2017-01-01 \n", + "11 2017-01-03 \n", + "12 2017-03-21 \n", + "18 2017-05-01 \n", + "19 2018-07-21 \n", + "22 2019-12-15 \n", + "34 2018-09-22 \n", + "100 2017-05-12 \n", + " (Total: 12)" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse.proj(date_of_birth='dob')" ] @@ -2133,9 +4060,103 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 48, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

age

\n", + " calculated attribute\n", + "
02017-05-1575
02018-01-15320
182018-01-15259
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date age \n", + "+----------+ +------------+ +-----+\n", + "0 2017-05-15 75 \n", + "0 2018-01-15 320 \n", + "18 2018-01-15 259 \n", + " (Total: 3)" + ] + }, + "execution_count": 48, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "(Mouse * Session).proj(age='datediff(session_date, dob)')" ] @@ -2150,9 +4171,133 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 49, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
\n", + "

age

\n", + " calculated attribute\n", + "
02017-05-152017-03-01M0Edgar Y. Walker75
02018-01-152017-03-01M100Jacob Reimer320
182018-01-152017-05-01F101Jacob Reimer259
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date dob sex experiment_set experimenter data_path age \n", + "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+ +-----+\n", + "0 2017-05-15 2017-03-01 M 0 Edgar Y. Walke 75 \n", + "0 2018-01-15 2017-03-01 M 100 Jacob Reimer 320 \n", + "18 2018-01-15 2017-05-01 F 101 Jacob Reimer 259 \n", + " (Total: 3)" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "(Mouse * Session).proj(..., age='datediff(session_date, dob)')" ] @@ -2161,7 +4306,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Fetch data" + "### Fetch data" ] }, { @@ -2175,21 +4320,118 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Fetch one or multiple entries: `fetch()`" + "### Fetch one or multiple entries: `fetch()`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All male mouse" + "All the male mice:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 51, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
12016-11-19M
192018-07-21M
342018-09-22M
\n", + " \n", + "

Total: 4

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +-----+\n", + "0 2017-03-01 M \n", + "1 2016-11-19 M \n", + "19 2018-07-21 M \n", + "34 2018-09-22 M \n", + " (Total: 4)" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "male_mouse = Mouse & 'sex = \"M\"'\n", "male_mouse" @@ -2204,9 +4446,24 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 52, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "array([( 0, datetime.date(2017, 3, 1), 'M'),\n", + " ( 1, datetime.date(2016, 11, 19), 'M'),\n", + " (19, datetime.date(2018, 7, 21), 'M'),\n", + " (34, datetime.date(2018, 9, 22), 'M')],\n", + " dtype=[('mouse_id', '\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
dobsex
mouse_id
02017-03-01M
12016-11-19M
192018-07-21M
342018-09-22M
\n", + "" + ], + "text/plain": [ + " dob sex\n", + "mouse_id \n", + "0 2017-03-01 M\n", + "1 2016-11-19 M\n", + "19 2018-07-21 M\n", + "34 2018-09-22 M" + ] + }, + "execution_count": 55, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "(Mouse & 'sex = \"M\"').fetch(format='frame')" ] @@ -2268,9 +4623,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 56, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "[{'mouse_id': 0}, {'mouse_id': 1}, {'mouse_id': 19}, {'mouse_id': 34}]" + ] + }, + "execution_count": 56, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "(Mouse & 'sex = \"M\"').fetch('KEY')" ] @@ -2284,7 +4650,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 57, "metadata": {}, "outputs": [], "source": [ @@ -2293,18 +4659,47 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 58, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "array(['M', 'M', 'unknown', 'F', 'F', 'F', 'F', 'F', 'M', 'F', 'M', 'F'],\n", + " dtype=object)" + ] + }, + "execution_count": 58, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "sex" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 59, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "array([datetime.date(2017, 3, 1), datetime.date(2016, 11, 19),\n", + " datetime.date(2016, 11, 20), datetime.date(2016, 12, 25),\n", + " datetime.date(2017, 1, 1), datetime.date(2017, 1, 3),\n", + " datetime.date(2017, 3, 21), datetime.date(2017, 5, 1),\n", + " datetime.date(2018, 7, 21), datetime.date(2019, 12, 15),\n", + " datetime.date(2018, 9, 22), datetime.date(2017, 5, 12)],\n", + " dtype=object)" + ] + }, + "execution_count": 59, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "dob" ] @@ -2318,9 +4713,31 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 60, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "[{'dob': datetime.date(2017, 3, 1), 'sex': 'M'},\n", + " {'dob': datetime.date(2016, 11, 19), 'sex': 'M'},\n", + " {'dob': datetime.date(2016, 11, 20), 'sex': 'unknown'},\n", + " {'dob': datetime.date(2016, 12, 25), 'sex': 'F'},\n", + " {'dob': datetime.date(2017, 1, 1), 'sex': 'F'},\n", + " {'dob': datetime.date(2017, 1, 3), 'sex': 'F'},\n", + " {'dob': datetime.date(2017, 3, 21), 'sex': 'F'},\n", + " {'dob': datetime.date(2017, 5, 1), 'sex': 'F'},\n", + " {'dob': datetime.date(2018, 7, 21), 'sex': 'M'},\n", + " {'dob': datetime.date(2019, 12, 15), 'sex': 'F'},\n", + " {'dob': datetime.date(2018, 9, 22), 'sex': 'M'},\n", + " {'dob': datetime.date(2017, 5, 12), 'sex': 'F'}]" + ] + }, + "execution_count": 60, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "info = Mouse.fetch('sex', 'dob', as_dict=True)\n", "info" @@ -2330,7 +4747,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Fetch data from only one entry: `fetch1()`" + "### Fetch data from only one entry: `fetch1()`" ] }, { @@ -2342,9 +4759,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 61, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "{'mouse_id': 0, 'dob': datetime.date(2017, 3, 1), 'sex': 'M'}" + ] + }, + "execution_count": 61, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "mouse_0 = (Mouse & {'mouse_id': 0}).fetch1() # \"fetch1()\" because we know there's only one\n", "mouse_0" @@ -2359,9 +4787,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 62, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "{'mouse_id': 0}" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "(Mouse & {'mouse_id': 0}).fetch1('KEY')" ] @@ -2375,7 +4814,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 63, "metadata": {}, "outputs": [], "source": [ @@ -2384,18 +4823,40 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 64, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "'M'" + ] + }, + "execution_count": 64, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "sex" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 65, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "datetime.date(2017, 3, 1)" + ] + }, + "execution_count": 65, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "dob" ] @@ -2404,7 +4865,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Deletion (`.delete()`) - deleting entries and their dependencies" + "### Deletion (`.delete()`) - deleting entries and their dependencies" ] }, { @@ -2457,30 +4918,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! You have successfully created your first DatJoint pipeline, using dependencies to establish the link between the tables. You have also learned to query and fetch the data.\n", + "Congratulations! You have successfully created your first DataJoint pipeline, using dependencies to establish the link between the tables. You have also learned to query and fetch the data.\n", "\n", "In the next session, we are going to extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", "\n", - "We will use both ephys and calcium imaging as example pipelines:\n", - "+ [02-electrophysiology](../02-Electrophysiology/02-Imported%20Tables%20-%20Interactive.ipynb)\n", - "+ [02-calcium imaging](../01-Calcium_Imaging/02-Imported%20Tables%20-%20Interactive.ipynb)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Clean up" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# schema.drop()" + "Please, continue to the next notebook:\n", + "+ [02-Calcium Imaging](../notebooks/02-Calcium%20Imaging%20Imported%20Tables.ipynb)\n" ] } ], From 6f34d6432c06b8304b054c9816d3a4e0d6d88290 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 15:38:49 +0000 Subject: [PATCH 16/70] Major changes in text and organization-Tutorial01 --- notebooks/01-DataJoint Basics.ipynb | 422 ++++++++++++++++------------ 1 file changed, 238 insertions(+), 184 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 98d14a0..8f204a3 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -11,7 +11,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint. " + "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint. \n", + "\n", + "- Essential concepts and setup\n", + " - Data pipelines\n", + " - Concept\n", + " - Practical examples\n", + " - Schemas and tables\n", + " - Concept\n", + " - Practical examples\n", + " - Basic relational operators\n", + " - Create tables with dependencies\n", + " - Querying data" ] }, { @@ -25,7 +36,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular [Python](https://www.python.org/) packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). Like any other package, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. The convention is to alias the package to `dj`." + "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular [Python](https://www.python.org/) packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). \n", + "\n", + "Like any other package, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. The convention is to alias the package to `dj`." ] }, { @@ -37,7 +50,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 68, "metadata": {}, "outputs": [], "source": [ @@ -63,21 +76,21 @@ "metadata": {}, "source": [ "\n", - "A data pipeline is a collection of processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex data acquisition sequences, processing, and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", + ">* A data pipeline is a collection of processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex data acquisition sequences, processing, and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", "\n", - "In other words, a data pipeline can be seen as a network where each node or entity set represents a **table**. The information involved in a research project or a whole research lab can be organized and stored in these entity sets or tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", + ">* In other words, a data pipeline can be seen as a network where each node or entity set represents a **table**. The information involved in a research project or a whole research lab can be organized and stored in these entity sets or tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", "\n", - "The data pipeline is formed by making these tables interdependent (as the nodes are connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms an entire cohesive data pipeline. \n", + ">* The data pipeline is formed by making these tables interdependent (as the nodes are connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms an entire cohesive data pipeline. \n", "\n", "A [DataJoint pipeline](https://datajoint.com/docs/core/datajoint-python/0.14/concepts/terminology/) contains database table definitions, dependencies, and associated computations, together with the transformations underlying a DataJoint workflow. \n", "\n", - "This is an example of a DataJoint pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", + "The following figure is an example of a DataJoint pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", "\n", "![pipeline](https://raw.githubusercontent.com/datajoint/element-optogenetics/main/images/pipeline.svg)\n", "\n", "A **well-designed data pipeline**: \n", - "- 1. Collects, organizes and stores **every relevant piece of information during the scientific research**\n", - "- 2. Integrates, processes and connects these pieces of information through **several steps**\n", + "- 1. Collects, organizes, and stores **every relevant piece of information during the scientific research**\n", + "- 2. Integrates, processes, and connects these pieces of information through **several steps**\n", "- 3. Analyses and transforms the input data into **valuable insights for the research**, bringing together logical clarity to the experiments\n" ] }, @@ -85,18 +98,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "##### Practical example" + "##### Practical examples" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The practical example that is used during these tutorials will be to design and compute a data pipeline for a scientific project of two experiments on rodents: \n", + "The practical examples that will be used in the next tutorials will allow you to design and compute a data pipeline for a scientific project of two experiments on rodents: \n", "- Single-electrode recording\n", "- Calcium imaging recording\n", "\n", - "Here is a brief description:" + "Let's start with a brief description of this project's context:" ] }, { @@ -104,15 +117,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> * Your lab houses many mice, and each mouse is identified by a unique ID. You also want to keep track of other information about each mouse such as their date of birth, and gender.\n", - "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day. However, on any given day, a mouse undergoes at most one recording session.\n", - "> * For each experimental session, you want to record what mouse you worked with and when you performed the experiment. You also want to keep track of other helpful information such as the experimental setup you worked on. \n", + "> * Your lab houses many mice, and a unique ID identifies each mouse. You also want to keep track of other information about each mouse, such as their date of birth and gender.\n", + "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse daily. However, a mouse undergoes at most one recording session on any given day.\n", + "> * For each experimental session, you want to record what mouse you worked with and when you experimented. You also want to keep track of other helpful information, such as the experimental setup you worked on. \n", "\n", "> * In a session of electrophysiology:\n", - ">> * You record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you recorded.\n", - ">> * Neuron's activities are recorded as raw traces. Neuron's spikes needs to be detected for further analysis to be performed.\n", + ">> * You record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you record.\n", + ">> * Neuron's activities are recorded as raw traces. Neuron's spikes need to be detected for further analysis to be perform.\n", + "\n", "> * In a session of calcium imaging:\n", - ">> * You scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", + ">> * You scan a brain region containing several neurons. You use recording equipment that produces separate data files for each scan you performed.\n", ">> * You need to segment the frames and get the regions of interest (ROIs), and save a mask for each ROI\n", ">> * In addition, you need to extract the trace from each segmented ROI" ] @@ -121,14 +135,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The design of a data pipeline starts by identifying the **entities** or **tables** in your research project. Common entities include experimental subjects (e.g. mouse), recording sessions, and two-photon scans." + "The design of a data pipeline starts by identifying the **entities** or **tables** in your research project. Common entities include experimental subjects (e.g., mouse), recording sessions, and two-photon scans." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's revisit the project description, this time paying special attention to **what** (e.g. nouns or entities) about your experiment. Here some particular entities are highlighted." + "Let's revisit the project description, this time paying special attention to **what** (e.g., nouns or entities) about your experiment. Here some particular entities are highlighted." ] }, { @@ -136,15 +150,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> * Your lab houses many **mice**, and each mouse is identified by a unique ID. You also want to keep track of other information about each mouse such as their date of birth, and gender.\n", - "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day. However, on any given day, a mouse undergoes at most one recording session.\n", - "> * For each **experimental session**, you want to record what mouse you worked with and when you performed the experiment. You also want to keep track of other helpful information such as the experimental setup you worked on. \n", + "> * Your lab houses many **mice**, and a unique ID identifies each mouse. You also want to keep track of other information about each mouse, such as their date of birth and gender.\n", + "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse daily. However, a mouse undergoes at most one recording session on any given day.\n", + "> * For each **experimental session**, you want to record what mouse you worked with and when you experimented. You also want to keep track of other helpful information, such as the experimental setup you worked on. \n", "\n", "> * In a session of electrophysiology:\n", - ">> * You record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you recorded.\n", - ">> * Neuron's activities are recorded as raw traces. **Neuron's spikes** needs to be detected for further analysis to be performed.\n", + ">> * You record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you record.\n", + ">> * Neuron's activities are recorded as raw traces. **Neuron's spikes** need to be detected for further analysis to be perform.\n", + "\n", "> * In a session of calcium imaging:\n", - ">> * You **scan** a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", + ">> * You **scan** a brain region containing several neurons. You use recording equipment that produces separate data files for each scan you performed.\n", ">> * You need to segment the frames and get the **regions of interest (ROIs)**, and save a mask for each ROI\n", ">> * In addition, you need to extract the **trace** from each segmented ROI" ] @@ -153,7 +168,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Just by going though the description, we can start to identify **entities** that need to be stored and represented in our data pipeline:\n", + "Just by going through the description, we can start to identify **entities** that needs to be stored and represented in our data pipeline:\n", "\n", ">* Mouse\n", ">* Experimental session\n", @@ -174,14 +189,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In the next section you will learn to design the tables and manipulate the data for `Mouse` and `Experimental sessions`. The rest of the pipeline (`Ephys` and `Calcium Imaging` will be addressed in the subsequent tutorials." + "In the next section, you will learn to design the tables and manipulate the data for `Mouse` and `Experimental sessions`. The rest of the pipeline (`Ephys` and `Calcium Imaging` will be addressed in the subsequent tutorials." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Schemas, tables and basic relational operators" + "### Schemas and tables" ] }, { @@ -195,11 +210,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In a data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each row of the table is a single example (instance) of the entity's category. \n", + "In a data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each table row is a single example (instance) of the entity's category. \n", "\n", - "For example, if we have a `Mouse` table, then each row in the mouse table represents a single mouse. \n", + "For example, if we have a `Mouse` table, each row in the mouse table represents a single mouse. \n", "\n", - "It is important to think what information will **uniquely identify** each entry. \n", + "It is essential to think about what information will **uniquely identify** each entry. \n", "\n", "In this case, the information that uniquely identifies the `Mouse` table is their **mouse IDs** - a unique ID number assigned to each animal in the lab. This attribute is named the **primary key** of the table.\n", "\n", @@ -217,7 +232,7 @@ "\n", "The mouse ID is then a column in the table or an **attribute** that can be used to **uniquely identify** each mouse. \n", "\n", - "Such attribute is called the **primary key** of the table: the subset of table attributes that uniquely identify each entity in the table. The **secondary attribute** refers to any field in a table not in the primary key.\n", + "Such an attribute is called the **primary key** of the table: the subset of table attributes uniquely identifying each entity in the table. The **secondary attribute** refers to any field in a table, not in the primary key.\n", "\n", "| Mouse_ID (*Primary key attribute*) \n", "|:--------:| \n", @@ -229,14 +244,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Once we have successfully identified the primary key of the table, we can now think about what other columns, or **non-primary key attributes** - additional information **about each entry in the table that need to be stored as well**." + "Once we have successfully identified the table's primary key, we can now think about what other columns, or **non-primary key attributes** - additional information **about each entry in the table that need to be stored as well**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "For the case of `Mouse`, what other information about the mouse you might want to store? \n", + "For the case of `Mouse`, what other information about the mouse might you want to store? \n", "\n", "Based on the project description, we would probably want to store information such as the mouse's **date of birth** (DOB) and **sex**." ] @@ -255,7 +270,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now we have an idea on how to represent information about mouse, let's create the table using **DataJoint**!" + "Now that we have an idea of how to represent information about the mouse, let's create the table using **DataJoint**!" ] }, { @@ -276,7 +291,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Every table lives inside a schema - a logical collection of one or more tables in your pipeline. Your final pipeline may consists of many tables spread across one or more schemas. Let's go ahead and create the first schema to house our `Mouse` table using DataJoint." + "Every table lives inside a schema - a logical collection of one or more tables in your pipeline. Your final pipeline will consist of many tables spread across one or more schemas. Let's go ahead and create the first schema to house our `Mouse` table using DataJoint." ] }, { @@ -284,23 +299,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We create the schema using `dj.schema()` function, passing in the name of the schema. For this tutorial, we create a schema called `tutorial`." + "We create the schema using `dj.schema()` function, passing in the schema's name. For this tutorial, we create a schema called `tutorial`." ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 69, "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "[2023-07-24 22:36:18,016][INFO]: Connecting root@fakeservices.datajoint.io:3306\n", - "[2023-07-24 22:36:18,073][INFO]: Connected root@fakeservices.datajoint.io:3306\n" - ] - } - ], + "outputs": [], "source": [ "schema = dj.schema('tutorial')" ] @@ -309,7 +315,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have a schema to place our table into, let's go ahead and define our first table. " + "Now that we have a schema to place our table into let's go ahead and define our first table. " ] }, { @@ -323,12 +329,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In DataJoint, you define each table as a `class`, and provide the table definition (e.g. attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)." + "In DataJoint, you define each table as a `class`, and provide the table definition (e.g., attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)." ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 70, "metadata": {}, "outputs": [], "source": [ @@ -347,12 +353,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's take a look at our brand new table" + "Let's take a look at our brand-new table" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 71, "metadata": {}, "outputs": [ { @@ -423,20 +429,55 @@ "

sex

\n", " sex\n", " \n", - " \n", + " 0\n", + "2017-03-01\n", + "M1\n", + "2016-11-19\n", + "M2\n", + "2016-11-20\n", + "unknown5\n", + "2016-12-25\n", + "F10\n", + "2017-01-01\n", + "F11\n", + "2017-01-03\n", + "F12\n", + "2017-03-21\n", + "F18\n", + "2017-05-01\n", + "F19\n", + "2018-07-21\n", + "M22\n", + "2019-12-15\n", + "F34\n", + "2018-09-22\n", + "M100\n", + "2017-05-12\n", + "F \n", " \n", " \n", - "

Total: 0

\n", + "

Total: 12

\n", " " ], "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +-----+ +-----+\n", - "\n", - " (Total: 0)" + "*mouse_id dob sex \n", + "+----------+ +------------+ +---------+\n", + "0 2017-03-01 M \n", + "1 2016-11-19 M \n", + "2 2016-11-20 unknown \n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "19 2018-07-21 M \n", + "22 2019-12-15 F \n", + "34 2018-09-22 M \n", + "100 2017-05-12 F \n", + " (Total: 12)" ] }, - "execution_count": 4, + "execution_count": 71, "metadata": {}, "output_type": "execute_result" } @@ -445,6 +486,13 @@ "Mouse()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Basic relational operators" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -456,7 +504,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The table was successfully defined, but without any content, the table is not too interesting. Let's go ahead and **insert some mouse information** into the table, one at a time using the `insert1` method." + "The table was successfully defined, but with content, the table will be more interesting. Let's go ahead and **insert some mouse information** into the table, one at a time, using the `insert1` method." ] }, { @@ -471,16 +519,30 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 72, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "DuplicateError", + "evalue": "(\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[72], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1((\u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39m2017-03-01\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m))\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", + "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" + ] + } + ], "source": [ "Mouse.insert1((0, '2017-03-01', 'M'))" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -584,7 +646,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -597,7 +659,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -606,7 +668,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -713,7 +775,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -726,7 +788,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -742,7 +804,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -757,7 +819,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -881,12 +943,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "DataJoint checks for data integrity, and ensures that you don't insert a duplicate by mistake. Let's try inserting another mouse with `mouse_id: 0` and see what happens!" + "DataJoint checks for data integrity and ensures you don't insert a duplicate by mistake. Let's try inserting another mouse with `mouse_id: 0` and see what happens!" ] }, { "cell_type": "code", - "execution_count": 14, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -915,12 +977,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Go ahead and insert a few more mice into your table before moving on." + "Let's insert a few more mice into your table before moving on." ] }, { "cell_type": "code", - "execution_count": 15, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -938,7 +1000,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1068,7 +1130,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1086,7 +1148,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! We have successfully created your first table! We are now ready to tackle and include other **entities** in the project into our data pipeline. \n", + "Congratulations! We have successfully created your first table! We are ready to tackle and include other **entities** into the project's data pipeline. \n", "\n", "Let's now have a look at representing an `experimental session`." ] @@ -1095,29 +1157,27 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As with `mouse`, we should think about **what information (i.e. attributes) is needed to uniquely identify an `experimental session`**. Here is the relevant section of the project description:\n", + "As with `mouse`, we should consider **what information (i.e., attributes) is needed to identify an `experimental session`** uniquely. Here is the relevant section of the project description:\n", "\n", - "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with **more than one mouse in a day**. However, on any given day, **a mouse undergoes at most one recording session**.\n", - "> * For each **experimental session**, you want to record **what mouse you worked with** and **when you performed the experiment**. You also want to keep track of other helpful information such as the **experimental setup** you worked on. " + "> * As a hard-working neuroscientist, you perform experiments daily, sometimes working with **more than one mouse in a day**. However, on any given day, **a mouse undergoes at most one recording session**.\n", + "> * For each **experimental session**, you want to record **what mouse you worked with** and **when you performed the experiment**. You also want to keep track of other helpful information, such as the **experimental setup** you worked on. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Based on the above, it seems that you need to know:\n", + "Based on the above, it seems that you need to know these two data to uniquely identify a single experimental session:\n", "\n", "* the date of the session\n", - "* the mouse you recorded from in that session\n", - "\n", - "to uniquely identify a single experimental session." + "* the mouse you recorded from in that session" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note that, to uniquely identify an experimental session (or simply a **session**), we need to know the mouse that the session was about. In other words, a session cannot existing without a corresponding mouse! \n", + "Note that to uniquely identify an experimental session (or simply a **session**), we need to know the mouse used in that the session. In other words, a session cannot be existing without a corresponding mouse! \n", "\n", "With **mouse** already represented as a table in our pipeline, we say that the session **depends on** the mouse! We could graphically represent this in an **entity relationship diagram (ERD)** by drawing the line between two tables, with the one below (**session**) depending on the one above (**mouse**)." ] @@ -1126,14 +1186,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Thus, we will need both **mouse** and a new attribute **session_date** to uniquely identify a single `session`. \n", + "Thus, we will need both the **mouse** and the new attribute **session_date** to identify a single `session` uniquely. \n", "\n", - "Remember that a **mouse** is already uniquely identified by its primary key - **mouse_id**. In DataJoint, you can declare that **session** depends on the mouse, and DataJoint will automatically include the mouse's primary key (`mouse_id`) as part of the session's primary key, along side any additional attribute(s) you specificy." + "Remember that a **mouse** is uniquely identified by its primary key - **mouse_id**. In DataJoint, you can declare that **session** depends on the mouse, and DataJoint will automatically include the mouse's primary key (`mouse_id`) as part of the session's primary key, alongside any additional attribute(s) you specify." ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1154,12 +1214,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can actually generate something similar to an entity relationship diagram (ERD) on the fly by calling `dj.Diagram` with the schema object. Many of the symbols and features are the same as the ERD standard." + "You can generate something similar to an entity relationship diagram (ERD) on the fly by calling `dj.Diagram` with the schema object. Many of the symbols and features are the same as the ERD standard." ] }, { "cell_type": "code", - "execution_count": 19, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1212,12 +1272,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's try inserting a few sessions manually." + "Let's insert a few sessions manually." ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1233,7 +1293,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1345,7 +1405,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1466,12 +1526,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "And another session done on the same date but on a different mouse" + "And another session is done on the same date but on a different mouse:" ] }, { "cell_type": "code", - "execution_count": 23, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1488,7 +1548,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1610,7 +1670,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1624,7 +1684,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": null, "metadata": { "scrolled": true }, @@ -1660,9 +1720,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Often times, you don't want all the data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your own parser, it is far more efficient to narrow your data to the subset before fetching.\n", + "Oftentimes, you want only some of the data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your parser, narrowing your data to the subset before fetching is far more efficient.\n", "\n", - "For this, DataJoint offers a very powerful yet intuitive **querying syntax** that will let you select exactly the data you want before you fetch it.\n", + "For this, DataJoint offers a very powerful yet intuitive **querying syntax** that lets you select the data you want before you fetch it.\n", "\n", "It is also critical to note that the result of any DataJoint query represents a valid entity." ] @@ -1671,18 +1731,19 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We will introduce four major types of queries used in DataJoint:\n", - "* 1. Restriction (`&`) and negative restriction (`-`): filter data\n", - "* 2. Join (`*`): bring fields from different tables together\n", + "We will introduce significant types of queries used in DataJoint:\n", + "* 1. Restriction (`&`) and negative restriction (`-`): filter data with certain conditions\n", + "* 2. Joining (`*`): bring fields from different tables together\n", "* 3. Projection (`.proj()`): focus on a subset of attributes\n", - "* 4. Aggregation (`.aggr()`): simple computation of one table against another table" + "* 4. Fetching (`.fetch()`): pull the data from the database\n", + "* 5. Deletion (`.delete()`): delete entries and their dependencies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Restrictions (`&`) - filter data with certain conditions" + "### 1. Restrictions (`&`) - filter data with certain conditions" ] }, { @@ -1708,7 +1769,7 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1812,7 +1873,7 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1925,7 +1986,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2042,12 +2103,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can also use as a dictionary as a restrictor, with one field or multiple fields" + "We can also use a dictionary as a restrictor, with one field or multiple fields:" ] }, { "cell_type": "code", - "execution_count": 30, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2165,7 +2226,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2290,7 +2351,7 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2403,7 +2464,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2523,7 +2584,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can easily combine multiple restrictions to narrow down the entities based on multiple attributes." + "You can easily combine multiple restrictions to narrow the entities based on various attributes." ] }, { @@ -2535,7 +2596,7 @@ }, { "cell_type": "code", - "execution_count": 34, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2644,7 +2705,7 @@ }, { "cell_type": "code", - "execution_count": 35, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2760,7 +2821,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -2883,7 +2944,7 @@ }, { "cell_type": "code", - "execution_count": 37, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -2913,7 +2974,7 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3022,19 +3083,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All the above queries could be combined:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Male mice that had a session" + "All the above queries can be combined, for example, based on the male mice that are in a session:" ] }, { "cell_type": "code", - "execution_count": 39, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3133,12 +3187,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Selec the mice that participated in an experimental session done on or before 2017-05-19" + "Another example of how to select the mice that participated in an experimental session done on or before 2017-05-19:" ] }, { "cell_type": "code", - "execution_count": 40, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3249,7 +3303,7 @@ }, { "cell_type": "code", - "execution_count": 41, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3380,7 +3434,7 @@ }, { "cell_type": "code", - "execution_count": 42, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -3391,21 +3445,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Joining (*) - bring fields from different tables together" + "### 2. Joining (*) - bring fields from different tables together" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Sometimes you want to see information from multiple tables combined together to be viewed (and queried!) simultaneously. You can do this using the join `*` operator." + "Sometimes you want to view and query information simultaneously from multiple tables combined. You can do this using the join `*` operator." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "This is how the Join operator works:\n", + "The Join operator works as follows:\n", "\n", "1. Match the common field(s) of the primary keys in the two tables\n", "2. Do a combination of the non-matched part of the primary key\n", @@ -3415,7 +3469,7 @@ }, { "cell_type": "code", - "execution_count": 43, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3545,7 +3599,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Here each row represents a unique (and valid!) combination of a mouse and a session." + "Each row represents a unique (and valid!) combination of a mouse and a session." ] }, { @@ -3557,7 +3611,7 @@ }, { "cell_type": "code", - "execution_count": 44, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3671,7 +3725,7 @@ }, { "cell_type": "code", - "execution_count": 45, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3793,20 +3847,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Projection .proj(): focus on attributes of interest\n", - "Beside restriction (`&`) and join (`*`) operations, DataJoint offers another type of operation: projection (`.proj()`). Projection is used to select attributes (columns) from a table, to rename them, or to create new calculated attributes. " + "### 3. Projection .proj(): focus on attributes of interest\n", + "Besides restriction (`&`) and join (`*`) operations, DataJoint offers another type of operation: projection (`.proj()`). Projection is used to select attributes (columns) from a table, rename them, or create new calculated attributes. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "From the ***Mouse*** table, suppose we want to focus only on the `sex` attribute and ignore the others, this can be done as:" + "From the ***Mouse*** table, suppose we want to focus only on the `sex` attribute and ignore the others. This can be done as:" ] }, { "cell_type": "code", - "execution_count": 46, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -3932,12 +3986,12 @@ "metadata": {}, "source": [ "### Rename attribute with proj()\n", - "Say we want to rename the existing attribute `dob` of the `Mouse` table to `date_of_birth`, this can be done using `.proj()`" + "Say we want to rename the existing attribute `dob` of the `Mouse` table to `date_of_birth`. This can be done using `.proj()`:" ] }, { "cell_type": "code", - "execution_count": 47, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4055,12 +4109,12 @@ "metadata": {}, "source": [ "### Perform simple computations with proj()\n", - "Projection is perhaps most useful to perform simple computations on the attributes, especially on attributes from multiple tables by using in conjunction with the join (`*`) operation" + "Projection is perhaps most useful to perform simple computations on the attributes, especially on attributes from multiple tables, by using it in conjunction with the join (`*`) operation" ] }, { "cell_type": "code", - "execution_count": 48, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4166,12 +4220,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note: as you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation, while removing all other non-primary attributes. To Keep all other attributes, you can use the `...` syntax" + "Note: As you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation while removing all other non-primary attributes. To Keep all other attributes, you can use the `...` syntax" ] }, { "cell_type": "code", - "execution_count": 49, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4306,14 +4360,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Fetch data" + "### 4. Fetching (`.fetch()`): pull the data from the database" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Once you have successfully narrowed down to the entities you want, you can fetch the query results just by calling fetch on it!" + "Once you have narrowed down to the entities you want, you can fetch the query results just by calling fetch on it!" ] }, { @@ -4332,7 +4386,7 @@ }, { "cell_type": "code", - "execution_count": 51, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4441,12 +4495,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch it!" + "Fetch it!:" ] }, { "cell_type": "code", - "execution_count": 52, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4472,12 +4526,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "or all in one step" + "Or all in one step:" ] }, { "cell_type": "code", - "execution_count": 53, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4503,12 +4557,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch as a list of dictionaries" + "Fetch as a list of dictionaries:" ] }, { "cell_type": "code", - "execution_count": 54, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4533,12 +4587,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch as a pandas dataframe" + "Fetch as a [Pandas dataframe](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) if needed:" ] }, { "cell_type": "code", - "execution_count": 55, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4618,12 +4672,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Fetch the primary key" + "Fetch the primary key:" ] }, { "cell_type": "code", - "execution_count": 56, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4645,12 +4699,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Fetch specific fields" + "Fetch specific fields:" ] }, { "cell_type": "code", - "execution_count": 57, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -4659,7 +4713,7 @@ }, { "cell_type": "code", - "execution_count": 58, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4680,7 +4734,7 @@ }, { "cell_type": "code", - "execution_count": 59, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4708,12 +4762,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Or fetch them together as a list of dictionaries" + "Or fetch them together as a list of dictionaries:" ] }, { "cell_type": "code", - "execution_count": 60, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4754,12 +4808,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "When knowing there's only 1 result to be fetched back, we can use `.fetch1()`. `fetch1` will always return the fetched result in a dictionary format" + "When there is only one result to be fetched back, we can use `.fetch1()`. `fetch1` will always return the fetched result in a dictionary format:" ] }, { "cell_type": "code", - "execution_count": 61, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4782,12 +4836,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "`fetch1()` could also fetch the primary key" + "`fetch1()` can also fetch the primary key:" ] }, { "cell_type": "code", - "execution_count": 62, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4809,12 +4863,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "or fetch specific fields:" + "Or fetch specific fields:" ] }, { "cell_type": "code", - "execution_count": 63, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -4823,7 +4877,7 @@ }, { "cell_type": "code", - "execution_count": 64, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4843,7 +4897,7 @@ }, { "cell_type": "code", - "execution_count": 65, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -4865,21 +4919,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Deletion (`.delete()`) - deleting entries and their dependencies" + "### 5. Deletion (`.delete()`) - delete entries and their dependencies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now we have a good idea on how to restrict table entries, this is a good time to introduce how to **delete** entries from a table." + "Now that we have a good idea of how to restrict table entries, this is an excellent time to introduce how to **delete** entries from a table." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "To delete a specific entry, you restrict the table down to the target entry, and call `delete` method." + "To delete a specific entry, you restrict the table to the target entry, and call the `delete` method." ] }, { @@ -4918,9 +4972,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! You have successfully created your first DataJoint pipeline, using dependencies to establish the link between the tables. You have also learned to query and fetch the data.\n", + "Congratulations! You have successfully created your first DataJoint pipeline, using dependencies to establish the link among the tables. You have also learned to query and fetch the data.\n", "\n", - "In the next session, we are going to extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", + "In the next session, we will extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", "\n", "Please, continue to the next notebook:\n", "+ [02-Calcium Imaging](../notebooks/02-Calcium%20Imaging%20Imported%20Tables.ipynb)\n" From feb084534e79d20176b508ec95469ac37df2addd Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 16:04:38 +0000 Subject: [PATCH 17/70] Minor: Summary added in the index --- notebooks/01-DataJoint Basics.ipynb | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 8f204a3..59326d0 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -11,7 +11,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint. \n", + "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint: \n", "\n", "- Essential concepts and setup\n", " - Data pipelines\n", @@ -22,7 +22,8 @@ " - Practical examples\n", " - Basic relational operators\n", " - Create tables with dependencies\n", - " - Querying data" + " - Querying data\n", + "- Summary" ] }, { From 2e03eda3f2bff8bfce58582f203e1fad25a39fac Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 16:47:48 +0000 Subject: [PATCH 18/70] Update Tutorial01 to add minor typos --- notebooks/01-DataJoint Basics.ipynb | 44 ++++++++++++++--------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 59326d0..3e00f94 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -642,7 +642,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can also `insert1` as a dictionary" + "You can also `insert1` as a dictionary:" ] }, { @@ -771,7 +771,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can also insert multiple **mice** together using the `insert` method, passing in a list of data." + "We can also insert multiple **mice** together using the `insert` method, passing in a list of data:" ] }, { @@ -800,7 +800,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Of course, you can `insert` a list of dictionaries" + "Of course, you can `insert` a list of dictionaries:" ] }, { @@ -978,7 +978,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert a few more mice into your table before moving on." + "Let's insert a few more mice into your table before moving on:" ] }, { @@ -1273,7 +1273,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert a few sessions manually." + "Let's insert a few sessions manually:" ] }, { @@ -1401,7 +1401,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert another session for `mouse_id = 0` but on a different date." + "Let's insert another session for `mouse_id = 0` but on a different date:" ] }, { @@ -1666,7 +1666,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "What happens if we try to insert a session for a mouse that doesn't exist?" + "What happens if we try to insert a session for a mouse that doesn't exist?:" ] }, { @@ -1765,7 +1765,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse with id 0" + "Mouse with id 0:" ] }, { @@ -1869,7 +1869,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All male mice (`'sex = \"M\"'`)" + "All the male (`M`) mice:" ] }, { @@ -1982,7 +1982,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All female mice (`'sex=\"F\"'`)" + "All the female (`F`) mice:" ] }, { @@ -2222,7 +2222,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse that is born **after 2017-01-01**" + "Mouse that is born **after 2017-01-01**:" ] }, { @@ -2347,7 +2347,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse that is born within a range of dates" + "Mouse that is born within a range of dates:" ] }, { @@ -2460,7 +2460,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse that is **not** male" + "Mice that are **not** male:" ] }, { @@ -2592,7 +2592,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's find all mice that **are not male** AND **born after 2017-01-01**." + "Let's find all mice that **are not male** AND **born after 2017-01-01**:" ] }, { @@ -2817,7 +2817,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The result of one query can be used in another query! Let's first find **all the female mice** and store the result." + "The result of one query can be used in another query! Let's first find **all the female mice** and **store the result**:" ] }, { @@ -2940,7 +2940,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "and among these mice, find the ones with **mouse_id > 10**" + "It's your turn! Find and store the mice with a **mouse_id > 10**:" ] }, { @@ -3299,7 +3299,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All mice that do not have any session" + "All the mice that do not have any session:" ] }, { @@ -3430,7 +3430,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Male mice that do not have any session" + "It's your turn! Find and store the male mice that do not have any session:" ] }, { @@ -3856,7 +3856,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "From the ***Mouse*** table, suppose we want to focus only on the `sex` attribute and ignore the others. This can be done as:" + "From the **Mouse** table, suppose we want to focus only on the `sex` attribute and ignore the others. This can be done as:" ] }, { @@ -3978,7 +3978,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that `.proj()` will always retain all attributes that are part of the primary key" + "Note that `.proj()` will always retain all attributes that are part of the primary key." ] }, { @@ -4110,7 +4110,7 @@ "metadata": {}, "source": [ "### Perform simple computations with proj()\n", - "Projection is perhaps most useful to perform simple computations on the attributes, especially on attributes from multiple tables, by using it in conjunction with the join (`*`) operation" + "Projection is perhaps most useful to perform simple computations on the attributes, especially on attributes from multiple tables, by using it in conjunction with the join (`*`) operation." ] }, { @@ -4221,7 +4221,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note: As you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation while removing all other non-primary attributes. To Keep all other attributes, you can use the `...` syntax" + "Note: As you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation while removing all other non-primary attributes. To keep all the other attributes, you can use the `...` syntax." ] }, { From 0914f0090358a1aba5204eda6c59242a39dd7f94 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 17:03:35 +0000 Subject: [PATCH 19/70] test commit --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c644192..42cd978 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ This document will guide you as a new DataJoint user through interactive tutoria *Please note that these hands-on DataJoint tutorials are friendly to non-expert users, and advanced programming skills are not required.* -## Table of contents +## Table of contents - Notebooks: interactive notebooks to be a master of DataJoint. The Calcium Imaging and Electrophysiology tutorials are common examples of data structure and analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! - 01-DataJoint Basics - 02-Calcium Imaging Imported Tables From 8ebe8bfcb6e6f8bcf1ff62fac3bc481130651f25 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Tue, 25 Jul 2023 17:18:43 -0500 Subject: [PATCH 20/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 42cd978..9f7034c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Welcome to DataJoint tutorials! -DataJoint is an open-source tool library to design, build and automate data analysis, pipelines, and data sharing for neuroscience experiments and research labs. +DataJoint is an open-source library for science labs to design and build data pipelines for automated data analysis and sharing. This document will guide you as a new DataJoint user through interactive tutorials organized in [Jupyter notebooks](https://jupyter-notebook.readthedocs.io/en/stable/) and written in [Python](https://www.python.org/). From f3ec2d2a44dd953af829a4eebc984aad44d2638c Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Tue, 25 Jul 2023 17:19:59 -0500 Subject: [PATCH 21/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9f7034c..ae79fd3 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ This document will guide you as a new DataJoint user through interactive tutoria *Please note that these hands-on DataJoint tutorials are friendly to non-expert users, and advanced programming skills are not required.* ## Table of contents -- Notebooks: interactive notebooks to be a master of DataJoint. The Calcium Imaging and Electrophysiology tutorials are common examples of data structure and analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! +- In the `tutorials` folder are interactive Jupyter notebooks to learn DataJoint. The Calcium Imaging and Electrophysiology tutorials are relevant examples of data structure and analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! - 01-DataJoint Basics - 02-Calcium Imaging Imported Tables - 03-Calcium Imaging Computed Tables From a3d18c142405a0c3a319110715842ac17e14e437 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Tue, 25 Jul 2023 17:20:26 -0500 Subject: [PATCH 22/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/README.md b/README.md index ae79fd3..088ae27 100644 --- a/README.md +++ b/README.md @@ -20,11 +20,6 @@ This document will guide you as a new DataJoint user through interactive tutoria - DataJoint in 30min - University -- Tutorial pipeline - - Ephys cell activity - - Imaging - - Mouse session - ## Key learnings from the tutorials After completing this set of tutorials, you will gain real experience in the basics of the DataJoint framework. These skills will allow you to design, implement and manage data pipelines effectively applied to your scientific research. From c725694114f9bad5c98c1aad9a45d66eed041a7f Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Tue, 25 Jul 2023 17:20:35 -0500 Subject: [PATCH 23/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 088ae27..dbb1f89 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ This document will guide you as a new DataJoint user through interactive tutoria - 04-Electrophysiology Imported Tables - 05-Electrophysiology Computed Tables -- Completed exercises: notebooks with the code sections completed and solved. +- In the `completed_tutorials` folder are Jupyter notebooks with the code sections completed and solved. - Short tutorials: - DataJoint in 30min From 8037c8f19a786c039792e65c13707db83ebd44ed Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Tue, 25 Jul 2023 17:20:44 -0500 Subject: [PATCH 24/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index dbb1f89..a5b3add 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ This document will guide you as a new DataJoint user through interactive tutoria - In the `completed_tutorials` folder are Jupyter notebooks with the code sections completed and solved. -- Short tutorials: +- You will find the following notebooks in the `short_tutorials` folder: - DataJoint in 30min - University From 367aa5136915ebca60db4b0c0ea917d6ffe5aac3 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Tue, 25 Jul 2023 17:21:01 -0500 Subject: [PATCH 25/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a5b3add..025d9d5 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ Here is a summary of the content that you can expect to have learned: - Retrieve selective attributes - Delete operations -- DataJoint advanced topics: workflow automation (~1 hour) +- DataJoint advanced topics: pipeline automation (~1 hour) - `Imported` and `Computed` tables - `make()` function - `.populate()` for automated computation From f1c1d9c29842bbfdcea260ff5e66682908a0442e Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 19:12:56 +0000 Subject: [PATCH 26/70] 01-Basics: Next 02-tutorial link deleted --- notebooks/01-DataJoint Basics.ipynb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 3e00f94..65b4585 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -4977,8 +4977,8 @@ "\n", "In the next session, we will extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", "\n", - "Please, continue to the next notebook:\n", - "+ [02-Calcium Imaging](../notebooks/02-Calcium%20Imaging%20Imported%20Tables.ipynb)\n" + "\n", + "Please, continue to the next notebook \"02-Calcium Imaging.ipynb\"." ] } ], From afec99217d5547f1f6854f82c85f4775be6ab585 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 19:16:12 +0000 Subject: [PATCH 27/70] 01-Basics: Minor change - new line --- notebooks/01-DataJoint Basics.ipynb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 65b4585..84ec623 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -11,7 +11,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. This tutorial will walk you through the major concepts and steps to use DataJoint: \n", + "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. \n", + "\n", + "This tutorial will walk you through the major concepts and steps to use DataJoint: \n", "\n", "- Essential concepts and setup\n", " - Data pipelines\n", From 89d23f1bb07a278fb5a0d85a199c5e6d978cdf60 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Tue, 25 Jul 2023 17:38:48 -0500 Subject: [PATCH 28/70] Update tutorial02: new sections and explanations --- .../02-Calcium Imaging Imported Tables.ipynb | 719 ++++++++++++++++-- 1 file changed, 668 insertions(+), 51 deletions(-) diff --git a/notebooks/02-Calcium Imaging Imported Tables.ipynb b/notebooks/02-Calcium Imaging Imported Tables.ipynb index 6114d4b..2e0fb93 100644 --- a/notebooks/02-Calcium Imaging Imported Tables.ipynb +++ b/notebooks/02-Calcium Imaging Imported Tables.ipynb @@ -4,31 +4,39 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Working with automated computations: Imported tables" + "# Working with automated computations: Imported tables\n", + "# Application to Calcium Imaging" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Welcome back! In this session, we are going to continue working with the pipeline for the mouse calcium imaging example. \n", + "Welcome back! The practical example of this session is Calcium Imaging! \n", "\n", - "In this session, we will learn to:\n", + "During this session you will learn:\n", "\n", - "* import neuron imaging data from data files into an `Imported` table\n", - "* automatically trigger data importing and computations for all missing entries with `populate`" + "* To import neuron imaging data from data files into an `Imported` table\n", + "* To automatically trigger data importing and computations for all the missing entries with `Populate`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "First thing first, let's import `datajoint` again." + "## Importing libraries" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First thing first, let's import `DataJoint` again." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -39,12 +47,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As we are going to perform some computations, let's go ahead and import NumPy and Matplotlib" + "As we are going to perform some computations, let's go ahead and import `NumPy` and `Matplotlib`." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -57,14 +65,34 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now we would like to continue working with the tables we defined in the previous notebook. To do so, we would need the classes for each table: `Mouse` and `Session`. We can either redefine it here, but for your convenience, we have included the schema and table class definitions in a package called `tutorial_pipeline.mouse_session`, from which you can import the classes as well as the schema object. We will use the schema object again to define more tables." + "## Schema, Mouse & Session" ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], + "source": [ + "We will continue working with the tables that we defined in the previous session. To do so, we will need the classes for each table: `Mouse` and `Session`. \n", + "\n", + "We can either redefine it here, but for your convenience, we have included the schema and table class definitions in a package called `tutorial_pipeline.mouse_session`, from which you can import the classes as well as the schema object. \n", + "\n", + "We will use the schema object again to define more tables." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "[2023-07-25 20:54:56,914][INFO]: Connecting root@fakeservices.datajoint.io:3306\n", + "[2023-07-25 20:54:56,935][INFO]: Connected root@fakeservices.datajoint.io:3306\n" + ] + } + ], "source": [ "from tutorial_pipeline.mouse_session import schema, Mouse, Session" ] @@ -73,23 +101,240 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Take a quick look at the tables Mouse and Session" + "Take a quick look at the tables `Mouse` and `Session`:" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experimental animals\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

dob

\n", + " date of birth\n", + "
\n", + "

sex

\n", + " sex\n", + "
02017-03-01M
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
1002017-05-12F
\n", + " \n", + "

Total: 7

\n", + " " + ], + "text/plain": [ + "*mouse_id dob sex \n", + "+----------+ +------------+ +---------+\n", + "0 2017-03-01 M \n", + "1 2016-11-19 M \n", + "2 2016-11-20 unknown \n", + "5 2016-12-25 F \n", + "10 2017-01-01 F \n", + "11 2017-01-03 F \n", + "100 2017-05-12 F \n", + " (Total: 7)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Mouse()" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experiment session\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-150Edgar Y. Walker/workspaces/datajoint-tutorials/data
02017-05-190Edgar Y. Walker/workspaces/datajoint-tutorials/data
52017-01-051Fabian Sinz/workspaces/datajoint-tutorials/data
1002017-05-25100Jacob Reimer/workspaces/datajoint-tutorials/data
\n", + " \n", + "

Total: 4

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +------------+ +------------+\n", + "0 2017-05-15 0 Edgar Y. Walke /workspaces/da\n", + "0 2017-05-19 0 Edgar Y. Walke /workspaces/da\n", + "5 2017-01-05 1 Fabian Sinz /workspaces/da\n", + "100 2017-05-25 100 Jacob Reimer /workspaces/da\n", + " (Total: 4)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Session()" ] @@ -98,26 +343,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The `mouse_session.py` also fills each table with data to make sure we are all on the same page." + "The `mouse_session.py` file located in the folder `tutorial_pipeline` also fills each table with example data to make sure that we are all on the same page." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Define table `Scan` for meta information of each calcium imaging scan" + "## Scan table and primary key attributes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's define a table `Scan` that describes a scanning in an experimental session that stores the meta information of a particular scan." + "First we define a table named `Scan` that describes a scanning in an experimental session of Calcium Imaging that will store the meta information of the scans." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -139,14 +384,74 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This table is dependent on the table `Session`, inheriting its primary key attributes, with an additional primary key attribute `scan_idx`. One session could contain multiple scans, which is another example of **one-to-many** relationship. We could take a look at the Diagram again." + "The table `Scan` is dependent on the table `Session`, inheriting its primary key attributes. In addition, `Scan` has an additional primary key attribute `scan_idx`. \n", + "\n", + "One session might contain multiple scans - This is another example of **one-to-many** relationship. \n", + "\n", + "Take a look at the `Diagram` again:" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session->Scan\n", + "\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "dj.Diagram(schema)" ] @@ -155,23 +460,43 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The thin solid line connecting `Mouse`-`Session`, and `Session`-`Scan` indicates **one-to-many relationship**. \n", + "The thin solid line connecting [`Mouse`-`Session`] and [`Session`-`Scan`] indicates **one-to-many relationship**. \n", "\n", - "The `____` indicates **additional primary key attribute(s)** apart from the ones inherited from its parents." + "The underline `____` indicates **additional primary key attribute(s)** apart from the ones inherited from its parents." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Here we have prepared two tif files of scanning in the `data` folder `example_scan_01.tif` and `example_scan_02.tif` " + "## Calcium imaging datasets" ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], + "source": [ + "In the `data` folder in this `DataJoint-Tutorials`, you can find three calcium imaging datasets. Raw data are stored as *.tif* files.\n", + "\n", + "Let's list these three files:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PosixPath('/workspaces/datajoint-tutorials/data')" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "from tutorial_pipeline import data_dir\n", "data_dir" @@ -179,40 +504,322 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "/workspaces/datajoint-tutorials/data/example_scan_02.tif\n", + "/workspaces/datajoint-tutorials/data/example_scan_03.tif\n", + "/workspaces/datajoint-tutorials/data/example_scan_01.tif\n" + ] + } + ], "source": [ "for f in data_dir.glob('*.tif'):\n", " print(f)" ] }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

depth

\n", + " depth of this scan\n", + "
\n", + "

wavelength

\n", + " wavelength used\n", + "
\n", + "

laser_power

\n", + " power of the laser used\n", + "
\n", + "

fps

\n", + " frames per second\n", + "
\n", + "

file_name

\n", + " name of the tif file\n", + "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
02017-05-153200.0920.024.015.0example_scan_02.tif
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", + "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", + "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", + "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", + "0 2017-05-15 3 200.0 920.0 24.0 15.0 example_scan_0\n", + " (Total: 3)" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Scan()" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert these meta information manually." + "Now we manually `insert` the meta information of the datasets and their file names in the table `Scan`:" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "ename": "DuplicateError", + "evalue": "(\"Duplicate entry '0-2017-05-15-1' for key 'scan.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Scan\u001b[39m.\u001b[39;49minsert([\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39msession_date\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2017-05-15\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mscan_idx\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m1\u001b[39;49m, \n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdepth\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m150\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mwavelength\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m920\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mlaser_power\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m26\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfps\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m15\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfile_name\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mexample_scan_01.tif\u001b[39;49m\u001b[39m'\u001b[39;49m},\n\u001b[1;32m 4\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39msession_date\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2017-05-15\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mscan_idx\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m2\u001b[39;49m, \n\u001b[1;32m 5\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdepth\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m200\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mwavelength\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m920\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mlaser_power\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m24\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfps\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m15\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfile_name\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mexample_scan_02.tif\u001b[39;49m\u001b[39m'\u001b[39;49m},\n\u001b[1;32m 6\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39msession_date\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2017-05-15\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mscan_idx\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m3\u001b[39;49m, \n\u001b[1;32m 7\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdepth\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m200\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mwavelength\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m920\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mlaser_power\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m24\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfps\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m15\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfile_name\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mexample_scan_03.tif\u001b[39;49m\u001b[39m'\u001b[39;49m} \n\u001b[1;32m 8\u001b[0m ])\n", + "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", + "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0-2017-05-15-1' for key 'scan.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" + ] + } + ], "source": [ "Scan.insert([\n", " {'mouse_id': 0, 'session_date': '2017-05-15', 'scan_idx': 1, \n", " 'depth': 150, 'wavelength': 920, 'laser_power': 26, 'fps': 15, 'file_name': 'example_scan_01.tif'},\n", " {'mouse_id': 0, 'session_date': '2017-05-15', 'scan_idx': 2, \n", " 'depth': 200, 'wavelength': 920, 'laser_power': 24, 'fps': 15, 'file_name': 'example_scan_02.tif'},\n", + " {'mouse_id': 0, 'session_date': '2017-05-15', 'scan_idx': 3, \n", + " 'depth': 200, 'wavelength': 920, 'laser_power': 24, 'fps': 15, 'file_name': 'example_scan_03.tif'} \n", "])" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

depth

\n", + " depth of this scan\n", + "
\n", + "

wavelength

\n", + " wavelength used\n", + "
\n", + "

laser_power

\n", + " power of the laser used\n", + "
\n", + "

fps

\n", + " frames per second\n", + "
\n", + "

file_name

\n", + " name of the tif file\n", + "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
02017-05-153200.0920.024.015.0example_scan_02.tif
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", + "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", + "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", + "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", + "0 2017-05-15 3 200.0 920.0 24.0 15.0 example_scan_0\n", + " (Total: 3)" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "Scan()" ] @@ -221,21 +828,29 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Looking at the raw data" + "## Looking into the raw data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's first load one raw data and take a look at the data:" + "Let's first load and look at the shape (frames, height and width) of one of the example TIFF of the calcium imaging dataset:" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(100, 128, 128)\n" + ] + } + ], "source": [ "import os\n", "from skimage import io\n", @@ -247,7 +862,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This tiff file contains 100 frames. Let's take the average of the images over frames and look at it." + "Particularly, this example contains 100 frames. \n", + "\n", + "Let's take the average of the images over frames and look at it." ] }, { @@ -714,7 +1331,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.16" + "version": "3.9.17" }, "vscode": { "interpreter": { From f516eecff2716d850784f21835af3160c1bdae46 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 19:58:49 +0000 Subject: [PATCH 29/70] 01-Basics Major changes in text, new section'Drop' --- notebooks/01-DataJoint Basics.ipynb | 494 ++++++++++++++++------------ 1 file changed, 287 insertions(+), 207 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 84ec623..648c6be 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -53,7 +53,7 @@ }, { "cell_type": "code", - "execution_count": 68, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -307,7 +307,7 @@ }, { "cell_type": "code", - "execution_count": 69, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -337,7 +337,7 @@ }, { "cell_type": "code", - "execution_count": 70, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -361,7 +361,7 @@ }, { "cell_type": "code", - "execution_count": 71, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -432,55 +432,20 @@ "

sex

\n", " sex\n", " \n", - " 0\n", - "2017-03-01\n", - "M1\n", - "2016-11-19\n", - "M2\n", - "2016-11-20\n", - "unknown5\n", - "2016-12-25\n", - "F10\n", - "2017-01-01\n", - "F11\n", - "2017-01-03\n", - "F12\n", - "2017-03-21\n", - "F18\n", - "2017-05-01\n", - "F19\n", - "2018-07-21\n", - "M22\n", - "2019-12-15\n", - "F34\n", - "2018-09-22\n", - "M100\n", - "2017-05-12\n", - "F \n", + " \n", " \n", " \n", - "

Total: 12

\n", + "

Total: 0

\n", " " ], "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +---------+\n", - "0 2017-03-01 M \n", - "1 2016-11-19 M \n", - "2 2016-11-20 unknown \n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "19 2018-07-21 M \n", - "22 2019-12-15 F \n", - "34 2018-09-22 M \n", - "100 2017-05-12 F \n", - " (Total: 12)" + "*mouse_id dob sex \n", + "+----------+ +-----+ +-----+\n", + "\n", + " (Total: 0)" ] }, - "execution_count": 71, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -522,30 +487,16 @@ }, { "cell_type": "code", - "execution_count": 72, + "execution_count": 13, "metadata": {}, - "outputs": [ - { - "ename": "DuplicateError", - "evalue": "(\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[72], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1((\u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39m2017-03-01\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m))\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", - "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" - ] - } - ], + "outputs": [], "source": [ "Mouse.insert1((0, '2017-03-01', 'M'))" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 14, "metadata": {}, "outputs": [ { @@ -631,7 +582,7 @@ " (Total: 1)" ] }, - "execution_count": 6, + "execution_count": 14, "metadata": {}, "output_type": "execute_result" } @@ -649,7 +600,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -662,7 +613,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 16, "metadata": {}, "outputs": [], "source": [ @@ -671,7 +622,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -760,7 +711,7 @@ " (Total: 2)" ] }, - "execution_count": 9, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } @@ -778,7 +729,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 18, "metadata": {}, "outputs": [], "source": [ @@ -791,7 +742,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 19, "metadata": {}, "outputs": [], "source": [ @@ -807,7 +758,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [ @@ -822,7 +773,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 21, "metadata": {}, "outputs": [ { @@ -926,7 +877,7 @@ " (Total: 7)" ] }, - "execution_count": 13, + "execution_count": 21, "metadata": {}, "output_type": "execute_result" } @@ -946,12 +897,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "DataJoint checks for data integrity and ensures you don't insert a duplicate by mistake. Let's try inserting another mouse with `mouse_id: 0` and see what happens!" + "DataJoint checks for data integrity and ensures you don't insert a duplicate by mistake. Let's try inserting another mouse with `mouse_id: 0` and see what happens!\n", + "\n", + "*Note that the following code cell is intended to give an error code.* " ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 23, "metadata": {}, "outputs": [ { @@ -961,7 +914,7 @@ "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1(\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m,\n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdob\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2018-01-01\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 4\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39msex\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 5\u001b[0m })\n", + "Cell \u001b[0;32mIn[23], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1(\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m,\n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdob\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2018-01-01\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 4\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39msex\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 5\u001b[0m })\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" @@ -976,6 +929,26 @@ "})" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The error message suggests to include a new argument `skip_duplicates=True` in the `.insert1()` method to skip inserting duplicated entries. " + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "Mouse.insert1(\n", + "{'mouse_id': 0,\n", + " 'dob': '2018-01-01',\n", + " 'sex': 'M',\n", + "}, skip_duplicates=True)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -985,7 +958,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 26, "metadata": {}, "outputs": [], "source": [ @@ -1003,7 +976,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 27, "metadata": {}, "outputs": [ { @@ -1122,7 +1095,7 @@ " (Total: 12)" ] }, - "execution_count": 16, + "execution_count": 27, "metadata": {}, "output_type": "execute_result" } @@ -1133,7 +1106,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 28, "metadata": {}, "outputs": [], "source": [ @@ -1196,7 +1169,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 29, "metadata": {}, "outputs": [], "source": [ @@ -1222,7 +1195,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 30, "metadata": {}, "outputs": [ { @@ -1232,21 +1205,21 @@ "\n", "%3\n", "\n", - "\n", + "\n", "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", + "Session\n", + "\n", + "\n", + "Session\n", "\n", "\n", "\n", - "\n", + "\n", "\n", - "Session\n", - "\n", - "\n", - "Session\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", "\n", "\n", "\n", @@ -1259,10 +1232,10 @@ "" ], "text/plain": [ - "" + "" ] }, - "execution_count": 19, + "execution_count": 30, "metadata": {}, "output_type": "execute_result" } @@ -1280,7 +1253,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 31, "metadata": {}, "outputs": [], "source": [ @@ -1296,7 +1269,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 32, "metadata": {}, "outputs": [ { @@ -1390,7 +1363,7 @@ " (Total: 1)" ] }, - "execution_count": 21, + "execution_count": 32, "metadata": {}, "output_type": "execute_result" } @@ -1408,7 +1381,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 33, "metadata": {}, "outputs": [ { @@ -1507,7 +1480,7 @@ " (Total: 2)" ] }, - "execution_count": 22, + "execution_count": 33, "metadata": {}, "output_type": "execute_result" } @@ -1534,7 +1507,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 34, "metadata": {}, "outputs": [], "source": [ @@ -1551,7 +1524,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 35, "metadata": {}, "outputs": [ { @@ -1655,7 +1628,7 @@ " (Total: 3)" ] }, - "execution_count": 24, + "execution_count": 35, "metadata": {}, "output_type": "execute_result" } @@ -1673,7 +1646,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 36, "metadata": {}, "outputs": [], "source": [ @@ -1685,9 +1658,16 @@ "}" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*Note: the following code line is intended to give an error code:*" + ] + }, { "cell_type": "code", - "execution_count": null, + "execution_count": 38, "metadata": { "scrolled": true }, @@ -1699,7 +1679,7 @@ "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mIntegrityError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[26], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Session\u001b[39m.\u001b[39;49minsert1(bad_data)\n", + "Cell \u001b[0;32mIn[38], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Session\u001b[39m.\u001b[39;49minsert1(bad_data)\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:440\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 424\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m 425\u001b[0m query \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39m{command}\u001b[39;00m\u001b[39m INTO \u001b[39m\u001b[39m{destination}\u001b[39;00m\u001b[39m(`\u001b[39m\u001b[39m{fields}\u001b[39;00m\u001b[39m`) VALUES \u001b[39m\u001b[39m{placeholders}\u001b[39;00m\u001b[39m{duplicate}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m 426\u001b[0m command\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mREPLACE\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mif\u001b[39;00m replace \u001b[39melse\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mINSERT\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m 427\u001b[0m destination\u001b[39m=\u001b[39m\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mfrom_clause(),\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 438\u001b[0m ),\n\u001b[1;32m 439\u001b[0m )\n\u001b[0;32m--> 440\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mconnection\u001b[39m.\u001b[39;49mquery(\n\u001b[1;32m 441\u001b[0m query,\n\u001b[1;32m 442\u001b[0m args\u001b[39m=\u001b[39;49m\u001b[39mlist\u001b[39;49m(\n\u001b[1;32m 443\u001b[0m itertools\u001b[39m.\u001b[39;49mchain\u001b[39m.\u001b[39;49mfrom_iterable(\n\u001b[1;32m 444\u001b[0m (v \u001b[39mfor\u001b[39;49;00m v \u001b[39min\u001b[39;49;00m r[\u001b[39m\"\u001b[39;49m\u001b[39mvalues\u001b[39;49m\u001b[39m\"\u001b[39;49m] \u001b[39mif\u001b[39;49;00m v \u001b[39mis\u001b[39;49;00m \u001b[39mnot\u001b[39;49;00m \u001b[39mNone\u001b[39;49;00m) \u001b[39mfor\u001b[39;49;00m r \u001b[39min\u001b[39;49;00m rows\n\u001b[1;32m 445\u001b[0m )\n\u001b[1;32m 446\u001b[0m ),\n\u001b[1;32m 447\u001b[0m )\n\u001b[1;32m 448\u001b[0m \u001b[39mexcept\u001b[39;00m UnknownAttributeError \u001b[39mas\u001b[39;00m err:\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/connection.py:340\u001b[0m, in \u001b[0;36mConnection.query\u001b[0;34m(self, query, args, as_dict, suppress_warnings, reconnect)\u001b[0m\n\u001b[1;32m 338\u001b[0m cursor \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_conn\u001b[39m.\u001b[39mcursor(cursor\u001b[39m=\u001b[39mcursor_class)\n\u001b[1;32m 339\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 340\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_execute_query(cursor, query, args, suppress_warnings)\n\u001b[1;32m 341\u001b[0m \u001b[39mexcept\u001b[39;00m errors\u001b[39m.\u001b[39mLostConnectionError:\n\u001b[1;32m 342\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m reconnect:\n", @@ -1735,7 +1715,7 @@ "metadata": {}, "source": [ "We will introduce significant types of queries used in DataJoint:\n", - "* 1. Restriction (`&`) and negative restriction (`-`): filter data with certain conditions\n", + "* 1. Restriction (`&`) and negative restriction (`-`): filter the data with certain conditions\n", "* 2. Joining (`*`): bring fields from different tables together\n", "* 3. Projection (`.proj()`): focus on a subset of attributes\n", "* 4. Fetching (`.fetch()`): pull the data from the database\n", @@ -1746,7 +1726,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 1. Restrictions (`&`) - filter data with certain conditions" + "### 1. Restrictions (`&` or `-`): filter the data with certain conditions" ] }, { @@ -1767,12 +1747,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse with id 0:" + "Mouse with `ID = 0`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 39, "metadata": {}, "outputs": [ { @@ -1858,7 +1838,7 @@ " (Total: 1)" ] }, - "execution_count": 27, + "execution_count": 39, "metadata": {}, "output_type": "execute_result" } @@ -1876,7 +1856,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 40, "metadata": {}, "outputs": [ { @@ -1971,7 +1951,7 @@ " (Total: 4)" ] }, - "execution_count": 28, + "execution_count": 40, "metadata": {}, "output_type": "execute_result" } @@ -1989,7 +1969,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 41, "metadata": {}, "outputs": [ { @@ -2093,7 +2073,7 @@ " (Total: 7)" ] }, - "execution_count": 29, + "execution_count": 41, "metadata": {}, "output_type": "execute_result" } @@ -2111,7 +2091,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 42, "metadata": {}, "outputs": [ { @@ -2197,7 +2177,7 @@ " (Total: 1)" ] }, - "execution_count": 30, + "execution_count": 42, "metadata": {}, "output_type": "execute_result" } @@ -2224,12 +2204,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse that is born **after 2017-01-01**:" + "Mouse that is born `after 2017-01-01`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 43, "metadata": {}, "outputs": [ { @@ -2336,7 +2316,7 @@ " (Total: 8)" ] }, - "execution_count": 31, + "execution_count": 43, "metadata": {}, "output_type": "execute_result" } @@ -2354,7 +2334,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 44, "metadata": {}, "outputs": [ { @@ -2449,7 +2429,7 @@ " (Total: 4)" ] }, - "execution_count": 32, + "execution_count": 44, "metadata": {}, "output_type": "execute_result" } @@ -2462,12 +2442,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mice that are **not** male:" + "Mice that are `not male`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 45, "metadata": {}, "outputs": [ { @@ -2574,7 +2554,7 @@ " (Total: 8)" ] }, - "execution_count": 33, + "execution_count": 45, "metadata": {}, "output_type": "execute_result" } @@ -2594,12 +2574,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's find all mice that **are not male** AND **born after 2017-01-01**:" + "Let's find all mice that `are not male` and born `after 2017-01-01`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 46, "metadata": {}, "outputs": [ { @@ -2697,7 +2677,7 @@ " (Total: 5)" ] }, - "execution_count": 34, + "execution_count": 46, "metadata": {}, "output_type": "execute_result" } @@ -2708,7 +2688,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 47, "metadata": {}, "outputs": [ { @@ -2806,7 +2786,7 @@ " (Total: 5)" ] }, - "execution_count": 35, + "execution_count": 47, "metadata": {}, "output_type": "execute_result" } @@ -2819,12 +2799,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The result of one query can be used in another query! Let's first find **all the female mice** and **store the result**:" + "The result of one query can be used in another query! Let's first find `all the female mice` and `store the result`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 48, "metadata": {}, "outputs": [ { @@ -2928,7 +2908,7 @@ " (Total: 7)" ] }, - "execution_count": 36, + "execution_count": 48, "metadata": {}, "output_type": "execute_result" } @@ -2942,12 +2922,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "It's your turn! Find and store the mice with a **mouse_id > 10**:" + "It's your turn! Find and store the mice with a `mouse_id > 10`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 49, "metadata": {}, "outputs": [], "source": [ @@ -2965,19 +2945,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Restriction of one table with another" + "### Restriction operator (`&`): all entities from one table for which there exist a matching entity in other table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that when restricting, for example, table A with table B (written A & B), the two tables must have common attributes (join-compatible). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All the mice that have a session:" + "To select all the `mice` that have a `session`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 54, "metadata": {}, "outputs": [ { @@ -3066,7 +3053,7 @@ " (Total: 2)" ] }, - "execution_count": 38, + "execution_count": 54, "metadata": {}, "output_type": "execute_result" } @@ -3079,19 +3066,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Combining restrictions" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "All the above queries can be combined, for example, based on the male mice that are in a session:" + "All the above queries can be combined, for example, based on the `male mice` that are in a `session`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 55, "metadata": {}, "outputs": [ { @@ -3177,7 +3157,7 @@ " (Total: 1)" ] }, - "execution_count": 39, + "execution_count": 55, "metadata": {}, "output_type": "execute_result" } @@ -3190,12 +3170,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Another example of how to select the mice that participated in an experimental session done on or before 2017-05-19:" + "Another example of how to select the `mice` that participated in an `experimental session` done `on or before 2017-05-19`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 56, "metadata": {}, "outputs": [ { @@ -3281,7 +3261,7 @@ " (Total: 1)" ] }, - "execution_count": 40, + "execution_count": 56, "metadata": {}, "output_type": "execute_result" } @@ -3294,19 +3274,19 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Negative restriction - with the `-` operator" + "### Negative restriction (`-`): subset of entities from one table for which there are no matching entities in other table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All the mice that do not have any session:" + "All the `mice` that do `not have any session`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 57, "metadata": {}, "outputs": [ { @@ -3419,7 +3399,7 @@ " (Total: 10)" ] }, - "execution_count": 41, + "execution_count": 57, "metadata": {}, "output_type": "execute_result" } @@ -3437,7 +3417,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 58, "metadata": {}, "outputs": [], "source": [ @@ -3448,7 +3428,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 2. Joining (*) - bring fields from different tables together" + "### 2. Joining (`*`): bring fields from different tables together" ] }, { @@ -3472,7 +3452,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 59, "metadata": {}, "outputs": [ { @@ -3588,7 +3568,7 @@ " (Total: 3)" ] }, - "execution_count": 43, + "execution_count": 59, "metadata": {}, "output_type": "execute_result" } @@ -3614,7 +3594,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 60, "metadata": {}, "outputs": [ { @@ -3716,7 +3696,7 @@ " (Total: 1)" ] }, - "execution_count": 44, + "execution_count": 60, "metadata": {}, "output_type": "execute_result" } @@ -3728,7 +3708,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 61, "metadata": {}, "outputs": [ { @@ -3837,7 +3817,7 @@ " (Total: 2)" ] }, - "execution_count": 45, + "execution_count": 61, "metadata": {}, "output_type": "execute_result" } @@ -3850,7 +3830,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 3. Projection .proj(): focus on attributes of interest\n", + "### 3. Projection (`.proj()`): focus on attributes of interest\n", "Besides restriction (`&`) and join (`*`) operations, DataJoint offers another type of operation: projection (`.proj()`). Projection is used to select attributes (columns) from a table, rename them, or create new calculated attributes. " ] }, @@ -3863,7 +3843,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 62, "metadata": {}, "outputs": [ { @@ -3967,7 +3947,7 @@ " (Total: 12)" ] }, - "execution_count": 46, + "execution_count": 62, "metadata": {}, "output_type": "execute_result" } @@ -3994,7 +3974,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 63, "metadata": {}, "outputs": [ { @@ -4098,7 +4078,7 @@ " (Total: 12)" ] }, - "execution_count": 47, + "execution_count": 63, "metadata": {}, "output_type": "execute_result" } @@ -4117,7 +4097,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 64, "metadata": {}, "outputs": [ { @@ -4209,7 +4189,7 @@ " (Total: 3)" ] }, - "execution_count": 48, + "execution_count": 64, "metadata": {}, "output_type": "execute_result" } @@ -4228,7 +4208,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 65, "metadata": {}, "outputs": [ { @@ -4350,7 +4330,7 @@ " (Total: 3)" ] }, - "execution_count": 49, + "execution_count": 65, "metadata": {}, "output_type": "execute_result" } @@ -4384,12 +4364,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All the male mice:" + "All the `male mice`:" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 66, "metadata": {}, "outputs": [ { @@ -4484,7 +4464,7 @@ " (Total: 4)" ] }, - "execution_count": 51, + "execution_count": 66, "metadata": {}, "output_type": "execute_result" } @@ -4503,7 +4483,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 68, "metadata": {}, "outputs": [ { @@ -4516,7 +4496,7 @@ " dtype=[('mouse_id', ' Date: Wed, 26 Jul 2023 20:16:27 +0000 Subject: [PATCH 30/70] 01-Basics New subsection 6.Drop added --- notebooks/01-DataJoint Basics.ipynb | 409 ++++++++++++++++++---------- 1 file changed, 258 insertions(+), 151 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 648c6be..68e9aec 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -307,9 +307,18 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 2, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "[2023-07-26 20:13:43,743][INFO]: Connecting root@fakeservices.datajoint.io:3306\n", + "[2023-07-26 20:13:43,759][INFO]: Connected root@fakeservices.datajoint.io:3306\n" + ] + } + ], "source": [ "schema = dj.schema('tutorial')" ] @@ -337,7 +346,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -361,7 +370,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -445,7 +454,7 @@ " (Total: 0)" ] }, - "execution_count": 12, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -487,7 +496,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ @@ -496,7 +505,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 6, "metadata": {}, "outputs": [ { @@ -582,7 +591,7 @@ " (Total: 1)" ] }, - "execution_count": 14, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -600,7 +609,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -613,7 +622,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 8, "metadata": {}, "outputs": [], "source": [ @@ -622,7 +631,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 9, "metadata": {}, "outputs": [ { @@ -711,7 +720,7 @@ " (Total: 2)" ] }, - "execution_count": 17, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } @@ -729,7 +738,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -742,7 +751,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -758,7 +767,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 12, "metadata": {}, "outputs": [], "source": [ @@ -773,7 +782,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 13, "metadata": {}, "outputs": [ { @@ -877,7 +886,7 @@ " (Total: 7)" ] }, - "execution_count": 21, + "execution_count": 13, "metadata": {}, "output_type": "execute_result" } @@ -904,7 +913,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 14, "metadata": {}, "outputs": [ { @@ -914,7 +923,7 @@ "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[23], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1(\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m,\n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdob\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2018-01-01\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 4\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39msex\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 5\u001b[0m })\n", + "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1(\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m,\n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdob\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2018-01-01\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 4\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39msex\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 5\u001b[0m })\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" @@ -938,7 +947,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -958,7 +967,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 16, "metadata": {}, "outputs": [], "source": [ @@ -976,7 +985,7 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -1095,7 +1104,7 @@ " (Total: 12)" ] }, - "execution_count": 27, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } @@ -1106,7 +1115,7 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 18, "metadata": {}, "outputs": [], "source": [ @@ -1169,7 +1178,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 19, "metadata": {}, "outputs": [], "source": [ @@ -1195,7 +1204,7 @@ }, { "cell_type": "code", - "execution_count": 30, + "execution_count": 20, "metadata": {}, "outputs": [ { @@ -1205,21 +1214,21 @@ "\n", "%3\n", "\n", - "\n", + "\n", "\n", - "Session\n", - "\n", - "\n", - "Session\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", "\n", "\n", "\n", - "\n", + "\n", "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", + "Session\n", + "\n", + "\n", + "Session\n", "\n", "\n", "\n", @@ -1232,10 +1241,10 @@ "" ], "text/plain": [ - "" + "" ] }, - "execution_count": 30, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } @@ -1253,7 +1262,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": 21, "metadata": {}, "outputs": [], "source": [ @@ -1269,7 +1278,7 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": 22, "metadata": {}, "outputs": [ { @@ -1363,7 +1372,7 @@ " (Total: 1)" ] }, - "execution_count": 32, + "execution_count": 22, "metadata": {}, "output_type": "execute_result" } @@ -1381,7 +1390,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": 23, "metadata": {}, "outputs": [ { @@ -1480,7 +1489,7 @@ " (Total: 2)" ] }, - "execution_count": 33, + "execution_count": 23, "metadata": {}, "output_type": "execute_result" } @@ -1507,7 +1516,7 @@ }, { "cell_type": "code", - "execution_count": 34, + "execution_count": 24, "metadata": {}, "outputs": [], "source": [ @@ -1524,7 +1533,7 @@ }, { "cell_type": "code", - "execution_count": 35, + "execution_count": 25, "metadata": {}, "outputs": [ { @@ -1628,7 +1637,7 @@ " (Total: 3)" ] }, - "execution_count": 35, + "execution_count": 25, "metadata": {}, "output_type": "execute_result" } @@ -1646,7 +1655,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 26, "metadata": {}, "outputs": [], "source": [ @@ -1667,7 +1676,7 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 27, "metadata": { "scrolled": true }, @@ -1679,7 +1688,7 @@ "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mIntegrityError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[38], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Session\u001b[39m.\u001b[39;49minsert1(bad_data)\n", + "Cell \u001b[0;32mIn[27], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Session\u001b[39m.\u001b[39;49minsert1(bad_data)\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:440\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 424\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m 425\u001b[0m query \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39m{command}\u001b[39;00m\u001b[39m INTO \u001b[39m\u001b[39m{destination}\u001b[39;00m\u001b[39m(`\u001b[39m\u001b[39m{fields}\u001b[39;00m\u001b[39m`) VALUES \u001b[39m\u001b[39m{placeholders}\u001b[39;00m\u001b[39m{duplicate}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m 426\u001b[0m command\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mREPLACE\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mif\u001b[39;00m replace \u001b[39melse\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mINSERT\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m 427\u001b[0m destination\u001b[39m=\u001b[39m\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mfrom_clause(),\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 438\u001b[0m ),\n\u001b[1;32m 439\u001b[0m )\n\u001b[0;32m--> 440\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mconnection\u001b[39m.\u001b[39;49mquery(\n\u001b[1;32m 441\u001b[0m query,\n\u001b[1;32m 442\u001b[0m args\u001b[39m=\u001b[39;49m\u001b[39mlist\u001b[39;49m(\n\u001b[1;32m 443\u001b[0m itertools\u001b[39m.\u001b[39;49mchain\u001b[39m.\u001b[39;49mfrom_iterable(\n\u001b[1;32m 444\u001b[0m (v \u001b[39mfor\u001b[39;49;00m v \u001b[39min\u001b[39;49;00m r[\u001b[39m\"\u001b[39;49m\u001b[39mvalues\u001b[39;49m\u001b[39m\"\u001b[39;49m] \u001b[39mif\u001b[39;49;00m v \u001b[39mis\u001b[39;49;00m \u001b[39mnot\u001b[39;49;00m \u001b[39mNone\u001b[39;49;00m) \u001b[39mfor\u001b[39;49;00m r \u001b[39min\u001b[39;49;00m rows\n\u001b[1;32m 445\u001b[0m )\n\u001b[1;32m 446\u001b[0m ),\n\u001b[1;32m 447\u001b[0m )\n\u001b[1;32m 448\u001b[0m \u001b[39mexcept\u001b[39;00m UnknownAttributeError \u001b[39mas\u001b[39;00m err:\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n", "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/connection.py:340\u001b[0m, in \u001b[0;36mConnection.query\u001b[0;34m(self, query, args, as_dict, suppress_warnings, reconnect)\u001b[0m\n\u001b[1;32m 338\u001b[0m cursor \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_conn\u001b[39m.\u001b[39mcursor(cursor\u001b[39m=\u001b[39mcursor_class)\n\u001b[1;32m 339\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 340\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_execute_query(cursor, query, args, suppress_warnings)\n\u001b[1;32m 341\u001b[0m \u001b[39mexcept\u001b[39;00m errors\u001b[39m.\u001b[39mLostConnectionError:\n\u001b[1;32m 342\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m reconnect:\n", @@ -1752,7 +1761,7 @@ }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 28, "metadata": {}, "outputs": [ { @@ -1838,7 +1847,7 @@ " (Total: 1)" ] }, - "execution_count": 39, + "execution_count": 28, "metadata": {}, "output_type": "execute_result" } @@ -1856,7 +1865,7 @@ }, { "cell_type": "code", - "execution_count": 40, + "execution_count": 29, "metadata": {}, "outputs": [ { @@ -1951,7 +1960,7 @@ " (Total: 4)" ] }, - "execution_count": 40, + "execution_count": 29, "metadata": {}, "output_type": "execute_result" } @@ -1969,7 +1978,7 @@ }, { "cell_type": "code", - "execution_count": 41, + "execution_count": 30, "metadata": {}, "outputs": [ { @@ -2073,7 +2082,7 @@ " (Total: 7)" ] }, - "execution_count": 41, + "execution_count": 30, "metadata": {}, "output_type": "execute_result" } @@ -2091,7 +2100,7 @@ }, { "cell_type": "code", - "execution_count": 42, + "execution_count": 31, "metadata": {}, "outputs": [ { @@ -2177,7 +2186,7 @@ " (Total: 1)" ] }, - "execution_count": 42, + "execution_count": 31, "metadata": {}, "output_type": "execute_result" } @@ -2209,7 +2218,7 @@ }, { "cell_type": "code", - "execution_count": 43, + "execution_count": 32, "metadata": {}, "outputs": [ { @@ -2316,7 +2325,7 @@ " (Total: 8)" ] }, - "execution_count": 43, + "execution_count": 32, "metadata": {}, "output_type": "execute_result" } @@ -2334,7 +2343,7 @@ }, { "cell_type": "code", - "execution_count": 44, + "execution_count": 33, "metadata": {}, "outputs": [ { @@ -2429,7 +2438,7 @@ " (Total: 4)" ] }, - "execution_count": 44, + "execution_count": 33, "metadata": {}, "output_type": "execute_result" } @@ -2447,7 +2456,7 @@ }, { "cell_type": "code", - "execution_count": 45, + "execution_count": 34, "metadata": {}, "outputs": [ { @@ -2554,7 +2563,7 @@ " (Total: 8)" ] }, - "execution_count": 45, + "execution_count": 34, "metadata": {}, "output_type": "execute_result" } @@ -2579,7 +2588,7 @@ }, { "cell_type": "code", - "execution_count": 46, + "execution_count": 35, "metadata": {}, "outputs": [ { @@ -2677,7 +2686,7 @@ " (Total: 5)" ] }, - "execution_count": 46, + "execution_count": 35, "metadata": {}, "output_type": "execute_result" } @@ -2688,7 +2697,7 @@ }, { "cell_type": "code", - "execution_count": 47, + "execution_count": 36, "metadata": {}, "outputs": [ { @@ -2786,7 +2795,7 @@ " (Total: 5)" ] }, - "execution_count": 47, + "execution_count": 36, "metadata": {}, "output_type": "execute_result" } @@ -2804,7 +2813,7 @@ }, { "cell_type": "code", - "execution_count": 48, + "execution_count": 37, "metadata": {}, "outputs": [ { @@ -2908,7 +2917,7 @@ " (Total: 7)" ] }, - "execution_count": 48, + "execution_count": 37, "metadata": {}, "output_type": "execute_result" } @@ -2927,7 +2936,7 @@ }, { "cell_type": "code", - "execution_count": 49, + "execution_count": 38, "metadata": {}, "outputs": [], "source": [ @@ -2964,7 +2973,7 @@ }, { "cell_type": "code", - "execution_count": 54, + "execution_count": 39, "metadata": {}, "outputs": [ { @@ -3053,7 +3062,7 @@ " (Total: 2)" ] }, - "execution_count": 54, + "execution_count": 39, "metadata": {}, "output_type": "execute_result" } @@ -3071,7 +3080,7 @@ }, { "cell_type": "code", - "execution_count": 55, + "execution_count": 40, "metadata": {}, "outputs": [ { @@ -3157,7 +3166,7 @@ " (Total: 1)" ] }, - "execution_count": 55, + "execution_count": 40, "metadata": {}, "output_type": "execute_result" } @@ -3175,7 +3184,7 @@ }, { "cell_type": "code", - "execution_count": 56, + "execution_count": 41, "metadata": {}, "outputs": [ { @@ -3261,7 +3270,7 @@ " (Total: 1)" ] }, - "execution_count": 56, + "execution_count": 41, "metadata": {}, "output_type": "execute_result" } @@ -3286,7 +3295,7 @@ }, { "cell_type": "code", - "execution_count": 57, + "execution_count": 42, "metadata": {}, "outputs": [ { @@ -3399,7 +3408,7 @@ " (Total: 10)" ] }, - "execution_count": 57, + "execution_count": 42, "metadata": {}, "output_type": "execute_result" } @@ -3417,7 +3426,7 @@ }, { "cell_type": "code", - "execution_count": 58, + "execution_count": 43, "metadata": {}, "outputs": [], "source": [ @@ -3452,7 +3461,7 @@ }, { "cell_type": "code", - "execution_count": 59, + "execution_count": 44, "metadata": {}, "outputs": [ { @@ -3568,7 +3577,7 @@ " (Total: 3)" ] }, - "execution_count": 59, + "execution_count": 44, "metadata": {}, "output_type": "execute_result" } @@ -3594,7 +3603,7 @@ }, { "cell_type": "code", - "execution_count": 60, + "execution_count": 45, "metadata": {}, "outputs": [ { @@ -3696,7 +3705,7 @@ " (Total: 1)" ] }, - "execution_count": 60, + "execution_count": 45, "metadata": {}, "output_type": "execute_result" } @@ -3708,7 +3717,7 @@ }, { "cell_type": "code", - "execution_count": 61, + "execution_count": 46, "metadata": {}, "outputs": [ { @@ -3817,7 +3826,7 @@ " (Total: 2)" ] }, - "execution_count": 61, + "execution_count": 46, "metadata": {}, "output_type": "execute_result" } @@ -3843,7 +3852,7 @@ }, { "cell_type": "code", - "execution_count": 62, + "execution_count": 47, "metadata": {}, "outputs": [ { @@ -3947,7 +3956,7 @@ " (Total: 12)" ] }, - "execution_count": 62, + "execution_count": 47, "metadata": {}, "output_type": "execute_result" } @@ -3974,7 +3983,7 @@ }, { "cell_type": "code", - "execution_count": 63, + "execution_count": 48, "metadata": {}, "outputs": [ { @@ -4078,7 +4087,7 @@ " (Total: 12)" ] }, - "execution_count": 63, + "execution_count": 48, "metadata": {}, "output_type": "execute_result" } @@ -4097,7 +4106,7 @@ }, { "cell_type": "code", - "execution_count": 64, + "execution_count": 49, "metadata": {}, "outputs": [ { @@ -4189,7 +4198,7 @@ " (Total: 3)" ] }, - "execution_count": 64, + "execution_count": 49, "metadata": {}, "output_type": "execute_result" } @@ -4208,7 +4217,7 @@ }, { "cell_type": "code", - "execution_count": 65, + "execution_count": 50, "metadata": {}, "outputs": [ { @@ -4330,7 +4339,7 @@ " (Total: 3)" ] }, - "execution_count": 65, + "execution_count": 50, "metadata": {}, "output_type": "execute_result" } @@ -4369,7 +4378,7 @@ }, { "cell_type": "code", - "execution_count": 66, + "execution_count": 51, "metadata": {}, "outputs": [ { @@ -4464,7 +4473,7 @@ " (Total: 4)" ] }, - "execution_count": 66, + "execution_count": 51, "metadata": {}, "output_type": "execute_result" } @@ -4483,7 +4492,7 @@ }, { "cell_type": "code", - "execution_count": 68, + "execution_count": 52, "metadata": {}, "outputs": [ { @@ -4496,7 +4505,7 @@ " dtype=[('mouse_id', '\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 68, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "dj.Diagram(schema)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Remember, again, that after running the following method, you will be asked to confirm to commit the delete:" + ] + }, { "cell_type": "code", - "execution_count": null, + "execution_count": 69, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "[2023-07-26 20:14:02,741][INFO]: `tutorial`.`session` (3 tuples)\n" + ] + } + ], "source": [ "Session.drop()" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 70, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 70, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "dj.Diagram(schema)" ] }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": {}, + "outputs": [], + "source": [ + "schema.drop()" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -5055,12 +5162,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! You have successfully created your first DataJoint pipeline, using dependencies to establish the link among the tables. You have also learned to query and fetch the data.\n", + "Congratulations! You have successfully created your first DataJoint pipeline, using dependencies to establish the link among the tables. You have also learned to query, fetch and delete the data.\n", "\n", "In the next session, we will extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", "\n", "\n", - "Please, continue to the next notebook \"02-Calcium Imaging.ipynb\"." + "Please, continue to the next notebook `02-Calcium Imaging.ipynb`." ] } ], From c16498146444f01a1b9bd7cc7e6c5f4389cf11a2 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 20:26:34 +0000 Subject: [PATCH 31/70] 01-Basics: new section 6.Drop listed --- notebooks/01-DataJoint Basics.ipynb | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/notebooks/01-DataJoint Basics.ipynb b/notebooks/01-DataJoint Basics.ipynb index 68e9aec..2782058 100644 --- a/notebooks/01-DataJoint Basics.ipynb +++ b/notebooks/01-DataJoint Basics.ipynb @@ -1727,8 +1727,9 @@ "* 1. Restriction (`&`) and negative restriction (`-`): filter the data with certain conditions\n", "* 2. Joining (`*`): bring fields from different tables together\n", "* 3. Projection (`.proj()`): focus on a subset of attributes\n", - "* 4. Fetching (`.fetch()`): pull the data from the database\n", - "* 5. Deletion (`.delete()`): delete entries and their dependencies" + "* 4. Fetch (`.fetch()`): pull the data from the database\n", + "* 5. Deletion (`.delete()`): delete entries and their dependencies\n", + "* 6. Drop (`.drop()`): drop the table from the schema" ] }, { @@ -4352,7 +4353,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 4. Fetching (`.fetch()`): pull the data from the database" + "### 4. Fetch (`.fetch()`): pull the data from the database" ] }, { @@ -4911,7 +4912,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 5. Deletion (`.delete()`) - delete entries in the table " + "### 5. Deletion (`.delete()`): delete entries in the table " ] }, { @@ -5004,7 +5005,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 6. Drop (`.drop()`) - drop the table from the schema" + "### 6. Drop (`.drop()`): drop the table from the schema" ] }, { From 3cfea317864500c68f24e0f679d5e982ca54a30c Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 20:31:21 +0000 Subject: [PATCH 32/70] 02Tut: section added,code moved from mouse_session --- .../02-Calcium Imaging Imported Tables.ipynb | 282 +++++++++++++----- 1 file changed, 203 insertions(+), 79 deletions(-) diff --git a/notebooks/02-Calcium Imaging Imported Tables.ipynb b/notebooks/02-Calcium Imaging Imported Tables.ipynb index 2e0fb93..0e46a2c 100644 --- a/notebooks/02-Calcium Imaging Imported Tables.ipynb +++ b/notebooks/02-Calcium Imaging Imported Tables.ipynb @@ -36,7 +36,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -52,7 +52,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -65,36 +65,185 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Schema, Mouse & Session" + "## Calcium imaging dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "We will continue working with the tables that we defined in the previous session. To do so, we will need the classes for each table: `Mouse` and `Session`. \n", + "In the `data` folder in this `DataJoint-Tutorials`, you can find a small dataset of three different cases of calcium imaging scans:\n", "\n", - "We can either redefine it here, but for your convenience, we have included the schema and table class definitions in a package called `tutorial_pipeline.mouse_session`, from which you can import the classes as well as the schema object. \n", + "- `/workspaces/datajoint-tutorials/data/example_scan_02.tif`\n", + "- `/workspaces/datajoint-tutorials/data/example_scan_03.tif`\n", + "- `/workspaces/datajoint-tutorials/data/example_scan_01.tif`\n", "\n", - "We will use the schema object again to define more tables." + "As you might know, calcium imaging scans (raw data) are stored as *.tif* files. \n", + "\n", + "*NOTE: For this tutorial there is no need to deeper explore this small dataset. Nevertheless, if you are curious about visualizing these example scans, we recommend you to open the TIFF with [ImageJ](https://imagej.nih.gov/ij/download.html).*" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Pipeline design: Schema, Mouse & Session" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The DataJoint pipeline commonly starts with a `schema` and the following classes for each table: `Mouse` and `Session`. Let's quickly create this pipeline's first steps as we learned it in the previous session:" ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 4, "metadata": {}, "outputs": [ { - "name": "stderr", - "output_type": "stream", - "text": [ - "[2023-07-25 20:54:56,914][INFO]: Connecting root@fakeservices.datajoint.io:3306\n", - "[2023-07-25 20:54:56,935][INFO]: Connected root@fakeservices.datajoint.io:3306\n" + "ename": "NameError", + "evalue": "name 'Mouse' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39mdelete()\n", + "\u001b[0;31mNameError\u001b[0m: name 'Mouse' is not defined" ] } ], "source": [ - "from tutorial_pipeline.mouse_session import schema, Mouse, Session" + "Mouse.delete()" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "schema = dj.schema('tutorial')" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class Mouse(dj.Manual):\n", + " definition = \"\"\"\n", + " # Experimental animals\n", + " mouse_id : int # Unique animal ID\n", + " ---\n", + " dob=null : date # date of birth\n", + " sex=\"unknown\" : enum('M','F','unknown') # sex\n", + " \"\"\"\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Insert the following example data into the table:" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [], + "source": [ + "mouse_data = [\n", + " {'dob': \"2017-03-01\", 'mouse_id': 0, 'sex': 'M'},\n", + " {'dob': \"2016-11-19\", 'mouse_id': 1, 'sex': 'M'},\n", + " {'dob': \"2016-11-20\", 'mouse_id': 2, 'sex': 'unknown'},\n", + " {'dob': \"2016-12-25\", 'mouse_id': 5, 'sex': 'F'},\n", + " {'dob': \"2017-01-01\", 'mouse_id': 10, 'sex': 'F'},\n", + " {'dob': \"2017-01-03\", 'mouse_id': 11, 'sex': 'F'},\n", + " {'dob': \"2017-05-12\", 'mouse_id': 100, 'sex': 'F'}\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "Mouse.insert(mouse_data, skip_duplicates=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class Session(dj.Manual):\n", + " definition = \"\"\"\n", + " # Experiment session\n", + " -> Mouse\n", + " session_date : date # date\n", + " ---\n", + " experiment_setup : int # experiment setup ID\n", + " experimenter : varchar(100) # experimenter name\n", + " data_path='' : varchar(255) #\n", + " \"\"\"\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Insert the following example data into the table:" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "session_data = [\n", + " {'experiment_setup': 0,\n", + " 'experimenter': 'Edgar Y. Walker',\n", + " 'mouse_id': 0,\n", + " 'session_date': \"2017-05-15\",\n", + " 'data_path': data_dir.as_posix()\n", + " },\n", + " {'experiment_setup': 0,\n", + " 'experimenter': 'Edgar Y. Walker',\n", + " 'mouse_id': 0,\n", + " 'session_date': \"2017-05-19\",\n", + " 'data_path': data_dir.as_posix()\n", + " },\n", + " {'experiment_setup': 1,\n", + " 'experimenter': 'Fabian Sinz',\n", + " 'mouse_id': 5,\n", + " 'session_date': \"2017-01-05\",\n", + " 'data_path': data_dir.as_posix()\n", + " },\n", + " {'experiment_setup': 100,\n", + " 'experimenter': 'Jacob Reimer',\n", + " 'mouse_id': 100,\n", + " 'session_date': \"2017-05-25\",\n", + " 'data_path': data_dir.as_posix()\n", + " }\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "Session.insert(session_data, skip_duplicates=True)" ] }, { @@ -106,7 +255,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 29, "metadata": {}, "outputs": [ { @@ -189,12 +338,22 @@ "2017-01-01\n", "F11\n", "2017-01-03\n", - "F100\n", + "F12\n", + "2017-03-21\n", + "F18\n", + "2017-05-01\n", + "F19\n", + "2018-07-21\n", + "M22\n", + "2019-12-15\n", + "F34\n", + "2018-09-22\n", + "M100\n", "2017-05-12\n", "F \n", " \n", " \n", - "

Total: 7

\n", + "

Total: 12

\n", " " ], "text/plain": [ @@ -206,11 +365,16 @@ "5 2016-12-25 F \n", "10 2017-01-01 F \n", "11 2017-01-03 F \n", + "12 2017-03-21 F \n", + "18 2017-05-01 F \n", + "19 2018-07-21 M \n", + "22 2019-12-15 F \n", + "34 2018-09-22 M \n", "100 2017-05-12 F \n", - " (Total: 7)" + " (Total: 12)" ] }, - "execution_count": 4, + "execution_count": 29, "metadata": {}, "output_type": "execute_result" } @@ -221,7 +385,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 30, "metadata": {}, "outputs": [ { @@ -302,35 +466,45 @@ "2017-05-15\n", "0\n", "Edgar Y. Walker\n", - "/workspaces/datajoint-tutorials/data0\n", + "0\n", "2017-05-19\n", "0\n", "Edgar Y. Walker\n", - "/workspaces/datajoint-tutorials/data5\n", + "/workspaces/datajoint-tutorials/data0\n", + "2018-01-15\n", + "100\n", + "Jacob Reimer\n", + "5\n", "2017-01-05\n", "1\n", "Fabian Sinz\n", - "/workspaces/datajoint-tutorials/data100\n", + "/workspaces/datajoint-tutorials/data18\n", + "2018-01-15\n", + "101\n", + "Jacob Reimer\n", + "100\n", "2017-05-25\n", "100\n", "Jacob Reimer\n", "/workspaces/datajoint-tutorials/data \n", " \n", " \n", - "

Total: 4

\n", + "

Total: 6

\n", " " ], "text/plain": [ "*mouse_id *session_date experiment_set experimenter data_path \n", "+----------+ +------------+ +------------+ +------------+ +------------+\n", - "0 2017-05-15 0 Edgar Y. Walke /workspaces/da\n", + "0 2017-05-15 0 Edgar Y. Walke \n", "0 2017-05-19 0 Edgar Y. Walke /workspaces/da\n", + "0 2018-01-15 100 Jacob Reimer \n", "5 2017-01-05 1 Fabian Sinz /workspaces/da\n", + "18 2018-01-15 101 Jacob Reimer \n", "100 2017-05-25 100 Jacob Reimer /workspaces/da\n", - " (Total: 4)" + " (Total: 6)" ] }, - "execution_count": 6, + "execution_count": 30, "metadata": {}, "output_type": "execute_result" } @@ -465,62 +639,12 @@ "The underline `____` indicates **additional primary key attribute(s)** apart from the ones inherited from its parents." ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Calcium imaging datasets" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In the `data` folder in this `DataJoint-Tutorials`, you can find three calcium imaging datasets. Raw data are stored as *.tif* files.\n", - "\n", - "Let's list these three files:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "PosixPath('/workspaces/datajoint-tutorials/data')" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from tutorial_pipeline import data_dir\n", - "data_dir" - ] - }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "/workspaces/datajoint-tutorials/data/example_scan_02.tif\n", - "/workspaces/datajoint-tutorials/data/example_scan_03.tif\n", - "/workspaces/datajoint-tutorials/data/example_scan_01.tif\n" - ] - } - ], - "source": [ - "for f in data_dir.glob('*.tif'):\n", - " print(f)" - ] + "outputs": [], + "source": [] }, { "cell_type": "code", From 3f7f66ed37409dd6615d402873a1777f1506e9d2 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 20:38:56 +0000 Subject: [PATCH 33/70] 'notebooks' directory renamed to 'tutorials' --- {notebooks => tutorials}/01-DataJoint Basics.ipynb | 0 {notebooks => tutorials}/02-Calcium Imaging Imported Tables.ipynb | 0 {notebooks => tutorials}/03-Calcium Imaging Computed Tables.ipynb | 0 .../04-Electrophysiology Imported Tables.ipynb | 0 .../05-Electrophysiology Computed Tables.ipynb | 0 5 files changed, 0 insertions(+), 0 deletions(-) rename {notebooks => tutorials}/01-DataJoint Basics.ipynb (100%) rename {notebooks => tutorials}/02-Calcium Imaging Imported Tables.ipynb (100%) rename {notebooks => tutorials}/03-Calcium Imaging Computed Tables.ipynb (100%) rename {notebooks => tutorials}/04-Electrophysiology Imported Tables.ipynb (100%) rename {notebooks => tutorials}/05-Electrophysiology Computed Tables.ipynb (100%) diff --git a/notebooks/01-DataJoint Basics.ipynb b/tutorials/01-DataJoint Basics.ipynb similarity index 100% rename from notebooks/01-DataJoint Basics.ipynb rename to tutorials/01-DataJoint Basics.ipynb diff --git a/notebooks/02-Calcium Imaging Imported Tables.ipynb b/tutorials/02-Calcium Imaging Imported Tables.ipynb similarity index 100% rename from notebooks/02-Calcium Imaging Imported Tables.ipynb rename to tutorials/02-Calcium Imaging Imported Tables.ipynb diff --git a/notebooks/03-Calcium Imaging Computed Tables.ipynb b/tutorials/03-Calcium Imaging Computed Tables.ipynb similarity index 100% rename from notebooks/03-Calcium Imaging Computed Tables.ipynb rename to tutorials/03-Calcium Imaging Computed Tables.ipynb diff --git a/notebooks/04-Electrophysiology Imported Tables.ipynb b/tutorials/04-Electrophysiology Imported Tables.ipynb similarity index 100% rename from notebooks/04-Electrophysiology Imported Tables.ipynb rename to tutorials/04-Electrophysiology Imported Tables.ipynb diff --git a/notebooks/05-Electrophysiology Computed Tables.ipynb b/tutorials/05-Electrophysiology Computed Tables.ipynb similarity index 100% rename from notebooks/05-Electrophysiology Computed Tables.ipynb rename to tutorials/05-Electrophysiology Computed Tables.ipynb From f0a0785ed83e213a9fe8b35152136217f690460f Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 20:43:12 +0000 Subject: [PATCH 34/70] 'completed_exercises' renamed --- .../01-DataJoint Basics.ipynb | 0 .../02-Calcium Imaging Imported Tables.ipynb | 0 .../03-Calcium ImagingComputed Tables.ipynb | 0 .../04-Electrophysiology Imported Tables.ipynb | 0 .../05-Electrophysiology Computed Tables.ipynb | 0 5 files changed, 0 insertions(+), 0 deletions(-) rename {completed_exercises => completed_tutorials}/01-DataJoint Basics.ipynb (100%) rename {completed_exercises => completed_tutorials}/02-Calcium Imaging Imported Tables.ipynb (100%) rename {completed_exercises => completed_tutorials}/03-Calcium ImagingComputed Tables.ipynb (100%) rename {completed_exercises => completed_tutorials}/04-Electrophysiology Imported Tables.ipynb (100%) rename {completed_exercises => completed_tutorials}/05-Electrophysiology Computed Tables.ipynb (100%) diff --git a/completed_exercises/01-DataJoint Basics.ipynb b/completed_tutorials/01-DataJoint Basics.ipynb similarity index 100% rename from completed_exercises/01-DataJoint Basics.ipynb rename to completed_tutorials/01-DataJoint Basics.ipynb diff --git a/completed_exercises/02-Calcium Imaging Imported Tables.ipynb b/completed_tutorials/02-Calcium Imaging Imported Tables.ipynb similarity index 100% rename from completed_exercises/02-Calcium Imaging Imported Tables.ipynb rename to completed_tutorials/02-Calcium Imaging Imported Tables.ipynb diff --git a/completed_exercises/03-Calcium ImagingComputed Tables.ipynb b/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb similarity index 100% rename from completed_exercises/03-Calcium ImagingComputed Tables.ipynb rename to completed_tutorials/03-Calcium ImagingComputed Tables.ipynb diff --git a/completed_exercises/04-Electrophysiology Imported Tables.ipynb b/completed_tutorials/04-Electrophysiology Imported Tables.ipynb similarity index 100% rename from completed_exercises/04-Electrophysiology Imported Tables.ipynb rename to completed_tutorials/04-Electrophysiology Imported Tables.ipynb diff --git a/completed_exercises/05-Electrophysiology Computed Tables.ipynb b/completed_tutorials/05-Electrophysiology Computed Tables.ipynb similarity index 100% rename from completed_exercises/05-Electrophysiology Computed Tables.ipynb rename to completed_tutorials/05-Electrophysiology Computed Tables.ipynb From fc3d04052e969ed14a059d61912e7c050dbc024f Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 20:46:26 +0000 Subject: [PATCH 35/70] Capitalized title in short tutorial file --- ...atajoint_in_30mins.ipynb => DataJoint tutorial in 30min.ipynb} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename short_tutorials/{datajoint_in_30mins.ipynb => DataJoint tutorial in 30min.ipynb} (100%) diff --git a/short_tutorials/datajoint_in_30mins.ipynb b/short_tutorials/DataJoint tutorial in 30min.ipynb similarity index 100% rename from short_tutorials/datajoint_in_30mins.ipynb rename to short_tutorials/DataJoint tutorial in 30min.ipynb From 1f9fe6fdcea3eb5af2f700095d5e2fc39a82b53b Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 20:49:22 +0000 Subject: [PATCH 36/70] README:instructions moved to cloud & local env --- README.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 025d9d5..b1f2448 100644 --- a/README.md +++ b/README.md @@ -66,6 +66,8 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - The start time for Codespaces is **~30s**. This will pull the built codespace from the cache when you need it. - *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of computing and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete Codespaces when not actively in use and recreate them when needed. We'll soon be creating prebuilds to avoid larger build times. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. - *Tip*: GitHub auto names the Codespaces, but you can rename the Codespaces so that it is easier to identify later. + - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. + - Once you are done, see the options in the menu in the bottom-left corner. In Codespaces, you can `Stop Current Codespace`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. - **Local environment**: - We highly recommend this option for users that who want to apply DataJoint to **their own neuroscience experiments** and lab research after exploring the tutorials. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have the following: @@ -73,16 +75,13 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - [Docker](https://docs.docker.com/get-docker/) - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - + - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. + - Once you are done, see the options in the menu in the bottom-left corner. When running DevContainer on your machine, you can `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. + For more detailed instructions, please check out the [User Guide in DataJoint Documentation](https://datajoint.com/docs/elements/user-guide/). Before we start, remember that all the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that interest you in keeping the changes. -### Instructions -- To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - -- Once you are done, see the options in the menu in the bottom-left corner. For example, in Codespaces you can `Stop Current Codespace`, but when running DevContainer on your machine the equivalent option is `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. - ## Support If you need help getting started or run into any errors, please open a GitHub Issue or contact our team by email at support@datajoint.com. From 5c874c5359e12b7829236b60cfd071819b34d355 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Wed, 26 Jul 2023 23:43:22 +0000 Subject: [PATCH 37/70] 02Tut:new Mouse&Session,rel.path updated,no output --- .../02-Calcium Imaging Imported Tables.ipynb | 833 ++---------------- 1 file changed, 90 insertions(+), 743 deletions(-) diff --git a/tutorials/02-Calcium Imaging Imported Tables.ipynb b/tutorials/02-Calcium Imaging Imported Tables.ipynb index 0e46a2c..e70ec6d 100644 --- a/tutorials/02-Calcium Imaging Imported Tables.ipynb +++ b/tutorials/02-Calcium Imaging Imported Tables.ipynb @@ -36,7 +36,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -52,7 +52,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -87,7 +87,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Pipeline design: Schema, Mouse & Session" + "## First steps of the pipeline design: Schema, Mouse & Session" ] }, { @@ -99,40 +99,12 @@ }, { "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "ename": "NameError", - "evalue": "name 'Mouse' is not defined", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39mdelete()\n", - "\u001b[0;31mNameError\u001b[0m: name 'Mouse' is not defined" - ] - } - ], - "source": [ - "Mouse.delete()" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "metadata": {}, - "outputs": [], - "source": [ - "schema = dj.schema('tutorial')" - ] - }, - { - "cell_type": "code", - "execution_count": 23, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ + "schema = dj.schema('tutorial')\n", + "\n", "@schema\n", "class Mouse(dj.Manual):\n", " definition = \"\"\"\n", @@ -141,48 +113,8 @@ " ---\n", " dob=null : date # date of birth\n", " sex=\"unknown\" : enum('M','F','unknown') # sex\n", - " \"\"\"\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Insert the following example data into the table:" - ] - }, - { - "cell_type": "code", - "execution_count": 24, - "metadata": {}, - "outputs": [], - "source": [ - "mouse_data = [\n", - " {'dob': \"2017-03-01\", 'mouse_id': 0, 'sex': 'M'},\n", - " {'dob': \"2016-11-19\", 'mouse_id': 1, 'sex': 'M'},\n", - " {'dob': \"2016-11-20\", 'mouse_id': 2, 'sex': 'unknown'},\n", - " {'dob': \"2016-12-25\", 'mouse_id': 5, 'sex': 'F'},\n", - " {'dob': \"2017-01-01\", 'mouse_id': 10, 'sex': 'F'},\n", - " {'dob': \"2017-01-03\", 'mouse_id': 11, 'sex': 'F'},\n", - " {'dob': \"2017-05-12\", 'mouse_id': 100, 'sex': 'F'}\n", - "]" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": {}, - "outputs": [], - "source": [ - "Mouse.insert(mouse_data, skip_duplicates=True)" - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": {}, - "outputs": [], - "source": [ + " \"\"\"\n", + "\n", "@schema\n", "class Session(dj.Manual):\n", " definition = \"\"\"\n", @@ -192,57 +124,47 @@ " ---\n", " experiment_setup : int # experiment setup ID\n", " experimenter : varchar(100) # experimenter name\n", - " data_path='' : varchar(255) #\n", - " \"\"\"\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Insert the following example data into the table:" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": {}, - "outputs": [], - "source": [ + " data_path='' : varchar(255) # relative path\n", + " \"\"\"\n", + "\n", + "mouse_data = [\n", + " {'dob': \"2017-03-01\", 'mouse_id': 0, 'sex': 'M'},\n", + " {'dob': \"2016-11-19\", 'mouse_id': 1, 'sex': 'M'},\n", + " {'dob': \"2016-11-20\", 'mouse_id': 2, 'sex': 'unknown'},\n", + " {'dob': \"2016-12-25\", 'mouse_id': 5, 'sex': 'F'},\n", + " {'dob': \"2017-01-01\", 'mouse_id': 10, 'sex': 'F'},\n", + " {'dob': \"2017-01-03\", 'mouse_id': 11, 'sex': 'F'},\n", + " {'dob': \"2017-05-12\", 'mouse_id': 100, 'sex': 'F'}\n", + "]\n", + "\n", "session_data = [\n", " {'experiment_setup': 0,\n", " 'experimenter': 'Edgar Y. Walker',\n", " 'mouse_id': 0,\n", " 'session_date': \"2017-05-15\",\n", - " 'data_path': data_dir.as_posix()\n", + " 'data_path': '../data/'\n", " },\n", " {'experiment_setup': 0,\n", " 'experimenter': 'Edgar Y. Walker',\n", " 'mouse_id': 0,\n", " 'session_date': \"2017-05-19\",\n", - " 'data_path': data_dir.as_posix()\n", + " 'data_path': '../data/'\n", " },\n", " {'experiment_setup': 1,\n", " 'experimenter': 'Fabian Sinz',\n", " 'mouse_id': 5,\n", " 'session_date': \"2017-01-05\",\n", - " 'data_path': data_dir.as_posix()\n", + " 'data_path': '../data/'\n", " },\n", " {'experiment_setup': 100,\n", " 'experimenter': 'Jacob Reimer',\n", " 'mouse_id': 100,\n", " 'session_date': \"2017-05-25\",\n", - " 'data_path': data_dir.as_posix()\n", + " 'data_path': '../data/'\n", " }\n", - "]" - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": {}, - "outputs": [], - "source": [ + "]\n", + "\n", + "Mouse.insert(mouse_data, skip_duplicates=True)\n", "Session.insert(session_data, skip_duplicates=True)" ] }, @@ -255,288 +177,39 @@ }, { "cell_type": "code", - "execution_count": 29, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", - " \n", - "

Total: 12

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +---------+\n", - "0 2017-03-01 M \n", - "1 2016-11-19 M \n", - "2 2016-11-20 unknown \n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "19 2018-07-21 M \n", - "22 2019-12-15 F \n", - "34 2018-09-22 M \n", - "100 2017-05-12 F \n", - " (Total: 12)" - ] - }, - "execution_count": 29, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse()" ] }, { "cell_type": "code", - "execution_count": 30, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experiment session\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-150Edgar Y. Walker
02017-05-190Edgar Y. Walker/workspaces/datajoint-tutorials/data
02018-01-15100Jacob Reimer
52017-01-051Fabian Sinz/workspaces/datajoint-tutorials/data
182018-01-15101Jacob Reimer
1002017-05-25100Jacob Reimer/workspaces/datajoint-tutorials/data
\n", - " \n", - "

Total: 6

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +------------+ +------------+\n", - "0 2017-05-15 0 Edgar Y. Walke \n", - "0 2017-05-19 0 Edgar Y. Walke /workspaces/da\n", - "0 2018-01-15 100 Jacob Reimer \n", - "5 2017-01-05 1 Fabian Sinz /workspaces/da\n", - "18 2018-01-15 101 Jacob Reimer \n", - "100 2017-05-25 100 Jacob Reimer /workspaces/da\n", - " (Total: 6)" - ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Session()" - ] - }, - { - "cell_type": "markdown", + "execution_count": null, "metadata": {}, + "outputs": [], "source": [ - "The `mouse_session.py` file located in the folder `tutorial_pipeline` also fills each table with example data to make sure that we are all on the same page." + "Session()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Scan table and primary key attributes" + "## Define the Scan table and its primary key attributes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "First we define a table named `Scan` that describes a scanning in an experimental session of Calcium Imaging that will store the meta information of the scans." + "First we define a table named `Scan` that describes a scanning in an experimental session of Calcium Imaging. This table will store the scans' metadata." ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -567,65 +240,9 @@ }, { "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session->Scan\n", - "\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "dj.Diagram(schema)" ] @@ -644,137 +261,6 @@ "execution_count": null, "metadata": {}, "outputs": [], - "source": [] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

depth

\n", - " depth of this scan\n", - "
\n", - "

wavelength

\n", - " wavelength used\n", - "
\n", - "

laser_power

\n", - " power of the laser used\n", - "
\n", - "

fps

\n", - " frames per second\n", - "
\n", - "

file_name

\n", - " name of the tif file\n", - "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
02017-05-153200.0920.024.015.0example_scan_02.tif
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", - "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", - "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", - "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", - "0 2017-05-15 3 200.0 920.0 24.0 15.0 example_scan_0\n", - " (Total: 3)" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], "source": [ "Scan()" ] @@ -783,27 +269,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now we manually `insert` the meta information of the datasets and their file names in the table `Scan`:" + "Now we manually `insert` the metadata of the datasets and their file names in the table `Scan`:" ] }, { "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "ename": "DuplicateError", - "evalue": "(\"Duplicate entry '0-2017-05-15-1' for key 'scan.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Scan\u001b[39m.\u001b[39;49minsert([\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39msession_date\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2017-05-15\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mscan_idx\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m1\u001b[39;49m, \n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdepth\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m150\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mwavelength\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m920\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mlaser_power\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m26\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfps\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m15\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfile_name\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mexample_scan_01.tif\u001b[39;49m\u001b[39m'\u001b[39;49m},\n\u001b[1;32m 4\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39msession_date\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2017-05-15\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mscan_idx\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m2\u001b[39;49m, \n\u001b[1;32m 5\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdepth\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m200\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mwavelength\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m920\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mlaser_power\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m24\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfps\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m15\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfile_name\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mexample_scan_02.tif\u001b[39;49m\u001b[39m'\u001b[39;49m},\n\u001b[1;32m 6\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39msession_date\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2017-05-15\u001b[39;49m\u001b[39m'\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mscan_idx\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m3\u001b[39;49m, \n\u001b[1;32m 7\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdepth\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m200\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mwavelength\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m920\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mlaser_power\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m24\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfps\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m15\u001b[39;49m, \u001b[39m'\u001b[39;49m\u001b[39mfile_name\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mexample_scan_03.tif\u001b[39;49m\u001b[39m'\u001b[39;49m} \n\u001b[1;32m 8\u001b[0m ])\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", - "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0-2017-05-15-1' for key 'scan.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Scan.insert([\n", " {'mouse_id': 0, 'session_date': '2017-05-15', 'scan_idx': 1, \n", @@ -817,169 +290,32 @@ }, { "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

depth

\n", - " depth of this scan\n", - "
\n", - "

wavelength

\n", - " wavelength used\n", - "
\n", - "

laser_power

\n", - " power of the laser used\n", - "
\n", - "

fps

\n", - " frames per second\n", - "
\n", - "

file_name

\n", - " name of the tif file\n", - "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
02017-05-153200.0920.024.015.0example_scan_02.tif
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", - "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", - "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", - "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", - "0 2017-05-15 3 200.0 920.0 24.0 15.0 example_scan_0\n", - " (Total: 3)" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Scan()" - ] - }, - { - "cell_type": "markdown", + "execution_count": null, "metadata": {}, + "outputs": [], "source": [ - "## Looking into the raw data" + "Scan()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's first load and look at the shape (frames, height and width) of one of the example TIFF of the calcium imaging dataset:" + "### Calculation of the average frame\n", + "\n", + "Let's first load and look at the number of frames of the calcium imaging TIF files:" ] }, { "cell_type": "code", - "execution_count": 18, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "(100, 128, 128)\n" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "import os\n", "from skimage import io\n", - "im = io.imread(data_dir / 'example_scan_01.tif')\n", - "print(im.shape)" + "im = io.imread('../data/example_scan_01.tif')\n", + "print('Number of frames = ',im.shape[0])" ] }, { @@ -988,7 +324,8 @@ "source": [ "Particularly, this example contains 100 frames. \n", "\n", - "Let's take the average of the images over frames and look at it." + "Let's calculate the average of the images over the frames and plot the result.\n", + "\n" ] }, { @@ -997,23 +334,30 @@ "metadata": {}, "outputs": [], "source": [ - "# ENTER YOUR CODE! - compute the avg frame with np.mean of axis=0\n", - "avg_image = np.mean(im, axis=0)\n", - "plt.imshow(avg_image, cmap=plt.cm.gray)" + "# ENTER YOUR CODE! \n", + "av_frame = np.mean(im,axis=0)\n", + "plt.imshow(av_frame)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*TIP: compute the `average frame` of the `im` image using the mean function from NumPy (np.mean) with axis = 0. Then, plot the result with `imshow`*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Defining table for average fluorescence across frames" + "## Define the table for the average fluorescence " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's create a table `AverageFrame` to compute and save the average fluorescence. \n", + "Now let's create a table `AverageFrame` to compute and save the average fluorescence across the frames. \n", "\n", "For each scan, we have one average frame. Therefore, the table shares the exact same primary key as the table `Scan`" ] @@ -1060,7 +404,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We defined `average_frame` as a `longblob` so that it can store a NumPy array. This NumPy array will be imported and computed from the file corresponding to each scan." + "We defined `average_frame` as a `longblob`, which allow us to store a NumPy array. This NumPy array will be imported and computed from the file corresponding to each scan." ] }, { @@ -1083,7 +427,7 @@ "source": [ "In DataJoint, the tier of the table indicates **the nature of the data and the data source for the table**. So far we have encountered two table tiers: `Manual` and `Imported`, and we will encounter the two other major tiers in this session. \n", "\n", - "DataJoint tables in `Manual` tier, or simply **Manual tables** indicate that its contents are **manually** entered by either experimenters or a recording system, and its content **do not depend on external data files or other tables**. This is the most basic table type you will encounter, especially as the tables at the beggining of the pipeline. In the Diagram, `Manual` tables are depicted by green rectangles.\n", + "DataJoint tables in `Manual` tier, or simply **Manual tables**, indicate that its content is **manually** entered by either experimenters or a recording system, and its content **do not depend on external data files or other tables**. This is the most basic table type that you will encounter, especially as the tables at the beggining of the pipeline. In the Diagram, `Manual` tables are depicted by green rectangles.\n", "\n", "On the other hand, **Imported tables** are understood to pull data (or *import* data) from external data files, and come equipped with functionalities to perform this importing process automatically, as we will see shortly! In the Diagram, `Imported` tables are depicted by blue ellipses." ] @@ -1108,7 +452,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Rather than filling out the content of the table manually using `insert1` or `insert` methods, we are going to make use of the `make` and `populate` logic that comes with `Imported` tables to automatically figure out what needs to be imported and perform the import!" + "Rather than filling out the content of the table manually using `insert1` or `insert` methods, we are going to make use of the `make` and `populate` logic that comes with `Imported` tables. These two methods automatically figure it out what needs to be imported, and perform the import." ] }, { @@ -1122,18 +466,17 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "`Imported` table comes with a special method called `populate`. Let's try calling it." + "`Imported` table comes with a special method called `populate`. Let's call it for `AverageFrame`:\n", + "\n", + "*Note that the following code line is intended to generate a code error.*" ] }, { "cell_type": "code", "execution_count": null, - "metadata": { - "scrolled": true - }, + "metadata": {}, "outputs": [], "source": [ - "# ENTER YOUR CODE! - call `populate` on the table\n", "AverageFrame.populate()" ] }, @@ -1141,7 +484,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Notice that `populate` call complained that a method called `make` is not implemented. Let me show a simple `make` method that will help elucidate what this is all about." + "Notice that the `populate` call complained that a method called `make` is not implemented. Let me show you a simple `make` method that will help elucidate what this is all about." ] }, { @@ -1174,7 +517,7 @@ "metadata": {}, "outputs": [], "source": [ - "# ENTER YOUR CODE! - call `populate` on the table\n", + "# ENTER YOUR CODE! - call `populate` on the table AverageFrame\n", "AverageFrame.populate()" ] }, @@ -1263,14 +606,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Notice that we added the missing attribute information `average_frame` into the `key` dictionary, and finally **inserted the entry** into `self` = `AverageFrame` table. The `make` method's job is to create and insert a new entry corresponding to the `key` into this table!" + "Notice that we added the missing attribute information `average_frame` into the `key` dictionary, and finally **inserted the entry** into `self` (in this case, `self` corresponds to the `AverageFrame` table). The `make` method's job is to create and insert a new entry. This new entry corresponds to the `key` into this table." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Finally, let's go ahead and call `populate` to actually populate the `AverageFrame` table, filling it with data loaded and computed from data files!" + "Finally, let's go ahead and call `populate` to actually populate the `AverageFrame` table with the new content, i.e., filling the table with the data loaded and computed from data files!" ] }, { @@ -1350,7 +693,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can find all `Scan` without corresponding `AverageFrame` entry with the **negative restriction operator** `-`" + "We can find all the `Scan` entries without their corresponding `AverageFrame` entries using the **negative restriction operator** `-`" ] }, { @@ -1359,7 +702,7 @@ "metadata": {}, "outputs": [], "source": [ - "# select all Scan entries *without* a corresponding entry in AverageFrame\n", + "# select all the `Scan` entries *without* a corresponding entry in `AverageFrame`\n", "Scan - AverageFrame" ] }, @@ -1392,7 +735,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now instead of loading from the raw tif file, we are able fetch the average fluorescence image from this table." + "Now instead of loading from the raw *.tif* file, we are able to fetch the average fluorescence image from this table." ] }, { @@ -1417,7 +760,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! You have successfully extended your pipeline with a table to represent processed data (`AverageFrame` as `Imported` table), learned and implemented the `make()` and `populate()` call to load external data to your tables." + "Congratulations! You have successfully:\n", + "- Extended your pipeline with a new table to represent the processed data (`AverageFrame` as `Imported` table)\n", + "- Learned and implemented the `make()` and `populate()` calls to load the external data to your tables" ] }, { @@ -1433,9 +778,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "At this point, our pipeline contains the core elements with data populated, ready for further downstream analysis.\n", + "At this point, our pipeline contains the core elements with the data populated, ready for further downstream analysis.\n", "\n", - "In the next [session](./03-Computed%20Table,%20Lookup%20Table,%20and%20Part%20Table%20-%20Interactive.ipynb), we are going to introduce the concept of `Computed` table, and `Lookup` table, as well as learning to set up a automated computation routine." + "In the next session `03-Calcium Imaging Computed Tables`:\n", + "- We will introduce the concept of `Computed` table and `Lookup` table\n", + "- We will also set up an automated computation routine, essential to develop advanced data analyses in your experiments!" ] } ], From e4e249a05be44260d02bfc90ced7e985eca97bd0 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 27 Jul 2023 01:19:57 +0000 Subject: [PATCH 38/70] Mouse_session.py file deleted --- tutorial_pipeline/mouse_session.py | 76 ------------------------------ 1 file changed, 76 deletions(-) delete mode 100644 tutorial_pipeline/mouse_session.py diff --git a/tutorial_pipeline/mouse_session.py b/tutorial_pipeline/mouse_session.py deleted file mode 100644 index 92f251d..0000000 --- a/tutorial_pipeline/mouse_session.py +++ /dev/null @@ -1,76 +0,0 @@ -import datajoint as dj -from . import data_dir - - -# fail-safe user name retrieval -username = dj.conn().conn_info['user'] -schema = dj.schema('{}_tutorial_pipeline'.format(username)) - - -# Table definitions - -@schema -class Mouse(dj.Manual): - definition = """ - # Experimental animals - mouse_id : int # Unique animal ID - --- - dob=null : date # date of birth - sex="unknown" : enum('M','F','unknown') # sex - """ - - -@schema -class Session(dj.Manual): - definition = """ - # Experiment session - -> Mouse - session_date : date # date - --- - experiment_setup : int # experiment setup ID - experimenter : varchar(100) # experimenter name - data_path='' : varchar(255) # - """ - - -# Insert the following data into the table - -mouse_data = [ - {'dob': "2017-03-01", 'mouse_id': 0, 'sex': 'M'}, - {'dob': "2016-11-19", 'mouse_id': 1, 'sex': 'M'}, - {'dob': "2016-11-20", 'mouse_id': 2, 'sex': 'unknown'}, - {'dob': "2016-12-25", 'mouse_id': 5, 'sex': 'F'}, - {'dob': "2017-01-01", 'mouse_id': 10, 'sex': 'F'}, - {'dob': "2017-01-03", 'mouse_id': 11, 'sex': 'F'}, - {'dob': "2017-05-12", 'mouse_id': 100, 'sex': 'F'} -] - -session_data = [ - {'experiment_setup': 0, - 'experimenter': 'Edgar Y. Walker', - 'mouse_id': 0, - 'session_date': "2017-05-15", - 'data_path': data_dir.as_posix() - }, - {'experiment_setup': 0, - 'experimenter': 'Edgar Y. Walker', - 'mouse_id': 0, - 'session_date': "2017-05-19", - 'data_path': data_dir.as_posix() - }, - {'experiment_setup': 1, - 'experimenter': 'Fabian Sinz', - 'mouse_id': 5, - 'session_date': "2017-01-05", - 'data_path': data_dir.as_posix() - }, - {'experiment_setup': 100, - 'experimenter': 'Jacob Reimer', - 'mouse_id': 100, - 'session_date': "2017-05-25", - 'data_path': data_dir.as_posix() - } -] - -Mouse.insert(mouse_data, skip_duplicates=True) -Session.insert(session_data, skip_duplicates=True) \ No newline at end of file From 223d85c41458fab06dae9af770f61dbef86a7b88 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 27 Jul 2023 14:44:20 +0000 Subject: [PATCH 39/70] New folder 'images' with two example pipelines --- images/pipeline-calcium-imaging.svg | 172 ++++++++++++++++++++++++++ images/pipeline-electrophysiology.svg | 117 ++++++++++++++++++ 2 files changed, 289 insertions(+) create mode 100644 images/pipeline-calcium-imaging.svg create mode 100644 images/pipeline-electrophysiology.svg diff --git a/images/pipeline-calcium-imaging.svg b/images/pipeline-calcium-imaging.svg new file mode 100644 index 0000000..053c336 --- /dev/null +++ b/images/pipeline-calcium-imaging.svg @@ -0,0 +1,172 @@ + + +%3 + + + +Fluorescence.Trace + + +Fluorescence.Trace + + + + + +Fluorescence + + +Fluorescence + + + + + +Fluorescence->Fluorescence.Trace + + + + +Mouse + + +Mouse + + + + + +Session + + +Session + + + + + +Mouse->Session + + + + +Scan + + +Scan + + + + + +Session->Scan + + + + +Segmentation + + +Segmentation + + + + + +Segmentation->Fluorescence + + + + +Segmentation.Roi + + +Segmentation.Roi + + + + + +Segmentation->Segmentation.Roi + + + + +SegmentationParam + + +SegmentationParam + + + + + +SegmentationParam->Segmentation + + + + +Segmentation.Roi->Fluorescence.Trace + + + + +AverageFrame + + +AverageFrame + + + + + +AverageFrame->Segmentation + + + + +Scan->AverageFrame + + + + \ No newline at end of file diff --git a/images/pipeline-electrophysiology.svg b/images/pipeline-electrophysiology.svg new file mode 100644 index 0000000..8364ec2 --- /dev/null +++ b/images/pipeline-electrophysiology.svg @@ -0,0 +1,117 @@ + + +%3 + + +Spikes.Waveform + + +Spikes.Waveform + + + + +Neuron + + +Neuron + + + + +ActivityStatistics + + +ActivityStatistics + + + + +Neuron->ActivityStatistics + + + +Spikes + + +Spikes + + + + +Neuron->Spikes + + + +Spikes->Spikes.Waveform + + + +Mouse + + +Mouse + + + + +Session + + +Session + + + + +Mouse->Session + + + +SpikeDetectionParam + + +SpikeDetectionParam + + + + +SpikeDetectionParam->Spikes + + + +Session->Neuron + + + + \ No newline at end of file From 11ae9adb1e62a24a870cd1aeef5c68a62ff04339 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 27 Jul 2023 14:51:23 +0000 Subject: [PATCH 40/70] Minor update in the `basic operators` list --- README.md | 11 ++++------- tutorials/01-DataJoint Basics.ipynb | 4 ++-- 2 files changed, 6 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index b1f2448..d58143b 100644 --- a/README.md +++ b/README.md @@ -31,16 +31,13 @@ Here is a summary of the content that you can expect to have learned: - Table tiers (`Lookup`, `Manual`, `Imported`, `Computed`) - Insert entries and view entries in tables - Table dependency and data integrity - - Query operations + - Basic operations - Restriction - `&` - Join - `*` - Projection - `.proj()` - - Aggregation - `.aggr()` - - Fetch operations - - Retrieve everything - - Retrieve primary key - `.fetch("KEY")` - - Retrieve selective attributes - - Delete operations + - Fetch - `.fetch()` + - Deletion - `.delete()` + - Drop - `.drop()` - DataJoint advanced topics: pipeline automation (~1 hour) - `Imported` and `Computed` tables diff --git a/tutorials/01-DataJoint Basics.ipynb b/tutorials/01-DataJoint Basics.ipynb index 2782058..f067963 100644 --- a/tutorials/01-DataJoint Basics.ipynb +++ b/tutorials/01-DataJoint Basics.ipynb @@ -1725,7 +1725,7 @@ "source": [ "We will introduce significant types of queries used in DataJoint:\n", "* 1. Restriction (`&`) and negative restriction (`-`): filter the data with certain conditions\n", - "* 2. Joining (`*`): bring fields from different tables together\n", + "* 2. Join (`*`): bring fields from different tables together\n", "* 3. Projection (`.proj()`): focus on a subset of attributes\n", "* 4. Fetch (`.fetch()`): pull the data from the database\n", "* 5. Deletion (`.delete()`): delete entries and their dependencies\n", @@ -3438,7 +3438,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 2. Joining (`*`): bring fields from different tables together" + "### 2. Join (`*`): bring fields from different tables together" ] }, { From 136eb0e59eae45233bfd4116a077247f95686ab5 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 27 Jul 2023 15:00:29 +0000 Subject: [PATCH 41/70] New pipeline figures added in three notebooks --- tutorials/01-DataJoint Basics.ipynb | 4 ++-- tutorials/02-Calcium Imaging Imported Tables.ipynb | 2 ++ tutorials/04-Electrophysiology Imported Tables.ipynb | 4 +++- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/tutorials/01-DataJoint Basics.ipynb b/tutorials/01-DataJoint Basics.ipynb index f067963..8edd08c 100644 --- a/tutorials/01-DataJoint Basics.ipynb +++ b/tutorials/01-DataJoint Basics.ipynb @@ -87,9 +87,9 @@ "\n", "A [DataJoint pipeline](https://datajoint.com/docs/core/datajoint-python/0.14/concepts/terminology/) contains database table definitions, dependencies, and associated computations, together with the transformations underlying a DataJoint workflow. \n", "\n", - "The following figure is an example of a DataJoint pipeline from [Optogenetics Element](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", + "The following figure is an example pipeline using [DataJoint Element for Multi-photon Calcium Imaging](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", "\n", - "![pipeline](https://raw.githubusercontent.com/datajoint/element-optogenetics/main/images/pipeline.svg)\n", + "![pipeline](../images/pipeline-calcium-imaging.svg)\n", "\n", "A **well-designed data pipeline**: \n", "- 1. Collects, organizes, and stores **every relevant piece of information during the scientific research**\n", diff --git a/tutorials/02-Calcium Imaging Imported Tables.ipynb b/tutorials/02-Calcium Imaging Imported Tables.ipynb index e70ec6d..99faaee 100644 --- a/tutorials/02-Calcium Imaging Imported Tables.ipynb +++ b/tutorials/02-Calcium Imaging Imported Tables.ipynb @@ -14,6 +14,8 @@ "source": [ "Welcome back! The practical example of this session is Calcium Imaging! \n", "\n", + "![pipeline](../images/pipeline-calcium-imaging.svg)\n", + "\n", "During this session you will learn:\n", "\n", "* To import neuron imaging data from data files into an `Imported` table\n", diff --git a/tutorials/04-Electrophysiology Imported Tables.ipynb b/tutorials/04-Electrophysiology Imported Tables.ipynb index 5c3e85d..83a37d3 100644 --- a/tutorials/04-Electrophysiology Imported Tables.ipynb +++ b/tutorials/04-Electrophysiology Imported Tables.ipynb @@ -11,7 +11,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example. \n", + "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example.\n", + "\n", + "![pipeline](../images/pipeline-calcium-imaging.svg)\n", "\n", "In this session, we will learn to:\n", "\n", From 495681bfe6bd66256904e846daaf0b5bdd4716b0 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 27 Jul 2023 15:33:40 +0000 Subject: [PATCH 42/70] 01Tut: clear outputs --- tutorials/01-DataJoint Basics.ipynb | 3825 ++------------------------- 1 file changed, 183 insertions(+), 3642 deletions(-) diff --git a/tutorials/01-DataJoint Basics.ipynb b/tutorials/01-DataJoint Basics.ipynb index 8edd08c..9d07fdf 100644 --- a/tutorials/01-DataJoint Basics.ipynb +++ b/tutorials/01-DataJoint Basics.ipynb @@ -53,7 +53,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -307,18 +307,9 @@ }, { "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "[2023-07-26 20:13:43,743][INFO]: Connecting root@fakeservices.datajoint.io:3306\n", - "[2023-07-26 20:13:43,759][INFO]: Connected root@fakeservices.datajoint.io:3306\n" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "schema = dj.schema('tutorial')" ] @@ -346,7 +337,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -370,95 +361,9 @@ }, { "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +-----+ +-----+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse()" ] @@ -496,7 +401,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -505,97 +410,9 @@ }, { "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - " (Total: 1)" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse()" ] @@ -609,7 +426,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -622,7 +439,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -631,100 +448,9 @@ }, { "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
1002017-05-12F
\n", - " \n", - "

Total: 2

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - "100 2017-05-12 F \n", - " (Total: 2)" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse()" ] @@ -738,7 +464,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -751,7 +477,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -767,7 +493,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -782,115 +508,9 @@ }, { "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
1002017-05-12F
\n", - " \n", - "

Total: 7

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +---------+\n", - "0 2017-03-01 M \n", - "1 2016-11-19 M \n", - "2 2016-11-20 unknown \n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "100 2017-05-12 F \n", - " (Total: 7)" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse()" ] @@ -913,23 +533,9 @@ }, { "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "ename": "DuplicateError", - "evalue": "(\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mDuplicateError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39;49minsert1(\n\u001b[1;32m 2\u001b[0m {\u001b[39m'\u001b[39;49m\u001b[39mmouse_id\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m0\u001b[39;49m,\n\u001b[1;32m 3\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39mdob\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39m2018-01-01\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 4\u001b[0m \u001b[39m'\u001b[39;49m\u001b[39msex\u001b[39;49m\u001b[39m'\u001b[39;49m: \u001b[39m'\u001b[39;49m\u001b[39mM\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m 5\u001b[0m })\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:453\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n\u001b[1;32m 452\u001b[0m \u001b[39mexcept\u001b[39;00m DuplicateError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 453\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 454\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore duplicate entries in insert, set skip_duplicates=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 455\u001b[0m )\n", - "\u001b[0;31mDuplicateError\u001b[0m: (\"Duplicate entry '0' for key 'mouse.PRIMARY'\", 'To ignore duplicate entries in insert, set skip_duplicates=True')" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse.insert1(\n", "{'mouse_id': 0,\n", @@ -947,7 +553,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -967,7 +573,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -985,137 +591,16 @@ }, { "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", - " \n", - "

Total: 12

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +---------+\n", - "0 2017-03-01 M \n", - "1 2016-11-19 M \n", - "2 2016-11-20 unknown \n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "19 2018-07-21 M \n", - "22 2019-12-15 F \n", - "34 2018-09-22 M \n", - "100 2017-05-12 F \n", - " (Total: 12)" - ] - }, - "execution_count": 17, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse()" ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1178,7 +663,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1204,51 +689,9 @@ }, { "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "dj.Diagram(schema)" ] @@ -1262,7 +705,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1278,105 +721,9 @@ }, { "cell_type": "code", - "execution_count": 22, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experiment session\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-150Edgar Y. Walker
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +------------+ +-----------+\n", - "0 2017-05-15 0 Edgar Y. Walke \n", - " (Total: 1)" - ] - }, - "execution_count": 22, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Session()" ] @@ -1390,110 +737,9 @@ }, { "cell_type": "code", - "execution_count": 23, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experiment session\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-150Edgar Y. Walker
02018-01-15100Jacob Reimer
\n", - " \n", - "

Total: 2

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +------------+ +-----------+\n", - "0 2017-05-15 0 Edgar Y. Walke \n", - "0 2018-01-15 100 Jacob Reimer \n", - " (Total: 2)" - ] - }, - "execution_count": 23, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "data = {\n", " 'mouse_id': 0,\n", @@ -1516,7 +762,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1533,115 +779,9 @@ }, { "cell_type": "code", - "execution_count": 25, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experiment session\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-150Edgar Y. Walker
02018-01-15100Jacob Reimer
182018-01-15101Jacob Reimer
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +------------+ +-----------+\n", - "0 2017-05-15 0 Edgar Y. Walke \n", - "0 2018-01-15 100 Jacob Reimer \n", - "18 2018-01-15 101 Jacob Reimer \n", - " (Total: 3)" - ] - }, - "execution_count": 25, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Session()" ] @@ -1655,7 +795,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1676,27 +816,11 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "ename": "IntegrityError", - "evalue": "Cannot add or update a child row: a foreign key constraint fails (`tutorial`.`session`, CONSTRAINT `session_ibfk_1` FOREIGN KEY (`mouse_id`) REFERENCES `mouse` (`mouse_id`) ON DELETE RESTRICT ON UPDATE CASCADE)", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mIntegrityError\u001b[0m Traceback (most recent call last)", - "Cell \u001b[0;32mIn[27], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Session\u001b[39m.\u001b[39;49minsert1(bad_data)\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:337\u001b[0m, in \u001b[0;36mTable.insert1\u001b[0;34m(self, row, **kwargs)\u001b[0m\n\u001b[1;32m 330\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minsert1\u001b[39m(\u001b[39mself\u001b[39m, row, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 331\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m 332\u001b[0m \u001b[39m Insert one data record into the table. For ``kwargs``, see ``insert()``.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m \n\u001b[1;32m 334\u001b[0m \u001b[39m :param row: a numpy record, a dict-like object, or an ordered sequence to be inserted\u001b[39;00m\n\u001b[1;32m 335\u001b[0m \u001b[39m as one row.\u001b[39;00m\n\u001b[1;32m 336\u001b[0m \u001b[39m \"\"\"\u001b[39;00m\n\u001b[0;32m--> 337\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49minsert((row,), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/table.py:440\u001b[0m, in \u001b[0;36mTable.insert\u001b[0;34m(self, rows, replace, skip_duplicates, ignore_extra_fields, allow_direct_insert)\u001b[0m\n\u001b[1;32m 424\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m 425\u001b[0m query \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39m{command}\u001b[39;00m\u001b[39m INTO \u001b[39m\u001b[39m{destination}\u001b[39;00m\u001b[39m(`\u001b[39m\u001b[39m{fields}\u001b[39;00m\u001b[39m`) VALUES \u001b[39m\u001b[39m{placeholders}\u001b[39;00m\u001b[39m{duplicate}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m 426\u001b[0m command\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mREPLACE\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mif\u001b[39;00m replace \u001b[39melse\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mINSERT\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[1;32m 427\u001b[0m destination\u001b[39m=\u001b[39m\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mfrom_clause(),\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 438\u001b[0m ),\n\u001b[1;32m 439\u001b[0m )\n\u001b[0;32m--> 440\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mconnection\u001b[39m.\u001b[39;49mquery(\n\u001b[1;32m 441\u001b[0m query,\n\u001b[1;32m 442\u001b[0m args\u001b[39m=\u001b[39;49m\u001b[39mlist\u001b[39;49m(\n\u001b[1;32m 443\u001b[0m itertools\u001b[39m.\u001b[39;49mchain\u001b[39m.\u001b[39;49mfrom_iterable(\n\u001b[1;32m 444\u001b[0m (v \u001b[39mfor\u001b[39;49;00m v \u001b[39min\u001b[39;49;00m r[\u001b[39m\"\u001b[39;49m\u001b[39mvalues\u001b[39;49m\u001b[39m\"\u001b[39;49m] \u001b[39mif\u001b[39;49;00m v \u001b[39mis\u001b[39;49;00m \u001b[39mnot\u001b[39;49;00m \u001b[39mNone\u001b[39;49;00m) \u001b[39mfor\u001b[39;49;00m r \u001b[39min\u001b[39;49;00m rows\n\u001b[1;32m 445\u001b[0m )\n\u001b[1;32m 446\u001b[0m ),\n\u001b[1;32m 447\u001b[0m )\n\u001b[1;32m 448\u001b[0m \u001b[39mexcept\u001b[39;00m UnknownAttributeError \u001b[39mas\u001b[39;00m err:\n\u001b[1;32m 449\u001b[0m \u001b[39mraise\u001b[39;00m err\u001b[39m.\u001b[39msuggest(\n\u001b[1;32m 450\u001b[0m \u001b[39m\"\u001b[39m\u001b[39mTo ignore extra fields in insert, set ignore_extra_fields=True\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m 451\u001b[0m )\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/connection.py:340\u001b[0m, in \u001b[0;36mConnection.query\u001b[0;34m(self, query, args, as_dict, suppress_warnings, reconnect)\u001b[0m\n\u001b[1;32m 338\u001b[0m cursor \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_conn\u001b[39m.\u001b[39mcursor(cursor\u001b[39m=\u001b[39mcursor_class)\n\u001b[1;32m 339\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 340\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_execute_query(cursor, query, args, suppress_warnings)\n\u001b[1;32m 341\u001b[0m \u001b[39mexcept\u001b[39;00m errors\u001b[39m.\u001b[39mLostConnectionError:\n\u001b[1;32m 342\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m reconnect:\n", - "File \u001b[0;32m/usr/local/lib/python3.9/site-packages/datajoint/connection.py:296\u001b[0m, in \u001b[0;36mConnection._execute_query\u001b[0;34m(cursor, query, args, suppress_warnings)\u001b[0m\n\u001b[1;32m 294\u001b[0m cursor\u001b[39m.\u001b[39mexecute(query, args)\n\u001b[1;32m 295\u001b[0m \u001b[39mexcept\u001b[39;00m client\u001b[39m.\u001b[39merr\u001b[39m.\u001b[39mError \u001b[39mas\u001b[39;00m err:\n\u001b[0;32m--> 296\u001b[0m \u001b[39mraise\u001b[39;00m translate_query_error(err, query)\n", - "\u001b[0;31mIntegrityError\u001b[0m: Cannot add or update a child row: a foreign key constraint fails (`tutorial`.`session`, CONSTRAINT `session_ibfk_1` FOREIGN KEY (`mouse_id`) REFERENCES `mouse` (`mouse_id`) ON DELETE RESTRICT ON UPDATE CASCADE)" - ] - } - ], + "outputs": [], "source": [ "Session.insert1(bad_data)" ] @@ -1762,97 +886,9 @@ }, { "cell_type": "code", - "execution_count": 28, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - " (Total: 1)" - ] - }, - "execution_count": 28, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'mouse_id = 0'" ] @@ -1866,106 +902,9 @@ }, { "cell_type": "code", - "execution_count": 29, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
12016-11-19M
192018-07-21M
342018-09-22M
\n", - " \n", - "

Total: 4

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - "1 2016-11-19 M \n", - "19 2018-07-21 M \n", - "34 2018-09-22 M \n", - " (Total: 4)" - ] - }, - "execution_count": 29, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'sex = \"M\"'" ] @@ -1979,115 +918,9 @@ }, { "cell_type": "code", - "execution_count": 30, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", - " \n", - "

Total: 7

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "22 2019-12-15 F \n", - "100 2017-05-12 F \n", - " (Total: 7)" - ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'sex = \"F\"'" ] @@ -2101,97 +934,9 @@ }, { "cell_type": "code", - "execution_count": 31, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
52016-12-25F
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "5 2016-12-25 F \n", - " (Total: 1)" - ] - }, - "execution_count": 31, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & dict(mouse_id=5)" ] @@ -2219,118 +964,9 @@ }, { "cell_type": "code", - "execution_count": 32, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
112017-01-03F
122017-03-21F
182017-05-01F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", - " \n", - "

Total: 8

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "19 2018-07-21 M \n", - "22 2019-12-15 F \n", - "34 2018-09-22 M \n", - "100 2017-05-12 F \n", - " (Total: 8)" - ] - }, - "execution_count": 32, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'dob > \"2017-01-01\"'" ] @@ -2344,106 +980,9 @@ }, { "cell_type": "code", - "execution_count": 33, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
122017-03-21F
182017-05-01F
1002017-05-12F
\n", - " \n", - "

Total: 4

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "100 2017-05-12 F \n", - " (Total: 4)" - ] - }, - "execution_count": 33, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'dob between \"2017-03-01\" and \"2017-08-23\"'" ] @@ -2457,118 +996,9 @@ }, { "cell_type": "code", - "execution_count": 34, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", - " \n", - "

Total: 8

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +---------+\n", - "2 2016-11-20 unknown \n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "22 2019-12-15 F \n", - "100 2017-05-12 F \n", - " (Total: 8)" - ] - }, - "execution_count": 34, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'sex != \"M\"'" ] @@ -2589,218 +1019,18 @@ }, { "cell_type": "code", - "execution_count": 35, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", - " \n", - "

Total: 5

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "22 2019-12-15 F \n", - "100 2017-05-12 F \n", - " (Total: 5)" - ] - }, - "execution_count": 35, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'sex != \"M\"' & 'dob > \"2017-01-01\"'" ] }, { "cell_type": "code", - "execution_count": 36, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", - " \n", - "

Total: 5

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "22 2019-12-15 F \n", - "100 2017-05-12 F \n", - " (Total: 5)" - ] - }, - "execution_count": 36, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & 'sex != \"M\" and dob > \"2017-01-01\"'" ] @@ -2814,115 +1044,9 @@ }, { "cell_type": "code", - "execution_count": 37, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
182017-05-01F
222019-12-15F
1002017-05-12F
\n", - " \n", - "

Total: 7

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "18 2017-05-01 F \n", - "22 2019-12-15 F \n", - "100 2017-05-12 F \n", - " (Total: 7)" - ] - }, - "execution_count": 37, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "female_mice = Mouse & 'sex = \"F\"'\n", "female_mice" @@ -2937,7 +1061,7 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -2974,100 +1098,9 @@ }, { "cell_type": "code", - "execution_count": 39, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
182017-05-01F
\n", - " \n", - "

Total: 2

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - "18 2017-05-01 F \n", - " (Total: 2)" - ] - }, - "execution_count": 39, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & Session " ] @@ -3081,97 +1114,9 @@ }, { "cell_type": "code", - "execution_count": 40, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - " (Total: 1)" - ] - }, - "execution_count": 40, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & Session & 'sex = \"M\"'" ] @@ -3185,97 +1130,9 @@ }, { "cell_type": "code", - "execution_count": 41, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - " (Total: 1)" - ] - }, - "execution_count": 41, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse & (Session & 'session_date <= \"2017-05-19\"')" ] @@ -3296,124 +1153,9 @@ }, { "cell_type": "code", - "execution_count": 42, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
12016-11-19M
22016-11-20unknown
52016-12-25F
102017-01-01F
112017-01-03F
122017-03-21F
192018-07-21M
222019-12-15F
342018-09-22M
1002017-05-12F
\n", - " \n", - "

Total: 10

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +---------+\n", - "1 2016-11-19 M \n", - "2 2016-11-20 unknown \n", - "5 2016-12-25 F \n", - "10 2017-01-01 F \n", - "11 2017-01-03 F \n", - "12 2017-03-21 F \n", - "19 2018-07-21 M \n", - "22 2019-12-15 F \n", - "34 2018-09-22 M \n", - "100 2017-05-12 F \n", - " (Total: 10)" - ] - }, - "execution_count": 42, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse - Session" ] @@ -3427,7 +1169,7 @@ }, { "cell_type": "code", - "execution_count": 43, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -3462,127 +1204,9 @@ }, { "cell_type": "code", - "execution_count": 44, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-152017-03-01M0Edgar Y. Walker
02018-01-152017-03-01M100Jacob Reimer
182018-01-152017-05-01F101Jacob Reimer
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date dob sex experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+\n", - "0 2017-05-15 2017-03-01 M 0 Edgar Y. Walke \n", - "0 2018-01-15 2017-03-01 M 100 Jacob Reimer \n", - "18 2018-01-15 2017-05-01 F 101 Jacob Reimer \n", - " (Total: 3)" - ] - }, - "execution_count": 44, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "# looking at the combination of mouse and session\n", "Mouse * Session" @@ -3604,113 +1228,9 @@ }, { "cell_type": "code", - "execution_count": 45, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02018-01-152017-03-01M100Jacob Reimer
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date dob sex experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+\n", - "0 2018-01-15 2017-03-01 M 100 Jacob Reimer \n", - " (Total: 1)" - ] - }, - "execution_count": 45, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "# Find 'experimenter = \"Jacob Reimer\"' and 'sex = \"M\"'\n", "Mouse * Session & 'experimenter = \"Jacob Reimer\"' & 'sex = \"M\"'" @@ -3718,120 +1238,9 @@ }, { "cell_type": "code", - "execution_count": 46, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02018-01-152017-03-01M100Jacob Reimer
182018-01-152017-05-01F101Jacob Reimer
\n", - " \n", - "

Total: 2

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date dob sex experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+\n", - "0 2018-01-15 2017-03-01 M 100 Jacob Reimer \n", - "18 2018-01-15 2017-05-01 F 101 Jacob Reimer \n", - " (Total: 2)" - ] - }, - "execution_count": 46, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse * Session & 'session_date > \"2017-05-19\"'" ] @@ -3853,115 +1262,9 @@ }, { "cell_type": "code", - "execution_count": 47, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

sex

\n", - " sex\n", - "
0M
1M
2unknown
5F
10F
11F
12F
18F
19M
22F
34M
100F
\n", - " \n", - "

Total: 12

\n", - " " - ], - "text/plain": [ - "*mouse_id sex \n", - "+----------+ +---------+\n", - "0 M \n", - "1 M \n", - "2 unknown \n", - "5 F \n", - "10 F \n", - "11 F \n", - "12 F \n", - "18 F \n", - "19 M \n", - "22 F \n", - "34 M \n", - "100 F \n", - " (Total: 12)" - ] - }, - "execution_count": 47, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse.proj('sex')" ] @@ -3984,115 +1287,9 @@ }, { "cell_type": "code", - "execution_count": 48, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

date_of_birth

\n", - " date of birth\n", - "
02017-03-01
12016-11-19
22016-11-20
52016-12-25
102017-01-01
112017-01-03
122017-03-21
182017-05-01
192018-07-21
222019-12-15
342018-09-22
1002017-05-12
\n", - " \n", - "

Total: 12

\n", - " " - ], - "text/plain": [ - "*mouse_id date_of_birth \n", - "+----------+ +------------+\n", - "0 2017-03-01 \n", - "1 2016-11-19 \n", - "2 2016-11-20 \n", - "5 2016-12-25 \n", - "10 2017-01-01 \n", - "11 2017-01-03 \n", - "12 2017-03-21 \n", - "18 2017-05-01 \n", - "19 2018-07-21 \n", - "22 2019-12-15 \n", - "34 2018-09-22 \n", - "100 2017-05-12 \n", - " (Total: 12)" - ] - }, - "execution_count": 48, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse.proj(date_of_birth='dob')" ] @@ -4107,103 +1304,9 @@ }, { "cell_type": "code", - "execution_count": 49, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

age

\n", - " calculated attribute\n", - "
02017-05-1575
02018-01-15320
182018-01-15259
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date age \n", - "+----------+ +------------+ +-----+\n", - "0 2017-05-15 75 \n", - "0 2018-01-15 320 \n", - "18 2018-01-15 259 \n", - " (Total: 3)" - ] - }, - "execution_count": 49, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "(Mouse * Session).proj(age='datediff(session_date, dob)')" ] @@ -4218,133 +1321,9 @@ }, { "cell_type": "code", - "execution_count": 50, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
\n", - "

age

\n", - " calculated attribute\n", - "
02017-05-152017-03-01M0Edgar Y. Walker75
02018-01-152017-03-01M100Jacob Reimer320
182018-01-152017-05-01F101Jacob Reimer259
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date dob sex experiment_set experimenter data_path age \n", - "+----------+ +------------+ +------------+ +-----+ +------------+ +------------+ +-----------+ +-----+\n", - "0 2017-05-15 2017-03-01 M 0 Edgar Y. Walke 75 \n", - "0 2018-01-15 2017-03-01 M 100 Jacob Reimer 320 \n", - "18 2018-01-15 2017-05-01 F 101 Jacob Reimer 259 \n", - " (Total: 3)" - ] - }, - "execution_count": 50, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "(Mouse * Session).proj(..., age='datediff(session_date, dob)')" ] @@ -4379,106 +1358,9 @@ }, { "cell_type": "code", - "execution_count": 51, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experimental animals\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

dob

\n", - " date of birth\n", - "
\n", - "

sex

\n", - " sex\n", - "
02017-03-01M
12016-11-19M
192018-07-21M
342018-09-22M
\n", - " \n", - "

Total: 4

\n", - " " - ], - "text/plain": [ - "*mouse_id dob sex \n", - "+----------+ +------------+ +-----+\n", - "0 2017-03-01 M \n", - "1 2016-11-19 M \n", - "19 2018-07-21 M \n", - "34 2018-09-22 M \n", - " (Total: 4)" - ] - }, - "execution_count": 51, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "male_mouse = Mouse & 'sex = \"M\"'\n", "male_mouse" @@ -4493,24 +1375,9 @@ }, { "cell_type": "code", - "execution_count": 52, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([( 0, datetime.date(2017, 3, 1), 'M'),\n", - " ( 1, datetime.date(2016, 11, 19), 'M'),\n", - " (19, datetime.date(2018, 7, 21), 'M'),\n", - " (34, datetime.date(2018, 9, 22), 'M')],\n", - " dtype=[('mouse_id', '\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
dobsex
mouse_id
02017-03-01M
12016-11-19M
192018-07-21M
342018-09-22M
\n", - "" - ], - "text/plain": [ - " dob sex\n", - "mouse_id \n", - "0 2017-03-01 M\n", - "1 2016-11-19 M\n", - "19 2018-07-21 M\n", - "34 2018-09-22 M" - ] - }, - "execution_count": 55, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "(Mouse & 'sex = \"M\"').fetch(format='frame')" ] @@ -4670,20 +1439,9 @@ }, { "cell_type": "code", - "execution_count": 56, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'mouse_id': 0}, {'mouse_id': 1}, {'mouse_id': 19}, {'mouse_id': 34}]" - ] - }, - "execution_count": 56, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "(Mouse & 'sex = \"M\"').fetch('KEY')" ] @@ -4697,7 +1455,7 @@ }, { "cell_type": "code", - "execution_count": 57, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -4706,47 +1464,18 @@ }, { "cell_type": "code", - "execution_count": 58, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array(['M', 'M', 'unknown', 'F', 'F', 'F', 'F', 'F', 'M', 'F', 'M', 'F'],\n", - " dtype=object)" - ] - }, - "execution_count": 58, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "sex" ] }, { "cell_type": "code", - "execution_count": 59, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([datetime.date(2017, 3, 1), datetime.date(2016, 11, 19),\n", - " datetime.date(2016, 11, 20), datetime.date(2016, 12, 25),\n", - " datetime.date(2017, 1, 1), datetime.date(2017, 1, 3),\n", - " datetime.date(2017, 3, 21), datetime.date(2017, 5, 1),\n", - " datetime.date(2018, 7, 21), datetime.date(2019, 12, 15),\n", - " datetime.date(2018, 9, 22), datetime.date(2017, 5, 12)],\n", - " dtype=object)" - ] - }, - "execution_count": 59, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "dob" ] @@ -4760,31 +1489,9 @@ }, { "cell_type": "code", - "execution_count": 60, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'dob': datetime.date(2017, 3, 1), 'sex': 'M'},\n", - " {'dob': datetime.date(2016, 11, 19), 'sex': 'M'},\n", - " {'dob': datetime.date(2016, 11, 20), 'sex': 'unknown'},\n", - " {'dob': datetime.date(2016, 12, 25), 'sex': 'F'},\n", - " {'dob': datetime.date(2017, 1, 1), 'sex': 'F'},\n", - " {'dob': datetime.date(2017, 1, 3), 'sex': 'F'},\n", - " {'dob': datetime.date(2017, 3, 21), 'sex': 'F'},\n", - " {'dob': datetime.date(2017, 5, 1), 'sex': 'F'},\n", - " {'dob': datetime.date(2018, 7, 21), 'sex': 'M'},\n", - " {'dob': datetime.date(2019, 12, 15), 'sex': 'F'},\n", - " {'dob': datetime.date(2018, 9, 22), 'sex': 'M'},\n", - " {'dob': datetime.date(2017, 5, 12), 'sex': 'F'}]" - ] - }, - "execution_count": 60, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "info = Mouse.fetch('sex', 'dob', as_dict=True)\n", "info" @@ -4806,20 +1513,9 @@ }, { "cell_type": "code", - "execution_count": 61, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'mouse_id': 0, 'dob': datetime.date(2017, 3, 1), 'sex': 'M'}" - ] - }, - "execution_count": 61, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "mouse_0 = (Mouse & {'mouse_id': 0}).fetch1() # \"fetch1()\" because we know there's only one\n", "mouse_0" @@ -4834,20 +1530,9 @@ }, { "cell_type": "code", - "execution_count": 62, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'mouse_id': 0}" - ] - }, - "execution_count": 62, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "(Mouse & {'mouse_id': 0}).fetch1('KEY')" ] @@ -4861,7 +1546,7 @@ }, { "cell_type": "code", - "execution_count": 63, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -4870,40 +1555,18 @@ }, { "cell_type": "code", - "execution_count": 64, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'M'" - ] - }, - "execution_count": 64, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "sex" ] }, { "cell_type": "code", - "execution_count": 65, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "datetime.date(2017, 3, 1)" - ] - }, - "execution_count": 65, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "dob" ] @@ -4932,28 +1595,9 @@ }, { "cell_type": "code", - "execution_count": 66, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "[2023-07-26 20:14:02,017][INFO]: Deleting 1 rows from `tutorial`.`mouse`\n", - "[2023-07-26 20:14:02,121][WARNING]: Deletes cancelled\n" - ] - }, - { - "data": { - "text/plain": [ - "1" - ] - }, - "execution_count": 66, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "(Mouse & 'mouse_id = 100').delete()" ] @@ -4967,33 +1611,22 @@ }, { "cell_type": "code", - "execution_count": 67, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "[2023-07-26 20:14:02,271][INFO]: Deleting 3 rows from `tutorial`.`session`\n", - "[2023-07-26 20:14:02,289][INFO]: Deleting 12 rows from `tutorial`.`mouse`\n", - "[2023-07-26 20:14:02,377][WARNING]: Deletes cancelled\n" - ] - }, - { - "data": { - "text/plain": [ - "12" - ] - }, - "execution_count": 67, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Mouse.delete()" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Mouse()" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -5005,65 +1638,23 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 6. Drop (`.drop()`): drop the table from the schema" + "### 6. Drop (`.drop()`): remove the table from the schema" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Contrary to the `.delete()` method - where the table is preserved but its content is deleted - with `.drop()` we drop (\"delete\") the whole table from the pipeline. \n", + "Contrary to the `.delete()` method - where the table is preserved but its content is deleted - with `.drop()` we remove the whole table from the pipeline. \n", "\n", "Again, `drop()` method not only drop the whole table, but also all the subsequent (downstream) tables." ] }, { "cell_type": "code", - "execution_count": 68, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 68, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "dj.Diagram(schema)" ] @@ -5077,75 +1668,25 @@ }, { "cell_type": "code", - "execution_count": 69, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "[2023-07-26 20:14:02,741][INFO]: `tutorial`.`session` (3 tuples)\n" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "Session.drop()" ] }, { "cell_type": "code", - "execution_count": 70, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 70, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "dj.Diagram(schema)" ] }, { "cell_type": "code", - "execution_count": 71, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ From e841910e598e87fb554456d3b6cd769b1f9010d7 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:29:47 +0200 Subject: [PATCH 43/70] Rename including a space --- .../03-Calcium Imaging Computed Tables.ipynb | 3089 +++++++++++++++++ 1 file changed, 3089 insertions(+) create mode 100644 completed_tutorials/03-Calcium Imaging Computed Tables.ipynb diff --git a/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb b/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb new file mode 100644 index 0000000..8d4c529 --- /dev/null +++ b/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb @@ -0,0 +1,3089 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Working with automated computations: Computed tables" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example. \n", + "\n", + "In this session, we will learn to:\n", + "\n", + "* compute various statistics for each neuron by defining a `Computed` table\n", + "* define a `Lookup` table to store parameters for computation\n", + "* define another `Computed` table to perform spike detection and store the detected spikes\n", + "* automatically trigger computations for all missing entries with `populate`\n", + "* define a `Part` table to save the results computed with the master `Computed` table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's import `datajoint` again." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import datajoint as dj" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we are going to perform some computations, let's go ahead and import NumPy and Matplotlib." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Similarly as before, to continue working with the tables we defined in the previous notebook, we can either redefine the classes for each table `Mouse`, `Session`, `Scan`, `AverageFrame` and populate them. Or, again for your convenience, we can import them from the `tutorial_pipeline.imaging` module. " + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Connecting shan@workshop-db.datajoint.io:3306\n" + ] + } + ], + "source": [ + "from tutorial_pipeline.imaging import schema, Mouse, Session, Scan, AverageFrame" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experiment session\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-150Edgar Y. Walkerdata
02017-05-190Edgar Y. Walkerdata
52017-01-051Fabian Sinzdata
1002017-05-25100Jacob Reimerdata
\n", + " \n", + "

Total: 4

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +------------+ +-----------+\n", + "0 2017-05-15 0 Edgar Y. Walke data \n", + "0 2017-05-19 0 Edgar Y. Walke data \n", + "5 2017-01-05 1 Fabian Sinz data \n", + "100 2017-05-25 100 Jacob Reimer data \n", + " (Total: 4)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Session()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

depth

\n", + " depth of this scan\n", + "
\n", + "

wavelength

\n", + " wavelength used\n", + "
\n", + "

laser_power

\n", + " power of the laser used\n", + "
\n", + "

fps

\n", + " frames per second\n", + "
\n", + "

file_name

\n", + " name of the tif file\n", + "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
1002017-05-251150.0920.025.015.0example_scan_03.tif
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", + "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", + "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", + "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", + "100 2017-05-25 1 150.0 920.0 25.0 15.0 example_scan_0\n", + " (Total: 3)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Scan()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

average_frame

\n", + " average fluorescence across frames\n", + "
02017-05-151=BLOB=
02017-05-152=BLOB=
1002017-05-251=BLOB=
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx average_fr\n", + "+----------+ +------------+ +----------+ +--------+\n", + "0 2017-05-15 1 =BLOB= \n", + "0 2017-05-15 2 =BLOB= \n", + "100 2017-05-25 1 =BLOB= \n", + " (Total: 3)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "AverageFrame()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `imaging.py` also fill each table by inserting manually and loading data from the external tiff files." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Computations in data pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now it's time to perform more complicated analyses. \n", + "\n", + "When you perform computations in the DataJoint data pipeline, you focus and design tables in terms of **what** is it that you are computing rather than the **how**. You should think in terms of the \"things\" that you are computing!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we would like to detect cells from the average image. The final product we would like is the binary **mask** for each individual cell, with the 1 in the region of interest (ROI) and 0 in other places." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So the new \"thing\" or entity here is `Roi`, where each entry corresponds the mask of one ROI. Let's start designing the table, paying special attention to the dependencies." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Detect cells from the average fluorescence image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's perform the segmentation to isolate ROIs. Let's start by taking a look at one average fluoresence image." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "keys = AverageFrame.fetch('KEY')\n", + "\n", + "# pick one key\n", + "key = keys[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

average_frame

\n", + " average fluorescence across frames\n", + "
02017-05-151=BLOB=
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx average_fr\n", + "+----------+ +------------+ +----------+ +--------+\n", + "0 2017-05-15 1 =BLOB= \n", + " (Total: 1)" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# ENTER YOUR CODE! - preview an AverageFrame for a particular key\n", + "AverageFrame & key" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "avg_image = AverageFrame.fetch('average_frame')" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", + " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", + " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", + " ...,\n", + " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", + " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", + " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]]),\n", + " array([[ 29.53, 30.8 , 30.96, ..., 230.09, 230.78, 228.19],\n", + " [ 29.61, 29.69, 30.71, ..., 224.86, 227.21, 227.22],\n", + " [ 29.61, 28.28, 30.19, ..., 220.51, 224.43, 224.85],\n", + " ...,\n", + " [ 23.67, 23.05, 24.06, ..., 32.54, 32.07, 35.28],\n", + " [ 24.03, 22.45, 22.4 , ..., 31.67, 30.75, 32.72],\n", + " [ 25.78, 23.27, 21.92, ..., 31.12, 31.29, 32.94]]),\n", + " array([[41.55, 40.2 , 38.3 , ..., 55.67, 57.17, 58.65],\n", + " [43.66, 42.2 , 38.59, ..., 50.35, 54.83, 57.9 ],\n", + " [42.07, 41.46, 38.3 , ..., 49.46, 52.67, 55.61],\n", + " ...,\n", + " [30.04, 31.05, 30.92, ..., 45.04, 44.93, 44.81],\n", + " [28.88, 29.52, 29.93, ..., 46.5 , 45.65, 45.13],\n", + " [27.84, 27.34, 28.5 , ..., 42.75, 43.28, 45.17]])], dtype=object)" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "avg_image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It's a bit subtle, but `fetch` returns a NumPy array of the attribute, even if the attribute contains a NumPy array. So here, we actually got a NumPy array of NumPy array. We can of course just index into it," + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", + " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", + " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", + " ...,\n", + " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", + " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", + " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "avg_image[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "but if we knew that there was only one item, we can use `fetch1` instead to save some trouble" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "avg_image = (AverageFrame & key).fetch1('average_frame')" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", + " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", + " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", + " ...,\n", + " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", + " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", + " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "avg_image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's plot to take a quick look:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO19fayt2V3Ws/Y+e5+vO7cz04/h2ja2TUa0kigNwQJGG+oHVEJjAqRIdISaRgMoqLGt/AEmkoAaBKMBJxQtpnRaPrQNoogVYkxkZBDEQimMgO3Qse3A3Hvmno+9zzl7+cfev/c8+3mftfbpvXPO3Z2znuTmPXfvd693vetd7/r9fs/vY6WcMxoaGq4uBve6Aw0NDfcWbRFoaLjiaItAQ8MVR1sEGhquONoi0NBwxdEWgYaGK44LWwRSSl+WUvpoSunJlNI7Luo6DQ0Nd4d0EXECKaUhgN8A8GcBPAXgFwB8bc751573izU0NNwVNi6o3S8E8GTO+bcAIKX0GIA3A7CLwHA4zBsbG+AFKaW0dMw5wy1Y8f1gMOiO8dl5wG3G33wtPerfpf6UrlE7Tz9z13aI71NK1TbOc83ZbHau+9PjZ4Lab2vjd6fjzvce8ySlhOFwCADY2Ji/Bpubm9je3u7+BoDRaNT9Ls6PNgDg5OQEAHB6enquvs5ms6Xv3XyNtk5OTrq/43ez2ax3rfjOXT/n3H3/7LPPPpNzfqn26aIWgZcD+Dj9/ykAf4JPSCm9DcDbAGA4HOLGjRs4OTnpBpgHH1gekDgCwHg8BgBcu3YNALCzs9P9Rh+Emyiz2ax7kHycTqcAgOPj46XvZrPZ0gOJ68Q1Y0INBoPu+jpRUkpdv7lPPEH52ny/fL5OkI2Nja7dwGQy6c6NScxtRX+jr5PJpLt3vi4wf07xXKKt0WjU/a2TkhclfhZxzTgOh8Ole+Bjzrm7hxhHJyyGw2Hv+cZ9nJycdP2Ol3xra6ubMy95yUsAAA8//DA+7/M+DwDwmte8BgDw0EMPAQCuX7+O++67D8B8jsX4PPvsswCAvb293jV1/s1ms969jMdjbG1tLY3b7du3AQDPPPNM1/7BwUF3jO8PDw+7a8az0jkzmUxwdHQEAHjf+973f2FwUYuAEw9LI5JzfhTAowAwHo/z8fExjo+Pey8wr7q9BmmV4xc4HrhOQF5FefLE+W6R0TbiOvyd+577rQ/ISZXT09PeQqJtu/4wTk5OelI2xuf09LTrW1w7pbQ0afX+9PosUePIi7PTyhQlKar3zM81rs8LrFuI475U2mp/gfmLFi9HPBfg7CX+5Cc/CQD4nM/5HADAAw880C0g/Nzj5bx582bXLoCubeBMqxgOh70Fdjwed/Mv+hu/vXXrFp577rmlPuace0Ll5OSk+z6O/Ny5Lw4XtQg8BeCV9P9XAPhE6eScM05PT5ekbMCp+fwCqXTjF50nexx10eCJHZ8dHx/3+uHMEu6HUwNLWg2rojwGKuni6BYlp/rHGAJ+IVGtxo3HbDbrLVDupeJr6v2GNhL3zW2U4DSAgKrtvAjES8UanS4QvPC4MY1xOTo6wq1btwCcvbj8YurLyr+Nlz9e2tu3b3d9i/HY2trqveiz2cxqvQCwv7/fLTLxXfSLr80LvAoyXqRLuCjvwC8AeDil9OqU0hjAWwB88IKu1dDQcBe4EE0g53ySUvomAD8NYAjgh3LOv1o6P6XUSXWVgryqq7q8sbFRVddVNXfqKa+UvLLq6lkjX5xd71T+EicR/VCJy+frd2wD18hLRxC59lXL0nuO35VUf3efg8GgahKopsbfsUbjTBwn3VQVdn0MbGxsdFKV+Zn4bbQVdvdgMOiZjY47ivP39/c7aR/z9fj4uBt7tud1Xkcfjo6OetrEeDxeMiGBM96HP4vfMV9QwkWZA8g5/xSAn7qo9hsaGp4fXNgi8JlCV8EASztd0UKDAJZJJmWrAyXJ5K7pbEdgLr2czazSh6+vNi1LcdZcVEKzVqF2LrfrGHLXfyd51W520pDvQ8ePORX3O7bjdfy4H8qH8HfKbzhNzUnqAHMw7vlEf/f29nrSNWzy7e3trh9xzuHh4RKpyPfBUjx+d/v2besxUu2H+Y44P+6Nr8fEprYb3zkvhaKFDTc0XHGshSbAnIBKGrZ/1AfKDDm7FmMFPk9wEUtPlti6Kjs7N9qfTqfVIBDHjKtd59xvztZnDiEkTJzP9rP2Z2Njw3IjTvKqdON7dx6XANvK0Qf1VnBMBWsf+lt+Fuq2dM91MBj0PAtubHnc1We/v7/feQdCA9jf3wcwZ+XVBbm/v9/91nljlNdi3ke9IdwP7re28eyzz/bmJrcbbcQ7cB6sxSIQpFUtJgDov8C8CDgVV3/HJgWruEG68PVLkXH8cq+K1HOTPaCBPnyePuTT09Pey8rkmDMb9D5dZBovuqwaqxmj/eN+cKDPecdF70+vw22xKl8juGazWe9Z8aKgrjN2p/KLHPcY6jT7/d3iHwsJRx3G/+NaLh7BBUrpYsdRtLxgOVey3p9b3Eto5kBDwxXHWmgCITE46EZXx1LAjFNxVaPg8zUyjkmmOI9ND12dmchhOGJT++bUcD5Xg12cOlsjefheVDo7d6BToTl0W+/DSXHXJ1blnfngXISqTteIRO6ji4hkKRvnKHHGpgr3V/sRUpzNRjbDoh8cDRpHdde5sZ1MJr3n7gKmnKuX29Ux0vuooWkCDQ1XHGujCYxGoyVJrUTRqoAHbivgEjhUUvKKzaSKS94JqMuvFNDibPtSH13oZ+2emDRyocrKQ7iAHO4bcwcu6EevqQFWfD5rRWqbjsdjyx2oW5dtXHXJOlcoj7cLNKvNJxcgxRpA9DHuKxJ+RqNR91lI5SAUOQgt2uLAINY6Vdt0AWc8b+NazBOECzH65p5xCWuxCABnE0H91szmqtrEGXO1F4xfQuf314k3nU6XYtL5GL8BluO4HSuvE5tj9h0J6BaouJ7LjHRxBbXoSCWJImmL731VKrFem5npmufA5V7wfZbMQE6K4qg5zZZkj47eOy8QfJ/uhdQxYhPDEaW8eALLKrrz2SvcYu7mHJ/vxkjP5byNVeRgMwcaGq441kYTAHw6La+OrphCbZVTYoZXetYwnBsrUCPCWOqqpB4Ohz0/fi1ewLXBWYjaljuPXVzafz4/JMhkMlmqN1Dqm4MbFxfpWMubqJkl3JaT4qp9ODejA2tXrq6Bi+6M3+lnTCDrs2VNoCTRow29dza1XFatmycuGxTwbtremFS/bWhoeMFjLTSBsHPYTtMjr9y1QAiOfef6AAEXh66uGUdUsQ3vuIbSPXE/WXq5CDO36gdU4vHqz+eUog1ZMoX0Pz4+7mlX3N8aD8HQiMuwR0suQgXbxQrmQ3gcNWCGn6PLftQ+uiCxjY2NjliL75wGFnCBOw48l2vagYPLBtXPWNq781dhLRYB4EzdL3kBeILzy+VIFRcVGCjFEPD5rPoxacR95fM5oi/AE0QnoIu8cyG5vJhpIomLi2A/t6r+0+m0I6iYRHVmV6m/zkziz1S9Zq+JMwt4fHScuX2NK+BFoEaiBbh9Zz5E+0w41rwrfM+uZFuco3NiOp1W1fXzRJGyCcJ90EQt7k9LIGpoaKhibTQBYFmlcySTfsaqOfuVnW864PziToVS6RNwPnn+3Em6WsQgH/X+uB31c/N5rLqWNB0XGcmEI6cB8/c6BkrIuRqALG2dRC3lN8T1+T5ZW+FUXmc2qtReRRa6HBPtW5DKjuQspUVz//n+WIrXSD33jGuuci72qtesmVrdNarfNjQ0vOCxNppASEJd+ZzdzSt+iRA572f8HUsmtbFYSjuJze68+L9KGleOnO+vxIeUtJX4O6QLS17nsgyNwUUHOptd3WX6W/1OJbAr/1YKjlESVzMk+Xx+7o5/cEVlzhNE5bgdPsdpGKWAsFVELM8NzQBkbbjGTbh8DOUXWMMsoWkCDQ1XHGulCTC7zRljwHLhDmdfulWa7dy4hgsMUp7AuesCTvtwATOj0ci68IAyN1A7/zyuHi5HpQUwnHeFr6+aDN8LHx33UdImWBtiW9vVH3Dehvi/jnMpmEuZdLat1X3pJCoHIamm48K6c849roalvu4F4PIsXIFZho4V3wv/X+d1KVfEYW0WgXDLRTy+LgLD4bBHGjn1jQekVGiB249rM3iyuwUiwJ/VEmpqLsISAalj4NJeXay5ywWIo77o/PLV1HunSvPv3IsV7UcfNReD++j8+E6FdQsJuzhLLwKTi0yO6pjzvbvIyxhbXjRqKbv6/NmU5HlU2m+C79ndC0PNJx7HVYtAMwcaGq441kITiNVtPB73zAFe2WJ14zrrjiysBd3wNaN91Saceu9WcCfFmSAquarcyjwYDKz0dufFsWbGqOt0Npv1XH58TUcyaeCMi0hkycRurIC6aQ8PD7vn53I6XICSjoeroMuagH6XUlrSKOOzz6QQh4tSdIFSaoYxXMYqa5Gq2XEAmZszjlTWOenqdiqaJtDQcMWxNprA1tbWksR0pbbUhcYrtyP/akQck1gu4MiRXHHUNhwx49xvbt+BQC3TzGW3MSnqXErcj2ijFszjSM44P6R4KTdB78+RhxGyPBqNbOCTI8C4Lb0n51ZzbrQ4vxb+G3Nnc3NzKe+B4Qjh09P+RqquKKoLIHPamysOex6OxHFY0Zaba4o7XgRSSq8E8MMAPgfADMCjOefvSyk9COB9AF4F4HcAfE3O+dlaW4PBoEvc4CoswLLf1ak8Svicxy8KLKtcOnD8AJ26rG24+AbOP3Apq64Nx8Zzm/yZU/1d4YuaWjibzXqTlglEXUzZc+AKZAT4PI1zd+PCz1YXNl5kXCwDLyjaXxYCq9TpOHKf+BzurzMHAmxauGShGtGsz5sjOvn3pYWRj3xvFxkncALg7+Sc/wiA1wP4xpTSawG8A8CHcs4PA/jQ4v8NDQ1rijvWBHLOTwN4evH3cymljwB4OYA3A3jD4rR3A/g5AG9f1V6oeLFS6j7uLouP1SC30rvUXEcClvIVFvcGYDmWPVAioqJdR1rqb/U6fH0XLencgXxN/S3fi44ll3Pj66vpEZJ9PB4XCS83HtzvOGdra6snUV3xDO4z9zfuSWMCuD0dKycJWXuLcTw4OFjKFeBrcnx+LcqT+6gkoXNLukhENq+c9qbXLEWxRl9X1ed8XojBlNKrAHw+gMcBPLRYIGKheFnhN29LKT2RUnqiplo2NDRcLO6aGEwpXQPw4wC+Jee8dx57HAByzo8CeBQAdnd3c2RHqURgW1FXSidteeUrrdLxW2A5EMe5ilQT4PPY5VbKK5d7Xro2f+Yi6fgcNy7uM0fK6T3z+GjxDM4xiPvb3t4GsLwtNmskOkZuGzgmNLVCr9NIXDCNBgjpefoZj7e2z5oAn6ckJxeTVfAzcG5VnX/sAufx0+xBF6HJ/XZuWtUA+BnXai0Ad7kIpJRGmC8A78k5/8Ti40+mlG7knJ9OKd0A8KlV7cxms24ThtK+c44gcvXZnI9f2yqd7yLjlBh0LDQ/GCavnCqnfePEGVcxWcGqo5sMblGM+3Bqp5aqHo/H9rM46qJxcnLS+f21qq5L9dbvo6+qOvOzcGp4oLboObOEx1HHyIUN8/kKF6vBz1jnsktpB/rkKfffLQLOc6Htcn8ujBhM85bfBeAjOefvoa8+COCRxd+PAPjAnV6joaHh4nE3msCXAPjLAP53SumXF5/9fQDfBeD9KaW3AvgYgK8+T2M5z3cTdumUQLlARYBXVi3BxZs16MrKkkb7w23UUlAdqcaSWldi5w50BCXfm5NktUiwEhkGLLssd3Z2AKA7ssrKm2tqv/kz3YKLJbdKSC6xpSQcg9Vrl4TkNAAn2fmo57s2XKQlf859Y+1U55BTwZ27jok+Z8a42Bin+tfMqQtLIMo5/zcAJT3jjXfabkNDw+ViLSIGB4MBdnd3kVLC4eEhgLOVOOwojpAr5QAA8xVZMxFrv3N2Vy0FdTQa9SLNWFK7bDx1uXEGYCC2YWOwJHN8gbPxlfxjjcNFMGoA02g06sYvjizlol3er0AzFx0x61xYrt+q7fHz4cAgJ6FLmgAHkPH5mvnJc8xxA+qq5DGt2d0ukzNwfHzcKwC7KsI1rsXaiiM+Aa85KFruQEPDFcdaaAIbGxt48YtfjO3tbdy6dQvAGdPMkkq1BOfmcTnbbF/qys1SlguDllw/wZhzH4+Pj21gTcBpB+dhyJ19V7P5OBhKj6tiyOO78Xjc4wLYLRhjxZ4A1bTC1o8jg8OG+Z7U9nWegFrWJvMEKg1LktAV9lQpG2AXdRxdiXL3XDgEWbkCtx8ka2WufVcno+bNWOUdWItFgHMHSu6SOA/whS+44IhOPvbr68NiVbE2aVj1UuKMyTROtnETCZi/HLGguUhE3TmXH7IjEvk+1Z3G967JMZG+DQC7u7vdMc7TlyRcucDZ7rtxr9xfXgTUt88mCN+v8/dHm26B0HiFnHMv3t8liZWuoe3qYsB9dO5LNcPOi5IgA+ZjxXkY0Q/+rfbDba5zKRGDDQ0Nn71YC00gVBxeiTWbq+QCjM9DE5hMJj1Vvpa95yKq3ArPGopG1HEaNGsRqg5GG9PptLeRZUrJVqIN1AKUnFajblJWLdl0CW0mogLjyP1m9VPr5TmpzL+rqaIu47KWb6G/0/Ep5QyUIu9cMFfJJZxS6saUzSU1Jd2eAS6aNZBzXtKS+HecFRr9drtMOdLXjUsJTRNoaLjiWBtNYDKZLNnySohMp9OevTMej7uVN4gqJ4nZFnfx4o5IUk2A7VwnsTWTjvuhRN9wOOwkLhNbjkjjc7R9vZYLaWao+8sFLTliKySfc1VGzgfgyc4An6NSizkVfZ5chISlufIPzl3nOAEmf1V74+xEV5PA5a5otivPJdXo2N3JKGlv3K5yTnwtJkW1jFqtZFpgbRaBIJH0IbiUUX5oGsE2Ho97LylPztKmItGPODrSLdrQl2k6nfYmDRNxOimd75ZfDvbBB7R9B35JS5FsCl3sJpOJjY2IPoYZEOPIOQ8uYUvVZd7mLMjgnZ2dLmJRiUQm5PjFZ88MsGwKBZzppLEjPAa8COiRX2o25dzmoPE7zQ9x8SdM5qqHiceTyWhXmcl9Bix7e0po5kBDwxXH2mgC6ussuU2AZVLP1SJwKbMBjbJjqc/nl9x73CcmyaK9WHU5AlC1Gad9cJ84SzL+79RBFxsf0ExARwxubW31MgVradSc5Vkjm1yUostmdPEPTpuIceF+uBh/F0WobQQcaelMIf6d+vP5N+dxM5e0D/0ta2K1LEnuq9NEtf/F3688o6Gh4QWNtdAEAi5GvWRvBs4TBcf/d662AEtWldq1OHTHZbjtogNsR3K7pUg31gTYlnQRZqUIR46WjPO3t7c7W/y+++7r2tD6AEF+HR8fd985DUzHmyMYmbtRd910Oi3az871p2MT96TPxcXPO9ewc7W5eaIao4vCZGnu5onOKw5yCihvxePBn3OBEv0snr/LQFWsxSKQc8bR0dHSw9JqwyklXLt2DcDyNlrxQsVNHxwc9MJXA65oiWPZOf7AvZhOxVVSx20TVgsfdT5+t8UWx0C4F10nQbTFpCG3ob7v8Xjci5bjCaubw3A/ddKzWcWTVCclk6KOKVezhxe0OI9jL1Rg8ELC9+TMhpJ3xcVlOG9MbU6klHoeFF6MlBR18yTuX8+L6ysJyBv1lNDMgYaGK4610AROT09x+/ZtbG5u9iReHHd2dvDAAw8AOItqOz4+7mLwWSqGGru/v9+dF0d2KQHLEVi1aDxWOx3JpOopk5bqQ+a/nYbhymm5xKG45xiX++67rxdrHjg6OurGiqWQbvLK5cUin4B33I3fcu6DxkhomTEeF4Yje10aszMNXQxI5DNoaq7Ls3AktEtMcqq0M09KZgG3W7oXnQur8mdc2rWai27OFdtceUZDQ8MLGmuhCQBngRcqMTiQQiWOszk5mMJdwxXdUK2Dd8RROJcO/81uNS1a4dKGXbSatunO39zcxIte9CIAZ5rA9evXu+sruXd8fNxJZrZVXVksDSByJKAL2AowuedILheoFXCurtB4HEnMXIPLXYjruLmjgWaOJ+Dn6VyQqjHws2buSu/XaZ21cQkw0ee2cdNndXp6utJN2DSBhoYrjrXRBNjWif8Dy3aahopOJpPuM0bJHmJpVAsDZnddLSCHJYjLSSgFLTlXVOkaeq2Qijs7O523JDiS4XDYjU2My3PPPQdgbsNH3zhzMDwLPN4ax88hqeox4EAVzo0o3Q9zKvy90xiiTReAo+eNRqPuXvS6Lry2VPqsFHDETL3TBEvFQhnOo8LBWW7PSsc/qJbKwVOqdfA+EiWszSIQKPnnT09POxWX49bdLq+qfqvvnM9n9YpdORqL7YpGsDurtgjovXG7tRwGvo5rK1ygt2/f7vqmCU9qFgDLbiRdbLmQRS0y0m0mouYPk17sW68tdgF+WV3uhfZja2uri3kIQjPGZX9/v5fOzQv9eZ6BS3d20ab8ErooSHVtspBTM8ClubObm00Al6QW1yslpnV9qn7b0NDwgsfaaAKh9pXIEV5NXe4AawTq9uLVWUkSDlRhNUwDjlxl4YBLVWVJo0E6JbeTahMub8Ih+ri1tdVJmJI2xP1hNZKJTdWc4siVcfk5hWYR7cb50+m06xvfu8vQ0xRiVpNVegJnkk5disCZ9sNZkOE+jO+cRsIBVQE3d1gCq4bh3J41QpjnvJsnbq6o9sFBS25+h6u8hKYJNDRccayNJhCrpLptXHYb255a2dYRck6Ks9R32X0ulDigJBZLj1Jed1wf8FK/Ziezbc2SJD4L6TmZTIpFJHZ2djrJGLbztWvXOgnqCmU4uPBYLpum965BPas0L/0d58MzZ6PPh0k0nUNcIZolpkpq3qFKtTanATpJzfepmhw/M8ctOU1AJbzLpeD5pP3gd6SE52NX4iGAJwD8bs75K1JKrwbwGIAHAfxPAH8551ztRcTBO3bWeQmYtdYXnfe6L1W84Xb5xeeJqA+LycNYmJg4O0+EGX9XilHnz/g+NKae2+OXz5XFBuaegPvvvx8AuviC3d1dW+Zcx5R3Gdb2T05OepF6ceT71UWPwVvMqbo8Go06ok9Tirk9Lm7ikpy03xx16Co+B5zpxC+kkoU1VZ7Hlp9/ydzj72rxJGwO6MLGAqSE58Mc+FsAPkL//24A/zTn/DCAZwG89Xm4RkNDwwXhbrcmfwWAvwDgOwH87TRfjr4UwF9anPJuAN8B4PtXtRX+ZpXALo7axWw79VQ1AF6J2UVXW7GVKHKuInbRsOuqFCfA7fM9qb+dfx+rf9wbkz2uDJWaLJw3wWShq42ouRSu1j1H8TkzIM51lXG1jY2NjaJmxK5hvheVkCcnJ73UZx5v1Q6cqr2xsdGZDi7yzhG22gb3y0Vj6jiyCazPgseK57lqyTyf7oUm8L0A/h6AuLsXA7iZc44RfArAy90PU0pvSyk9kVJ6wtnPDQ0Nl4M71gRSSl8B4FM5519MKb0hPjan2mUo5/wogEcBYHd3N5eIMbdyO1uTV0VXGz+gKyVHsPFKrGRhLVaetQmWCM59GddW3oLtYseLxDXZZemiw0pBIywpQ4vY2trq5by7MmQs6dUVxoFSPB5xVPLURd6xJqCaGu8oxePjAnciOIg3GA3os3BBOpubm708BZ4TzI1EP9wziP/X+CGehyrtnVZUi1wttRvHVcFZd2MOfAmAr0wpvQnAFoDrmGsG96eUNhbawCsAfOIurtHQ0HDBuONFIOf8TgDvBICFJvB3c85fl1L6UQBfhbmH4BEAH1jV1mw260pd10qCK0/AwRrOY1Crg88rtgtLrQXuuDhxDRbiABhXV8AF6ZxHIjhtQplhN37D4bBXQpwltXNjqXfg6Oio0ybcWKoNzOG9AdZg2I7lLeh5zBg8VvE910iIvzkgKI4qDZmvCLgxDY7APWNXcLQmdR3bz7+paUhOq+BnXHJtcnslXEScwNsBPJZS+ocAfgnAu87zo4gRdxFu+v+4wclk0puozv/KA6Qvh4vxduQYPyhdBFhNdrHjtcQTB/Wjr4JbjGox/uxmVOI1pdS9RHt7ewDOYvC5hBf3VSP6eBK7cmTqJnP3zsd4Edlk0BwJVyGar6NjyeZXYDY720fAEcJ6f/EboO9S5nFx7kA2G0uLgIuudOaD/q39vpRFIOf8cwB+bvH3bwH4wuej3YaGhovH2kQMhuro1FhgTto4CRlSykn7Wqklp2I7ItGRdWo+sAajaj6f51ALMuK+Oq3CSZpA3HNIysPDQ+s2jDTkiCLkTEuNz+eMPpbA0Z5zr+n+B3yvbh8BR8iqWXV0dNQbL84j0f67jT1dWS9uw7mmdV6xVqgarHPvlUho5xqMtnTu8Fzja5cC49xW8L37qn7b0NDwgsdaaAJs96kNxMEvumEj16sP1OxM/ttJYJVQfL4LpuG2HCFTc+eoFGJpr1KxFlbKRxfyGyG9ri9s93LGYDwL3isQmLsWOUcfmEtlJwUDmuXHQVRBMnKZeL23II35fFf6jQO2eN+++L9mKXJ/WdsrBYmx5sXPTrUDx8/w3DlPWTn+nUpxvk/WWlQTqLWvWItFIKV5OioTIVp0Iefcsb9cNScmOavyJZ8qkySrWHnnFYijW6h4swfAM8erUHtY7gVzC4+aL/y5JtRwnAD7ypkRZ5ycnPSY99FoZCs4x9GZGVqNuFbTkT1AzqzjF8FF0sVYaKLZ6elpb38H5x0IOE9HLbqSyUuX78HkqfaXyUXnSXFkpC7EjkgvoZkDDQ1XHGuhCQwGA2xvb+P09LSnygUODw87TSBcV3t7e91nvMLWNIGA8+uyaqdqHpsAujrzb50P2Wkf+p3eAx+du4lXf3YLar9ZzQ8SMGoTXrt2bUki8RE487drYRDtY6ma7dbWVnctHh+n8ehnLLlVzXdRcM6PH1oi1/tfFVOv4+G2eGNTRc1WF+fgyEInsWupxJwaXJvXtRiTEpom0NBwxbE2msDOzs7Sysp76AHehnMbTTpChqFS3PEQjsCpxXG7LC5uVyWls+FddlgtX8HdZykAJo6u+IfuWOSy5ejGF3QAACAASURBVNw+BXGdo6OjYh7/aDSydjdviBr91nJu0ZZz5TFqrkSO3nTkmz5HHovatVg7VOLOkcWqVcT3cSxJa37Gmg/D1xqNRsUK2+7eFWuzCGxtba1MdNDNMzc3N7vJ4iZNoKYanTchgyeAtsOJJOwHdhtexFHVwtoiwP2pReMxoen6qJGAg8GgKzAScBWFOaEoSMN4Fhy1Gep3eA5cLAP/HYvA5uZmLzmHx87V/dNnzC9dwL2QPB76cvCzdaSoxg7w+W7hduniLtVXTTG+tkYwusIxbkHTuVRDMwcaGq441kITAM6kmEpNTn8NScOJIloN2KmPKpEBT6Y4wk9/yyq3K3PlyEVdpUtx5SUXl8sJYFcek1GqwnMNwfiOzQGVZKxhaD+m02kvxZa3MndRa1psZTQa2W3CXD/iqH5xzhNgMjA+cySnSlQ2S1hDK917zrlX3ITNI+1/iSx18yRQIwsDTMTGeW5/B75OcxE2NDRUsTaaQEgNl3oKzO3Y2FKLo9WYCwB80QqnCayy8dXuYoJLpXgpOozb4+9Go5HNUlRCjqWWk/DhfgvEWABnAUEc96/lxVzmHW9NHueH5jWdTm3Mflw3zuex0/PZ9cvlwFRCsjTXKDh2j7L2UXrunHnH0DZ4/DRjsBSkpS485yIOOBeuywthjcBpNY5o1nGuaROKpgk0NFxxrIUmEDHSbI+GVAnpz+WxahKEWd87jcF3MeGucEYtmIeDlvR8LuHl4v5dJp0GsQDL4aVx1GuyxNEyZ9PptMsFYDdgaBGcgQj4jT2n02kxe489JIEIDItrRRulopyrymO5YC8NGnLhwKWgr4BqEyW7usTKu9+UcgGU73H36zZs5XuK+RTPjvkOlxG51Hb120vCYDDA9evXl9xBbjMKfYldUkfNrcYEVI10q7mK2Nzgl+o8lWJXvdS6Eairrxg4PT3tiFJHKGnF5cFg0Kni3H60we3GeZoHwS81u2bjujERV6miMVFjP4GTkxNb5Sf66iruuIQdfRE5bkHHiJ+7Lr5xr9yWuw+XcOTaYoGi5oMjsl26OM8N7QtXjXL1Ids2ZA0NDVWshSYwHA5x7do1nJyc4ObNmwDOioVwTTsl6ZwKDdTVfkf0aHAMk38uAs+RP+q24Wuoqsu1FDmaTLUPJ8m4H+fJkmRTyEVBalrvbDbrZURGW9euXetFGAJn0icyBgNM3MZ9himg46Obn8aRtQQOKIrfcnaiS7GNzx2Z6+C0H2A+ZjouXAnZBYQ5klO1PIbTNpzLkqMvA+5agVUl/Zsm0NBwxbEWmgBwFmYZJBQXtwTmq6RqAuwqci4/dbnp38ByLQBXhVfrGrAE5uAU57YpBYxsbGwsFbzQfqmE55Vcfxf9jWNpByK9vrahUlT/jvN1bGtZkEBfovJ58awdeeV4Bd1hiM8bj8fVkFnlK7iUGPe7lHvh7Hku5qF9zTn3yF+gnmVaIxldOTK+ZolLc+HRirVYBGazebnoyWTSDUhM9pgowPLLDyynVbpYABdtFeC4eN2Zl1lcfZk4ao4TVFRFY9+0Y5q14i77uXmSaVtu4XGxCa6CjVOFdVFkc0f7c/v27d7GHvwyherP3zmTRUlL3g1YX3guc87pxVogJaXUu76L8mS1WseUfxNw5CxHqbqNZeI6mvzDnhSdtzpGOn4cp+LI6pr5WhJG3fWq3zY0NLzgsTaawN7e3hKZEgQSR6PV4q5Zyjk1Vs+Pv7e2tnpkF6vVLtOM+w0sS0NWqzkugOEkJLub9J5c/Dy7uFgL0f6yWaNmzGw267klOcNRtQku9cXjo3s+sJbjog81h4ELh8Q1uTAImw1xjhKI/L3eO48Vj7uLt1fzxWlqHKWomgBDNR52B7JWqJocmwca4+KyKl2+B2txq8yBpgk0NFxxrIUmcHp6iv39/aU4aicVnaTRIBOWZK7qrErPra2tns3kiJxYWV38PBfscHnfjhNgPiE+cxl3wFzaqduQiVInhVyko3ICTIBqleK4Lo+Vq5vAefOaQ8ARjJy5qIVD2A2oROnR0dESERjXdhqPunr12TGcBObnyNeK79Qd7bQ31j7cBqYl253PY83EBYm5QDoXJanXLOGuFoGU0v0AfhDA5wHIAL4BwEcBvA/AqwD8DoCvyTk/W2snXgpWLTVtmEkYl4LqEkS0zp6LwBqPx/bB8KSN68f/NV2TXw5WGV1aZ/xfJyC/yMoWu3uZzWbWZNEFhE0RNaP4O/Z3q9qrBCGwvDDo4szkof52Op32wr+BM4JPffG86PKz1hLivMCrGeM8NUymscmiqem8KDkTLuCS0BzL78ZSF2cePw0lZgGiC7ieF21ctDnwfQD+Y875DwP4YwA+AuAdAD6Uc34YwIcW/29oaFhT3LEmkFK6DuBPAfirAJBzngKYppTeDOANi9PejfkehW9f1V6oz7qS8f+5oAZQTtekPi5951Z/F+HniD42MZwm4IpQqObC57iU0oDzFzvy0kmVaI9V8vhcNyTlKrzsmlU1VmsT8j0Nh8NOrdeNTjY2NnoaFZsDrNayxsfX5nP42WoBE5fizc/WlWJTEi3n3Kvlx1LZmV21CEQXp6Juyel02pPUrDnq+aXrqNta51wNd6MJvAbApwH8q5TSL6WUfjCltAvgoZzz04vOPQ3gZe7HKaW3pZSeSCk9oTZfQ0PD5eFuOIENAK8D8M0558dTSt+Hz0D1zzk/CuBRALh+/XqeTCZLEWlq77oAHl4BWSvQFd7tOsOSRstMsY3f3ayJ8a/tfsOSQ0lA/ru2YnOmnpPiCtYO9NpbW1s9baUUqRfnafVg5mXCFmcehz8DlglN5lScbV0KOGL+hIlT1RTZ3enyCtz4qrZXciVGmyrZmZNy3I6Cz9eAsLgvbqsWSMTX4rmvbkmXzt1rs/ptHU8BeCrn/Pji/z+G+aLwyZTSDQBYHD91F9doaGi4YNyxJpBz/n8ppY+nlD435/xRAG8E8GuLf48A+K7F8QPnaS9cXioheZV2edQqxQHvKQDKee4akOGyE52bKa7NocTODq0VnODrlHIeOItQ8xz4PC58qTbidDrtMe/uWrNZfwPQWnAMM/XOQ6KBQaw1cR9ceLGOmZNuLndBXYWsIbH0170OOMDLFfjQ58PPTMH9dqHbNS9BgN3AAZfD4PJZarkoiruNE/hmAO9JKY0B/BaAr8dcu3h/SumtAD4G4KtXNZLSfEPS4XDYqTOqirKKzi+OplA6kiTg1CKeDC7JpeT+YrCvnN03SkLW1E73IrgYglobLpqM23RxFnq//OLqS8KJKudZHFn1d9GBTuV3ppmOB88Fl/SlR2eaTafTXlyDu5Yz4QLOtHHuXf5OF13uty48fJ/OvezaDfAiuarG4F0tAjnnXwbwBearN95Nuw0NDZeHtYgYDE0g59yrHuwkGbuRdFciPs+pna7AgnNHlWLCS9F5Tt0sSSaWtk66OWKo5jLj71ST4nM06pCj7GpwLkjWlEo5D6yGK2Gl96f3ye0r+VZqoxR1WHL5llxzfA/cH1eUxZGFcU7tXvg653Hjcb9VS3Gmh4tOLaHlDjQ0XHGshSYAnJEbTkICy+4pjtNX+3IwGPRW/Rohdx5JyOc5+4tt9pqbkVdwxwmoXexWcw4GCaKP7egIunFuMicRdBej09OzfQRUM+Hz+DnpvbvyZU4Cs8TW587Xq4WGB7iP6iLma3HItJLQnOXnOBjVSFwOg+tjrQ3Hy7hgIT7WApTc+auwFovAbDbDwcEBUko9koYnmPvMqWPnIdiYyFMVilN3daLwtdzkcXCRY9oGP1w9h1+c0suin7l8CI055xeB/fQa0eeIRB2fUp+0P67AB7PbNULOxYxwXoiOqSPfuDiMi0moRea5eeWKeMTv3Rg56DMtlbXXtpz5EuDxXnX9Zg40NFxxrI0mcHR0tORzLqVoAt4HyjHeuqLWiks4KeT86DWp74gZFx3G11btwGkJrDa7fQ0cUeW2E9f2VbpwH7mwS8C50LivTkrFd3ot9xzdeLAkc2XUuNyWtuekud4Tk6hMNJci//jenWnjfPxOa3MmYm2Mas+Kz9F+8O+aJtDQ0FDFWmgCQN8l6NxOjvRwgSGl2GuX7eeCkLgftRhydx0mJZXAqbknWSPRVd3Zp4404kxB7Q9fO845PT3t7Uo0Ho97bsCIHDw+PrbSSvvL7lIXjenG2z2X6KtqAtyG3hO3UYvsY5vdcQfumTmXnwvm0fYdHNGnGgxre6XfAp6g5LYuup5AQ0PDZznWQhMIFxhLQw4zBZYlNts7zi4uSVInQRwLPZvNeu4uV2aK6xtoSSuWIMquc6l0x4artD0v0+x4CM7T135vb2/jvvvuA3DmKnRjw6XenUQtudX4eTI0fJlrAWj4NUuymjuXA5NcmKyWSuf7Ys3IeYP4GvqdahNOaytxRqU2AszsO02qttsQa2Cr3OBrsQjMZvN9Bziyy0WYBVgNci+6I2ni6FRAVTeZHFP/Mn/HLkVXj09JS+6H85WXCKKaK4jhVO3ow9HRUa/QiEtQ4d/oojsajXqf8eLi6vOpeVLqt778rj6gKwnnNlBxC49+5iI0ObVaI0adSu3mmlPzeQF3MSCl8XZuYwd2izt346pn0MyBhoYrjrXSBDijzxGDNTcTr9yqOtfqvrMLiqVQKe6/pJ6qys/qaS0YpBaNx206s0cxm51Vy9U0Zo7xZ1OHC3Vqv5XUc2nSHFilmoDbQty5WJlsrQVd8b24jVFVk2JtQjUjTot26cUaWFULXip9plrEYHC2nwXPTReJCMw1Hj3fRVc6rYZdxU0TaGhoqGItNAHn/tNyWm6F5Zj6AK/6jrByKzb/FljmDoKYZMmqEpulJ7dVIqrYpeOChLTQJwfH1MJ0WSKoFsL3FDg5OemVBCttdAoshxkHkehCcuM6bremWgAPg7Umd+9Om9DzXN6Je8bcVk1qOhd1jcdR6cxS2RGfOuedJsDzm+dVaV/K83ACa7MIxLZjOinj85RSz2NQqiakL0KAJywPqouyUrKLTQzH4q7yxXL7bNrwguaqEgPeV+7ILyaIAky46cuZcz91ezQadbED8aKzqaAbgbp4AVeJiPuoz2AwGPReGDZ/tK4i94NR8iLwC+mIPjd31ETUe4ij8+5of2I8x+Nx9TzdK4K9TkxM6watvACqB2NVQRGgmQMNDVcea6EJDAYD7OzsYDbrb6PMBJBKZZdKzESVEltcpZbJlfNkcfFR1Xx2tbkYBr7PuLYjwrQ2Hv/f7URTy2HQ/pckn0oK3qo9tDBXtdcV8+DCLsCyFOV+q+rsxpu1It19aXNzs0f6uh2fXPSm0wBrfvSa67kGNmNiXFLql/riupCa9zEej3sEaEqpR/oOBoOlZ8TfsclXQtMEGhquONZCEwD6RT4c2aQr5WAw6NlsbM+rDcznOynBbbisPcDHhDuCku1cDRBhVx7ftwskif+7oCgXoRfQ/pdIOiWjdnZ2sLOzA2BZgkUbSlRxsFAt85M/0/vk8Xf7K6o2w9KR6yDoODB3VHKpMtw5zD3oWLImqloNcyul3JJo0/0WWA6KYiLR7cmhmlFgVQ4DsCaLQMQJAH0yhZnTIKMi1JVVHVbRtYos4zzppm6yc191cWGTgh8yJ8Ho+brdlv6t13SRabVoSRd9xu3p+Tx+LvQUWCa2ov3JZNKNt94vE7E86UuMuvaDj6XxYLXXxQLwEVgWCC5E2PVJ++0EAoeQA/MXmYVV6T6dOcibs/I4x/mu/mZAvQ8tYrChoWEl1kYTiKIimtQR5NTm5mavpt5kMln6G5i7GHVLqwCTei52wLl+nLtJYxlcnIBL4WSJrb7hVf5/BUuygJNoLLVqWkGozrdv3+7FRrAZpgQUaw7Rbjwzp7az+cAakrp1eX8AlXyTyaSo/sa96lE1P1a13Vxw0lPnBD/fGJeQ4kz48YYxauq5xDgXjRlt8PbpvIenPisev1XmQNMEGhquONZCEwDOiD+XHgmUs+fUFufoMGcTansu4o2hEsRJWyb1+H5Ycum1XRahtss2q9sHQW1rRxYGXAqqC5KZTCbdParUB/pjyeOnrkV2GbJ25tJ6azwBFzXh/zNc3gmPgc6T0WhkA2xc4JiCn13cIwcEAcu2eIzH1tZWj1PhPjqOR3kixw85zoNJxlo+BnCXmkBK6VtTSr+aUvpwSum9KaWtlNKrU0qPp5R+M6X0vjTfoqyhoWFNcceaQErp5QD+JoDX5pwPU0rvB/AWAG8C8E9zzo+llH4AwFsBfP+KtjAcDpdipdVe4y3EeVXX+gOTyaS440rJZq6xvo71d8yxW7FLLO7GxkYvzp2lkHOTRvuuKGYtZLrk/eAjn89ait4nu8S4jxzEE/cXbZXctfwZS3G9NmuEXBhUg8TYTavam+METk5Oelon7yRV87w4TYAZ/WhLx5s5L5dXou5A9m654DIeK/Xo8Jy76NyBDQDbKaVjADsAngbwpQD+0uL7dwP4DqxYBELtce4p56cNHB4e4uDgAACW3II68fjh6eAzWcgEjbqmlLzR75xbSomhAKuK7mXVazg3j1sE3GLgVEH3nfutuul4oWJVn/3mwPIOxDqObPLxYq2LvksY437oM2bzy+UH6OI4nU57AseRrTy27L+PMdAFMBaD4XDYGz9+qdkEcYldcdT7ZKLUzWvNBTlPUZc7Ngdyzr8L4J9gvvPw0wBuAfhFADdzzvF0nwLwcvf7lNLbUkpPpJSeWMVeNjQ0XBzuxhx4AMCbAbwawE0APwrgy82plpXIOT8K4FEAGI/HOQgjjaPmbbJ0RT04OOhIIlYFdQXm1VwlAks3/k774VKaa+p4SqnrexBDAZZanA+h2YMaEMP9YbW9RiSxGuwIpZoW4VRo16caUeXcXqpW18iuo6OjHjHI0pOfu2oRNbcrmw+uUjEHSAFzSa+uT85wjGfNmZfRPmur+mxZW1LTs6QJqLnrittwWxdJDP4ZAL+dc/50zvkYwE8A+GIA96eUYlRfAeATd3GNhoaGC8bdcAIfA/D6lNIOgEMAbwTwBICfBfBVAB4D8AiAD5ynsSCHNFiIV7Swd8LmPDg46K2Kzl4MuJBLxwm4eGsnMVlaqPaRc7baQ1xbuQmWhmqzl6Sc4yFK0tC5y1ZxAnrPnM3IGonLJ9BrsrSNvx3RF4hnvL+/30nSGlHGQTd6ZDdmwIXwcl5DPLuQ/teuXevC1ZnPUc2hRuA5sphdsqqBuSzM4+PjnsvZZc7ynF61NfkdLwI558dTSj8G4H8COAHwS5ir9/8ewGMppX+4+Oxd52nPFZpgsGrkCCVXUVhfbn756D66v2uVd5m9dtGEqvbytUKddWy1u5aqb5xAxEc1j1hdd6SXM4W0/zW/OBOUTIRFwpEuArwo8cKgpKhTiWOiHxwcdHkl/LLqgum8PHxUr4OLmuTkHFXvt7e3ex4AfsaxaDFBrX3kRYbnle4C7Uw4F6fCHqOSyl+qKM24K+9AzvnbAXy7fPxbAL7wbtptaGi4PKxNxCDgV+faJh4bGxtV1fY80VksQQKsQmmUHcec88YoTt10Kn+pj3yeRtSNRiOrbjpJo1pKLYbAuadqLsW4Rmk8AlwUQyUlg6WikmLOHeie8Xk9S87UchpjLWJQx4aj8VTrc6q802BOT0975fJq7mJ2DZdMGv1/yyJsaGioYi00gZxzt9mlrmSBkq2qAUH8t2tLSa+4Nn/nIswcycQ1/nUHIgdXKMNJJnVdDQaDzgZ3EWZhqzJx5iSIcg5ce8Ht+ONsa2339PR0aRyAOYkGzDUB3deAOQEXnFUiuPReXGSkCxIqjYf7zhVvCSk9Ho+XcvS1TY3y4xwWDtxxmoPLLVHwOKj7kvkeV77ss6LaMENVHUdq8ITRPeadzzngGOTSAqELiVPfa8STY47dvbhFwIWnKmvNixEvGqqyKjGn7Wu0n4s14Pt0qdVcqRhY9q2ricPqKROI8SLokUkvF+9Rmvx6nwGX9MXVjJ1XKo7OvON4Br6n09PT3hzjZ8CflaJRHYHMEZc8PxzZG/emtR8VzRxoaLjiWAtNIOKyV7lyArxKO02gJr1L/n8GS1kXMeiqCDuJpO467oOT9tpvJ/U58k6llYuIXLUHgJbFYglWizRzppZKRZZkbudcNkF0/GpmiTMDXdw/j4WOLVcDZk1BVX4eR+eSLW2cW9JgXZxKaE4uTsBpk7UEMAUnq5XQNIGGhiuOtdEEXJVZoJzuCngXIa+eLujGRZXV4uedG0lXXY4E48xBrTbLbjJXWVbdPM49VXNZnRcsPXXc2T2qUo4lmZYUA/rSaDKZ9HbV4fZZ44nIPHdPSty6vAnmcZxW5qI21X7moCXVkJh/YnteXcnsunSFT1UL4vR51d54bjoy0kHHb1W0INA0gYaGK4+10ASAs5BN3VvAheYyK66S7OjoyO4fGOcoG+7a5bh8F8rr+ArHZZQyEE9OTnphqWw/q73NOeRafIP/dq4gl9HHfdVcAKAf+8+2u8sLCGj/S33Tz9g1V9O8WEI6745zDcb5zkOj88M9T/5O+8YegwDnt8R37HGI8Qut8OTkpMfe8725DUmdhqveAdbePmtchIPBYKmard786elpb/I437CLkV8VHahwqiWrmI7Uc8UldENP9uuG+hvf8eYZNdKtNHal+wuUJoIrUMHjoJ/VXj6Nu9/e3u69mC63gxcXjsLU6zh3Kj+/80QPukrBzjzSRDYmbmumIVf5DfDcVJKOC+nUNozhcVdzhwuYOFPVRWsymjnQ0HDFsRaawGAwwO7u7pJa5FZYJ7U1ss8RVbWgHnaJOY1B1UgXnz8ejzvpFxl1Ozs7S6WmuD+j0aj7jt1OJWJwd3e3t/q7bDJWTzVAhb9jLUfVR5Y+TvMKsJsqzovnF2PAxVS45JhmzR0fH3fpwpExyOqse7b6fJxUZujcKWUiapGQ0N547ji3oc4TJnq5r9rudDrt5aew6q95BW5zWp4LjmAvmUnd99VvGxoaXvBYG01ge3t7qSSTCyhxIcWO6HOhsIB3LbErh6Wt0wCiDUew8b0A85U+JKH2h4N/SiGjwPKq7rLm4m8uLqm7L9X2XihlS7qMxTi67dOZtwHOpD7fk6uSzPa/tsvPx9nxjqQraYUl4tFJyFLYNZN6XNla5wmHIDveQuECn1xNCn4+riK3zkXm1i60nsDziXgxSrsSM2kT5/CD54mnD8SxyjzgOun5e8f664NnYtCVnHYLj2PZSwsPg4kr9XO7qEA3Abkfrky3VgNm8lDJK0dUBTY3N7vzmDjlTTv1d1xRCJgXFaltLMux+iUTjoUF9zX+5nuKew6zJNT3wWCwZLoFdE6ySediE9grEP0peUZ4zgccec51G/k8YD7eqxaBZg40NFxxrI0mACyv5rEC84qpqyLXZ3NulYD6mfl8F3XIq7OL2df9BHZ2dnD9+nUAZ2m0m5ubVgOI+2QJBvgMM1Wz+ZqsKgZY4rlYBiUBWXNgKVRLo1bCkdtnDS3+rxWXr1271v2GzZjob3zGY8ckbnzmtkMLuLiPmkruTENXqVqJQZb2zvVcy2IMsMbooJoGpwazxuMyCwFf9EXRNIGGhiuOtdIEuDhjLffeRWzFare1tVWUVm6bKW63lFEILJfM4msBcxdeVKIN9xjneGvs+/HxcY9k4siumguIORCnYWjQCEtzJQ1dgUp2tSlZx4FYTHIyGcZjllLqNIDQlHZ2dnqRdM4N6DgKtqe1H+ySc/ekUpmDdlwAVCkbk8HPWH9X0jyU/HOSmjUpPZ81TEdQKt/CbZSwNotATGAXehrfu+SIuEHeBy8+i8Hkyjuqhjv/MrPxOmk44YP3otdrsmkT14qX7/DwsBdZNpvNuntQFdNFyPHEjs+m06klnrQN97K4RVG/K53j1O/4v5pOPOmdX59DpYF5GHiQdJGizJ/FmLIqXyPfOKFJFxLujy5o3AZDf1uL3uR5xaakEqTRf15gNeqUrzWbzbrPY05y8tqqRaCZAw0NVxxroQnMZrMudl4JmQCnefJqq+7Ara2tnqRhtUmrwToijIkWVYnZReO2nOIEH04YArAk0ZSQ45h6LcThfPHOh3x8fNyTggGXAMXqrDPD1HV13tRW3e2Xr8/RlXyeJoyxdhPPjDeduX37NoBlTSDG99atWwB85V9HFroiHnoPjtyrpZ47bZI/Y81E8wnYDewke4CLkWjEKpsFzUXY0NBQxdpoAgcHB0s2mboDXYEIltgu9djBxZAr2NWmUn97e7tnp3EWF0sEJZVYorlMOrUvmQxS8g1Aj4dwwT8BF5TELj/WglQas4aiLiiWhro70fb2dieZWKIFSbi7u9uNcSlQajKZ9DILJ5NJF0zE937z5s2lfoRGwKnYLuuwllvi5hNLeHUp8u8d4ajaAWuA+uzc3OT8BtYEXJFX4HnSBFJKP5RS+lRK6cP02YMppZ9JKf3m4vjA4vOUUvpnKaUnU0q/klJ63ar2Gxoa7i3Oown8awD/HMAP02fvAPChnPN3pZTesfj/2zHfmvzhxb8/AeD7F8cqcs44OjpaKqLgXC0aLsnsM9vMnFXHx9ls1rPJnPbBocQuR17tYJf3z54Ix1bX8hpUMrAtyYjVP75jl59yDi5kmnkLtmVLmoD+NsZHpU9I/+vXr+PBBx/s/o5xjPPDrbq5udnLlitpLjpWAZaooSWE5lXKLnRx+W4+Act8QU2yOk8K5/OX3MCuX9PptBcoxV4Nfv6qhcUYs/ZWwspFIOf8X1NKr5KP3wzgDYu/3w3g5zBfBN4M4IfzvJc/n1K6P6V0I+f89IprdDHoTk0GluOoXeosEy36wjDZyBF3wLJqzhO7pPY6Qo6vx754V6OPr1NqI8B9VMKU+1gjqpyL0Km/DkoCutwONskiJuD+++8HADzwwAPdy8/uTyZqA87lB/hYELdB62Aw6BYVrWG4t7fXW1y4PiCPmS58/aZ5cwAAFkNJREFUbq6xH780fkw88lx2BKKSkHzNWECYxNR9HjgfQ58PL44l3Ckx+FC82IvjyxafvxzAx+m8pxaf9ZBSeltK6YmU0hOrOtnQ0HBxeL6JQReVYEOncs6PYr6VOUajUS6twC6Vk1d1XYl5804Xz63fsXsvMJvNeuRfgDWHpRs3gSEBlULu96WgpfhOC3G4+2RT6DzBVu5aHOSi9+Ri1FmbCE3gRS96EYC5CaDur5xzJ92CyEspVTWB6DcTjloGbDwe48UvfjGAvptsd3e3R87u7e111+S+uSCrgCvo4oJ4Ampycq6G27ZOTbOUktXeVLPkWpsxfmp21HCnmsAnU0o3Fp26AeBTi8+fAvBKOu8VAD5xh9doaGi4BNypJvBBAI8A+K7F8QP0+TellB7DnBC8tYoPAM6kibO3Xew7r7Caucb5B2qTu8AgV3UY6BeJcCG857Gn47p8LSeRWavR6sRMDLKkcuNRCvll8L3XQn0D3C8tmZZS6pXM4v67bDyVypPJpCsvpryJGyt2e3HYeGgicX12LUb7TNK6Ap8aiu20NtYINLgofs/x/A5O2mslaSZu1QXI5x8fH3fBU6H9sIuwpgEC51gEUkrvxZwEfElK6SkA3475y//+lNJbAXwMwFcvTv8pAG8C8CSAAwBfv6r9riMbG0spti4xwxEtAfYhuxc8zuFad9Guqk4ussvFu7sFhKP3HEOv9+JKQ+siwCojM/eqnro6jC4SkCePRkS6mHfeiENLpfOGl7qByHQ6tQsK51AA8wjAeEkDXJVJ23Ami3uZ+NnpwsMvMCch6eLizAL+TnNAdCMRHj/+LUOFBC8CzlOjROLp6elSrgWAbjxXbUEGnM878LWFr95ozs0AvnHlVRsaGtYGaxExGCs7qz8qxXnVDbD6q4RS7VrAsmR32VuOeFK46DyOTdC+8IqvUshlqbFkqxUJcRWFA+xWK+VUxPfRD3Vbsqak7kCuEq1prCyduc8hrcKff3h4WKyN7zZqTSn13Iyz2awjBNXs4nnFz9URbCqV+eiKrajJySXNHClaSzXWWpTD4bAX6zKdTnukNWsMMY7PPfdcdx2nfTBa7kBDwxXHWmgCbN8GlBjilZM/01z9k5OTYqFOboPtKV1tJ5NJz/XDdrlKC9YmtIQXX5clt7PVXRFPbYvvSd1HzGW4ICqNUnPBK/yZ40o0QImzAlk7iN87Dsbl8auWEr87PDzsEb3T6dRuPxcuQg2c4T5y3QkdX977QfvNWaz8/EvlyJiQYwmvGoMjOR0nwNJf5yYXE9WoSa5rUULTBBoarjjWRhMIe6zEbrNtw9KulgtQc8kxXO0AlQgsgbWmv2PU3XWdy8+5gxSc7ee0Aqf5KEPO8eUhVdgl6/rhJLx+5oKWWALHGHBVIC235rwxLFl1XNwW4hxaHeC2lD2fTCa9cms8HqoJcHn7WlWqwGw2s8Ft6jHicdB+D4dDm++h3NhwOFzaKYmvyaHZJazFIgCc+Y9VharFufOEZWLLqevxnZoXriIuD7QjHJUM4lRfdv3pi87/r6ny+uD53l2tO5dPoOm33Aa7/Nwk0/TpBx54oDvyGPER8JuPaBLVc88911tYS2m3wLIL0rm7ODaBXZPAmUp88+ZN/P7v/373N4AuYQ1YJnPVVHEvfCm2pHRP7sXn3zuBEMdSHgy3x/kBaha4vSgUzRxoaLjiWAtNIKXUubBUpWONwElxJe7YbeOi7Fzgjgs+ciRQXMeRgLXoMOfec0E0LkAqrqmqtgteYZXfpWSrxHHBRfy9/n97e7vL1GOtRfMV4v+8exC74+IzR47VIjlVsgLLG4YqKRuawP7+fi+9mDU13nFJny3PFx2/mplZ0hxcungpSpGrY3O72haPKe/lEO03TaChoaGKtdAEgOUAHcC7S1QTYHuRSy1pnXpnT/OKGp9HqCXbYiqxOaCEJZNKTy45pjHhnA/vNB0XbnyecmuhUcXYMNj95UKIA1xYQ/vDmgPfi+YMhDRi+59dZ0pasttLY/DZrRZt8Z4LcR4XK1GSmF3Jzj7nOeZ4gmhDx5b7pmASkElOd3+OA4r+qI0/Ho97GgwHYGkWJJ9XwlosAkFacUEQfdGAfh3BUpx7idzhc/lvfVg8sZWscUUaOJrR1d6rFcXQWoZAvyIOxyHwYuPURzU9dILx+Qzn9w9/P1cFcoRgQBduvj824dw2WrrA8/joC8m+b14gYhHXDVUnk0lv7wK3sPLYuEVAow6dN8vlhygJrPeu1+b/u99yVGIc47y4vwAvGiU0c6Ch4YpjbTSBIIdUdWfJoK4iV/KJ/67V8WNtIdpgCVVaPVmKuyxGVv1VdXZ713MbWs+QpYAzT7TQiNNuasSfK1E2Ho87DeChhx4CALz0pS9d6heDiVjV3liz43NUQvLGsjpmjGiLs/045oEzBIGzLMWbN29ib28PALqUW3YNu2et48L9ZtJQx56fpzP5PlPoWJUIStV6nUZaQtMEGhquONZCEwi42HReaTVKzLn3XBw/S1RXODTAK6ySYmx/aYQe2258VBuSIxP1/KOjo57GEJLKFZfgvHzmI9TOdQFK/DslLV2OhAtGYu5BSTd+Thp5x1oNR+epa5i1JhcZqXUQ+PpxfyH19/f3O74gtAPmIVj7ZI2Ir+kyP2tBQ6V6Ai4SVbWeWgAZk5E8ptpvJklLGZqBtVgEmFhzKnwcHVlTY9n1RWP2PMAPgCe2ElV85E0ttf+sttUWgUBMWCZ0Qh3n7bq4smx8Fn+7qEBngsTYsv9ckfNZPcPf+73fW2rrxo0bXSVhfkk0nTteOK64zGpqPLO4ZxcKy2SnEmEuUtSRkQGXJMamJJ9femZuM1snQPiophj3wxGl2gb3jRdOTgEH5vNEi73UwqkVzRxoaLjiWBtNQKP8zkOmMCHCmkMpkovPdy5IVb3ceSU1X80XbsPdi8tvcKof4JN0mCjlNp07MtpQuCQWjpqLNjgJRclZdoU5t13Ane8KtaiJ49TlcCdzfzY2Nmy0XPxfnxn3g7UQtwFpnO/IXJXeTvV3KecBt4cCt+W0IOdejpwOLfHGqfUlNE2goeGKYy00gQDb7Co9eTVnIsdlrpUCWnglZpeVcxuWAj2c/eqi1ZyWwtqOSkvmN9TdyFKI7VK1//g3LkCKI+7iXLWVgX7VYOUouB/MfTjpqeMymUx6EW8uBZrJNxdcFO1eu3ataz84hiitFcfj4+Put6E5OMnuxpmfnX7Gc001Vv6OtbMaJ6USfjQa9biJ8Xjc46SYZ9F2XaFWRdMEGhquONZCEwgNgFczV2jU5Zxz/HR8VwrljGsB3k5n6aMaAIcRq7RghpeZXpdFFtD2ebVWXsFl+7EEYS+BxuW7IqgsiR1rrhI9vtvb2+u0iOjvzs5OV29A3ZM5515dA+YmWPNye0RGm+r2Go1Gvc/cPXMBERfq7TJKdZz5Gep3PHfOEyLsuCMX6s32v5YSc1uNT6fTHh9TC3NXrMUiAJz5j5XUqUW8cUVXXgxcFd64hhJlLrmE1WSXkqsvKVeAdZFuzg+sKi6bNm4y60LC1Y/YZeQqBcXnTDTGdfgeYkzjN9HurVu3AMxfJs0rGA6H3SIU24/xs4jfxgvJz4AJSBcHEf93m5soKcquMCX82FXJL6suAnzvARfjX8vHcPEB/CyU/CstDDou/Dt94U9OTnqxAC4vo4RmDjQ0XHGshSbgVLOAU2XcausyqtTVxqjtuMOSWgM5HBFWygRz32sbvOo71TyOLuU3oBKe+8tkZC0un/dXUMKRI9Q4ZTvaclFqcdRxnEwmvZwHJsziM1aDYz8Bvg4H8QBzEvDZZ5/t/gbOgpYODg5szUUn0RW1vBOG0wQU7prcbsC5nrUd4Ow58ia5mgezsbFx95pASumHUkqfSil9mD77xymlX08p/UpK6d+mlO6n796ZUnoypfTRlNKfX9V+Q0PDvcV5NIF/DeCfA/hh+uxnALwz53ySUvpuAO8E8PaU0msBvAXAHwXwBwD855TSH8o5V+MWYzV0sdhu9WVXmu4Bx3aXs4trdjTbZBqbzm1piCZLMkcI1VyQzH0o1+BsZnYfau44S169J+eG4/0SOfBEaymw/apSn/PVNXafSUa29VWrYQJM7ejNzc2lfQz0XuI53r59u9MEopho5A64IiQu6IbtZ+c+VO2N+SQX3qv2P5PKTvtQvmpzc7OqpTDhqKHEgdlstrLi8Hn2IvyvKaVXyWf/if778wC+avH3mwE8lnOeAPjtlNKTAL4QwH9fdZ2IbS/FOTv/v3txHPnCRJRLKtKJV1LXAf8icDUergRTqzJbi3TU42w265kUzkQo5UFEHzUikReNAJtTGstwfHzce5lu3rzZRalF8ZGASz3e3d21qr9LTIrvdGE7PDy0m5pqHIQbd15g3aKiHh0WDBydGOOjbHyAF3qX3xBwZDWr8o4E1IW7RG7H/axKIHo+iMFvAPAfFn+/HMDH6bunFp/1kFJ6W0rpiZTSE6tcGA0NDReHuyIGU0rfBuAEwHviI3OaZSVyzo8CeBQANjc3c/j3SwQhq9yOiKuRUbGC8yYUrMaxmy6OJZ8w+/852k+vxZqAk9jqJuN7cWNQStflz3jFV03GST6Wsq46raasAv0MxIODg84sednLXgbgbJ+C3d3dnoQfj8edK9H52VVbYddpSP+bN2925F/c8+3bt3v1DHm+uH0E9FouD8LF5ztNQOFKg7nSahzX4ua3PgN+Fqy5xr1r/gRrmCXc8SKQUnoEwFcAeGM+u8pTAF5Jp70CwCfu9BoNDQ0XjztaBFJKXwbg7QD+dM75gL76IIAfSSl9D+bE4MMA/sd52oxgllK0X2k1c7akkmG8/ZYWoSzZaSX7/PT0tLND47fb29tLMelxdCXHtH2+/xIppefFUQlHDghyhFKNQ2BtQrkM1nj0t6PRqIvfj3EJl97Ozk7Hm7CmobHvrDUFWPpHuyH9b9261ZF/XEBUK+5qjoK2rwQsawdKaG5tbXXP2El2bYs1gQDzSVwiTzNoWQtyZKvjjHSnJ66V4Ob4Ur+q38479l4AbwDwkpTSUwC+HXNvwCaAn1l0/Odzzn895/yrKaX3A/g1zM2Eb1zlGQgwobe4bu/oFgSt1suhsPqisX+Zz4kHp6GrfB6rZdEuh6zqC+OIQW7Xqf5KWjrCisdLSTTnpeAJ68gjJcI4zNT1W8eUQ3jjWvFi8u9iEvMiw+OjJlBM6oODg471j63E9vb2emYJt6sLIadiu4IgfG1dOHQcuY+rQnJVCLm0aBcnwOapK1ajwsUR3jUvhOI83oGvNR+/q3L+dwL4zpVXbmhoWAusRcQgk3m68jkpx4jfcVSZkigugYOlkap0vHo6dUyTYbg2foBXbAWrdKyKlggip/ozeFxK9RXdhp3cZ5YupchFllhx73t7e931Y7zDNcjVhl2BEXZ/6VjF+awJPPPMMwDmJKA+YyZK3XN0Kb9OC3JkYZyrGib325kFalpwvoczrVRqs3bI80SlvdMEuI+rtIGWO9DQcMWxFppAgINutH66I3cGg8FSzDswJ+nCleNSedVOc1II6NuC0SZH2fG5zlXkIhYBLwWcVGHJq3Yux4SzRHUu0BgfdQc6ycdEldMYVDIdHh52tnoQhKGVbW1t9QJgOGWcyT9tlzNBIwcgiMH9/X2bCxBw9vB5bGW2zzWC0WkCrKEF3HeOLOZnp3OeNTB1+fE9MCdQ0qD5WiU0TaCh4YpjLTSBnOclu1lacXAJsGxPBTjYhe1iV3YZmEsV1Sgmk4mVKi7YJtpytmFttXVSR8/n+6vVGnCbtnKbKklLrHa0G3D5BNoGe3A4P18lUzD2m5ubS1pBtKUeAOYh1MaPuaGf6Rg4tytrOep+TSZ7lEODNX+C3a8Mp1lGW+qR4Ou556kepqOjIzs39RmzFum4tFWcwFosAoGNjY1O7dYNODimPcDqZviIeeBc8o+60DgCy8XZ64Pkdvn/zsev/XU5DJ9pNWA+6sNNKfXGjdVxfflYjXRuQ4234DgLV/QjwIuLqsscy+4IPFcD0vWjBjcuznWqZokzL93Cw/1xQgjwQsuh5jbme68JErco1kwWRTMHGhquONIq0uBSOpHSpwHsA3jmXvcFwEvQ+sFo/VjGZ3M//mDO+aX64VosAgCQUnoi5/wFrR+tH60fl9uPZg40NFxxtEWgoeGKY50WgUfvdQcWaP1YRuvHMl5w/VgbTqChoeHeYJ00gYaGhnuAtgg0NFxxrMUikFL6sjTfp+DJlNI7Lumar0wp/WxK6SMppV9NKf2txecPppR+JqX0m4vjA5fUn2FK6ZdSSj+5+P+rU0qPL/rxvpTSeFUbz0Mf7k8p/Via7ynxkZTSF92L8UgpfevimXw4pfTelNLWZY1H8vts2DFIc/yzxbz9lZTS6y64Hxez30fEVd+rfwCGAP4PgNcAGAP4XwBeewnXvQHgdYu/7wPwGwBeC+AfAXjH4vN3APjuSxqHvw3gRwD85OL/7wfwlsXfPwDgb1xCH94N4K8t/h4DuP+yxwPz6tS/DWCbxuGvXtZ4APhTAF4H4MP0mR0DAG/CvNJ2AvB6AI9fcD/+HICNxd/fTf147eK92QTw6sX7NDz3tS56Yp3jZr8IwE/T/9+J+cYml92PDwD4swA+CuDG4rMbAD56Cdd+BYAPAfhSAD+5mFTP0ANfGqML6sP1xcuX5PNLHQ+cla1/EPPclp8E8OcvczwAvEpePjsGAP4lgK91511EP+S7vwjgPYu/l94ZAD8N4IvOe511MAfOvVfBRSHNN1f5fACPA3go5/w0ACyOL7uELnwvgL8HILJJXgzgZs45MnEuY0xeA+DTAP7Vwiz5wZTSLi55PHLOvwvgnwD4GICnAdwC8Iu4/PFglMbgXs7dO9rvw2EdFoFz71VwIRdP6RqAHwfwLTnnvcu6Ll3/KwB8Kuf8i/yxOfWix2QDc/Xz+3POn495Lsel8DOMhb39ZszV2j8AYBfAl5tT18G3fU/mbrqL/T4c1mERuGd7FaSURpgvAO/JOf/E4uNPppRuLL6/AeBTF9yNLwHwlSml3wHwGOYmwfcCuD+lFDnLlzEmTwF4Kuf8+OL/P4b5onDZ4/FnAPx2zvnTOedjAD8B4Itx+ePBKI3Bpc/ddLbfx9flhe5/t/1Yh0XgFwA8vGB/x5hvaPrBi75omidevwvAR3LO30NffRDAI4u/H8GcK7gw5JzfmXN+Rc75VZjf+3/JOX8dgJ/F2R6Pl9GP/wfg4ymlz1189EbMS8df6nhgbga8PqW0s3hG0Y9LHQ9BaQw+COCvLLwErwdwK8yGi0A62+/jK3N/v4+3pJQ2U0qvxmew3weAe08MLhazN2HOzv8fAN92Sdf8k5irTL8C4JcX/96EuT3+IQC/uTg+eInj8AaceQdes3iQTwL4UQCbl3D9Pw7gicWY/DsAD9yL8QDwDwD8OoAPA/g3mLPelzIeAN6LORdxjLmEfWtpDDBXw//FYt7+bwBfcMH9eBJz2z/m6w/Q+d+26MdHAXz5Z3KtFjbc0HDFsQ7mQENDwz1EWwQaGq442iLQ0HDF0RaBhoYrjrYINDRccbRFoKHhiqMtAg0NVxz/H/i1hqnqZEePAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.imshow(avg_image, cmap=plt.cm.gray)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are multiple ways to perform the segementation. To keep it simple, we just detect the cells by setting up the threshold on the average image." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAASdUlEQVR4nO3dfYxc1XnH8e+vBuNAYhlDQH5TMZJJSxExyOIlVC3CSQ0U4VSCCooSN6GyWtGWJK2CXf6glRoptBGhkVrSFZA4FeE1tFiIdktdUFSpOKzBMQbHsIEUFjsYKiBVXLkmefrH3BXj9d3d2bkvc++c30da7cydO3PPnJn7nOece+ceRQRmlq5fGHQBzGywHATMEucgYJY4BwGzxDkImCXOQcAscZUFAUmXStoraVzSpqq2Y2bFqIrzBCTNA14EPgFMAE8D10bEC6VvzMwKOaai1z0PGI+IlwEk3QesB3KDwHwdFws4oaKimM3sjLMPDroItdix69BbEfHhqcurCgLLgNe67k8A53evIGkjsBFgAcdzvtZWVBSzmY2O7hx0EWoxb8n4f+Utr2pMQDnLjuh3RMRIRKyJiDXHclxFxTCb3bqlq1m3dPWgizEwVQWBCWBF1/3lwL6KtmVmBVQVBJ4GVklaKWk+cA2wtaJtmVkBlYwJRMR7kv4QGAXmAXdHxPNVbMvMiqlqYJCIeAx4rKrXN7NyVBYEzNpm3dLVjO5r95GCmQc4x3OX+rRhs8Q5EzDrMrUlLTMzyGulu19/UIcpHQTMZjC5Y+YFg5l22tF9O3vaqZtwfoK7A2aJcyZg1oMmtNhVcSZgljgHAbMKtClzcBAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS5yBgljgHAbPEOQiYJc5BwCxxDgJmiXMQMEtc30FA0gpJT0jaI+l5STdmyxdLelzSS9n/E8srrpmVrUgm8B7wJxHxy8AFwA2SzgQ2AdsiYhWwLbtvZg3VdxCIiP0R8Ux2+3+APcAyYD2wJVttC/DJooU0s+qUMiYg6TTgHGA7cGpE7IdOoABOmeY5GyWNSRo7zKEyimFmfSg874CkDwLfAT4XET+R1NPzImIEGAFYqMVRtBxWvemm5GrTlXXtaIWCgKRj6QSAeyLi4WzxG5KWRMR+SUuAA0ULaYM123x8k487GLRTkaMDAu4C9kTEbV0PbQU2ZLc3AI/0Xzwzq1qRTOAi4FPAc5Imm4o/A74MPCDpeuBV4OpiRbRBmeuMvM4I2qnvIBAR/wFMNwCwtt/XNbN6eUJSO8pcM4BJzgDayacNmyWuEUHgjLMP9t36WLn6+RzWLV3tLKDFGtUdGN2301+mlvDnNDwakQmY2eA0KhOwwem1G+AMYPg4EzBLXKOCgFuZ5hvdt9ODuEOmEUHgxV3HOwCYDUgjgoCZDY4HBhPn1N6cCZglrlFBwK1S8/nswOHTqCDgL1e9+gm6PjowfBoVBMysfq0aGJypBXIWMXvLPrWO1i1d7VbdnAmYpa5VmYDl66U1z8uU2pAFdJfR2V41nAmYJa7VmUDqLUO/GUBbdV/IdOp7H6b3WbdWBYHJD7oNaWwdUqiPXrsx7jb0z90Bs8S1KhOY5Eh/pO76qOva/01OzZtWnqZzJmCWuFZmAja9Xlq9MsYQ8rZTV/Zh5SpjVuJ5wBjwekRcIWklcB+wGHgG+FRE/F/R7VhxZe5E3d0B7/ztVkZ34EZgT9f9W4GvRsQq4G3g+hK2YWYVKRQEJC0HfhO4M7sv4BLgoWyVLcAni2zDmq2JrfRkduKfPfemaCZwO/BF4OfZ/ZOAdyLivez+BLAs74mSNkoakzR2mEMFi2Fm/ep7TEDSFcCBiNgh6eLJxTmrRt7zI2IEGAFYqMW561i52npy0UytuadDL67IwOBFwJWSLgcWAAvpZAaLJB2TZQPLgX3Fi2lmVek7CETEZmAzQJYJ/GlEXCfpQeAqOkcINgCPlFBOK0HbMoBeOAMororzBG4C7pP0l8CzwF0VbMMGZOpON4yBJTWlBIGIeBJ4Mrv9MnBeGa9rZtXzGYMJKWNgsOh5+Z5+vnn82wGzxCli8EfnFmpxnK+1s67nw0Hlqqo/n/erxl7Xt+r8Wzy0IyLWTF3equ6AvyzlqHowz4OF7eLugFniGpcJTNeKOAswq4YzAbPENSIInHH2wVnnuHM/c3j5sx2sRgQBq1cTf2LriU4Hx0HALHGtCQJNa7msGs4I6teaIGBm1WjcIcKpnAGYVcuZgFniHAQS1uQsy2MD9XEQMEucg4BZ4hwErNHcJaieg4BZ4hp7iLDJg1bDwq2sQYMzAY8OG7gxqENjg4CZ1aOx3QGr3rqlq3OvHlx3BubWfrCcCZglzplAIvKu9z/bpdzmmhF44tB2KhQEJC0C7gTOojP78GeBvcD9wGnAj4Dfjoi35/ra/rKUo3tHrmKnLuO1bLCKdgf+BviXiPgl4KPAHmATsC0iVgHbsvtm1lB9Tz4iaSHwfeD06HoRSXuBiyNiv6QlwJMR8ZGZXmvNRxfE90ZXHLHMLUf5em29XffDabrJR4pkAqcDbwLfkPSspDslnQCcGhH7AbL/p+Q9WdJGSWOSxt78758VKIaZFVEkCBwDnAvcERHnAD9lDql/RIxExJqIWPP26x9y62M2IEWCwAQwERHbs/sP0QkKb2TdALL/B4oV0cyq1HcQiIgfA69JmuzvrwVeALYCG7JlG4BHCpXQStPES43b4BU9T+CPgHskzQdeBj5DJ7A8IOl64FXg6tle5IyzDzI66t8JNIGDRHoKBYGI2AkcNdpIJyswsxbwGYMJ6m7t/UtN828HzBLXuEzAfdJ69fs7ARsejQgCL+463jv/gPiHPebugFniHATMEucgYJa4RowJ2OB4LMCcCZglzkHALHEOAmaJcxAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS16qfEuddB88/hbVhU/f33JmAWeJakwn4arjt0/2ZpZyxlfHdrfKCsIUyAUmfl/S8pN2S7pW0QNJKSdslvSTp/myKMjNrqL6DgKRlwB8DayLiLGAecA1wK/DViFgFvA1cX0ZB8yJgyq2LNd/ovp2tyGCLdgeOAT4g6TBwPLAfuAT4nezxLcCfA3cU3A7gnb5tUvy8ytzp66q/IlOTvw58hc7Mw/uBd4EdwDsR8V622gSwLO/5kjZKGpM0dphD/RbDzArqOxOQdCKwHlgJvAM8CFyWs2rkPT8iRoARgIVanLuOWVsUyQAGnTEVGRj8OPBKRLwZEYeBh4GPAYskTQaX5cC+gmU0swoVGRN4FbhA0vHA/wJrgTHgCeAq4D5gA/BI0UKaNUEVg3yDzgKg2JjAduAh4Bnguey1RoCbgC9IGgdOAu4qoZxmVpFCRwci4hbglimLXwbOK/K6Zlaf1pwxaNYm06X5TTxvwL8dMEucMwGrzOi+nY0Y+KrTbO+3ifXhIGCl6055q/zhS5O0+f25O2CWOGcCVrrJVnHYugPd76v7fts5EzBLnDMBq8ywtJRTDdv7ciZgljgHAbPEOQgMWFuuPmPDy0HALHEeGByAVFp+X224HZwJmCXOmUCFymrxp3udpreuTS+fdTgINNhsQWTYzlyz8uR9d+YtyV/X3QGzxDkTKNlcuwAzteJ5j7V9UHG28jur6U+R74UzAbPEORMoyVwicZHWrq0tZa/143GO+jkI1Mhf7N7lBQ3XXzXnXrg7YJY4ZwIlmRqVfbZc+YZ1UHGmLtBM77msQWJnAmaJcyZQkba2SmWr85Dm1G215TMYdDlnzQQk3S3pgKTdXcsWS3pc0kvZ/xOz5ZL0NUnjknZJOrfKwptZcb10B74JXDpl2SZgW0SsArZl96EzNfmq7G8jcEc5xbQ2avuJTU2wbunqyjOFWbsDEfFdSadNWbweuDi7vQV4ks5EpOuBb0VEAE9JWiRpSUTsL6vA1nxN2fmHaXC2u/xl12+/A4OnTu7Y2f9TsuXLgNe61pvIlh1F0kZJY5LGDnOoz2KYWVFlDwwqZ1nkrRgRI3SmMmehFueuY+3SlAwgj08+ml6/mcAbkpYAZP8PZMsngBVd6y0H9vVfPDOrWr9BYCuwIbu9AXika/mns6MEFwDvejzAmqhtWcDkBWmryGhm7Q5IupfOIODJkiaAW4AvAw9Iuh54Fbg6W/0x4HJgHDgIfKZQ6awVmtwNGBZTp0ArUy9HB66d5qG1OesGcEPRQplZfXzGoPXFrf9g5B0qLPpZ+LcDZolzELAkDUMmU9bZhA4CZonzmIDNyTC0oNC+Q4QzWbd09Yyfy/vvdTz3cQcBm9Ww7PgwXDt/L3p5v+4OmCXOQcBsCBQZJHQQMEucxwRsVlWeslqXVMYC+nmfDgI2qzbv/DY7dwfMEudMwKY1LBlAKl2BfjkTMEucg4BZ4twdaJkyU/TZ0uRhOCqQmn4mYHEmYJY4BwFrlDom2xhG011/sJcszkHALHEeE2iZMvrpTW5pp07TPazvs0mcCZglzkGgpYa9lZuuj2v5phtL6eV74u5Ai/XyAc80WDTsgSRF/XymzgTMEudMYMjltQzDnmY7w5mbWTMBSXdLOiBpd9eyv5b0A0m7JP2jpEVdj22WNC5pr6R1VRXczMrRS3fgm8ClU5Y9DpwVEWcDLwKbASSdCVwD/Er2nL+TNK+00lophvWEnGF9X1XrZS7C70o6bcqyf+26+xRwVXZ7PXBfRBwCXpE0DpwH/GcppbWByJv6yoZHGQODnwX+Obu9DHit67GJbNlRJG2UNCZp7DCHSiiGmfWj0MCgpJuB94B7JhflrBZ5z42IEWAEYKEW565jzTM13W5KZuBuQP/6DgKSNgBXAGuzKcmh0/Kv6FptObCv/+KZWdX6CgKSLgVuAn49Ig52PbQV+Lak24ClwCrge4VLaTaFW/7yzBoEJN0LXAycLGkCuIXO0YDjgMclATwVEb8fEc9LegB4gU434YaI+FlVhbc0OQCUq5ejA9fmLL5rhvW/BHypSKHMrD4+bdgKqbNV9nkA1XAQMEucfztghVV52NAtf/WcCZglbigzgelaonVLV/d1SWabm15/uei6b4ZWBwF/sdrDn0tzuTtglji9f8bvAAshvQn8FHhr0GUBTsbl6OZyHKnN5fjFiPjw1IWNCAIAksYiYo3L4XK4HPWWw90Bs8Q5CJglrklBYGTQBci4HEdyOY40dOVozJiAmQ1GkzIBMxsABwGzxDUiCEi6NJunYFzSppq2uULSE5L2SHpe0o3Z8sWSHpf0Uvb/xJrKM0/Ss5Ieze6vlLQ9K8f9kubXUIZFkh7K5pTYI+nCQdSHpM9nn8luSfdKWlBXfUwzz0ZuHajja9n3dpekcysuRzXzfUTEQP+AecAPgdOB+cD3gTNr2O4S4Nzs9ofozJ9wJvBXwKZs+Sbg1prq4QvAt4FHs/sPANdkt78O/EENZdgC/F52ez6wqO76oHN16leAD3TVw+/WVR/ArwHnAru7luXWAXA5nSttC7gA2F5xOX4DOCa7fWtXOc7M9pvjgJXZ/jSv521V/cXq4c1eCIx23d8MbB5AOR4BPgHsBZZky5YAe2vY9nJgG3AJ8Gj2pXqr6wM/oo4qKsPCbOfTlOW11gfvX7Z+MZ3ftjwKrKuzPoDTpux8uXUA/D1wbd56VZRjymO/BdyT3T5inwFGgQt73U4TugM9z1VQlWxylXOA7cCpEbEfIPt/Sg1FuB34IvDz7P5JwDsR8V52v446OR14E/hG1i25U9IJ1FwfEfE68BXgVWA/8C6wg/rro9t0dTDI725f833kaUIQ6Hmugko2Ln0Q+A7wuYj4SV3b7dr+FcCBiNjRvThn1arr5Bg66ecdEXEOnd9y1DI+0y3rb6+nk9YuBU4ALstZtQnHtgfy3S0y30eeJgSBgc1VIOlYOgHgnoh4OFv8hqQl2eNLgAMVF+Mi4EpJPwLuo9MluB1YJGnyp9511MkEMBER27P7D9EJCnXXx8eBVyLizYg4DDwMfIz666PbdHVQ+3e3a76P6yLL/YuWowlB4GlgVTb6O5/OhKZbq96oOtdKvwvYExG3dT20FdiQ3d5AZ6ygMhGxOSKWR8RpdN77v0fEdcATvD/HYx3l+DHwmqSPZIvW0rl0fK31QacbcIGk47PPaLIctdbHFNPVwVbg09lRgguAdye7DVXomu/jyjh6vo9rJB0naSVzne+jykGeOQyAXE5ndP6HwM01bfNX6aRMu4Cd2d/ldPrj24CXsv+La6yHi3n/6MDp2Qc5DjwIHFfD9lcDY1md/BNw4iDqA/gL4AfAbuAf6Ix611IfwL10xiIO02lhr5+uDuik4X+bfW+fA9ZUXI5xOn3/ye/r17vWvzkrx17gsrlsy6cNmyWuCd0BMxsgBwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeL+HzgSnDgghdkpAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "threshold = 50\n", + "mask = avg_image > threshold\n", + "plt.imshow(mask)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The outcome is different across different threholds we set. Therefore, this threshold is a parameter we could potentially tweak." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQhUlEQVR4nO3df4wc5X3H8fen/hmTItsQkH+pGMlJS6vGoBMxoYoQTmpCEaYSSEYocVNXVivakqRSsMsfqFIjhTZKaNSK1AISp6IQ6tDaQrRX4xBFlYrDAa4xGPAFWjjsYFCBREFy7ObbP+a5eO/Y8+3N7Mzu7fN5Sdbuzs7ufD13+5nv8+zejiICM8vXL/W6ADPrLYeAWeYcAmaZcwiYZc4hYJY5h4BZ5moLAUlXSXpB0qikbXVtx8yqUR2fE5A0B3gR+AQwBjwB3BgRz3V9Y2ZWydyanvdSYDQiXgKQ9ACwEWgbAvO1IBZyVk2lmHXmg7/5bq9LqMWLBxcB8BPeejMiPjD5/rpCYAXwasvtMeAjrStI2gpsBVjIIj6i9TWVYtaZ4eEDvS6hFhuWrwXg0dj1P+3ur2tOQG2WTRh3RMSOiBiKiKF5LKipDLPOjb9YBkkn/6e6OoExYFXL7ZXA0Zq2ZWZJmSCrqxN4AlgjabWk+cAmYE9N2zKzCmrpBCLilKQ/BoaBOcC9EfFsHdsyG2TjR/bho2eer6gylKlrOEBEPAI8Utfzm1l31BYCZrPR5CPqdEfgTp+nqjonLf2xYbPMuRMwO4NBfNtwMncCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa50CEhaJekxSYclPSvplrR8qaS9ko6kyyXdK9fMuq1KJ3AK+LOI+DVgHXCzpIuAbcC+iFgD7Eu3zaxPlQ6BiDgWEU+l6z8BDgMrgI3AzrTaTuC6qkWaWX26Micg6QLgYmA/cH5EHIMiKIDzpnjMVkkjkkZOcqIbZZhZCZVDQNL7ge8An42IH3f6uIjYERFDETE0jwVVyzCzkiqFgKR5FAFwX0Q8lBa/LmlZun8ZcLxaiWZWpyrvDgi4BzgcEV9puWsPsDld3wzsLl+emdWtylmJLwc+BTwjafwk7n8OfAl4UNIW4BXghmolWj8aPlr8yHM4a++gKx0CEfEfgKa4e33Z5zWzZlXpBCxj7gAGhz82bJY5h4BZ5hwCZplzCJhlzhODNsH4W3/gyb9cuBMwy5w7AZvAR//8uBOwaQ0fPTBhmGCDxSFgljmHgFnmHAJmmfPEoAG0HfOPTxJ6snCwuROwKXlCMA8OAbPMeThg0/KnCAebOwGzzLkTGCB1H7HdBQwmdwJmmXMnkDnP/ptDYIC4XbcyPBwwy5xDwDrWz0OH8Q829XON/cohYJY5h4BZ5ipPDEqaA4wAr0XENZJWAw8AS4GngE9FxM+qbse6ZxBbZk+KlteNTuAW4HDL7TuAr0bEGuAtYEsXtmFmNal6avKVwO8Ad6fbAq4EdqVVdgLXVdmGmdWraidwJ/AF4Ofp9jnA2xFxKt0eA1a0e6CkrZJGJI2c5ETFMsysrNIhIOka4HhEPNm6uM2q0e7xEbEjIoYiYmgeC8qWYSVsWL7WY2j7hSoTg5cD10q6GlgInE3RGSyWNDd1AyuBo9XLNLO6lO4EImJ7RKyMiAuATcB3I+Im4DHg+rTaZmB35SqtL7h7GEx1/O3ArcADkv4SeBq4p4ZtWAUzfYvQL/7B1pUQiIjvAd9L118CLu3G85pZ/fxXhBlqPbJ30hWMr+OOYDD5Y8NmmXMnkLnxo/tMOoLWx9ns5xDI3CD+HYHNjIcDZplzJ5CxJrqAM53ezPqDOwGzzLkTsFI8STg4HAIZ27B8bVeGBJOfw6Ewu3g4YJY5h4B13XTf+utvBe4vDgGzzDkErDbTHe3dEfQHh4BZ5hTR9tu/GjX04YXxg+FVE5Z5hrlZvT4i++ddv0dj15MRMTR5ud8iNGBmf0jU7nFlHtvKf67cOx4OmGXOnYBNcKYj8VRH+m4OJdwRNM+dgFnm+i4E/J34/cs/l8Hk4YDNSLsgmEk4TDV0cMD0Tt91AmbWLHcC1igf8fuPOwGzzPVFJ/DiwUU+Qpj1SKVOQNJiSbskPS/psKTLJC2VtFfSkXS5pFvFmln3VR0O/A3wbxHxq8CHgcPANmBfRKwB9qXbZtanSoeApLOBj5FOOBoRP4uIt4GNwM602k7guqpFmll9qnQCFwJvAN+Q9LSkuyWdBZwfEccA0uV57R4saaukEUkjJzlRoQwzq6JKCMwFLgHuioiLgZ8yg9Y/InZExFBEDM1jQYUyzKyKKiEwBoxFxP50exdFKLwuaRlAujxerUQzq1PpEIiIHwGvSvpQWrQeeA7YA2xOyzYDuytVaGa1qvo5gT8B7pM0H3gJ+AxFsDwoaQvwCnBDxW2YWY0qhUBEHADe83VFFF2Bmc0C/tiwWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYqhYCkz0l6VtIhSfdLWihptaT9ko5I+nY6RZmZ9anSISBpBfCnwFBE/AYwB9gE3AF8NSLWAG8BW7pRqJnVo+pwYC7wPklzgUXAMeBKitOUA+wErqu4DTOrUZVTk78GfJnizMPHgHeAJ4G3I+JUWm0MWNHu8ZK2ShqRNHKSE2XLMLOKqgwHlgAbgdXAcuAs4JNtVo12j4+IHRExFBFD81hQtgwzq6jKcODjwMsR8UZEnAQeAj4KLE7DA4CVwNGKNZpZjaqEwCvAOkmLJAlYDzwHPAZcn9bZDOyuVqKZ1anKnMB+ignAp4Bn0nPtAG4FPi9pFDgHuKcLdZpZTeZOv8rUIuJ24PZJi18CLq3yvGbWHH9i0CxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMlfpOwbNZmr46IFfXN+wfG0PK7FxDgHrmdZAmMwBUU27fTtnWft1PRwwy5w7AetLZ+oSwJ3CVKbbb+24EzDLnDsBa0SZI1TZ58uxS6iyf6ftBCTdK+m4pEMty5ZK2ivpSLpckpZL0tckjUo6KOmS0pWZWSM6GQ58E7hq0rJtwL6IWAPsS7ehODX5mvRvK3BXd8q02Wr46IGudwE2UdX9O20IRMT3gf+dtHgjsDNd3wlc17L8W1F4nOI05VO8MWGDrlcv/tyCp+rwp+zE4PkRcQwgXZ6Xlq8AXm1Zbywtew9JWyWNSBo5yYmSZZhZVd2eGFSbZdFuxYjYQXEqc87W0rbr2OzUL0fhdnUM6qTh+P+rybcIXx9v89Pl8bR8DFjVst5K4GjJbZhZA8qGwB5gc7q+GdjdsvzT6V2CdcA748MGM+tP0w4HJN0PXAGcK2kMuB34EvCgpC3AK8ANafVHgKuBUeBd4DM11GwD5kwter8MLWaLMsOCaUMgIm6c4q71bdYN4OaOt25mPedPDFrXzPSo3ckkXes67go6N5P95r8dMMucOwGrrKkjtLuCck7vt9G297sTMMucOwGrpFdH5MnzCe4MynMI2IxUfbF1+xN7ndYzqJ8U7AYPB8wy5xCwjs3WlttdwJk5BMwy5xCwWW3D8rU+0lfkEDDLnEPALHN+i9Cm5QnBweZOwCxz7gRs4LgDmBl3Ajatbs7A13ESktk6XOkXDgGzzHk4YI0bP3JX6S589O8edwJmmXMnYNOq66hbtiNwF9Bd7gTMMucQsGnV/fn8To/sfiegHg4B61jdQTDVC9wv/no5BMwy54lBm5EqJ74sYybb8ScFy5m2E5B0r6Tjkg61LPtrSc9LOijpnyUtbrlvu6RRSS9I2lBX4WbWHZ10At8E/hb4VsuyvcD2iDgl6Q5gO3CrpIuATcCvA8uBRyV9MCL+r7tlW690owM40zcFl31+dwHldXIuwu9LumDSsn9vufk4cH26vhF4ICJOAC9LGgUuBf6zK9Vaz3VjOOBJvv7SjYnB3wf+NV1fAbzact9YWvYekrZKGpE0cpITXSjDzMqoNDEo6TbgFHDf+KI2q0W7x0bEDmAHwNla2nYd61+9PiWY2//uKR0CkjYD1wDr0ynJoTjyr2pZbSVwtHx5Zla3UsMBSVcBtwLXRsS7LXftATZJWiBpNbAG+EH1Ms1OcxfQXdN2ApLuB64AzpU0BtxO8W7AAmCvJIDHI+IPI+JZSQ8Cz1EME272OwPWLX7x16OTdwdubLP4njOs/0Xgi1WKMrPm+BODVlmdnyL00b9+/tsBs8y5E7Cu6fSo3Y2vF7PucSdgljl3AtY4dwD9xZ2AWeYcAmaZ0+lP/PawCOkN4KfAm72uBTgX19HKdUw0m+v4lYj4wOSFfRECAJJGImLIdbgO19FsHR4OmGXOIWCWuX4KgR29LiBxHRO5jokGro6+mRMws97op07AzHrAIWCWub4IAUlXpfMUjEra1tA2V0l6TNJhSc9KuiUtXyppr6Qj6XJJQ/XMkfS0pIfT7dWS9qc6vi1pfgM1LJa0K51T4rCky3qxPyR9Lv1MDkm6X9LCpvbHFOfZaLsPVPha+r09KOmSmuuo53wfEdHTf8Ac4IfAhcB84L+AixrY7jLgknT9l4EXgYuAvwK2peXbgDsa2g+fB/4ReDjdfhDYlK5/HfijBmrYCfxBuj4fWNz0/qD4duqXgfe17Iffa2p/AB8DLgEOtSxruw+Aqym+aVvAOmB/zXX8NjA3Xb+jpY6L0utmAbA6vZ7mdLytun+xOvjPXgYMt9zeTnFik6br2A18AngBWJaWLQNeaGDbK4F9wJXAw+mX6s2WH/iEfVRTDWenF58mLW90f3D6a+uXUvyB28PAhib3B3DBpBdf230A/D1wY7v16qhj0n2/C9yXrk94zQDDwGWdbqcfhgMdn6ugLunkKhcD+4HzI+IYQLo8r4ES7gS+APw83T4HeDsiTqXbTeyTC4E3gG+kYcndks6i4f0REa8BXwZeAY4B7wBP0vz+aDXVPujl726p83200w8h0PG5CmrZuPR+4DvAZyPix01tt2X71wDHI+LJ1sVtVq17n8ylaD/vioiLKf6Wo5H5mVZpvL2Roq1dDpwFfLLNqv3w3nZPfnernO+jnX4IgZ6dq0DSPIoAuC8iHkqLX5e0LN2/DDhecxmXA9dK+m/gAYohwZ3AYknj3/fQxD4ZA8YiYn+6vYsiFJreHx8HXo6INyLiJPAQ8FGa3x+tptoHjf/utpzv46ZIvX/VOvohBJ4A1qTZ3/kUJzTdU/dGVXxX+j3A4Yj4Sstde4DN6fpmirmC2kTE9ohYGREXUPzfvxsRNwGPcfocj03U8SPgVUkfSovWU3x1fKP7g2IYsE7SovQzGq+j0f0xyVT7YA/w6fQuwTrgnfFhQx1qO99HnZM8M5gAuZpidv6HwG0NbfO3KFqmg8CB9O9qivH4PuBIulza4H64gtPvDlyYfpCjwD8BCxrY/lpgJO2TfwGW9GJ/AH8BPA8cAv6BYta7kf0B3E8xF3GS4gi7Zap9QNGG/136vX0GGKq5jlGKsf/47+vXW9a/LdXxAvDJmWzLHxs2y1w/DAfMrIccAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhl7v8B+wWmT91sIhwAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "threshold = 60\n", + "mask = avg_image > threshold\n", + "plt.imshow(mask)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we could use scipy.ndimage to detect the blobs from this binary mask\n", + "\n", + "For the detailed tutorial, please refer to https://scipy-lectures.org/advanced/image_processing/index.html#segmentation" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "from scipy import ndimage" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The method `label` marks each blob with different number, `label_im` is the image with different blobs marked with different number and `nb_labels` is the number of blobs that are detected" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "10\n" + ] + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQzUlEQVR4nO3df4wc9X3G8fdTn30Ep8g2BGR8qLaDm9aKUqBXMFAhhJPwowhTCSRTRK6pK6sVbQmpFNvlD1SpUeI2CjRSRWoBiRM5EOrQ2kK0FAxRVDV2OcA1BkN8mBQOOxhUICmRjJ18+sd+D+8de769nZ3Zvf0+L+m0O7OzOx+Pd5/5fGd/jCICM8vXr3S6ADPrLIeAWeYcAmaZcwiYZc4hYJY5h4BZ5koLAUlXSHpR0oik9WWtx8yKURmfE5A0C/gR8ClgFHgSuCEinm/7ysyskL6SHvd8YCQiDgBIuh9YBTQMgTnqj5OYW1IpZs1R/5xOl1CKOPIeAD/jrTcj4iMTby8rBBYBr9ZNjwIX1C8gaS2wFuAkTuYCrSypFLPm9A0s7nQJpTh24McAPBZb/6fR7WUdE1CDeePGHRGxKSIGI2JwNv0llWHWvLEXSy9p5t9UVicwCpxVNz0AHCxpXWaWtBJkZXUCTwLLJC2RNAdYDWwvaV1mVkApnUBEHJP0Z8AjwCzg3oh4rox1mfWysT1739LFTS3XirKGA0TEw8DDZT2+mbVHaSFgNhNN3KNOtQdu9nGKKvOgpT82bJY5dwJmJ9CLbxtO5E7ALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzLUcApLOkvSEpH2SnpN0S5q/QNKjkvany/ntK9fM2q1IJ3AM+MuI+E1gBXCzpOXAemBHRCwDdqRpM+tSLYdARByKiKfT9Z8B+4BFwCpgc1psM3Bt0SLNrDxtOSYgaTFwLrALOCMiDkEtKIDTJ7nPWknDkoaPcqQdZZhZCwqHgKQPA98DPhcRP232fhGxKSIGI2JwNv1FyzCzFhUKAUmzqQXAloh4MM1+XdLCdPtC4HCxEs2sTEXeHRBwD7AvIr5ad9N2YChdHwK2tV6emZWtyFmJLwZuAp6VtDvN+yvgy8ADktYArwDXFyvRutGxy34bgL7Hn+pwJVZUyyEQEf8BaJKbV7b6uGZWrSKdgGXMHUDv8MeGzTLnEDDLnEPALHMOAbPM+cCgjfP2TRe+f33et3/YwUqsKu4EzDLnTsDG8d4/P+4EbEqvrbuI19Zd1OkyrCQOAbPMOQTMMucQMMucDwwaACN3rPjAvLNv3QnAoo3/WXU5ViF3AjapkTtWNAwH6y0OAbPMeThgU6rvBsaGCNY73AmYZc6dQA858J1z3r++9A92n2DJ1rgL6E3uBMwy504gcz76bw6BHlLGEMB6n4cDZplzCFjTunrosGPg+J9Ni0PALHMOAbPMFT4wKGkWMAy8FhFXS1oC3A8sAJ4GboqI94qux9qnq9v6Vq0c7XQFM1Y7OoFbgH110xuBOyJiGfAWsKYN6zCzkhQ9NfkA8HvA3WlawGXA1rTIZuDaIusws3IV7QTuBL4A/DJNnwq8HRHH0vQosKjRHSWtlTQsafgoRwqWYWatajkEJF0NHI6I+jNTNjpLcTS6f0RsiojBiBicTX+rZVgLzr51p78HYO8rcmDwYuAaSVcBJwGnUOsM5knqS93AAHCweJlmVpaWO4GI2BARAxGxGFgNPB4RNwJPANelxYaAbYWrtK7g7qE3lfHdgXXA/ZL+BngGuKeEdVgB032L0C/+3taWEIiI7wPfT9cPAOe343HNrHz+FmGG6vfszXQFY8u4I+hN/tiwWebcCWRubO8+nY6g/n428zkEMteT3yOwafFwwCxz7gQyVkUXcKLTm1l3cCdgljl3AtYSHyTsHQ6BjJ196862DAkmPoZDYWbxcMAscw4Ba7upTmnuU553F4eAWeYcAlaaqfb27gi6g0PALHNdEQJzlwe/s/sX4/6sGt3wU2PuBjrLbxEaML0vEjW6Xyv3reevK3dOV3QCZtY57gRsnBPtiSfb07eznXdHUD13AmaZ67oQePKcWTx5zqxOl2ENeO/cmzwcsGlpFATTCYfJhg4OmM7puk7AzKrlTsAq5T1+93EnYJa5rugE3n1ePhho1iGFOgFJ8yRtlfSCpH2SLpS0QNKjkvany/ntKtbM2q/ocODvgX+LiN8AfgvYB6wHdkTEMmBHmjazLtVyCEg6BbiEdMLRiHgvIt4GVgGb02KbgWuLFmlm5SnSCSwF3gC+IekZSXdLmgucERGHANLl6Y3uLGmtpGFJw0c5UqAMMyuiSAj0AecBd0XEucC7TKP1j4hNETEYEYOz6S9QhpkVUSQERoHRiNiVprdSC4XXJS0ESJeHi5VoZmVqOQQi4ifAq5I+lmatBJ4HtgNDad4QsK1QhWZWqqKfE/hzYIukOcAB4LPUguUBSWuAV4DrC67DzEpUKAQiYjcw2OCmlUUe18yq448Nm2XOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuUAhIulXSc5L2SrpP0kmSlkjaJWm/pO+mU5SZWZdqOQQkLQL+AhiMiI8Ds4DVwEbgjohYBrwFrGlHoWZWjqLDgT7gQ5L6gJOBQ8Bl1E5TDrAZuLbgOsysREVOTf4a8BVqZx4+BLwDPAW8HRHH0mKjwKJG95e0VtKwpOGjHGm1DDMrqMhwYD6wClgCnAnMBa5ssGg0un9EbIqIwYgYnE1/q2WYWUFFhgOfBF6OiDci4ijwIHARMC8NDwAGgIMFazSzEhUJgVeAFZJOliRgJfA88ARwXVpmCNhWrEQzK1ORYwK7qB0AfBp4Nj3WJmAd8HlJI8CpwD1tqNPMStI39SKTi4jbgdsnzD4AnF/kcc2sOv7EoFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa7QbwyaTdeGl/a8f/1LH/1EByuxMQ4B65j6QJjIAVHMIwd3f2DerIWNl/VwwCxz7gSsK52oSwB3CpNp1AFMxZ2AWebcCVglptqzt/PxcuwSWukAxkzZCUi6V9JhSXvr5i2Q9Kik/elyfpovSV+TNCJpj6TzWq7MzCrRzHDgm8AVE+atB3ZExDJgR5qG2qnJl6W/tcBd7SnTZqoNL+1pexdg4xXpAqCJEIiIHwD/O2H2KmBzur4ZuLZu/reiZie105RP8saE9bpOvfhzC57Lzzyn0P1bPTB4RkQcAkiXp6f5i4BX65YbTfM+QNJaScOSho9ypMUyzKyodh8YVIN50WjBiNhE7VTmnKIFDZexmalb9sKN6ujVg4Zj3UCVbxG+Ptbmp8vDaf4ocFbdcgPAwRbXYWYVaDUEtgND6foQsK1u/mfSuwQrgHfGhg1m1p2mHA5Iug+4FDhN0ihwO/Bl4AFJa4BXgOvT4g8DVwEjwM+Bz5ZQs/WYE7Xo3TK0mClaGRZMGQIRccMkN61ssGwANze9djPrOH9i0NpmunvtZg7S1S/jrqB59W8bTtUV+LsDZplzJ2CFVbWHdlfQmuNdwUjD290JmGXOnYAV0qk98sTjCe4MWucQsGkp+mJr9yf2mq2nVz8p2A4eDphlziFgTZupLbe7gBNzCJhlziFgM9qXPvoJ7+kLcgiYZc4hYJY5v0VoU/IBwd7mTsAsc+4ErOe4A5gedwI2pXYegS/jJCQzdbjSLRwCZpnzcMAqN7bnLtJdeO/fPu4EzDLnTsCmVNZet9WOwF1Ae7kTMMucQ8CmVPbn85vds/udgHI4BKxpZQfBZC9wv/jL5RAwy5wPDNq0jHUDVe2Zp7Mef1KwNVN2ApLulXRY0t66eX8n6QVJeyT9s6R5dbdtkDQi6UVJl5dVuJm1h2pnDjvBAtIlwP8B34qIj6d5nwYej4hjkjYCRMQ6ScuB+4DzgTOBx4Bfj4hfnGgdp2hBXKAPnNXMulA7OoAyfinYXcDUHoutT0XE4MT5zZyL8AeSFk+Y9+91kzuB69L1VcD9EXEEeFnSCLVA+GGLdVuXacdwwAf5uks7Dgz+EfCv6foi4NW620bTvA+QtFbSsKThoxxpQxlm1opCBwYl3QYcA7aMzWqwWMPxRkRsAjZBbThQpA6rXqdPCeb2v31aDgFJQ8DVwMo4fmBhFDirbrEB4GDr5ZlZ2VoaDki6AlgHXBMRP6+7aTuwWlK/pCXAMuC/ipdpdpy7gPaashOQdB9wKXCapFHgdmAD0A88KglgZ0T8SUQ8J+kB4Hlqw4Sbp3pnwKxZfvGXo5l3B25oMPueEyz/ReCLRYoys+r4E4NWWJmfIvTev3z+7oBZ5twJWNs0u9dux8+LWfu4EzDLnDsBq5w7gO7iTsAscw4Bs8xN+VXiSoqQ3gDeBd7sdC3AabiOeq5jvJlcx69FxEcmzuyKEACQNNzou86uw3W4jnLr8HDALHMOAbPMdVMIbOp0AYnrGM91jNdzdXTNMQEz64xu6gTMrAMcAmaZ64oQkHRFOk/BiKT1Fa3zLElPSNon6TlJt6T5CyQ9Kml/upxfUT2zJD0j6aE0vUTSrlTHdyXNqaCGeZK2pnNK7JN0YSe2h6Rb0//JXkn3STqpqu0xyXk2Gm4D1XwtPW/3SDqv5DrKOd9HRHT0D5gFvAQsBeYA/w0sr2C9C4Hz0vVfBX4ELAf+Flif5q8HNla0HT4PfAd4KE0/AKxO178O/GkFNWwG/jhdnwPMq3p7UPt16peBD9Vthz+sansAlwDnAXvr5jXcBsBV1H5pW8AKYFfJdXwa6EvXN9bVsTy9bvqBJen1NKvpdZX9xGriH3sh8Ejd9AZgQwfq2AZ8CngRWJjmLQRerGDdA8AO4DLgofSkerPuP3zcNiqphlPSi08T5le6PTj+s/ULqH3B7SHg8iq3B7B4wouv4TYA/hG4odFyZdQx4bbfB7ak6+NeM8AjwIXNrqcbhgNNn6ugLOnkKucCu4AzIuIQQLo8vYIS7gS+APwyTZ8KvB0Rx9J0FdtkKfAG8I00LLlb0lwq3h4R8RrwFeAV4BDwDvAU1W+PepNtg04+d1s630cj3RACTZ+roJSVSx8Gvgd8LiJ+WtV669Z/NXA4Ip6qn91g0bK3SR+19vOuiDiX2nc5Kjk+Uy+Nt1dRa2vPBOYCVzZYtBve2+7Ic7fI+T4a6YYQ6Ni5CiTNphYAWyLiwTT7dUkL0+0LgcMll3ExcI2kHwP3UxsS3AnMkzT2ew9VbJNRYDQidqXprdRCoert8Ung5Yh4IyKOAg8CF1H99qg32Tao/Llbd76PGyP1/kXr6IYQeBJYlo7+zgFWUzt/QalU+630e4B9EfHVupu2A0Pp+hC1YwWliYgNETEQEYup/dsfj4gbgSc4fo7HKur4CfCqpI+lWSup/XR8pduD2jBghaST0//RWB2Vbo8JJtsG24HPpHcJVgDvjA0bylDa+T7KPMgzjQMgV1E7Ov8ScFtF6/xdai3THmB3+ruK2nh8B7A/XS6ocDtcyvF3B5am/8gR4J+A/grWfw4wnLbJvwDzO7E9gL8GXgD2At+mdtS7ku1B7azah4Cj1PawaybbBtTa8H9Iz9tngcGS6xihNvYfe75+vW7521IdLwJXTmdd/tiwWea6YThgZh3kEDDLnEPALHMOAbPMOQTMMucQMMucQ8Asc/8PbqC4fI92GrEAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "label_im, nb_labels = ndimage.label(mask)\n", + "print(nb_labels)\n", + "plt.imshow(label_im) " + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], dtype=int32)" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "np.unique(label_im)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "0 marks the background and 1 to 10 mark the detected blobs. Some of these are too small to be cells. We could set a cutoff on size to filter out the small blobs. Here the cutoff is another parameter that could be tweaked." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQEElEQVR4nO3de4xU93nG8e9TLktMigA7trip4ArS0kq1rZWD4yqyTFJfahkq2RKWlWxTLNTKbZ2kUgz1H1alRgptlLhRK6cIOyGVC6HELchySwkhiirV1Gub2mBswHYLa4jBqu1EiUQgefvH/DYM61l2ds5lZvf3fCQ0c86cmfNyduY57+/M5SgiMLN8/VK3CzCz7nIImGXOIWCWOYeAWeYcAmaZcwiYZa6yEJB0q6RXJR2TtL6q9ZhZMaricwKSpgBHgE8AQ8CzwD0R8XLpKzOzQqZW9LjXA8ci4nUASduAVUDLEJiuvpjBzIpKMWuP+vq6XUIl4uxZAH7EO29HxIdG3l5VCCwATjRNDwEfaV5A0jpgHcAMLuMjWllRKWbtmbL4V7tdQiV+duQ1AL4TO/631e1VHRNQi3kXjTsiYlNE9EdE/zQmZwLbxDL8YplM2vk/VdUJDAGLmqYXAicrWpeZJZ0EWVWdwLPAUklLJE0H1gC7KlqXmRVQSScQEecl/TGwG5gCPB4Rh6pYl9lkNrxnn7Ls0scrigxlqhoOEBFPA09X9fhmVo7KQsBsIhq5Rx1rD9zu4xRV5UFLf2zYLHPuBMwuYTK+bTiSOwGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy13EISFokaZ+kw5IOSXogzZ8raY+ko+lyTnnlmlnZinQC54E/i4hfB1YA90taDqwH9kbEUmBvmjazHtVxCETEqYh4Pl3/EXAYWACsArakxbYAq4sWaWbVKeWYgKTFwLXAfuCqiDgFjaAArhzlPuskDUoaPMfZMsowsw4UDgFJHwS+DXwmIn7Y7v0iYlNE9EdE/zT6ipZhZh0qFAKSptEIgCci4sk0+y1J89Lt84DTxUo0syoVeXdAwGPA4Yj4ctNNu4CBdH0A2Nl5eWZWtSJnJb4R+CTwkqQDad6fA18EtktaCxwH7i5WoplVqeMQiIj/ADTKzSs7fVwzq5c/MWiWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuyLcIbRI5srn/ffOW3TfYhUqsbu4EbFRHNve3DAebXBwCZpnzcMDG1NwNeIgw+bgTMMucQ8Da5i5gcnIImGXOxwQy56P/5k7ALHMOAbPMOQSsbR46TE4OAbPMOQTMMlf43QFJU4BB4M2IuEPSEmAbMBd4HvhkRPy06HqsPG7rrVkZncADwOGm6Y3AVyJiKfAOsLaEdZhZRYqemnwh8LvA5jQt4GZgR1pkC7C6yDrMrFpFO4FHgM8DP0/TlwPvRsT5ND0ELGh1R0nrJA1KGjzH2YJlmFmnOg4BSXcApyPiuebZLRaNVvePiE0R0R8R/dPo67QM68Cy+wb9PQD7hSIHBm8E7pR0OzADmEWjM5gtaWrqBhYCJ4uXaWZV6bgTiIgNEbEwIhYDa4DvRsS9wD7grrTYALCzcJXWE9w9TE5VfIHoQWCbpL8EXgAeq2AdVsB43yL0i39yKyUEIuJ7wPfS9deB68t4XDOrnr9KnKHmPXs7XcHwMu4IJid/bNgsc+4EMje8dx9PR9B8P5v4HAKZ8/cIzMMBs8y5E8hYHV2AT2/W+9wJmGXOnYB1xAcJJw+HQMaW3TdYypBg5GM4FCYWDwfMMucQsNKNdUpzn/K8tzgEzDLnELDKjLW3d0fQGxwCZplzCGSuF35qzN1Ad/ktQgPG90WiVvfr5L7N/HXl7nEnYJY5dwJ2kUvtiUfb05fZzrsjqJ87AbPMOQSsbd47T04eDti4tAqC8YTDaEMHB0z3uBMwy5w7AauV9/i9x52AWeYcAmaZKxQCkmZL2iHpFUmHJd0gaa6kPZKOpss5ZRVrZuUr2gn8DfBvEfFrwG8Bh4H1wN6IWArsTdNm1qM6DgFJs4CPkU44GhE/jYh3gVXAlrTYFmB10SLNrDpFOoGrgTPA1yW9IGmzpJnAVRFxCiBdXtnqzpLWSRqUNHiOswXKMLMiioTAVOA64NGIuBb4MeNo/SNiU0T0R0T/NPoKlGFmRRQJgSFgKCL2p+kdNELhLUnzANLl6WIlmlmVOg6BiPgBcELSh9OslcDLwC5gIM0bAHYWqtDMKlX0E4N/AjwhaTrwOvBpGsGyXdJa4Dhwd8F1mFmFCoVARBwAWn0jZGWRxzWz+vgTg2aZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZKxQCkj4r6ZCkg5K2SpohaYmk/ZKOSvpWOkWZmfWojkNA0gLgT4H+iPhNYAqwBtgIfCUilgLvAGvLKNTMqlF0ODAV+ICkqcBlwCngZhqnKQfYAqwuuA4zq1CRU5O/CXyJxpmHTwHvAc8B70bE+bTYELCg1f0lrZM0KGnwHGc7LcPMCioyHJgDrAKWAPOBmcBtLRaNVvePiE0R0R8R/dPo67QMMyuoyHDg48AbEXEmIs4BTwIfBWan4QHAQuBkwRrNrEJFQuA4sELSZZIErAReBvYBd6VlBoCdxUo0syoVOSawn8YBwOeBl9JjbQIeBD4n6RhwOfBYCXWaWUWmjr3I6CLiYeDhEbNfB64v8rhmVh9/YtAscw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDJX6DcGzcZr98kDv7h+y/xruliJDXMIWNc0B8JIDoj6eDhgljl3AtaTLtUlgDuFMrkTMMucOwGrxVh79jIfz13C+IzZCUh6XNJpSQeb5s2VtEfS0XQ5J82XpK9KOibpRUnXVVm8mRXXznDgG8CtI+atB/ZGxFJgb5qGxqnJl6Z/64BHyynTJqrdJw+U3gVYucYMgYj4PvB/I2avArak61uA1U3zvxkNz9A4Tfm8soq1iaVbL34Hz/h0emDwqog4BZAur0zzFwAnmpYbSvPeR9I6SYOSBs9xtsMyzKyosg8MqsW8aLVgRGyicSpzZmluy2VsYuqVvXCrOnzQ8P067QTeGm7z0+XpNH8IWNS03ELgZOflmVnVOg2BXcBAuj4A7Gya/6n0LsEK4L3hYYOZ9aYxhwOStgI3AVdIGgIeBr4IbJe0FjgO3J0Wfxq4HTgG/AT4dAU12yRzqRa9V4YWk9mYIRAR94xy08oWywZwf9GizKw+/sSglWa8e+12DtI1L+OuoBr+7oBZ5twJWGF17aHdFVTDnYBZ5twJWCHd2iOPPJ7gzqBzDgEbl6IvtrI/sdduPf6k4Og8HDDLnEPA2jZRW253AZfmEDDLnEPAJrRb5l/jPX1BDgGzzDkEzDLntwhtTD4gOLm5EzDLnDsBm3TcAYyPOwEbU5lH4Ks4CclEHa70CoeAWeY8HLDaDe+5i3QX3vuXx52AWebcCdiYqtrrdtoRuAsolzsBs8w5BGxMVX8+v909u98JqIZDwNpWdRCM9gL3i79aDgGzzPnAoI3LcDdQ1555POvxJwU7M2YnIOlxSaclHWya99eSXpH0oqR/ljS76bYNko5JelXSLVUVbmblaKcT+Abwt8A3m+btATZExHlJG4ENwIOSlgNrgN8A5gPfkbQsIn5WbtnWLWV0AJf6peBOH99dQOfaORfh9yUtHjHv35smnwHuStdXAdsi4izwhqRjwPXAf5ZSrXVdGcMBH+TrLWUcGPwD4F/T9QXAiabbhtK895G0TtKgpMFznC2hDDPrRKEDg5IeAs4DTwzParFYtLpvRGwCNgHM0tyWy1jv6vYpwdz+l6fjEJA0ANwBrEynJIfGnn9R02ILgZOdl2dmVetoOCDpVuBB4M6I+EnTTbuANZL6JC0BlgL/VbxMswvcBZRrzE5A0lbgJuAKSUPAwzTeDegD9kgCeCYi/jAiDknaDrxMY5hwv98ZsLL4xV+Ndt4duKfF7McusfwXgC8UKcrM6uNPDFphVX6K0Hv/6vm7A2aZcydgpWl3r13Gz4tZedwJmGXOnYDVzh1Ab3EnYJY5h4BZ5nThE79dLEI6A/wYeLvbtQBX4DqauY6LTeQ6fiUiPjRyZk+EAICkwYjodx2uw3XUW4eHA2aZcwiYZa6XQmBTtwtIXMfFXMfFJl0dPXNMwMy6o5c6ATPrAoeAWeZ6IgQk3ZrOU3BM0vqa1rlI0j5JhyUdkvRAmj9X0h5JR9PlnJrqmSLpBUlPpeklkvanOr4laXoNNcyWtCOdU+KwpBu6sT0kfTb9TQ5K2ippRl3bY5TzbLTcBmr4anrevijpuorrqOZ8HxHR1X/AFOA14GpgOvDfwPIa1jsPuC5d/2XgCLAc+CtgfZq/HthY03b4HPCPwFNpejuwJl3/GvBHNdSwBbgvXZ8OzK57e9D4deo3gA80bYffr2t7AB8DrgMONs1ruQ2A22n80raAFcD+iuv4HWBqur6xqY7l6XXTByxJr6cpba+r6idWG//ZG4DdTdMbaJzYpO46dgKfAF4F5qV584BXa1j3QmAvcDPwVHpSvd30B79oG1VUw6z04tOI+bVuDy78bP1cGl9wewq4pc7tASwe8eJruQ2AvwfuabVcFXWMuO33gCfS9YteM8Bu4IZ219MLw4G2z1VQlXRylWuB/cBVEXEKIF1eWUMJjwCfB36epi8H3o2I82m6jm1yNXAG+HoalmyWNJOat0dEvAl8CTgOnALeA56j/u3RbLRt0M3nbkfn+2ilF0Kg7XMVVLJy6YPAt4HPRMQP61pv0/rvAE5HxHPNs1ssWvU2mUqj/Xw0Iq6l8V2OWo7PNEvj7VU02tr5wEzgthaL9sJ721157hY530crvRACXTtXgaRpNALgiYh4Ms1+S9K8dPs84HTFZdwI3Cnpf4BtNIYEjwCzJQ3/3kMd22QIGIqI/Wl6B41QqHt7fBx4IyLORMQ54Engo9S/PZqNtg1qf+42ne/j3ki9f9E6eiEEngWWpqO/02mc0HRX1StV47fSHwMOR8SXm27aBQyk6wM0jhVUJiI2RMTCiFhM4//+3Yi4F9jHhXM81lHHD4ATkj6cZq2k8dPxtW4PGsOAFZIuS3+j4Tpq3R4jjLYNdgGfSu8SrADeGx42VKGy831UeZBnHAdAbqdxdP414KGa1vnbNFqmF4ED6d/tNMbje4Gj6XJujdvhJi68O3B1+kMeA/4J6Kth/dcAg2mb/AswpxvbA/gL4BXgIPAPNI5617I9gK00jkWco7GHXTvaNqDRhv9det6+BPRXXMcxGmP/4efr15qWfyjV8Spw23jW5Y8Nm2WuF4YDZtZFDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMvf/gBt0F9GgVmYAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "size_cutoff = 50\n", + "sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", + "\n", + "small_size_filter = sizes < size_cutoff\n", + "pixel_to_remove = small_size_filter[label_im]\n", + "\n", + "label_im[pixel_to_remove] = 0\n", + "plt.imshow(label_im)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now there are only two blobs left." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next let's separate out each mask." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "rois = []\n", + "for i in np.unique(label_im)[1:]: # 0 is the background\n", + " rois.append(label_im == i)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are two special notes about the analyses we performed above.\n", + "1. The segmentation results are dependent on the parameters `threshold` and `size_cutoff`. Rather than fixing the value of the threshold, we might want to try different values and see what works well.\n", + "2. For the segemtation analyses of each `AverageFrame`, the result is a number of ROIs instead of one. We need to perform the analyses on the level of each `AverageFrame`, but save the result on the granularity of each `Roi`.\n", + "\n", + "Next we would like to introduce two DataJoint table tiers to help handling these two needs." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Parameter `Lookup` table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We would like to perform the segmentation for a **combination** of `AverageFrame`s and different set of paremeters of `threshold` and `size_cutoff` values. To do this while still taking advantage of the `make` and `populate` logic, you would want to define a table to house parameters for segmentation in a `Lookup` table!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's define `Param` table to hold different parameter configuration for our spike detection algorithm. We are going to define this table as a `Lookup` table, rather than a `Manual` table. By now, you know that `Lookup` must be yet another **table tier** in DataJoint. `Lookup` tables are depicted by gray boxes in the Diagram.\n", + "\n", + "This tier indicates that the table will contain information:\n", + "* that will be referenced by other tables\n", + "* that doesn't change much - usually contains a few pre-known entries" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class SegmentationParam(dj.Lookup):\n", + " definition = \"\"\"\n", + " seg_param_id : int # unique id for cell segmentation parameter set\n", + " ---\n", + " threshold : float\n", + " size_cutoff : float\n", + " \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session->Scan\n", + "\n", + "\n", + "\n", + "\n", + "SegmentationParam\n", + "\n", + "\n", + "SegmentationParam\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "AverageFrame\n", + "\n", + "\n", + "AverageFrame\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Scan->AverageFrame\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dj.Diagram(schema)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So far the `SegmentationParam` is an extra table that did not relate with any of the existing, but the `Segmentation` will depend on this table." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Master `Computed` table `Segmentation` and `Part` table for `Roi`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As mentioned above, we performed the segmentation processing on each `AverageFrame`, but the product is the masks of each ROI. In this case, we could create a `Computed` table `Segmentation` containing the `make` method to drive the processing, while using a `Part` table to save results into `Roi`.\n", + "\n", + "`Computed` table and `Part` table are another two table tiers like `Manual`, `Lookup`, and `Imported` we introduced previously. \n", + "\n", + "`Computed` table is very similar to the `Imported` table, which also supports the definition of `make` function and `populate`. The only difference is that the computation in an `Imported` table is dependent on external data, while the computation in a `Computed` table only depends on data inside the database.\n", + "\n", + "Contents in a `Part` table is dependent on its **master** table and the master table could be any type of table. This current example is a very typical usage of Part table. The master `Computed` serves as the driver for computation and the major results with a smaller granularity are saved in the `Part` table." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class Segmentation(dj.Computed):\n", + " definition = \"\"\"\n", + " -> AverageFrame\n", + " -> SegmentationParam\n", + " ---\n", + " segmented_masks : longblob # overview of segmented masks\n", + " \"\"\"\n", + " class Roi(dj.Part):\n", + " definition = \"\"\"\n", + " -> master\n", + " roi_idx : int # index of an roi\n", + " ---\n", + " mask : longblob # mask of this roi\n", + " \"\"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that in the definition of `Roi`, apart from inheriting the primary from its master table `Segmentation`, the Roi has another primary key attribute `roi_idx`. The relationship between `Segmentation` and `Roi` is **one-to-many**." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session->Scan\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation\n", + "\n", + "\n", + "Segmentation\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation.Roi\n", + "\n", + "\n", + "Segmentation.Roi\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation->Segmentation.Roi\n", + "\n", + "\n", + "\n", + "\n", + "SegmentationParam\n", + "\n", + "\n", + "SegmentationParam\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "SegmentationParam->Segmentation\n", + "\n", + "\n", + "\n", + "\n", + "AverageFrame\n", + "\n", + "\n", + "AverageFrame\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "AverageFrame->Segmentation\n", + "\n", + "\n", + "\n", + "\n", + "Scan->AverageFrame\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dj.Diagram(schema)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `Computed` table is labeled as a pink oval and the `Part` table is bare text. We see that `Segmentation` is a `Computed` table that depends on **both AverageFrame and SegmentationParam**. Finally, let's go ahead and implement the `make` method for the `Segmenation` table. " + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class Segmentation(dj.Computed):\n", + " definition = \"\"\"\n", + " -> AverageFrame\n", + " -> SegmentationParam\n", + " ---\n", + " segmented_masks : longblob # overview of segmented masks\n", + " \"\"\"\n", + " class Roi(dj.Part):\n", + " definition = \"\"\"\n", + " -> master\n", + " roi_idx : int # index of an roi\n", + " ---\n", + " mask : longblob # mask of this roi\n", + " \"\"\"\n", + " \n", + " def make(self, key): # key is one of the primary keys of the join product of AverageFrame and ParameterSet\n", + " \n", + " print('Populating for: ', key)\n", + " \n", + " # fetch average image from the previous table AverageFrame\n", + " avg_image = (AverageFrame & key).fetch1('average_frame')\n", + " \n", + " # fetch the parameters threshold and size_cutoff\n", + " threshold, size_cutoff = (SegmentationParam & key).fetch1(\n", + " 'threshold', 'size_cutoff')\n", + " \n", + " # perform the thresholding and blob detection\n", + " mask = avg_image > threshold\n", + " label_im, nb_labels = ndimage.label(mask)\n", + " sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", + "\n", + " small_size_filter = sizes < size_cutoff\n", + " pixel_to_remove = small_size_filter[label_im]\n", + "\n", + " label_im[pixel_to_remove] = 0\n", + " \n", + " rois = []\n", + " for i in np.unique(label_im)[1:]: # 0 is the background\n", + " rois.append(\n", + " dict(**key, # inherit primary key from master table\n", + " roi_idx=i, \n", + " mask=label_im==i))\n", + " \n", + " # insert into the master table first\n", + " self.insert1(\n", + " dict(**key, segmented_masks=label_im)\n", + " )\n", + " print('Detected {} ROIs!\\n'.format(len(rois)))\n", + " # then insert into the part table\n", + " self.Roi.insert(rois)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The implementation of the segmentation is pretty much what we had above, except that we now fetch the value of `threshold` and `size_cutoff` from the `SegmentationParam` table.\n", + "\n", + "**Important note: always insert into the master table first and then insert the corresponding entries in the part table.** If a master table entry does not exist, its corresponding entries would not be valid." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looking at the `Segmentation` table, we see that it indeed inherits the primary key attributes from **both AverageFrame (`mouse_id`, `session_date`, `scan_idx`) and SegmentationParam (`seg_param_id`)**." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And for the part table `Segmenation.Roi`, there was an additional primary key attribute `roi_idx`:`" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

roi_idx

\n", + " index of an roi\n", + "
\n", + "

mask

\n", + " mask of this roi\n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id *roi_idx mask \n", + "+----------+ +------------+ +----------+ +------------+ +---------+ +--------+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation.Roi()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Populating `Segmentation` table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We are now ready to populate! When we call `populate` on `Segmentation`, DataJoint will automatically call `make` on **every valid combination of the parent tables - AverageFrame and SegmentationParam**." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "# ENTER YOUR CODE! - populate the Segmentation table\n", + "Segmentation.populate()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Hm... `populate` doesn't seem to be doing anything... What could be the cause?" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looking at `SegmentationParam` reveals the issue:" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "That's right! We have not added a parameter set yet. Let's go ahead and add one." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "SegmentationParam.insert1((0, 50, 50))" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
050.050.0
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "0 50.0 50.0 \n", + " (Total: 1)" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we should really be ready to perform the computation..." + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 0}\n", + "Detected 6 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 0}\n", + "Detected 6 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 0}\n", + "Detected 9 ROIs!\n", + "\n" + ] + } + ], + "source": [ + "# ENTER YOUR CODE! - populate the Segmenation table for real!\n", + "Segmentation.populate()" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "0 2017-05-15 1 0 =BLOB= \n", + "0 2017-05-15 2 0 =BLOB= \n", + "100 2017-05-25 1 0 =BLOB= \n", + " (Total: 3)" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "...and we now have spike detection running!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Trying out other parameter values" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's see how different thresholds affect the results." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [], + "source": [ + "SegmentationParam.insert1((1, 60, 50)) # add another threshold and size cutoff" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
050.050.0
160.050.0
\n", + " \n", + "

Total: 2

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "0 50.0 50.0 \n", + "1 60.0 50.0 \n", + " (Total: 2)" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam()" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", + "Detected 3 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", + "Detected 2 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n", + "Detected 3 ROIs!\n", + "\n" + ] + } + ], + "source": [ + "# ENTER YOUR CODE! - populate the \"missing\" entry in Segmentation table\n", + "Segmentation.populate()" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", + " \n", + "

Total: 6

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "0 2017-05-15 1 0 =BLOB= \n", + "0 2017-05-15 2 0 =BLOB= \n", + "100 2017-05-25 1 0 =BLOB= \n", + "0 2017-05-15 1 1 =BLOB= \n", + "0 2017-05-15 2 1 =BLOB= \n", + "100 2017-05-25 1 1 =BLOB= \n", + " (Total: 6)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can see that the results of segmentation under different parameter settings can live happily next to each other, without any confusion as to what is what." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Deleting entries from \"upstream\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's say that we decided that we don't like the first threshold of `50`. While there is really nothing wrong keeping those results around, you might decide that you'd rather delete all computations performed with that threshold to keep your tables clean." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "While you can restrict `Segmentation` table to the specific parameter id (i.e. `seg_param_id = 0`) and delete the entries:" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "About to delete:\n", + "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", + "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", + "Proceed? [yes, No]: No\n", + "Cancelled deletes.\n" + ] + } + ], + "source": [ + "# Select 'No' when it pops up\n", + "(Segmentation & 'seg_param_id = 0').delete()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can simply delete the unwanted paramter from the `SegmentationParam` table, and let DataJoint cascade the deletion:" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
050.050.0
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "0 50.0 50.0 \n", + " (Total: 1)" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam & 'seg_param_id = 0'" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "About to delete:\n", + "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", + "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", + "`shan_tutorial_pipeline`.`#segmentation_param`: 1 items\n", + "Proceed? [yes, No]: yes\n", + "Committed.\n" + ] + } + ], + "source": [ + "(SegmentationParam() & 'seg_param_id = 0').delete()" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "0 2017-05-15 1 1 =BLOB= \n", + "0 2017-05-15 2 1 =BLOB= \n", + "100 2017-05-25 1 1 =BLOB= \n", + " (Total: 3)" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Visualize ROIs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we have all ROI masks saved in the table `Segmentation.Roi`, let's quickly look at an example by fetching the `mask` from the table." + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPGklEQVR4nO3df+xddX3H8edrLS2CI1AVUloyStK5sSUT0iDoYozVgcwISzSBmNk5lmaL2/yxRMr8g+wPE9mMOrNF14haF0QZstEQN4YVY5bMzi/KEKjYCht8pVLMBI0mrJ3v/XFP5VK/pd/ec8/9fuHzfCTNvedzz73n3c/3e1/nc8493/tJVSGpXb+w1AVIWlqGgNQ4Q0BqnCEgNc4QkBpnCEiNGywEklyS5IEk+5JsG2o7kvrJENcJJFkBfBt4HTAPfA24sqrun/rGJPWycqDXvQDYV1UPAiT5LHAZsGAIrMrqOpGTBypFEsCP+MH3q+olR7YPFQLrgEfGlueBl4+vkGQrsBXgRE7i5dk8UCmSAL5YN//3Qu1DnRPIAm3POO6oqu1VtamqNp3A6oHKkHQsQ4XAPHDW2PJ64NGBtiWph6FC4GvAxiQbkqwCrgB2DrQtST0Mck6gqg4l+WPgdmAF8Imqum+IbUnqZ6gTg1TVF4AvDPX6kqbDKwalxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxk0cAknOSnJnkj1J7kvyjq59TZI7kuztbk+bXrmSpq3PSOAQ8GdV9avAhcDbk5wLbAN2VdVGYFe3LGmZmjgEqmp/VX29u/8jYA+wDrgM2NGttgO4vG+RkoYzlXMCSc4GzgN2A2dU1X4YBQVw+lGeszXJXJK5gzw1jTIkTaB3CCR5IfB54J1V9cPFPq+qtlfVpqradAKr+5YhaUK9QiDJCYwC4IaquqVrfizJ2u7xtcCBfiVKGlKfTwcCXA/sqaoPjj20E9jS3d8C3Dp5eZKGtrLHc18J/C7wzSR3d21/DrwfuCnJVcDDwJv7lShpSBOHQFX9G5CjPLx50teVNFteMSg1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1bhqzEq9I8o0kt3XLG5LsTrI3yeeSrOpfpqShTGMk8A5gz9jydcCHqmoj8APgqilsQ9JA+k5Nvh74beDj3XKA1wA3d6vsAC7vsw1Jw+o7Evgw8B7gp93yi4AnqupQtzwPrFvoiUm2JplLMneQp3qWIWlSE4dAkjcAB6rqrvHmBVathZ5fVduralNVbTqB1ZOWIamniacmB14JvDHJpcCJwCmMRganJlnZjQbWA4/2L1PSUCYeCVTVNVW1vqrOBq4AvlRVbwHuBN7UrbYFuLV3lZIGM8R1AlcD706yj9E5gusH2IakKelzOPAzVfVl4Mvd/QeBC6bxupKG5xWDUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuN6hUCSU5PcnORbSfYkuSjJmiR3JNnb3Z42rWIlTV/fkcBfA/9SVb8C/AawB9gG7KqqjcCublnSMjVxCCQ5BXgV3YSjVfW/VfUEcBmwo1ttB3B53yIlDafPSOAc4HHgk0m+keTjSU4Gzqiq/QDd7ekLPTnJ1iRzSeYO8lSPMiT10ScEVgLnAx+tqvOAH3McQ/+q2l5Vm6pq0wms7lGGpD76hMA8MF9Vu7vlmxmFwmNJ1gJ0twf6lShpSBOHQFV9D3gkyUu7ps3A/cBOYEvXtgW4tVeFkga1sufz/wS4Ickq4EHgbYyC5aYkVwEPA2/uuQ1JA+oVAlV1N7BpgYc293ldSbPjFYNS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS43qFQJJ3Jbkvyb1JbkxyYpINSXYn2Zvkc90UZZKWqYlDIMk64E+BTVX168AK4ArgOuBDVbUR+AFw1TQKlTSMvocDK4EXJFkJnATsB17DaJpygB3A5T23IWlAfaYm/y7wAUYzD+8HngTuAp6oqkPdavPAuoWen2Rrkrkkcwd5atIyJPXU53DgNOAyYANwJnAy8PoFVq2Fnl9V26tqU1VtOoHVk5Yhqac+hwOvBR6qqser6iBwC/AK4NTu8ABgPfBozxolDahPCDwMXJjkpCQBNgP3A3cCb+rW2QLc2q9ESUPqc05gN6MTgF8Hvtm91nbgauDdSfYBLwKun0Kdkgay8tirHF1VXQtce0Tzg8AFfV5X0ux4xaDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNa7XdwxKx+v2R+/+2f2Lz3zZElaiwwwBLZnxQDiSATE7Hg5IjXMkoGXp2UYJ4EhhmhwJSI1zJKCZONaefZqv5yjh+BxzJJDkE0kOJLl3rG1NkjuS7O1uT+vak+QjSfYluSfJ+UMWL6m/xRwOfAq45Ii2bcCuqtoI7OqWYTQ1+cbu31bgo9MpU89Vtz9699RHAZquY4ZAVX0F+J8jmi8DdnT3dwCXj7V/uka+ymia8rXTKlbPLUv15jd4js+kJwbPqKr9AN3t6V37OuCRsfXmu7afk2Rrkrkkcwd5asIyJPU17RODWaCtFlqxqrYzmsqcU7JmwXX03LRc9sIL1eFJw5836UjgscPD/O72QNc+D5w1tt564NHJy5M0tElDYCewpbu/Bbh1rP2t3acEFwJPHj5skLQ8HfNwIMmNwKuBFyeZB64F3g/clOQq4GHgzd3qXwAuBfYBPwHeNkDNep55tiH6cjm0eD47ZghU1ZVHeWjzAusW8Pa+RUmaHa8Y1NQc7157MSfpxtdxVDAM/3ZAapwjAfU2qz20o4JhOBKQGudIQL0s1R75yPMJjgwmZwjouPR9s037ir3F1uOVgkfn4YDUOENAi/ZcHXI7Cnh2hoDUOENAz2kXn/ky9/Q9GQJS4wwBqXF+RKhj8oTg85sjAalxjgT0vOMI4Pg4EtAxTfMM/BCTkDxXD1eWC0NAapyHA5q5w3vuPqML9/7T40hAapwjAR3TUHvdSUcEjgKmy5GA1DhDQMc09PX5i92z+0nAMAwBLdrQQXC0N7hv/mEZAlLjPDGo43J4NDCrPfPxbMcrBSdzzJFAkk8kOZDk3rG2v0ryrST3JPnHJKeOPXZNkn1JHkhy8VCFS5qOxYwEPgX8DfDpsbY7gGuq6lCS64BrgKuTnAtcAfwacCbwxSS/XFX/N92ytVSmMQJ4tm8KnvT1HQVMbjFzEX4lydlHtP3r2OJXgTd19y8DPltVTwEPJdkHXAD8+1Sq1ZKbxuGAJ/mWl2mcGPx94J+7++uAR8Yem+/afk6SrUnmkswd5KkplCFpEr1ODCZ5L3AIuOFw0wKr1ULPrartwHaAU7JmwXW0fC31lGAO/6dn4hBIsgV4A7C5m5IcRnv+s8ZWWw88Onl5koY20eFAkkuAq4E3VtVPxh7aCVyRZHWSDcBG4D/6lyk9zVHAdB1zJJDkRuDVwIuTzAPXMvo0YDVwRxKAr1bVH1bVfUluAu5ndJjwdj8Z0LT45h/GYj4duHKB5uufZf33Ae/rU5Sk2fGKQfU25FWE7v2H598OSI1zJKCpWexeexpfL6bpcSQgNc6RgGbOEcDy4khAapwhIDUuT1/xu4RFJI8DPwa+v9S1AC/GOsZZxzM9l+v4pap6yZGNyyIEAJLMVdUm67AO65htHR4OSI0zBKTGLacQ2L7UBXSs45ms45med3Usm3MCkpbGchoJSFoChoDUuGURAkku6eYp2Jdk24y2eVaSO5PsSXJfknd07WuS3JFkb3d72ozqWZHkG0lu65Y3JNnd1fG5JKtmUMOpSW7u5pTYk+SipeiPJO/qfib3JrkxyYmz6o+jzLOxYB9k5CPd7+09Sc4fuI5h5vuoqiX9B6wAvgOcA6wC/hM4dwbbXQuc393/ReDbwLnAXwLbuvZtwHUz6od3A58BbuuWbwKu6O5/DPijGdSwA/iD7v4q4NRZ9wejb6d+CHjBWD/83qz6A3gVcD5w71jbgn0AXMrom7YDXAjsHriO3wJWdvevG6vj3O59sxrY0L2fVix6W0P/Yi3iP3sRcPvY8jWMJjaZdR23Aq8DHgDWdm1rgQdmsO31wC7gNcBt3S/V98d+4M/oo4FqOKV78+WI9pn2B09/bf0aRn/gdhtw8Sz7Azj7iDffgn0A/B1w5ULrDVHHEY/9DnBDd/8Z7xngduCixW5nORwOLHqugqF0k6ucB+wGzqiq/QDd7ekzKOHDwHuAn3bLLwKeqKpD3fIs+uQc4HHgk91hyceTnMyM+6Oqvgt8AHgY2A88CdzF7Ptj3NH6YCl/dyea72MhyyEEFj1XwSAbT14IfB54Z1X9cFbbHdv+G4ADVXXXePMCqw7dJysZDT8/WlXnMfpbjpmcnxnXHW9fxmhYeyZwMvD6BVZdDp9tL8nvbp/5PhayHEJgyeYqSHICowC4oapu6ZofS7K2e3wtcGDgMl4JvDHJfwGfZXRI8GHg1CSHv+9hFn0yD8xX1e5u+WZGoTDr/ngt8FBVPV5VB4FbgFcw+/4Yd7Q+mPnv7th8H2+pbuzft47lEAJfAzZ2Z39XMZrQdOfQG83ou9KvB/ZU1QfHHtoJbOnub2F0rmAwVXVNVa2vqrMZ/d+/VFVvAe7k6TkeZ1HH94BHkry0a9rM6KvjZ9ofjA4DLkxyUvczOlzHTPvjCEfrg53AW7tPCS4Enjx82DCEweb7GPIkz3GcALmU0dn57wDvndE2f5PRkOke4O7u36WMjsd3AXu72zUz7IdX8/SnA+d0P8h9wD8Aq2ew/ZcBc12f/BNw2lL0B/AXwLeAe4G/Z3TWeyb9AdzI6FzEQUZ72KuO1geMhuF/2/3efhPYNHAd+xgd+x/+ff3Y2Prv7ep4AHj98WzLy4alxi2HwwFJS8gQkBpnCEiNMwSkxhkCUuMMAalxhoDUuP8Hh33yejtUZzUAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# show one example ROI\n", + "masks = (Segmentation.Roi).fetch('mask')\n", + "plt.imshow(masks[2])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Fluorescence trace of each segmented ROI" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we got masks of ROIs in the table `Segmetation.Roi` obtained with different parameter combinations. We would like to extract the fluorescence trace of each segmentation. \n", + "\n", + "The table design is similar to the `Segmentation` and `Roi`. The master table `Fluorescence` is the driver for the computation, with a secondary attribute `time` shared across traces of all ROIs. The part table `Trace` saves the extracted trace for each ROI over time (frame). " + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [], + "source": [ + "from skimage import io\n", + "import os\n", + "@schema\n", + "class Fluorescence(dj.Imported): # imported table because it also rely on the external tiff file.\n", + " definition = \"\"\"\n", + " -> Segmentation\n", + " ---\n", + " time : longblob # time for each frame\n", + " \"\"\"\n", + " \n", + " class Trace(dj.Part):\n", + " definition = \"\"\"\n", + " -> master\n", + " -> Segmentation.Roi\n", + " ---\n", + " trace : longblob # fluorescence trace of each ROI\n", + " \"\"\"\n", + " \n", + " # the master table is mainly to perform the computation, while the part table contains the result\n", + " def make(self, key):\n", + " \n", + " print('Populating: {}'.format(key))\n", + " # fetch data directory from table Session\n", + " data_path = '../01-Calcium_Imaging/' + (Session & key).fetch1('data_path')\n", + " \n", + " # fetch data file name from table Scan\n", + " file_name = (Scan & key).fetch1('file_name')\n", + " \n", + " # load the file\n", + " im = io.imread(os.path.join(data_path, file_name))\n", + " \n", + " # get dimensions of the image and reshape\n", + " n, w, h = np.shape(im)\n", + " im_reshaped = np.reshape(im, [n, w*h])\n", + " \n", + " # get frames per second to compute time\n", + " fps = (Scan & key).fetch1('fps')\n", + " \n", + " # insert into master table first\n", + " self.insert1(dict(**key, time=np.array(range(n))/fps))\n", + " \n", + " \n", + " # extract traces\n", + " roi_keys, masks = (Segmentation.Roi & key).fetch('KEY', 'mask')\n", + " \n", + " traces = []\n", + " for roi_key, mask in zip(roi_keys, masks):\n", + " \n", + " # reshape mask\n", + " mask_reshaped = np.reshape(mask, [w*h])\n", + " trace = np.mean(im_reshaped[:, mask_reshaped], axis=1)\n", + " \n", + " traces.append(dict(**roi_key, trace=trace))\n", + " \n", + " self.Trace.insert(traces)" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", + "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", + "Populating: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n" + ] + } + ], + "source": [ + "# ENTER YOUR CODE! - populate the Fluorescence table\n", + "Fluorescence.populate()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we could plot the traces of an example scan" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Text(0, 0.5, 'Fluorescence')" + ] + }, + "execution_count": 61, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEJCAYAAAB7UTvrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOydd3zU9f34n++7JJe9B2QPVgaEvYcyFCmCAxUp7lGrrdZWu6zV/r611tbWFkcrtiourOJCcCFDRPYmCSNkkr33zr1/f3xyIeOSXJK7DHw/H497JPcZ78/rIPd5fV5bSClRKBQKhQJAN9gCKBQKhWLooJSCQqFQKFpRSkGhUCgUrSiloFAoFIpWlFJQKBQKRStKKSgUCoWiFZspBSHEq0KIAiFEQoftPxVCnBVCJAoh/tJm+2+EEOdb9l1pK7kUCoVC0TV2Nlz7deAF4A3TBiHE5cBKYIKUsl4I4d+yPQZYDcQCgcDXQogxUspmG8qnUCgUig7YTClIKXcLIcI7bP4x8GcpZX3LMQUt21cC77ZsTxNCnAemA/u6u4avr68MD+94CYVCoVB0x5EjR4qklH7m9tnSUjDHGGCeEOIpoA54REp5CAgC9rc5LqtlWyeEEPcC9wKEhoZy+PBh20qsUCgUlxhCiIyu9g10oNkO8AJmAo8C7wkhBCDMHGu2/4aUcr2UcqqUcqqfn1lFp1AoFIo+MtBKIQv4UGocBIyAb8v2kDbHBQM5AyybQqFQfO8ZaKXwMbAQQAgxBnAAioDNwGohhEEIEQGMBg4OsGwKhULxvcdmMQUhxEbgMsBXCJEFPAG8CrzakqbaANwmtTatiUKI94AkoAl4QGUeKRQKxcAjhnPr7KlTp0oVaFYoFIreIYQ4IqWcam6fqmhWKBQKRStKKSgUCoWiFaUUFL2mpK6ED5M/ZDi7HhUKhXkGunhNcQnw7KFn+TT1UwKcA5gTNGewxVEoFFZEWQqKXpFcmsyW1C0AvH367UGWRqFQWBulFBS94oVjL+Bi78KacWv4NvtbMiq6rJZXKBTDEKUUFBZzsvAkOy7s4LbY27hnwj3Y6ezYeGbjYIulUCisiFIKCotZd3Qd3o7e3BJzC75OviwNX8rH5z+murF6sEVTKBRWQikFhUUcyD3AgbwD3D3+blzsXQBYM24N1Y3VfHL+k0GWTqFQWAuVfaSwiK2pW3FzcOPGsTe2bhvvN54JvhN4I+kNUstTSShKILc6lw1LNxDuET54wioUij6jLAVFj0gp2Ze7j5kjZ2LQG9rtuzX2VrKrstmSugWD3kBJXQnf5Xw3SJIqFIr+oiwFRY+kVaSRV53HvRPu7bTvyvArmew/GR8nHwSCRe8v4lTRqUGQUqFQWAOlFBQ9si9Hm4o6a+Qss/v9nC8OOxrvO55ThUopKBTDFeU+UvTIvpx9hLqFEuwW3OOxE/wmkFmZSVld2QBIplAorI1SCopuaWxu5GDeQWYFmrcSOjLBbwKAciEpFMMUpRQU3XKi8AS1TbUWK4VYn1h0QqeUgkIxTFFKQdEte3P2ohd6po+YbtHxzvbORHlGcbLopI0lUygUtkApBUW37M/dz3jf8bg5uFl8zgTfCSQUJajW2grFMEQpBUWXlNeXk1CUwOzA2b06b7zveMrry8mszLSRZAqFwlYopaDokgO5B5BIi+MJJsb7jQe0BnoKhWJ4oZSCokuO5B/Byc6JON+4Xp0X5RGFs52zUgoKxTBEKQVFl6SVpxHpEYmdrnc1jnqdnljfWJWBpFAMQ5RSUHRJekU6Ye5hfTp3vO94zpaepb653spSKRQKW6KUgsIstU215Fbn9rnb6QTfCTQZmzhdfNq6gikUCpuilILCLJkVWuZQhHtEn843xSESixOtJpNCobA9NlMKQohXhRAFQogEM/seEUJIIYRvy3shhFgnhDgvhDgphJhsK7kUlpFekQ7QZ0shwCUAPyc/Eoo6/fcrFIohjC0thdeBpR03CiFCgCVA2yT2q4DRLa97gX/ZUC6FBaSXpwMQ6hba5zVifWOVpaBQDDNsphSklLuBEjO7ngN+CbQtd10JvCE19gOeQoiRtpJN0TPpFemMcBmBs71zn9eI9YklvTydqoYqK0qmUChsyYDGFIQQK4BsKeWJDruCgAtt3me1bDO3xr1CiMNCiMOFhYU2klSRXp5OuHt4v9aI841DIkkqTrKOUAqFwuYMmFIQQjgDjwG/N7fbzDazjXOklOullFOllFP9/PzMHaLoJ1JK0iv6rxRifWIBFWxWKIYTAzl5LQqIAE4IIQCCgaNCiOlolkFIm2ODgZwBlE3RhuK6Yqoaq/ocZDbh5ehFkGuQCjYrFMOIAbMUpJSnpJT+UspwKWU4miKYLKXMAzYDt7ZkIc0EyqWUuQMlm6I9aeVpAP22FABifGKUpaBQDCNsmZK6EdgHjBVCZAkh7urm8M+AVOA88Apwv63kUvRMRkUG0Pd01LbE+caRXZVNaV1pv9dSKBS2x2buIynlzT3sD2/zuwQesJUsit6RXp6OQW9gpEv/E8DifLQitqTiJOYEzen3egqFwraoimZFJ9Ir0gl1D0Un+v/nEe0TDaDiCgrFMEEpBUUnrJF5ZMLNwY1w93ASipVSUCiGA0opKNrR2NxIVmWW1ZQCaJXNSUWqVkGhGA4opaBox4WqCzTLZiI8+tYIzxxxPnEU1BZQUFNgtTUVCoVtUEpB0Q5Tz6O+zlEwh6ljqoorKBRDH6UUFO3ob3dUc4zxGgNAcmmy1dZUKBS2QSkFRTsyKjLwdvTG3cHdams62zsz0mUkaRVpVltToVDYBqUUFO3IrMi0quvIRIRHBKllqVZfV6FQWBelFBTtyKzMJMQtpOcDe0mkRyTpFekYpdHqaysUCuuhlIKildqmWgpqCvo1WKcrIjwiqG2qJb863+prKxQK66GUgqKVrMosAELdbaMUAFLLlQtJoRjKKKWgaCWzUpuQagtLIdIjErjYgVWhUAxNlFJQtHKhQht+F+wWbPW1TRlNylJQKIY2SikoWsmszMTT4ImHwcPqawshiPSI7KQU1h1dx4HcA1a/nkKh6BtKKShayazMtInryESkZ2Q791FWZRavnHqF5448Z7NrKhSK3qGUgqKVCxUXCHG3fjqqiQj3CErqSiivLwdgd9ZuQJvhnFSsGuYpFEMBpRQUADQ0N5BbnWuTGgUTkZ7tg827s3Yz0mUkBr2BD859YLPrKhQKy1FKQQFAdlU2EmlT91HbtNSaxhoO5h1kSdgSrgy/kq1pW6lprLHZtRUKhWUopaAA4EKllnlkS0sh0CUQB50DqWWp7MvdR6OxkQXBC1g1ZhXVjdV8kf6Fza6tUCgsw2YzmhXDi8yKlhoFGxSumdDr9IR7hJNWkUZlYyWu9q5MCpiEnbBjlOcoNp3bxHWjr7PZ9RUKRc8oS0EBaJlHrvaueBm8bHqdSI9IUspS+DbrW+YEzcFeZ48QglVjVnGq6BRnSs7Y9PoKhaJ7lFJQABcb4QkhbHqdCI8IsquyKawtZEHwgtbtyyOXY9Ab+Cj5I5teX6FQdI9SCgpAS0e1pevIhKndhUAwN2hu63YPgwezAmfxTdY3SCltLodCoTCPUgoKGo2N5FTl2DTzyIQpAyneLx4vx/auqnlB88iuylbDeBSKQUQpBQV5VXk0ySabZh6ZCPcIx93BnSvDr+y0z2Q57MnaY3M5BpMDqcU88M5RGpvVbAnF0MNmSkEI8aoQokAIkdBm21+FEGeEECeFEB8JITzb7PuNEOK8EOKsEKLzHUNhM1q7ow6A+8igN/Dl9V+yJnpNp32BroFEeUTxbfa3NpdjMHnrQCZbT+by2ancwRZFoeiELS2F14GlHbZtA+KklBOAc8BvAIQQMcBqILblnJeEEHobyqZogy1bZpvD1cEVnTD/pzcveB5H8o9csoVsRqNkT3IhAK99lz64wigUZrCZUpBS7gZKOmz7SkrZ1PJ2P2Dq0bwSeFdKWS+lTAPOA9NtJZuiPZkVmTjZOeHr5DvYojA3aC6NxsZLtnNqQk45pTWNTA3z4viFMo5mlg62SApFOwYzpnAn8HnL70HAhTb7slq2dUIIca8Q4rAQ4nBhYaGNRfx+kFWVRZBrkM3TUS1hsv9knO2c2ZN9acYVdp/T/mb/fuNE3BztlLWgGHIMilIQQjwGNAFvmzaZOcxsXqKUcr2UcqqUcqqfn5+tRPxeUVRTRIBzwGCLAYC93p6ZI2eyJ3vPJZmauju5iNhAd0J9nLlpagifn8olt7x2sMVSKFoZcKUghLgNWA78UF781mcBbVNfgoGcgZbt+0pRXRE+Tj6DLUYrc4PnklOdc8lNaausa+RoRinzx2gPM7fNDscoJW/uyxhkyRSKiwyoUhBCLAV+BayQUraNJG4GVgshDEKICGA0cHAgZfu+IqWkqLZoSMQTTMwLmgdwybmQ9qUU02SUzB+tKYUQb2eWxATwzsFM6hqbB1k6hULDlimpG4F9wFghRJYQ4i7gBcAN2CaEOC6E+DeAlDIReA9IAr4AHpBSqm/JAFBeX06TsWlIKYURLiMY7TWarzO+HmxRrMru5EKcHfRMCbtYtLdmRhhlNY3sTSkaRMkUiovYMvvoZinlSCmlvZQyWEr5XynlKClliJRyYsvrvjbHPyWljJJSjpVSft7d2grrUVSr3YyGklIAWBm1kuOFxzlfen6wRbEa3yYXMSvSBwe7i1+7mZHeuDjo2ZZUMIiSKRQXURXN33OK6oamUrg66mrsdHZ8kHxpTGTLKK4mo7imNZ5gwmCnZ/4YP3acyb8kA+uK4YdSCt9zhqql4O3ozeLQxWxO2Ux9c/1gi9Nvdidr/84dlQLA4ugA8ivqSciuGGixFIpOKKXwPae4thgYekoB4Pox11PRUMG2jG2DLUq/ScqpwNvFgXAf5077Lh/nj07AttP5gyCZQtEepRS+5xTVFmHQG3C1dx1sUToxfcR0gl2D+eDc8HchZZXWEOLlZLZA0NvFgSlhXmxXSkExBLBIKQiNtUKI37e8DxVCqDYUlwCmdNShUM3cEZ3Qcf2Y6zmcf5i08uHdTju7tJZgr85WgolF0QEk5lSoQjbFoGOppfASMAu4ueV9JfCiTSRSDChFtUOrcK0j14y6Bjthx/vn3h9sUfqM0SjJKqsl2Mupy2MWR/sD8PVplYWkGFwsVQozpJQPAHUAUspSwMFmUikGjKLaInwdh148wYSvky9LI5byzul32J21e7DF6RNF1fU0NBm7VQpRfq6E+zgrF5Ji0LFUKTS2tLKWAEIIP0BNCLkEKK4tHpJB5rY8PvNxxnqP5ZFvHuFU4anBFqfXZJVqLqGgbpSCEIJF0QHsPV9MdX1Tl8cpFLbGUqWwDvgI8BdCPAXsAf5kM6kUA0KjsZHS+tIhrxSc7Z15cdGLeDt688D2B8isyBxskXqFSSl0F1MAWDTOn4ZmI/tTiwdCLIXCLBYpBSnl28AvgaeBXOAaKeXwdfIqACip1cZdDOWYgglfJ19eXvIyAD/f9fNhVeiVVaq1+Qry7NpSAJgS7oWjvY5vk1XLC8XgYWn20UwgW0r5opTyBSBLCDHDtqIpbM1QrWbuijD3MH425WecLT3L8cLjgy2OxWSV1uLt4oCLwa7b4wx2eqZH+PDdeaUUFIOHpe6jfwFVbd5Xt2xTDGOGcuFaVywNX4qLvQubzm0abFEsJru0tkcrwcTcUT4kF1SRV15nY6kUCvNYqhREm9kHSCmNQPePPYohz1BtcdEdzvbOLItYxlfpX1HRMDzaQmSV1nSbedSWOaO0/wtlLSgGC0uVQqoQ4kEhhH3L6yHg0pqA8j3EpBSGQ0yhLavGrKKuuY6tqVsHW5QekVKSVdp9jUJboke44+PiwB6lFBSDhKVK4T5gNpCNNiVtBnCvrYRSDAxFtUW4O7hj0BsGW5ReEeMTQ7R3NJvObbJ6wNlolDy1NYmE7HKrrFdU1UB9k7HHzCMTOp1g9ihf9pwvGlbBdMWlg6XZRwVSytVSSn8pZYCUco2UUpVeDnOG2sS13rBqzCrOlZ4joSjBquvmVtTxyrdp/OjNI5TXNvZ7PUszj9oyb5QvhZX1nMuv6vlghcLKWJp95CeE+K0QYr0Q4lXTy9bCKWzLcFYKyyKW4WTnZPV5CzllWk1Bdlktv/nwZL+f1rNb1gv2tlwpzBmt/Z8oF5JiMLDUffQJ4AF8DWxt81JYifyKOjYezORkVhnNxoFxGwz1vkfd4ergylURV7E1dWtrFpU1MCmFVVOC+exUHu8eutCv9VqrmXthKQR5OhHp68Ke5MJ+XVuh6AuWZhA5Syl/ZVNJvsfklNVy0/p9XCjRbiBujnYsGOPH326Mx2Cnt9l1h7OlAHB77O18lPwRbyS9wcNTHrbKmqab+JMrYsmvqOMPnyYyNcyL0QFufVyvBk9ne9wc7Xt13pxRvnxwNIuGJmO78Z0Kha2x9K9tixBimU0l+Z6SV17Hza/sp6y6kTfunM66mycxf7QfW07mcji91GbXrWmsobapdlgrhQiPCJZGLGXjmY3sSE7liU8SMPbTysopq8XL2R5Xgx1/uzEee52Ol3dbnmj31v4M/rvnYpvvrF7UKLRl7mhfahqaOZRe0utzFYr+YKlSeAhNMdQJISqEEJVCiOGRJD6EKajUFEJxVQMb7prO/DF+rIgP5M/Xj0cn4GCa7W4Iw7FGwRz3jr+XuqY6/rb/P2zYl8HX/ewymlNWS2DLTdzfzZGF0f7sOFNgkUvvxZ3n+d3HCfzps9OtAebsXqSjtmX+aD/cDHZ8cDSr1+cqFP3B0uwjNymlTkrpKKV0b3nvbmvhLnX++sVZcstr2XDnNCaHerVud3O0J3qku02fEluVwhBum20Jo7xGsSRsCZlNX4Guhpd2pfQrOJxTVteqFACuiBlBSXUDRzK6t9pe/iaFv355lsXRAQjgte/S29QoWJaO2hYnBz3L4wP5/FQeVaprqmIA6e3ktcdb3oeoyWv9o7ymkU9P5nDd5GCmhHl32j8t3JtjmWU0NtumQ/lwLVwzx1XBt4CunrCIwxy/UMa+fnQZzSlr7+5ZMNYPB72ObUl5XZ7z1v4Mnv78DMsnjOTfayfzgwkjefdgJunFNdQ2NvfJUgC4YWowtY3NbD2Z06fzFYq+0NvJa2ta3lehJq/1iw+PZVHXaGTN9FCz+6dHeFPb2Gy1IqqOXCruI4DsAk8aK2Kpc96Nr5vgpZ0pfVqnoq6RyvomAj0dW7e5GuyYPcqHr5Lyu7RAXt2TxpQwL567aSJ2eh33zIukuqGZZ788C/TcMrsrJoV4EuXnwvuHlQtJMXCoyWuDgJSStw9kEh/iSVyQh9ljpoVr1oOt4gpFtUXohR5Pg2fXBzU1QPXQz5Xfl1KMR+MCapqquHxSMXvOF3Eyq6zX65jSUQM7BIaXxASQUVxDckHnYrLq+ibSiquZP9oPe732dYoL8mBWpA9bT+UCvUtHbYsQghumhnA4o5TUQlXIphgYbDZ5raXArUAIkdBmm7cQYpsQIrnlp1fLdiGEWCeEOC+EOCmEmNzHzzMsOJReyvmCKn44w7yVAODnZiDC18VmcYXiumK8Hb3R67pJed35FPxjAuQcs4kM1sBolOxPK2Zu8Ex8HH2oMxzG3dGuT9ZCl0ohOgCAbUmdg9incyuQEuKC2ofY7pkf0fp7dxPXeuK6SUHodYJNR5S1oBgYbDl57XVgaYdtvwa2SylHA9tb3gNcBYxued3LJd6W++0DGbg52nH1hMBuj5sW7sWh9NJ+p1maw6IaheSvoLEa3rkJyjpMOzPnSsk5DpsfhJqBS6M8k1dJWU0js6P8uDL8Svbm7GH1DD++SMwjp6yW/bn7ufqjq8mv7jkrKbtMa1fd8cne392RiSGefJXYOa5gcu/FBra3+C4b488of1fcHe3wcOpdjULHay8Y48eHR7MHrKhR8f3GZpPXpJS7gY53h5XAhpbfNwDXtNn+htTYD3gKIUZa9hGGF8VV9Xx+Ko/rJwfj5NB9Ydr0CB/KaxvNui36S251Lv7O/l0fUFUABUkQvwYa6+DtG7Sb/ZnP4NWl8Ed/2PoIVOSC0Qh7n4f/LIajG+DERqvL2xWmoPKsKB+uiriKBmMD/iOSAfgiIZtnDj5DekU6bya92eNaOWW12OsFfq6dGwQuiQngRFZ5pzkHiTkV+Lg4EODe/hydTvDsDfH88drxff1ordwwJZi8ijq+OafajSlsz0BPXguQUuYCtPw03ZWCgLb9BLJatpmT5V4hxGEhxOHCwuHXBuDDo9k0NBtZ043ryMR0U1zByi6kZmMzGeUZRHhEdH1Q2u4WIe6G1W9BcQo8Fwvv3gzl2RB9NRx5DdZNhPUL4KvfwZgrwW8cJH3SvQBVBZoisQL7UooJ93Em0NOJeL94glyDOFy0k9H+rvzvzCecLztPsGsw7597n/L67oP2OWW1jPBwRKcTnfZdGdviQupQB5GQU0FMoDtCdD5nYognK+K7twYtYVF0AP5uBjbszej3WgpFTwyVyWudv1Et8YtOG6VcL6WcKqWc6ufnZ0URBoZ9qcWM9ndljAVtE0K8nQhwN3DIysHm7KpsGowNRHpEdn1Q6i4weMDIiRAxH657GYKmwPX/hQePwapX4SeHIe56KL8Ay5+Dm96CuFVw4QBUdJFGWVMC/5wIW3/e78/RbJQcSCtmVpSWViuEYGn4Uvbn7mdetB1ZfES0VyzPXf4cNU01vHf2vW7XyymrJdDDvP8/ys+VCF+XdnGF+qZmkvMru0wWsBYOdjrWzgzjm3OFnLeB1ahQtGWgJ6/lm9xCLT9N9nAWENLmuGDgkkzOTswpt/gmIoRgWrg3B9NKrNpbP7Vca9vQo6UQPhdMgei46+H2LTB+Fehb/uu9I+Cal+BX6TD1ThACYlZo+05vMb9u0sdanOLIa10fYyFJORVU1jUxM/JircVVEVfRLJs5WPMsOvtyprqvZZz3OOYEzuGt029R19T1mMucsrouM4WEEFwRE8C+lCIq67SW2sn5VTQZJbGBtq/jXDMjFAe9jg17021+LcX3m4GevLYZuK3l99vQuq+att/akoU0Eyg3uZkuJYqq6smvqO/VTWR6hDd5FXWtjdqsgUkpRHp2YSmUpkNZBkQu6P3ifmO7dyGd2gQ+o2BkPGz+qRaTACg4A29eB/tesvhS+1K1dNlZbZTCGK8xRHlEkVF1Hn1dNGfSNbfPnXF3UlJXwuaUzWbXamo2kldR122m0JKYABqbJbvOam7LroLMtsDX1cCKiYF8cDTLKnMeFIqusNnkNSHERmAfMFYIkSWEuAv4M7BECJEMLGl5D/AZmpI5D7wC3N/LzzEsSMzR2kXF9EIpmNpfHLvQ+7z7rkgtS8XXyRd3hy7kSP1G+xnRB6UAELMSMvdqsYO2lGdBxncw4SbNDdVYCx/fB7ufhZfnQcp22P7/LiqKHjiVXUGwlxP+7heLzYQQLI9ajk7oWBhwO3uSi6iqb2LaiGnE+cTxeuLrNBubO61VUFlPs1F2Skdty6RQL3xcHFpdSIk5Fbga7Ajz7ltxWm+5fXY4NQ3NvNfPdt4KRXfYbPKalPJmKeVIKaW9lDJYSvlfKWWxlHKRlHJ0y8+SlmOllPIBKWWUlHK8lPKwNT7cUCMxp+XJcqTlT5ZjR7hhsNNx0opKIa08rft4Qto34DpCe+rvCzErQRrhTAf3UELLQJzxq8B3NCx9Wotd7Pg/GLsM7t4Oxib45s+dljQrZlEVUX6unbbfFnsbm6/ZzOr4mTQ0G9l5pgAhBHfE3cGFygt8l/Ndp3O6qlFoi14nWBwdwM4zBTQ0GUnMKSdmpLvZwLQtiAvyYHq4Nxv2pav0VIXNsDT76C9CCPcW19F2IUSREGKtrYW71EjM0Z5sPZwtz1u31+uIDXTnRB8qdM0hpSS1PLXreIKUWjwhYr4WI+gL/jHgHdXZhXTqfQiaCt4tCmnK7bD4D3DT23DjBgieClPvgKNvQlFyj58jrbCaCF+XTvvsdfaEuYcxJcwLX1cHvmypL7g85HI8DZ5sSekcyzBNSAtq0+LCHEtiAqisb2JvShGncyt7ZfVZgzvmhJNVWssW1Q9JYSMsdR9dIaWsAJajuY/GAI/aTKpLlKScCmJG9v4mMiHYk4TsCpqs0ByvqLaIqsaqri2FgiSoLuxbPMGEEJq1kPYtVLc0pys4A3mnYPwN7Y+b+zOIXn5x2/xHwc4Rdvyx20vkV9RT3dBMlF9npWBCrxMsidGe7Osam7HX23Nl+JXsvLCT6sbq1uOajc1sz9oKuhpGdpF9ZGLuaF+c7PW8/E0qtY3NNs886siSmADigtz5w6dJFFR2HTQfaGobmq2aDKEYPCxVCqZH22XARpPbR2E5VfVNpBVV9ykoGR/iQW1jM+et0P+mxyBzazxhfv8uFHe95kL67xJI2QEJm0DoIPba7s9z9YfZP9GylLKPdHlYapH2bxFpxn3UlitiR1Dd0MzeFC0ovTxyOXXNdWzP3N56zCcpn7Cz+Hncgz/FxdB9Up2jvZ75Y3xbi+YGIvOoLXZ6Hc/dOJHq+iZ+/cGp1htxYWU9P3/vONv7OU+iL6QWVhH35Jcs+OsuntycyHfni5SCGMZYqhQ+FUKcAaYC21t6Hw2dx5RhwJlcLcjcl5tIfLDWtO6EFeIKrUqhK0sh6WPwGQ2ePRfXdcuIOLjlQ0DCm9fCd+u0wLVbQM/nzvoJOPvAa8u0SuqDr3RqnZFaqD3pm3MftWV2lA+uBju+StRulqYCN5MLqa6pjpeOv4SQdkiXYxzNP9qjeEtiRgBa/cAo/+6Vki0YHeDGr68ax44zBWw8eIF9KcUsW/ctHx7N5o9bT9ukLUp37E8todkoCfZyYuPBTH74nwO8czCz5xMVQxJLA82/RmudPVVK2QjUoLWmUFiIKfMoNqj3SiHcxwU3RztOZPW/jXZqWSqu9q74OZkp/Ms+ohWeTbur39cBIGoh/HgfXP4Y6OwsX9fRHW7bosUcipLhs0dg4+p2h6QVVeNkr2eEe/cxAIOdnsvH+bMtKZ9mo9SykyKXcyDvAAU1Bbx75l3ya/LxrroHB7x5+uDTZrOT2rJonIVVRC0AACAASURBVD86AeNGuLV2Rh1obpsVztxRvvzh00R++J/9uBnseGjRaNKKqtl5dmDbYZzMKsPT2Z63757B8d9fwcQQT9bvTlXB8GGKpYFmZ+ABLlYxB6JZDQoLScwpx9vFocebmDl0OkF8sKdVLAVT5pG5tgzs/zc4uMHEH/b7Oq3YO8KCX8JvW1pjWEpADFz1jFY9vfB3mrIqvdjmIbWwinBfF4syf66MDaC4zfS0H0T+AKM08t7Z93jl1CvMCZxDcVEUE51v4UzJGT5I/qDb9bxcHLhzTgQ3Tg3p9jhbotMJ/nrDBLycHbg6PpDNP53LTxeOItDDsd2M6IHg+IUyJgR7IoTAyUHPvfMjySiuMdtVVjH0sfQx5zWgAa1WAbRgc/eRQEU7EnMqiO2iR44lTAj24GxeJXWNXTzFlmVCc89jG1PLUwn3CO+8oyIXEj+CSWu1J3Vr09dMJiG01hnQLsU1taiayG6CzG1ZMEabnmbqchrhEUGcTxzrT66noqGCu+MeoLKuiWl+lzE1YCrPH3u+xz5Jv1sew9qZYX37TFZipIcT+36zkH+unoSrwQ47vY5bZ4ezN6WY07kDM0K9pqGJ5IIqJgZfjJVdERNAsJcT/93Tl/pWxWBjqVKIklL+BWgEkFLWYr5fkcIMDU1GzuX3L31xQrAnTUZJkrkve1kmrJsMm27vttFcZUMlhbWF5uMJh/+r1QjM6LYmcXDwjoAR4yFJq0ZuaDJyoaSGqB7iCSbcHO2ZM8qHL5PyWgOgP4j8ARLJVeFX4dCsPfGH+rjw6+m/prKhkgd3PEhlQ6VtPo8V6fiQcfO0UJzs9bw6QNZCYk4FzUbJhOCLw5rs9DrunBPBofRSjluxvkYxMFiqFBqEEE5cHLITBdTbTKpLjOSCShqbZb/aIUwM6SbYnLQZjI1w+lPY2bUB12WQubEODr8GY6+6WEMw1IheobmQKvPILKnGKCHCQksBtCykCyW1nM7VbvRXR13N8sjl/GzKz0jI1hRtXKAHY73H8uf5f+Zk4Unu+vIuSutKbfJxbIWHsz2rpgTzyfEciqps/xU1/T1OCGn/t33jtBDcHO34z7fKWhhuWKoUngC+AEKEEG+jDcj5pc2kusRoDTL3w1IY4eGIv5uBk+aCzUmfQMB4LTD77d/guPl5BqllXaSjJmyCmiKYcV+f5bM50SsACWe2tGYeRfpanvmzODoAIeCrJM2F5GHw4Ol5TxPoGsip7HLcHe0I8dZqFJaGL+WfC/9Jankqd3xxB4U1w6tF++1zwmloNvLOAdtnAJ3MKifQwxF/t/axMleDHWumh/J5Qh5ZpTU2l0NhPXpUCkKzT88A1wG3AxvRspB22VSyS4jE7HKcHfRE+Fj+ZGuOCcGenSuby7Mh6yDEroRlz2r1BZt/ChcOdTo/rTwNe509Qa4dRlWceBd8x/a/NsGW+I3VUmVPf0pqUUs6ai8sBT83A1PDvPgysXPw09S5tq0rZn7wfP61+F/kVOfw1IGn+i//ABLl58qkUE92n7O9MjuRVdbOddSW22aHY5SSD45k21wOhfXoUSm0tMz+uKVv0VYp5RYp5dCf5j5EKKtp4KNj2cyO8u13j5yJIR6kFla375J5+lPtZ8w1oLeHG98AF1+tn1AHUstTCXMPw07XpkCrsQ6yDsHoJX0PBg8Eprbcad+Sl5uNr6sBd8fejbm8ImYEp3MruFBy8cm1sdnImVzzMxGmjZjGHXF3sD1zO6cKT/X7Iwwkk0O9OJVdTqMVquC7oqymgYziGuJDzCuFQE8nYgPdWwsHFcMDS91H+4UQ02wqySXKS7tSqKxv4hdXjOn3WqYnspNtrYWkT7ReQ76jtfdOXjDjR1pTu/zEdudnVJiZtpZ9BJrqIGxOv+WzOdFXg2zGL2enxZlHbVkcoxXO7WqTx38uv5KGZmOX7SpujbkVb0dv/nnsn32TeZCYGOJJfZOm8GyFqW4mPrjrWNmsSB+OZZZ1nTWnGHJYqhQuB/YJIVKEECeFEKeEECdtKdilQHZZLa/vTefaSUFE96HnUUcmh3lhpxPsTWnpJ1SZD5n7tD5D7Q68DeycYH/74XhFtUUEOHeoKM74DhAQNqvf8tmckRPBI5TJFduJ9Ol9u+pwH2dCvZ35po1bJbE1yGz+/8fF3oW7x9/NgdwD7M/d3ze5B4FJodoDxPELpXye9jl/3P9H/nXiX7x/7n1yq6wzquTkhTItY7gbpTA7ypeGZmNrjYhi6GPp9LSrbCrFJcrfvjoLwC+u6GML6g64GuyYFOrJd+dbzPEznwKys1Jw9ob41XD8HVj8JLj4UttUS1VjFT5OPu2PTd+jtaRw8rKKjDZFCOrib2PW7v+Duo+A+F6eLlgwxo8PjmbR0GTEwU5HQk45rgY7wruJ99w49kbeSHqDdUfXMWPZjD7XmgwkQZ5O+LkZ2J6+j6PnnsbRzpHappb24C6BbL1ua3s3Yh84kVVOpK9Lt268aRHe6HWCvSlFzBnl26/rKQYGS9tcZACewNUtL8+WbYouSMqp4KNj2dwxO7zLEY99Yc4oX05ll1NW06C5jnzHaJPOOjLjPmiu18ZeAsW1mnXh49hGKTQ1wIWDEDbXavLZmtNRd7CleQYzz/+jtW6hN8wf40dNQzOHM7ReSqeyy4kJ7H4mgkFv4P74+zlVdIodmTv6LPtAIoQgLtiBY7X/IsQthF037uLo2qM8M+8Zcqpz+Drj636tL6XkRFZZl/EEE64GO+KDPS5at4ohj6VtLh4C3gb8W15vCSF+akvBhiN/+eIMC5/dxeT/28bVL+zB3dGe+y8bZdVrzB3li5RwOOm89pQfs9J8gNh/HEQtgoP/gaYGiutalEJbSyHnKDTVQnjv4gkHUot55P0Tg+InTi2q5ReNP6Y+YBJ8eG+3nVTNMSvKB3u94JtzhTQ1GzmdW8F4C9pfXx11NeHu4bx04qVh0wG0zPldmnXl/Hba/+Fs74y93p6lEUsJdQtlQ+KGfn2OvIo6CivrW5s1dsfsKF9OZpVTVd9zxb1i8LE0pnAXMENK+Xsp5e+BmcA9thNraFPT0NTpC7XzTAEv7UohwN2Rq+JG8KP5kWy4c3qvBupYQnyIJy4OeopPfqG1ph7TjWdv5v1QlQfbHqcoQ5s25uvUxoRP36P9DJ1t5mTznM2r5O4Nh9l0JIvPEwZ+jHZaUTVNOgP6H76rtdl+dy00WJ4H72qwY2qYN9+cLSSlsJq6RiNxFjQptNPZce+EezlXeo5dF3b14xMMDFtTt3K+5lsaChfRUBPcul0ndNwScwsJxQkcLzze5/WPZWrJDj1ZCqAp4maj5FCa6rg/HLBUKQig7WNhM9/TNhfFVfXMe2Yn97xxuDXdr7ahmd9vTmCUvysb7pzOU9eO55dLx7VWIVsTe72OmZE+uOXsAUdPCJzY9cFRCzVr4cC/Kf5Gy7X3OfT6xf0Z32mZSy4+5s/vQH5FHXe8dhAnBz1Bnk68dyirH5+kd2SV1vD4xwms/zZV607qHgDX/hsqc+Dg+l6ttWCsH2fyKtl+RqtZiLOw0vyqiKsIdg3m5ZMvD3lr4fljzxPrM56mkstbb+AmVkStwMPgwYbEDX1e/2BaCc4OeosKMqeEeeGg16nU1GFCbxriHRBCPCmEeBLYD/zXZlINYf7xdTKlNQ18fbqAR98/gdEoeWFnMhdKavnjNXE42Nm+lfKcKB8mNh6jJngu6PRdH6jTaTMNHkmmOP5GALz3vwwn34PmRsg8YHEqanV9E3dtOERZbSOv3j6Nm6eHsC+1mIzi6p5P7gd1jc08uTmRy/66i3cPZXLdpCBevmWKtjNsNoxaAnuegzrL24ovGKO1DX91TxpO9voeB/WYsNPZcff4u0ksTjQ753moUNlQSXZVNovDFjImwKNT/yFne2duHHMjOzJ3cKHiQp+usT+1mClhXha1Dne01zM5zFPFFYYJlgaa/w7cAZQApcAdUsp/2FKwoUhyfiXvHMxk7cwwHr1yLB8fz+HBd4+xfncq108OZmakZU/c/eVyv3ICRQmJTpMtO8HVn2JXHzwNntiHzoZPH4ITG6Gx2uJ4woZ96SRkV/DimsnEBXlw/ZRgdALeP2w7a+FcfiUrX/iO1/emc9O0EL559HL+fP0Egr3apKMu/B3UlcHeFyxed9wIN/zdDBRVNRAT6I6+F0WFK6JWMNJlJC+fGLrWgqnHVZRHFJNCPTmeWdpp8M7qcavR6/S8efrNXq9fVtPA2fxKpod7W3zO7ChfknIrtAQJxZDG0kBzKFAEfAx8BBS3bPte8dRnp3F20POzxWO4/7Io7pobwZaTuTg72PHbZWYygGxEeJmWL7+1Ktric4pqi7R4wg2vgYMrbH5Q22Fh5tHJC+VE+Lpw+Th/QGvbPH+MH5uOZNlkmMonx7O5+vk9FFfXt7rkAs1lcQVO1Kq5978E1Za5J4QQzG+xFiwJMrfFXm/PnXF3crzwOAfzDvbq3IEipSwFgFGeo5gU4kVFXVNraxAT/s7+rIhawftn3+dkYfuSo2ZjM0W1Xf9bHk4vRUqYHmG5UpgV5YOU2pQ2xdDGUl/HVmBLy2s7kAp8biuhhiLfnCtk19lCHlw4Gm8XB4QQPLYsml8uHcvzN0/Cx9UwYLKI1F0U2gfxaaa9xaMXi2uLtXRUtxGw6lUtY8l3LLiamcBmhtN5FUSPdGu37aapIeRV1LE72bo9duqbmnn84wRiAt35/KH5re6eLln4O2is0dxIFmJasy9NCq8dfS3+zv78cvcv2ZO9p9fn25rzZecx6A0EugYysbWIrXN33V9M/UXr56ho0Ir4ahpruO/r+1j8/mL+ceQf1DV1nrp7ML0EBzudRUFmE/HBWoLEjjNq8M5Qx1L30Xgp5YSW12hgOjD0vg02QkrJn7aeJszHmVtnXxysotMJ7r9sVOtT54DQ3Ajpe6gInEtxdQNn8ixrY1BUW3QxHTViHqx6Da78k0XnVtY1klFcQ0yHquxF0QF4uzjw/uG++aW7Yve5Iirqmnhw0Wj83CxQtr6jYeIaLeCcddiiayyJCeDhxWNYGjei1/IZ9AbWL1mPj5MPP/76x/zl0F9oaB46bpHUslQiPSLR6/SM8nPFzWDH0czOFcXuDu78ZcFfyK/O58m9T1JeX8492+7hUN4hZgfO5r8J/2XVp6s4lNe+ueKBtBImhnjiaN9NPKsDDnY6rowbwecJearlxRCnT1FRKeVR4HvTCymzpIaz+ZXcPTcCg53lXwSbkHUIGqrwmnAlwMXq5h4orituX6MQew2MXmzRuSbF07FVh4OdjmsnBbEtKZ9iK/bu/+R4Nt4uDsztTQXskv8Dt5Hwv7Va+48ecLTX89Di0bj1sqmeiSjPKN5Z9g43j7uZN5Pe5Fe7f9WndWzB+bLzRHlGAdqDy/QIb3adKTBrVcb7xfPTyT9lW8Y2Vn68ktPFp/nbgr/x0uKXeOWKV2g2NnPPV/dwpuQMAFX1TSRklzOjF64jE9dOCqKyromdZwZ2hrSid1gaU/h5m9cjQoh3gD77DIQQDwshEoUQCUKIjUIIRyFEhBDigBAiWQjxPyGEQ1/XtzZJLfMQemMu24yUnSB0eMcsIsLXhX2pPWd01DTWUNtU265G4XB6CUcyLPPvmkY7mpscd9O0EBqbJR8etU575Or6Jr4+nc+y8SMsymxpxdkbVr+jZSG9d6tWrW1jHO0c+e2M33LP+Hv4OvNr0soHdjayOaoaqsivyW9VCgArJgaSU17HwXTz/9+3x97O3KC51DTV8MKiF1gUtgiAmSNn8u7yd/EwePDU/qcwSiNHM0ppNspexRNMzI7yxc/NwEfHVCvtoYyl3zq3Ni8DWoxhZbdndIEQIgh4EG0mQxygB1YDzwDPtbinStEK5mxGb1IpE3Mq0OsEYwLcej7Y1qTsgKAp4OTJzEgfDqaV0NRDe+SOLS6MRskD7xzl5vUHLFIMSTkVeDnbM8LdsdO+MQFuTA71ZOOhTKtk42xLyqeu0cjKiUE9H9yREXGw8kW4sB++GLgn9zXRa7DX2fP26bcH7JpdkVKuBZmjPC4qhSUxATg76Pm4i5uxTuhYt3AdX17/JbMD2xcyehg8eHjKwxwvPM7mlM0cTCtBrxNMDu19ryy9TrAiPpCdZwtUFtIQxtKYwh/avJ6SUr4tpewcgbIcO8BJCGEHOAO5wEJgU8v+DcA1/Vi/Wz44ksVlz+7irIX++MScckb5ufbKh2p1Guvg/Ndaa4qohQDMjvLRzPmc7oe0F9VpLiaTpXDsQin5FfXodHDvG0fazRcwx+ncCqJHunfZCG719FBSC6s5lN7/TpifHM8myNOJKX246QAQdx3M+gkcfhWKkvstjyX4OvmyLGIZm1M2U15veb2ELWibeWTC2cGOpbEj2Hoqt50/v6HJ2Dqbw15nj5ej+X/zFVErmOg3keeOPMe+tAuMD/LAxdC3ZnrXTgqisVny2am8Pp2vsD3dKgUhxKdCiM1dvfpyQSllNvAskImmDMqBI0CZlNLUHCULMPuoKIS4VwhxWAhxuLCwbx6sRdH+ONvreWnXeYuOT8qt6NcozX5RVQgbb4a/RMBb14O9C8ReB9BaF7Gvh6Igk6VgUgpbT+bhYKfj/R/NpskoufP1Q1TUNZo9t6nZyJm8yk5B5rYsnzASN4Md7x7q3/jHkuoGvk0u4ur4wP4NJJr1gPYz6eN+ydMb1saspbaplo+SPxqwa5ojpSylNfOoLde0+PNNsySajZLbXztI/B++Yuk/dvPk5kQOdeFe0gkdj818jLL6Mk7Xv9+neIKJ2EB3Rvm7dmm1DBZSSp7+/DSbT+QMtiiDTk+WwrPA37p59RohhBea6ykCCARcMN+a26wvQkq5Xko5VUo51c+vb1k/ns4OrJ0Zxqcnckgv6t6NVFRVT35FvVl/+oCQ/BWc/QzG3wBr3oNHzmrN7tBGTI72d+0xrmDKOfdx8sFolHyekMv80X6MD/bgX2snk1ZUzSPvnTB7blpRNfVNxm7nQTg72LFiYiCfncptPxWul3x2Kpcmo2RFfGDPB3eHeyCEzIDET/q3Ti8Y5z2OqQFT2XhmI03GwWv8llKW0pp51JbZUT74uhr4+Jh20/vn9mT2phRz09QQ/NwM/O/QBW5ev5+EbPOWzjjvccwfsRKdxz58fft+QxdCcM3EQA6mlwyp2c3ZZbW8/E0qD248xiPvn6Cm4fvbvK8npZAmpfymq1cfr7m4Zd1CKWUj8CEwG/BscScBBAM2Vdl3zYvATq/jX7tSuj3OFGQeNKWQn6gNzFn+HIy5Ehza9/2fHeXD4fQSGpq6jisU1RahEzq8DF6cyCojt7yOZeNHtJzvy8NLxvBVUr7Z+EJSN0HmtqyeFkpdo5FPjvf9hrH5eA6j/V071UP0iZiVkH8Kirv//7Uma6PXklOdw84LOwfsmh1JKU8h0jOy03Y7vY4V8YHsOFPAZ6dyeX5HMqumBPPMqgm8edcM9v1mIb6uBh7+3/EuU0arcpcgmnzYmP40pXV9dxWa4kWfHB86T+UJLcOWro4P5IOjWSx/fg8phVWDLNXg0JNSaLW/hRAfWOmamcBMIYSz0JzUi4AkYCewquWY2wCbPub5uzmyeloIHx7LIqestsvjEluUQuzI3lW+Wo2CRM0y6KLH0awoH2oamtuP6OxAcV0xngZP9Do9nyfkYa8XLIq+OIHtjjnh+Lo68LevznU6Nym3Anu9IKqH/kDjgz2IDXRn48ELfQo4F1fVcyijhGXjR1pniE30Cu3nALqQLgu5jCDXIP594t9UN9q2J5Q5qhqqyKvOaxdPaMs1kwJpaDbyk3eOMsrPlf+3MrZ1n6ezA8+smkByQVXrcKi2nMoqZ+fpCq4N+jVl9aU8tucxjLJv859DvJ2ZGOLJV4lDJ66QkF2OXif466oJvH33DEqqG3hyc2LPJ16C9KQU2n47Oz9+9AEp5QG0gPJR4FSLDOuBXwE/F0KcB3wYgIZ7P1oQhZSwfndql8ck5pQT5Olk9RbYFpOfCP6xXe6eEeGDEN3HFUwtLqSUfHYql7mjfPFwuvh5nB3s+PFlo9ibUtxpndO5lYz2d7Oo0d/q6aGczq0wWz3bEzvPFiIlLI4O6PlgS/AMgaCp2iCiAUKv0/PbGb8lpSyFn2z/Seuks4Gibc8jc4wP8iDSzwUHOx0v/nAyzg7tg8ULxvixdmYo/9mTxv4OLsm/bzuLp7M9j16+kEenPcq32d/yeuLrfZZ1cbQ/J7LKKajsT76K9UjIKWe0v5ZMMjvKl7vmRPBtctH30lro6Zsuu/i9X0gpn5BSjpNSxkkpb5FS1kspU6WU06WUo6SUN0gprVcN1QVBnk5cOymIjQczya8w/8c5uEHmAqguhICYLg/xcnFg3Aj3buMKJbUl+Dj6kJBdQVZpLVeNH9npmB/OCCXA3cDft51t96SflFNh8XzpayYG4mqw4/W96RYd35btp/MJcDdYNNvAYmKvgdwTUDJw9QPzg+fzp7l/4kj+ER7e9fCAVjqbMo/a1ii0RQjBy2unsOm+2V2mV/92WTSh3s48/L/jrdbn0cxSdp4t5N75kbg52rN67GqWhC1h3dF1rdfsLSZLdSgUskkpScguJ7ZNC/XV00Nx0Ot4c9/3b8BkT0ohXghRIYSoBCa0/F4hhKgUQnSfBzlM+MlCzdR+dNPJThWf1fVNpBVVD248ASCga0sBtLjCkYxS6pvM+4JNlsLWU7nY6QRXxHR+Gne01/OTy0dxKL2Ub5O1wHRBZR1FVZYH2d0c7blhajBbT+Z2qWTN0dBkZPe5QhaOC7Du/ONWF9LAWQsAyyKX8cSsJ/gu+zse/+7xAbuuqedRkGvXNR6jA9yI66YJoLODHS+u0brvXvfSXv61K4Xntp3Dx8WB22aFA5py+d3M3yGEYNO5TV2u1R3jRrgR6OHI16cHXykUVNZTVNXA+DYPJH5uBn4wYSSbjmR97ybGdasUpJR6KaW7lNJNSmnX8rvp/SDdKa1LmI8Lv1sew+5zhWzYl95u35m8CqSk3RPEgFKQpP3sxn0EMCvSh/omY6dhKqA9BZlaXGxLymNWlA+ezuaLxW+cFkKQpxO/+fAUb+xLb+1o2V06akdunx1Os5S8tf/iE1ZdYzN/3JLEizvPsy+luFNmx4G0Yqobmlkc7W/xdSzCKwwCJw9oXMHE9WOu5774+/gs7TOOF/R9wllvSClPIcIjolPmUW+JC/Lg84fmcUVsAM98cYZvk4u4b0FUu9oEb0dvLg+5nC2pW/pkDQmhxbX2JBcNei8kU8ZVR2V566wwquqb+PDowA2TGgrYfiLMMGDtjFAWR/vz9OdnOJN30QAyZR4NmvsoPwlc/HrsZDo90huDnY4PjnT+461qrKK+uR5nvScphdXMjuq6n5DBTs/fbozHw8me33+SyIMbjwG9UwphPi4sGufP2wcyqWtsRkrJLzed5D970vjrl2e5+ZX9THjyKzYevFjTsP10AQY7Xbey9ZmYlZBzbECzkEzcEXsHXgYv/n3i3za/VpOxiXMl57p0HfUWT2cHXlwzmb9cP4EfTBjJ2plhnY65bvR1lNWX9Xk86aJof2obmy1q1WJLTmWXI0Tn3l6TQr2ID/Zgw950pJQUVtazbnsyG/amt05dvBRRSgHtqeWZ6yfg4WTPgxuPtZqLiTkVeDrbM9Kjc3uHASE/oUfXEYC7oz1rZ4bx4bFsUjsExkyFa2WV2meYEtZ9pfDMSB8+e2geW346l9tnh3PbrLBeB9nvmBNBSXUDm4/n8NKuFDafyOHRK8dy7PElvHb7NKaGe/Hk5kSS8yuRUvL16XzmjvLFycEGFePxq0FnDwdsf2PuiLO9M7fH3c53Od/Z3Fp4K+ktCmsLWRxqWZNDSxBCcOO0EF5cM9ns/82skbMIcA7gw/Mf9mn9mZE+ODvo2THILqSE7AoifV3MVmnfOiuclMJq7n3zCHOf2cHft53jic2JLF+3p8tiv+GOUgot+LgaePaGeJILqljy92/4/FQuiTlakNmqfm5LMTZD4ZkeXUcm7lsQhYNexz+3t2/tYCpcyyu1x14vmBBsmSssLsiDJ1fE8oeVcb2TGy3GMTbAjb9+dZa/fnmWlRMDuf+yKLxcHLh8nD/rbp6Ei8GOh987TmKOFvxeZK2so464jdAK/469DbX9b8PRW1aPXW1zayGjIoMXjr/AwpCFLApdZLPrdESv07Ny1Er2Zu8lr7r36aWO9nrmjvJl++n8QZ1il5hT3uWwpeXxI/F1NfDN2UKunRTEjl8sYP0tU6iqb+KGf+/juW2d07gHgk+OZ9us+E8phTYsGOPHpvtm4enswI/fPsqp7PJeuU6sSkkqNNVZZCmAFhi7dXYYm0/kcC7/Yk+n4jrNUkjNE8QFeQxI/yYhBHfMCaewsp74YA+euX5CO8Xq7+bIn64dT0J2BT9++wiguRJsxqz7tdGjR/o+qL6v2NpaMEojT+x9AgedA4/NfGzAH2CuGXUNEskn5/sWzF8U7U9OeR2ncy3rQ2ZtiqrqyS2v6zL4brDTs/knc9jza20UbKSfK1fEjmDbz+ezNHYE/9qVQmn1wDb3K6is4xfvneC179Jtsr5SCh2YEubNpz+ZwxNXxxDo4cjCcTZ6gu2J1syjrtNRO/Kj+VE42+v5x9cXn15MlsLZHPreZK4PXDs5iMeXx/DKbVPNKqKlcSNYNSWYCyW1jA/yIMBMB1arMWI8RMzXhvA0970NR1+xpbWw6dwmjuQf4dFpj+LvbEPF2gUhbiFMHzGdj89/3KdiNtN418GayNZanNpNMkmgpxP+bu3/Pp0d7Hho8Wgamo18OMB9nN49eIEmozQb57EGSimYwU6v4445Eez9zSJmRfn0fIItKEgCoQM/y2c/e7s4cOfcCD47lUdijpZRUVxbjE7oaah3ZGr4wCkFg52eu+ZG7zuYVgAAIABJREFUdPoyteWJq2OIDXTnxqnBthdo5gNQkT3g6amgWQtrotfwXc53fXKzdEV1YzXPHXmOmSNncs0omzUV7pFrR19LVlUWR/KP9PpcfzdH4oM92DFI9QqmzKPYPtTHRI90Jz7Ek/9ZqW28JTQ1G3nnQCbzRvsS4evS8wl9QCmFoUp+InhHgb2ZYfXdcPe8SNwMdry0U8u2Ka4rxknnDuiY3EOQeaBxc7Rn64PzuKUl/92mjL4CfEbBvhdhEPzXyyKWAfBV+ldWW3NbxjaqGqt4YOIDgxP3amFhyEIcdA7syNzRp/MXjPXn+IWyQZmxkJBdTriPM+59nMB387QQzuVXcdRMOrgt+Pp0PnkVddxqw++MUgpDlfzEXrmOTHg42fPDmWF8npBLRnG1ln3U7E6ot3O3T+2XPDodzPyxNo8i2Xo3ZksJdQ9lnPc4tmVs6/KYqoYqPj7/MZtTNrM9YzuJRd333tmSsoVQt1Di/eItF6Q0w+rpuc72zswMnMnOCzv79MR82Vg/jBJ2J1s2WtaaJOSUE9tNMV9PLI8PxNlBz//62TbeUt7Yl0GQpxMLx9nOVaiUwlCkvgpK0yzOPOrIHXPC0esE//k2jaLaImpqnZg6xKyEQWHSLeAfA58+NCiZSEvClnC88LhZF1JicSI3brmRx797nMf2PMbPdv2M1VtXsyV1i9m18qrzOJh3kOWRyy23Esqz4JWFsP5yqyuGy0MuJ7sqm3Olvc/GiQ/2xMvZvnXWw0BRUdfIhZJa4vpRnOpqsGNFfCCfnsilsouZJNbifEEle1OKWTMjFH1/5o30gFIKQ5FCbUi6pZlHHQlwd+TaSUG8f+QCuVUFNNQ7DznX0aBgZ4BrXtJ6Sn3x2wG//BVhVwCwPXN76zYpJW+ffpu1n62lobmB9UvWs/Xarby3/D2ivaN54dgLNJoJjm9J3YJEsjxqufmL5Z2CsgsX3zfWwf/WQlO9ZjW9+0Pt4cNKXBZyGQLRp7bhep1g3mg/dp8r7NRqxpZkl2oNC8N8nPu1zk3TQqhtbObTE7nWEKtL3tqfiYNex03TQmx6HaUUhiJpu7WfI8b3eYl750dS19hESV0xxib3AQ0yD2kCJ8G8X8CJd+Ds59q24hQ48S7U2zYtMtwjnNFeo9vFFV44/gJ/Pvhn5gbOZdPVm5gVOItQ91CifaJ5aPJDZFdl8/6599utI6Xk05RPmeQ/iRC3DjcIYzPsegZeng/PT4GdT0NjLWz9uVbZfd16WPUaFJ2FT+63WnzF18mX8X7j+zxL4rKxfhRVNbTO7xgIcss1pdDf4tSJIZ6MG+HGOwczbBZwrm1o5oMjWSwbPwJfV4NNrmFCKYWhRnMjHPoPhM/Tevf0kVH+biyIdgJhxIAno/2tMLjmUmH+oxAQBx/fD+smwfOT4aMfwcc/tnkQ+oqwKzhWcIyCmgK2Z2xn/cn1XDvqWtYtXIeno2e7Y2cHzmbaiGm8fPJlahovFir9//bOPK6qanvg38UoIA4oIioOOKDmiDibOablnA2OWVk2WGn9Kptfs/Z6r/GV9nLIyrLUV+ZQauac8zxPOAOCoghcBIH9+2NfcGLmXg7E/n4+fO495+yzz+LCPeustdewL3Yf4XHh9A2+wUpIiIbv7oKV70GTu6FRP1g1CT66BXbMgttehIZ3Qt2u0OMNHYm1+l8O+926BnVl3/l9BYqw6txAl3IpShdSZJwu2hhYPn/BHDciIgxvW5M9Zy6x7aRz3JJ/HogmPjmVe8OcayWAUQrFj/0LdOhku8cLPVXfUN0Yp07F6k71QZY43Dxg0BTwKKsjku74AG59Tn/2W6Y79dK317odheKrXV/x8tqXaVq5abZJZyLCuNBxxF6O5dt932buX3h0Ie4u7vSq3evq4LRUmN4LTm6A/p/ZLYJpMGohlK+h+3rfNuHq+A5P60zvFe/AwmcgtfCRP92CugEUqBZS5bKeNKtRnpUHC9Z3vSBEXryMq4vg71v4J++7QmvgW8bNaQllC3ZG4O/rSdtg54fI31zsw2AtG6dAxdrQoHehp/KvoJ+EBjTNe65DqaFqU3hm99Xt9HTtXlnyMtRsr/NDds2GlROhZge4YxJ4Fd4FF1whmHoV6jH74Gz8yvjxYZcP8XTN/qbU3L853YK6MWPvDJJSk7iUconfj/9Ol6AulPe8ZoH0yB86C/6embqPRAZ1boVHV988sQgM+lL3s173CUTvh3u/gbIFj2qpU74OtcrVYsWpFQxpOCTf59/WwJ/PVxwhznalSJpaRcZdJsDX0yEPTD6ebgxpHcT0dceJjEsqtPVxLfGXr/DnwWiGtXHuAnMGxlIoTpzZCqc2QptHs22/mR9ikvRTV5/GRinkiouLth48y8GcUTC1m3YnufvA7jnwRXs4/IdDLtU3uC9u4sa/b/s3VX2q5jr+6dCnSVfpzNw3k+Unl+Pv5c/IxiOvH7T9W11Rt2GfvAvi4go934LB0yBiB0y5FQ4szudvcxURoWtQVzZFbSI+Jf/rMxmhqWuOFI21EHUpiaoOLHZ5f/vaKKUc3phn6d6zpKSm0695NYfOmx1GKRQnNkwBD19oOcIh00XbohGESl4WZWWXNMpWgbu+hHOHIf4s3PUVPP4XPLIcypSHWYNh/ReFvsxDTR5i+b3LCasalqfxdSvUZf3Q9WwbsY1V961i/sD5tKzS8uqAhBg49LuuCOtagCfspnfDw8vApzLMHgpzH4LEguUMdA3qSmp6KqtOr8r3uS2CKlLey73IXEiRFy8TWMFxT/RBft70bBzAD5tOOrRHxIJdEVSv4EVozQq5D3YARikUF+KjYO/P0HI4lHFMEb4YWwyVvCrh7mJRf+mSSN1uMHYjPLUFmt2rLYhqLWHMKqjXE1a8q2/ChUBE8Cvjl69zXF1cs89H2PUjpKdCi0I8TFRtCo+sgK6vwL5f4YN6MKkmfNwMfhgGKYl5mqZFlRYE+QYx5+Cc3AffgKuL0LmBPysPRjs9NFUpRWTcZQIdXHPrwY51uGC7wvwdjqmHFJuYwtrD5+jXvFqRZa0bpeAsjq2Bhc/CrHvhiw76NTY8+/HrPtVf7DZjHCbCWdtZ/L1ybtBjyAL/EPC4oa6MexnobQ/vXP2BNXJlhVLadVSjNVQppJvQzQNuewEeWwtdXoRmQ6B6Kzi4CDZ+macpXMSF+0LuY1v0tgIlsvVoVIVzCSnsOO3cshFxSVdIupLmUEsBoG0dPxpW9eXLVeGcT8h7m/mT522kpN5cUPD3PVGkpiv6Nb+5r7qzMErBWSwcDzt/gPgIqBAEJ9fD5I76y5V+wx///FFdwbPlCKjkmM5ZoC2FAG+Lqrz+HalcH0JH6gil2GNWS6M5s00nOzrI5Qho5dLlRbjzn3DPDKjfC9Z9DEl5u1EPqDsAT1dPfjzwY74v3aVBFVxdhOX7nVs19Wo4qmMtBRHhlT6NOHMxiUFf/MWR6NwTBP/Yd5bOH6yg2ZtLGPbVBj5cdogFOyPYdfoiv2w/Q7C/T5GW8DdKwRnEhsP5IzoW/LG1MOxHeGID1OoAv70A3w26PlHqj3+Aqwd0e9WhYkTbovH3NpaCQ7ntRXBxgxXvWSfDqc0Qvko/TGydDm5eOuTUWXR7FS7HwV+f5Wl4hTIV6F27NwvCF5CQkr+s6fLe7rSuXZHlTu7G5qjEtay4tb4/s8e0w5aSyl1frOOnzaeYsuooT36/jdFfb76pR/m0tceoWq4MQ9vUJC7pCp/9eZinfthO//+sY9PxWPo2KzrXEZiQVOeQEaVS75rWiOWrw/C5sG2mdit9dzcMn6PLEexfAF1f1V3CHERKWgoXki9YUmP/b025QJ1DsvZD6PAUBDYr2uvv+F5HRV1L86EOW4fKksBm0GQwbJgMbR/NU9jqkIZDmH90Pr8e/ZVhjYbl63I9GgXwzqL9nIq1EeRXuBIU2eGoxLXsaFmzIj8/0ZGHvt7MC/N2AVC9ghdnLiYxY91xxnatB8DBqHjWh59nQu+GPN5FewlsKamcjLVx4ryN6EuX6d+iulNkzA5jKTiDw0t12esbXUEi0OoBuHs6nN4M3w2GJS9BuerQfqxDRcgIRzVKwQl0HKdzForaWojcqRPNat8KoxbAwCnQ402HW5hZ0vUV3Qlwzb/zNLxJ5SY0qdSEHw/+mO/SDxmtWZ3pQnJk4lp2BPl5M//Jjsx9rD3bX+vJuhe70aNRAFNWHc0sEz5z/XE83VwYck09I28PNxpWLUevW6oysn1tynsVbaCIUQqOJsUGx9fo+v3ZcctA7auN2Ka/6D3eAA/HPhHF2IxScBpeFaDtY3DoN4g5WDTXtMXCjyPBy0/XLqrTGVoMhU7jdcays6lUV69bbJ6mrZU83Ojva3gf4XHhbI7anK9L1ansQ7C/D8ud2HjHkYlrOeHt4UZYbT8q+ngA8HyvEBKSU5m86ihxtiv8vO0MA1pUyzxeHLBEKYhIBRGZKyIHRGS/iLQXET8RWSYih+2vJbOC2/G1+omqfo+cxzUeAENn66fOJnc7XIyzNv2UZaKPnETrh8GtTJ797Pkm7gxM7Qmz7oHFL+gKp5ci7FnHFv1Nu7+uo5x+eRy+vw8u5VwVtHft3lT0rMg3+77J96V6NgpgQ/h5p5WjdnTiWl4JqerLoBbV+XrdcT778zBJV9IY1aF2kcuRE1ZZCp8AvyulGgLNgf3Ai8BypVR9YLl9u+RxZJle+KvVKfex9XvqjFIXx/8ZMiwFE33kJHwqQ4vhOkcg3nEtNjPZ8T2c3qRvvDtm6ZpGd/4Tglo7/lp5xacyPLAIek/SlXw/bwM/P6bzay7H3TS8jFsZhjYayqrTqzh84XC+LtW9UQBX0hSrDzmn8Y6jE9fywzM9G5CuFFPXHqNNbb8c+0NbQZErBREpB3QGpgEopVKUUheBAcBM+7CZgHVNZwuKUno9Ifg2HdduIdG2aDxcPK6vj2NwLO3H6qq2eYzhz0QpSDyf85g983TNpcfXwkun4eUzEPZQwWV1FBkd7B5fByF36PLjcx6AD+pD+Mqbhg8NGYqXmxdf7/06X5cJrVmBCt7ufL/pBDtOXeRK2s0x/AXFWYlreSXIz5thbWoCFDsrAayxFIKBGGCGiGwXkaki4gMEKKUiAeyvWTrDRWSMiGwRkS0xMQXMLD13GOY/qRuOOJLzR+DC8eujjiwiOkmHo1rZu/dvT6W60KgvbJmWv4Y1S16GD4LhszBY/PzV/hkZnN0HMfuhiT3MVCTfvbqdTqW6uhLr80fhwd91Ls6vT92U+VyhTAXuqn8Xi8MXE5mQ9yY0bq4ujGhbi3VHzjPw83U0fWMJ42ZvJzm18OUjMhLXrHAfZfB/vUKYeFdTejdxXMSho7BCKbgBocBkpVRLIJF8uIqUUv9VSoUppcL8/QvoW714UmeBrv+8YOdnx2F7/936PR07bwGItkUb11FR0GGcdp1s/zb3sQAHFsGGL6DBHboa7rZvYWY/nXeQwd7/gbjodafijqsb1GoP/T7R36uVE68eUwpObeL+enehUPleW3iuVwgbX+7O58NCGRxag/k7Injmxx2kFbIERkY4ajWL3EcA5cq4M7SIqp7mFyuUwmngtFJqo317LlpJnBWRQAD7q/NCD+p1h4Z9dbmCOMfUKOHyJV1quXKI/rJbTIwtxiSuFQVBrbWbZ82HWfrVr+PiKd3YJ7AF3DsTRsyFF45C+Zqw9BXdNU0p7Tqq07lQZayLnNqdIPR+/aAVsUNnP88bDdN6Um3WMO4I7Mi8w/OIS87lM7qBgHJl6NMskHcHNeWVOxuxeHcUr83fU6gOZxmJa1ZaCsWZIlcKSqko4JSIhNh3dQf2Ab8Co+z7RgHznSpIr3dBpcNSB8R4nz8K03pC1B7o/Fzh5yskSinO2s6acNSiotc7kBgDq/6Z/Zi0VJj3sL7x3zND94sGXWOpxz90EuPOHyByh86Id2aGsrPo+RZ4V9Zd7KZ0gr2/QNvHIekCD2xfSFJqEj8c+KHA0z/SOZjHbqvL9xtP8tGy/NdVyiDTUnBS4prTuHxJW5RrP4afRsGOgn+WOWFV9NFTwCwR2QW0AN4DJgE9ReQw0NO+7Twq1oZOz2hT/diags2Rkqj/8b/qBglnYeTPurKmxSReSSQpNYkqXkYpFAnVW+kY/o1Tss9b+OsTOLUB+n0MfsHXH2syWId6Ln9bu5Nc3HQrzZKGV0UdIRVzQJfwHr1MNyd6dBUhFerQ2ZbEdzsmY1s1SUdTpeU/3HRC7xDuCwvi0z+PMGfLqQKJWRSJa5nYYuHUpsLPc3YffNwEvumvy+JEbIdk5/SztkQpKKV22NcFmimlBiqlLiilziuluiul6ttfY50uSMdxUKGmXuzL6z+oUrqi6dQeurTwnFHgG6jLDgffViAxlFL8dPAnHvj9AbZHby/QHNcSbdOeN2MpFCHd/6Eb8vw24ebErqSLsPYTCOmjexfciAj0mggJUXrRum538M5fae1iQ+OBeuH50TVQo5XeV74GPPgbj9TsTRzpzNn6H9069KMmOis8Hy5cEeGdQU3oWK8SL/+8m43huURxZUFk3GWqODtxTSnYNQf+E6a9CAufLXhgiy0WfhiiQ92Hz4UXjsH4XbrkiBMo3RnN7l7Q+30d6ZHH9H1W/ROWvaZdTx2ehuHzYMxK8KtTIBGupF3hrQ1v8faGt9l7bi+jfhvF+5vev65Re37JTFwzawpFR1l/6PoyhK+AAwuvP7ZhMiTHQdeXsj8/qLW2GOBq1FFJREQvPHuWvX6/uxct+vyHtlXb8nX1uiQPnqprKq36J3zcVOc75JIMlzmVqwtfDGtFUEVvHvtuKyfO563XQwZRl5KcUggvkwsn4Pt74X8Pa49Em0e1sp92u45OzA9pqTrkNz4S7vtOB7E4+YHBFMRreKeuG7/qfd1gJahN9mN3/ggr34Pmw2DgF/oLUAguXL7AuBXj2B69nUeaPsJDTR7i0+2f8t3+71h5aiUfd/2YEL+Q3Ce6gYy6Ryb6qIhp/fDVgocBt2g3UdIFHW3UqJ9uZJMTvd7TVmej/kUjrwWMaTaG0UtH87N7GkOGz9E3yU1f6dLx+36F256Hdk9cXXPJhvLe7kx7oDUDP1/H0P9uoGvDKjSuVo42tf2oH+Cb47mRFy/TyBmlqG2x+uFy03+1C7DXRP007+IKwV3gl8fg83b6fb3u2u148aTuj207Dx2f1p6LDJTSa57HVsGAL4oscVEKs4pvNWFhYWrLli2Fn+jyJb0wBrrU9Y0VJ9PT4cRa+PYuqNkORvxPNyQpJJ9u+5Tpe6Yz6dZJ9K7TO3P/lqgtTFgzgfiUeCZ2mkj3Wt3zNe/U3VP5ZNsnbBq+CS+3EraYVtKJOQjTe4OnLzy0RD8hrv4AHlsHVZtYLZ3lKKUY+dtIYmwxLLxr4dWugLHhsORV3dCnXA1o/4SOZvLM+Qa/9UQsHyw5yN6IS8RfTsVFYPaY9rSpk/XTtFKKxq8vYXjbmrzat3Hhf6G0VN1X/eBivR6UEg8thkGXl26uSXXhuC6LcngZXLy2j7PoNRivirrMfrWWOu9l0bM6Y77t43ptxoGIyFalVJb9YI1SyODkRpjRW9chqt9TZ2qGr9AKQ9kTZiqHwOiluiCaA3hh1QvsOb+HxXfd3Cw9xhbD+BXj2XVuF2NbjOXRZo/mORHtvY3vsTB8IX8N/cshchryyZmtMLM/lA+CuNNQr5uuWWQAYPXp1YxdPpa3OrzFoPqDrj949E9Y/S84sQ48y+sbbMM7oWb7HPtPK6U4GWtj2FcbKevpxsKnO+HuerN3/KIthRZvLePVPo14+NbgLGbKheQErQQid+jQ2+NrtDXo4g4NeulqsgG5KBultBKM2gUV60DlBlpJzLpHWww93oTNX+lk2C4vwa3/p60NB5KTUjDuowxqtoXOz2s30u6fwKcKhNypexy4uOuyFc2GOEwhAEQkRlDNp1qWx/y9/Zneezpv/PUGn+/4nNrla9O7du8sx95ItC3aRB5ZSfVWMGSW/pKnXdGNeQyZ3Fr9Vhr5NeLLXV/SN7gv7tfe7Ot20z+nt8Jfn2pLa+NkKFNeu217vZvlDVJEqFXJh3/0a8yYb7cyY90xxnS+uYthRvOeYH+fm47lysWT8HUf/Qr6hl6/ly73Ubdb3ntaiOiM8GtL61dpBA8v12sRvz0PZQPg/vk6X6WIMUrhWjq/oH16/g2hWqhTCtVdS2RiJO0D22d73NPVk3c6vsPBCwf5dNundK/Z/aq5nQMxthgTeWQ1wV1gxDx9A8ntybGUISI8Hfo0j//xOPMOz2NIwyE3D6rRSif4JSdoi33ffK0cUpOg78f6xqqUdsds+q+O7ElP5Xa3Mnxf+Rbm/nELkSFPEBhwdV0tLukKE3/bT/OgCnRpkM/vR9wZnXl+OQ6G/KC7KDrwAREA3wB4cLEuhth4gGXJi0YpXIurm2N73ebAlbQrxNhiqFY2a0shUyQXV8aHjmfs8rHMPTSXoQ2H5jr3WdtZ2ga2dZSohoJiwVNeSaFjtY60CmjFl7u+pH/d/ni7Z9NPxLOsXqRv1E+749Z+qBPkOj+nay3tnqM/Z79gvbhri6Xt0RV0cFlG6uT/QIOe0OweaHAHnyw9jGviWd4f3AKX7MJRk+Nh+3ew9WvwLKetgJrtdK00Wyzc/4u2BJ2Fhw+0ecR58+cBoxQs4qztLApFoE9grmNvrX4rrau2ZsrOKfSv2x8f9+xN3/iUeM4lnaOqT/ErtGUwZCAijAsdx/2/3c8PB35gdNPRuZ/U/XXtc1/zL9j1E8Sdgm6vaZ/7NettrmmpzFvwC+e3/I97j26kwqHfUOLG6yqV1z2BH9Hlv9td09ZUKZ0zsXGKTgqr0QbSUmD5m/q4R1mdnOpMhVBMMErBIiITdUx2YNnclYKI8EzoMwxbPIyZe2fyRIsnsh27OHwxaSqNbkHdHCarweAMWlZpSecanZm+Zzr3hNxDOY9cfPIi0Pcj7cI5shyGfK8XoW/E1Y2B/QfzhW9z2q04REfXA/Rw38OFVHce7NkaryOL4I839cJwRnb5+s9h9T91OHCn8Vdv/pci4MgfULUZVGvh2A+gmFK6k9csJFMp5MFSAGjq35Tba93O13u/Zu+5vVmOUUox9/BcGvo1pHEl48c2FH+ebvk0l1IuMX339Lyd4OIK93wNzx3KWiHYcXURnupen9/HdyE5qBMvxd+Nf5/X8OrwiM4xcnWHX5+2V3LdrEtHNOqno8SutQbKVdOhsaVEIYBRCpYRkRABkC83z/jQ8Xi5eTFk0RBeXPNi5hwZ7Du/jwOxBxhcf7Dpo2AoEYT4hdAvuB/f7PuGE5dO5H4CaIshjz3Na1f24dvRbVj1fBfuCQvSO8tVg9vf1uGk6z6BuQ/pff3/U+iE1L8DRilYRFRiFJXKVMLTNe9FuYLKBbFw0EJGNxnNHyf+oN/P/Vh+Ynnm8bmH51LGtQx9gvs4Q2SDwSk8G/Ysnq6eTNw4sVAlsbMjI1z1OkJH6QXqP/6hS0jc/bXjo4lKKEYpWEREQkSukUdZ4evhy/hW41kwcAEN/RoyYc0EdsXswnbFxuLwxfSq3Qtfj5yzQA2G4kRlr8o82fJJ1kWsY/nJ5bmf4AhEoN+nUK469J54tXifwSgFq4hMjMzzekJWBJYN5LPun+Hv5c9Tfz7F1N1TsaXauLtBFlU4DYZizn0h9xFSMYT3NxeuGGS+8KsDz+y1PAS0uGGUggUopQqtFAD8yvgxucdk0lQaX+3+irrl69Lcv7mDpDQYig43FzdeafcKUYlRfLT1I6e4kbLErCHchFEKFhB7OZbktOQ8haPmRu3ytfm066d4uXkxsvFIs8BsKLG0rNKS4Y2GM/vgbF7/63WupOe/CY+h8Jg8BQuISowC8h6OmhuhAaGsHbIWD9fCV241GKxkQusJ+Hr4MmXnFKISo/iwy4dmjayIMZaCBUQk6lDSgiw0Z4dRCIa/AyLC2Ba6guqWqC2MWDyC8IvhVotVqjBKwQIiE/KXuGYwlDYG1R/ElJ5TuJh8kSGLhrA4/Oby8gbnYJSCBUQmRuLt5p17Wr/BUIppG9iWn/r+lBl6PWnTpKJbgC7FGKVgARmRR2ZR2GDImQCfAKb1msbwRsOZtX8Wn23/zGqR/vaYhWYLiEiIcEjkkcFQGnB3cWdC6wlcTr3MV7u/oqpPVe4Nuddqsf62GKVgAVGJUTSpbPr1Ggx5RUR4td2rxCTF8O7Gd/F09aSZfzO83LxIV+kciztGeFw4ZxPP4uPuQ1mPspT3LE+gTyA1ytagincVXB3c0vLvilEKRYztio0LyRccGnlkMJQG3Fzc+KDzB4xeMppX172a5RhPV0+S05Jv2u/l5sX/tfo/7g25t1i6bZPTkolPiSfxSiJp6WnUKV/HMjmNUihiomyOzVEwGEoT3u7eTOs1jY2RG7Gl2khKTQKgVrlaBJcPxq+MH+kqnYQrCVxMvsiZhDOcSTjDsuPLeGfjO+w6t4vX2r3GxeSLzNgzgwVHF9C+WntebPMi/t7+TpFZKcWxuGNsiNxAREIE7au1p03VNri7urP33F5m7J3BshPLSFfpmed0rN6RN9u/SYBPQOYcRy4eoZxHOap4V3GqwhCrVvNFxBXYApxRSvUVkTrAbMAP2AaMVEql5DRHWFiY2rJli/OFdSDrzqzjsT8eY2bvmYQGhFotjsFQKkhX6UzZOYXJOydTvWx1ztrOgtI33/UR6/F09WR8q/Hc3eBuXCR/8TdKKZYcX8LmqM2kqTRS01NJSUsh/ko88SnxnEk4w7mkc4C2dlLTU/H18KWmb032nt+Lr7svA+sPpJZvLbzdvYm2RfPlri9xc3HjmVbPEJsUy4LwBZmlxf3K+NHQryED6g7gzuDse0rkhIhsVUqFZXXMSkthHLAfyIjLfB/4SCk1W0SmAKOByUUhiO2KLfsesQ5YaDKgAAAJZElEQVQmo7mOcR8ZDEWHi7jwRIsnaFK5Ce9vep/B9QfzUJOHqFa2GsfjjvP2hrd5e8Pb/Hr0V15r9xohfiEApKWnsS16G5W8KhFcPvimefee38ukjZPYEbMDXw9fvFy9cHVxxcPVg7LuZfH18KVdYDvCAsJoG9gWf29/NkRsYNmJZRy6cIjnwp5jcP3BlPUoe928PWr14NW1r/LW+rcACAsIY9Qto0hJS+FA7AEOxB4gJinGKZ+VJZaCiNQAZgLvAs8C/YAYoKpSKlVE2gNvKKV65TRPYSyF5LRklh5fyk8Hf2JHzA5aBbRiWMNhdKvZDTeXwunKPef28O2+b1FKMaLxCJr5NwMg8Uoib294m9+P/c6WEVsKfR2DweAYlFL8evRX/r3l31xKucTQhkNxd3Vn0dFFRCdFA9AlqAsPN32YAO8A1p5Zy+rTq1l5aiUVy1RkXOg4BtQd4NDF7LT0NDZEbqB2+dpUL1vdYfNCzpaCVUphLjAR8AWeAx4ANiil6tmPBwG/KaVuCtERkTHAGICaNWu2OnEij92armHlqZW8vu51LiRfoFa5WnSu0Zk/T/7JmYQz+Hv5U6tcLXw9fLXmd/PC280bL3cvutfsToOKDbKcMz4lnvUR65l9cDabozbj6+4LoveHBYRRvWx1lp5YSlJqEh2rdWRKzyn5lttgMDiXuOQ4Ptn2CXMPzcVVXOlUvRN96vYh/GI43x/4nrjkuMyxVX2qcmedO3m46cMlrj5TsVIKItIXuFMp9YSIdEErhQeB9TcohcVKqaY5zVVQS+FY3DE+3vox9zW8j3aB7XARF9LS01h9ejWLji3ifNJ54lO0PzBjMSs5LRlB6Fe3H2NbjKWCZwV2ndvFtrPb2BC5gV0xu0hTaVTxrsL9je/PbIk579A8vtn3DfEp8fSu05tB9QbR3L95sYyAMBgMmlPxp/B286aSV6XMfbYrNuYfnU9yajKdqneiboW6JfZ7XNyUwkRgJJAKlEGvKfwM9KII3Uf5JS45jmm7pzFr/yzSSUcpRZpKQxAaV2pMh2od6Fi9I838m+Hu4n7duWnpaaSTftN+g8FgsIJipRSuu7jdUrBHH80B5l2z0LxLKfVFTudbEX0UlRjFt/u+xdPVk9CAUJr7Ny9xpqPBYCjdFNfooxuZAMwWkXeA7cA0i+XJkqo+VXm+9fNWi2EwGAxOwVKloJRaCay0vw8H2lgpj8FgMJR2TJVUg8FgMGRilILBYDAYMjFKwWAwGAyZGKVgMBgMhkyMUjAYDAZDJkYpGAwGgyEToxQMBoPBkImlGc2FRURigPxXxNNUBs45UJyipKTKbuQuWozcRUtJkruWUirLrkIlWikUBhHZkl2ad3GnpMpu5C5ajNxFS0mV+0aM+8hgMBgMmRilYDAYDIZMSrNS+K/VAhSCkiq7kbtoMXIXLSVV7usotWsKBoPBYLiZ0mwpGAwGg+EGjFIwGAwGQyalUimISG8ROSgiR0TkRavlyQsiMl1EokVkj9Wy5AcRCRKRFSKyX0T2isg4q2XKCyJSRkQ2ichOu9xvWi1TfhARVxHZLiILrZYlP4jIcRHZLSI7RKRo2yoWAhGpICJzReSA/X+9vdUyFZRSt6YgIq7AIaAncBrYDAxVSu2zVLBcEJHOQALwjVKqidXy5BURCQQClVLbRMQX2AoMLAGftwA+SqkEEXEH1gLjlFIbLBYtT4jIs0AYUE4p1ddqefKKiBwHwpRSJSUJDAARmQmsUUpNFREPwFspddFquQpCabQU2gBHlFLhSqkUYDYwwGKZckUptRqItVqO/KKUilRKbbO/jwf2A9WtlSp3lCbBvulu/ykRT1AiUgPoA0y1WpbSgIiUAzpjbyGslEopqQoBSqdSqA6cumb7NCXgJvV3QERqAy2BjdZKkjfsLpgdQDSwTClVIuQGPgZeANKtFqQAKGCpiGwVkTFWC5NHgoEYYIbdZTdVRHysFqqglEalIFnsKxFPgCUZESkLzAPGK6UuWS1PXlBKpSmlWgA1gDYiUuzddiLSF4hWSm21WpYC0lEpFQrcAYy1u02LO25AKDBZKdUSSARKxFplVpRGpXAaCLpmuwYQYZEspQK7T34eMEsp9T+r5ckvdlfASqC3xaLkhY5Af7tvfjbQTUS+s1akvKOUirC/RgM/o929xZ3TwOlrLMm5aCVRIimNSmEzUF9E6tgXhIYAv1os098W+4LtNGC/UupDq+XJKyLiLyIV7O+9gB7AAWulyh2l1EtKqRpKqdro/+0/lVIjLBYrT4iIjz0YAbv75Xag2EfbKaWigFMiEmLf1R0o1oEUOeFmtQBFjVIqVUSeBJYArsB0pdRei8XKFRH5AegCVBaR08A/lFLTrJUqT3QERgK77f55gJeVUostlCkvBAIz7dFqLsBPSqkSFd5ZAgkAftbPEbgB3yulfrdWpDzzFDDL/qAZDjxosTwFptSFpBoMBoMhe0qj+8hgMBgM2WCUgsFgMBgyMUrBYDAYDJkYpWAwGAyGTIxSMBgMBkMmRikYDICIVLJX5twhIlEicuaa7b+cdM2WIpJtfSJ7rkRJCck0/E0odXkKBkNWKKXOAy0AROQNIEEp9S8nX/Zl4J0cZIoRkUgR6aiUWudkWQwGwFgKBkOuiEiC/bWLiKwSkZ9E5JCITBKR4fa+C7tFpK59nL+IzBORzfafjlnM6Qs0U0rttG/fdo1lsj0jsxf4BRheRL+qwWCUgsGQT5oD44Cm6EztBkqpNugy1U/Zx3wCfKSUag0MJusS1mFcX8LhOWCsvQDfrUCSff8W+7bBUCQY95HBkD82K6UiAUTkKLDUvn830NX+vgfQ2F6uAaCciPja+0lkEIgut5zBOuBDEZkF/E8pddq+Pxqo5vhfw2DIGqMUDIb8kXzN+/RrttO5+n1yAdorpZLIniSgTMaGUmqSiCwC7gQ2iEgPpdQB+5ic5jEYHIpxHxkMjmcp8GTGhoi0yGLMfqDeNWPqKqV2K6XeR7uMGtoPNaAEVAo1/H0wSsFgcDxPA2EisktE9gGP3TjAbgWUv2ZBebyI7BGRnWjL4Df7/q7AoqIQ2mAAUyXVYLAMEXkGiFdK5ZSrsBoYoJS6UHSSGUozxlIwGKxjMtevUVyHiPgDHxqFYChKjKVgMBgMhkyMpWAwGAyGTIxSMBgMBkMmRikYDAaDIROjFAwGg8GQiVEKBoPBYMjk/wFURZjo5gIJGgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "key = dict(mouse_id=0, session_number=1, scan_idx=1, seg_param_id=1)\n", + "\n", + "# ENTER YOUR CODE! - fetch 'time' from the Fluorescence table using fetch1()\n", + "time = (Fluorescence & key).fetch1('time')\n", + "\n", + "# ENTER YOUR CODE! - fetch 'trace' from the Fluorescence.Trace table using fetch()\n", + "traces = (Fluorescence.Trace & key).fetch('trace')\n", + "\n", + "plt.plot(time, np.vstack(traces).T)\n", + "plt.xlabel('Time (s)')\n", + "plt.ylabel('Fluorescence')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Summary" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Congratulations! You have successfully extended your pipeline with all types of tables. This pipeline is simple, but reflecting a typical preprocessin pipeline of calcium imaging. We have all the table tiers and major dependencies between DataJoint tables.\n", + "\n", + "**Table tiers**: \n", + "Manual table: green box \n", + "Lookup table: gray box \n", + "Imported table: blue oval \n", + "Computed table: red circle \n", + "Part table: plain text\n", + "\n", + "**Dependencies**: \n", + "One-to-one primary: thick solid line, share the exact same primary key \n", + "One-to-many primary: thin solid line, inherit the primary key from the parent table, but have additional field(s) as part of the primary key as well" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "%3\n", + "\n", + "\n", + "\n", + "Fluorescence.Trace\n", + "\n", + "\n", + "Fluorescence.Trace\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Fluorescence\n", + "\n", + "\n", + "Fluorescence\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Fluorescence->Fluorescence.Trace\n", + "\n", + "\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "Mouse\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "Session\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Mouse->Session\n", + "\n", + "\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "Scan\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Session->Scan\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation\n", + "\n", + "\n", + "Segmentation\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation->Fluorescence\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation.Roi\n", + "\n", + "\n", + "Segmentation.Roi\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation->Segmentation.Roi\n", + "\n", + "\n", + "\n", + "\n", + "SegmentationParam\n", + "\n", + "\n", + "SegmentationParam\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "SegmentationParam->Segmentation\n", + "\n", + "\n", + "\n", + "\n", + "Segmentation.Roi->Fluorescence.Trace\n", + "\n", + "\n", + "\n", + "\n", + "AverageFrame\n", + "\n", + "\n", + "AverageFrame\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "AverageFrame->Segmentation\n", + "\n", + "\n", + "\n", + "\n", + "Scan->AverageFrame\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dj.Diagram(schema)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have covered most of the building elements of data pipeline design. Using these elements, we could design more sophiscated pipelines that facillitates your experimental recordings and data analyses." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.16" + }, + "vscode": { + "interpreter": { + "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} From 800e3d621f22bb70c7dde23dee14badcd276d389 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:30:07 +0200 Subject: [PATCH 44/70] Rename including a space --- .../03-Calcium ImagingComputed Tables.ipynb | 2691 ----------------- 1 file changed, 2691 deletions(-) delete mode 100644 completed_tutorials/03-Calcium ImagingComputed Tables.ipynb diff --git a/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb b/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb deleted file mode 100644 index 4436773..0000000 --- a/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb +++ /dev/null @@ -1,2691 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Working with automated computations: Computed tables" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example. \n", - "\n", - "In this session, we will learn to:\n", - "\n", - "* compute various statistics for each neuron by defining a `Computed` table\n", - "* define a `Lookup` table to store parameters for computation\n", - "* define another `Computed` table to perform spike detection and store the detected spikes\n", - "* automatically trigger computations for all missing entries with `populate`\n", - "* define a `Part` table to save the results computed with the master `Computed` table" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's import `datajoint` again." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import datajoint as dj" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As we are going to perform some computations, let's go ahead and import NumPy and Matplotlib." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "import numpy as np\n", - "import matplotlib.pyplot as plt\n", - "%matplotlib inline" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Similarly as before, to continue working with the tables we defined in the previous notebook, we can either redefine the classes for each table `Mouse`, `Session`, `Scan`, `AverageFrame` and populate them. Or, again for your convenience, we can import them from the `tutorial_pipeline.imaging` module. " - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Connecting shan@workshop-db.datajoint.io:3306\n" - ] - } - ], - "source": [ - "from tutorial_pipeline.imaging import schema, Mouse, Session, Scan, AverageFrame" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experiment session\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-150Edgar Y. Walkerdata
02017-05-190Edgar Y. Walkerdata
52017-01-051Fabian Sinzdata
1002017-05-25100Jacob Reimerdata
\n", - " \n", - "

Total: 4

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +------------+ +-----------+\n", - "0 2017-05-15 0 Edgar Y. Walke data \n", - "0 2017-05-19 0 Edgar Y. Walke data \n", - "5 2017-01-05 1 Fabian Sinz data \n", - "100 2017-05-25 100 Jacob Reimer data \n", - " (Total: 4)" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Session()" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

depth

\n", - " depth of this scan\n", - "
\n", - "

wavelength

\n", - " wavelength used\n", - "
\n", - "

laser_power

\n", - " power of the laser used\n", - "
\n", - "

fps

\n", - " frames per second\n", - "
\n", - "

file_name

\n", - " name of the tif file\n", - "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
1002017-05-251150.0920.025.015.0example_scan_03.tif
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", - "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", - "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", - "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", - "100 2017-05-25 1 150.0 920.0 25.0 15.0 example_scan_0\n", - " (Total: 3)" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Scan()" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

average_frame

\n", - " average fluorescence across frames\n", - "
02017-05-151=BLOB=
02017-05-152=BLOB=
1002017-05-251=BLOB=
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx average_fr\n", - "+----------+ +------------+ +----------+ +--------+\n", - "0 2017-05-15 1 =BLOB= \n", - "0 2017-05-15 2 =BLOB= \n", - "100 2017-05-25 1 =BLOB= \n", - " (Total: 3)" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "AverageFrame()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `imaging.py` also fill each table by inserting manually and loading data from the external tiff files." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Computations in data pipeline" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now it's time to perform more complicated analyses. \n", - "\n", - "When you perform computations in the DataJoint data pipeline, you focus and design tables in terms of **what** is it that you are computing rather than the **how**. You should think in terms of the \"things\" that you are computing!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, we would like to detect cells from the average image. The final product we would like is the binary **mask** for each individual cell, with the 1 in the region of interest (ROI) and 0 in other places." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So the new \"thing\" or entity here is `Roi`, where each entry corresponds the mask of one ROI. Let's start designing the table, paying special attention to the dependencies." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Detect cells from the average fluorescence image" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now let's perform the segmentation to isolate ROIs. Let's start by taking a look at one average fluoresence image." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "keys = AverageFrame.fetch('KEY')\n", - "\n", - "# pick one key\n", - "key = keys[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

average_frame

\n", - " average fluorescence across frames\n", - "
02017-05-151=BLOB=
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx average_fr\n", - "+----------+ +------------+ +----------+ +--------+\n", - "0 2017-05-15 1 =BLOB= \n", - " (Total: 1)" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# ENTER YOUR CODE! - preview an AverageFrame for a particular key\n", - "AverageFrame & key" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "avg_image = AverageFrame.fetch('average_frame')" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", - " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", - " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", - " ...,\n", - " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", - " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", - " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]]),\n", - " array([[ 29.53, 30.8 , 30.96, ..., 230.09, 230.78, 228.19],\n", - " [ 29.61, 29.69, 30.71, ..., 224.86, 227.21, 227.22],\n", - " [ 29.61, 28.28, 30.19, ..., 220.51, 224.43, 224.85],\n", - " ...,\n", - " [ 23.67, 23.05, 24.06, ..., 32.54, 32.07, 35.28],\n", - " [ 24.03, 22.45, 22.4 , ..., 31.67, 30.75, 32.72],\n", - " [ 25.78, 23.27, 21.92, ..., 31.12, 31.29, 32.94]]),\n", - " array([[41.55, 40.2 , 38.3 , ..., 55.67, 57.17, 58.65],\n", - " [43.66, 42.2 , 38.59, ..., 50.35, 54.83, 57.9 ],\n", - " [42.07, 41.46, 38.3 , ..., 49.46, 52.67, 55.61],\n", - " ...,\n", - " [30.04, 31.05, 30.92, ..., 45.04, 44.93, 44.81],\n", - " [28.88, 29.52, 29.93, ..., 46.5 , 45.65, 45.13],\n", - " [27.84, 27.34, 28.5 , ..., 42.75, 43.28, 45.17]])], dtype=object)" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "avg_image" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "It's a bit subtle, but `fetch` returns a NumPy array of the attribute, even if the attribute contains a NumPy array. So here, we actually got a NumPy array of NumPy array. We can of course just index into it," - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", - " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", - " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", - " ...,\n", - " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", - " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", - " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "avg_image[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "but if we knew that there was only one item, we can use `fetch1` instead to save some trouble" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [], - "source": [ - "avg_image = (AverageFrame & key).fetch1('average_frame')" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", - " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", - " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", - " ...,\n", - " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", - " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", - " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "avg_image" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's plot to take a quick look:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO19fayt2V3Ws/Y+e5+vO7cz04/h2ja2TUa0kigNwQJGG+oHVEJjAqRIdISaRgMoqLGt/AEmkoAaBKMBJxQtpnRaPrQNoogVYkxkZBDEQimMgO3Qse3A3Hvmno+9zzl7+cfev/c8+3mftfbpvXPO3Z2znuTmPXfvd693vetd7/r9fs/vY6WcMxoaGq4uBve6Aw0NDfcWbRFoaLjiaItAQ8MVR1sEGhquONoi0NBwxdEWgYaGK44LWwRSSl+WUvpoSunJlNI7Luo6DQ0Nd4d0EXECKaUhgN8A8GcBPAXgFwB8bc751573izU0NNwVNi6o3S8E8GTO+bcAIKX0GIA3A7CLwHA4zBsbG+AFKaW0dMw5wy1Y8f1gMOiO8dl5wG3G33wtPerfpf6UrlE7Tz9z13aI71NK1TbOc83ZbHau+9PjZ4Lab2vjd6fjzvce8ySlhOFwCADY2Ji/Bpubm9je3u7+BoDRaNT9Ls6PNgDg5OQEAHB6enquvs5ms6Xv3XyNtk5OTrq/43ez2ax3rfjOXT/n3H3/7LPPPpNzfqn26aIWgZcD+Dj9/ykAf4JPSCm9DcDbAGA4HOLGjRs4OTnpBpgHH1gekDgCwHg8BgBcu3YNALCzs9P9Rh+Emyiz2ax7kHycTqcAgOPj46XvZrPZ0gOJ68Q1Y0INBoPu+jpRUkpdv7lPPEH52ny/fL5OkI2Nja7dwGQy6c6NScxtRX+jr5PJpLt3vi4wf07xXKKt0WjU/a2TkhclfhZxzTgOh8Ole+Bjzrm7hxhHJyyGw2Hv+cZ9nJycdP2Ol3xra6ubMy95yUsAAA8//DA+7/M+DwDwmte8BgDw0EMPAQCuX7+O++67D8B8jsX4PPvsswCAvb293jV1/s1ms969jMdjbG1tLY3b7du3AQDPPPNM1/7BwUF3jO8PDw+7a8az0jkzmUxwdHQEAHjf+973f2FwUYuAEw9LI5JzfhTAowAwHo/z8fExjo+Pey8wr7q9BmmV4xc4HrhOQF5FefLE+W6R0TbiOvyd+577rQ/ISZXT09PeQqJtu/4wTk5OelI2xuf09LTrW1w7pbQ0afX+9PosUePIi7PTyhQlKar3zM81rs8LrFuI475U2mp/gfmLFi9HPBfg7CX+5Cc/CQD4nM/5HADAAw880C0g/Nzj5bx582bXLoCubeBMqxgOh70Fdjwed/Mv+hu/vXXrFp577rmlPuace0Ll5OSk+z6O/Ny5Lw4XtQg8BeCV9P9XAPhE6eScM05PT5ekbMCp+fwCqXTjF50nexx10eCJHZ8dHx/3+uHMEu6HUwNLWg2rojwGKuni6BYlp/rHGAJ+IVGtxo3HbDbrLVDupeJr6v2GNhL3zW2U4DSAgKrtvAjES8UanS4QvPC4MY1xOTo6wq1btwCcvbj8YurLyr+Nlz9e2tu3b3d9i/HY2trqveiz2cxqvQCwv7/fLTLxXfSLr80LvAoyXqRLuCjvwC8AeDil9OqU0hjAWwB88IKu1dDQcBe4EE0g53ySUvomAD8NYAjgh3LOv1o6P6XUSXWVgryqq7q8sbFRVddVNXfqKa+UvLLq6lkjX5xd71T+EicR/VCJy+frd2wD18hLRxC59lXL0nuO35VUf3efg8GgahKopsbfsUbjTBwn3VQVdn0MbGxsdFKV+Zn4bbQVdvdgMOiZjY47ivP39/c7aR/z9fj4uBt7tud1Xkcfjo6OetrEeDxeMiGBM96HP4vfMV9QwkWZA8g5/xSAn7qo9hsaGp4fXNgi8JlCV8EASztd0UKDAJZJJmWrAyXJ5K7pbEdgLr2czazSh6+vNi1LcdZcVEKzVqF2LrfrGHLXfyd51W520pDvQ8ePORX3O7bjdfy4H8qH8HfKbzhNzUnqAHMw7vlEf/f29nrSNWzy7e3trh9xzuHh4RKpyPfBUjx+d/v2besxUu2H+Y44P+6Nr8fEprYb3zkvhaKFDTc0XHGshSbAnIBKGrZ/1AfKDDm7FmMFPk9wEUtPlti6Kjs7N9qfTqfVIBDHjKtd59xvztZnDiEkTJzP9rP2Z2Njw3IjTvKqdON7dx6XANvK0Qf1VnBMBWsf+lt+Fuq2dM91MBj0PAtubHnc1We/v7/feQdCA9jf3wcwZ+XVBbm/v9/91nljlNdi3ke9IdwP7re28eyzz/bmJrcbbcQ7cB6sxSIQpFUtJgDov8C8CDgVV3/HJgWruEG68PVLkXH8cq+K1HOTPaCBPnyePuTT09Pey8rkmDMb9D5dZBovuqwaqxmj/eN+cKDPecdF70+vw22xKl8juGazWe9Z8aKgrjN2p/KLHPcY6jT7/d3iHwsJRx3G/+NaLh7BBUrpYsdRtLxgOVey3p9b3Eto5kBDwxXHWmgCITE46EZXx1LAjFNxVaPg8zUyjkmmOI9ND12dmchhOGJT++bUcD5Xg12cOlsjefheVDo7d6BToTl0W+/DSXHXJ1blnfngXISqTteIRO6ji4hkKRvnKHHGpgr3V/sRUpzNRjbDoh8cDRpHdde5sZ1MJr3n7gKmnKuX29Ux0vuooWkCDQ1XHGujCYxGoyVJrUTRqoAHbivgEjhUUvKKzaSKS94JqMuvFNDibPtSH13oZ+2emDRyocrKQ7iAHO4bcwcu6EevqQFWfD5rRWqbjsdjyx2oW5dtXHXJOlcoj7cLNKvNJxcgxRpA9DHuKxJ+RqNR91lI5SAUOQgt2uLAINY6Vdt0AWc8b+NazBOECzH65p5xCWuxCABnE0H91szmqtrEGXO1F4xfQuf314k3nU6XYtL5GL8BluO4HSuvE5tj9h0J6BaouJ7LjHRxBbXoSCWJImmL731VKrFem5npmufA5V7wfZbMQE6K4qg5zZZkj47eOy8QfJ/uhdQxYhPDEaW8eALLKrrz2SvcYu7mHJ/vxkjP5byNVeRgMwcaGq441kYTAHw6La+OrphCbZVTYoZXetYwnBsrUCPCWOqqpB4Ohz0/fi1ewLXBWYjaljuPXVzafz4/JMhkMlmqN1Dqm4MbFxfpWMubqJkl3JaT4qp9ODejA2tXrq6Bi+6M3+lnTCDrs2VNoCTRow29dza1XFatmycuGxTwbtremFS/bWhoeMFjLTSBsHPYTtMjr9y1QAiOfef6AAEXh66uGUdUsQ3vuIbSPXE/WXq5CDO36gdU4vHqz+eUog1ZMoX0Pz4+7mlX3N8aD8HQiMuwR0suQgXbxQrmQ3gcNWCGn6PLftQ+uiCxjY2NjliL75wGFnCBOw48l2vagYPLBtXPWNq781dhLRYB4EzdL3kBeILzy+VIFRcVGCjFEPD5rPoxacR95fM5oi/AE0QnoIu8cyG5vJhpIomLi2A/t6r+0+m0I6iYRHVmV6m/zkziz1S9Zq+JMwt4fHScuX2NK+BFoEaiBbh9Zz5E+0w41rwrfM+uZFuco3NiOp1W1fXzRJGyCcJ90EQt7k9LIGpoaKhibTQBYFmlcySTfsaqOfuVnW864PziToVS6RNwPnn+3Em6WsQgH/X+uB31c/N5rLqWNB0XGcmEI6cB8/c6BkrIuRqALG2dRC3lN8T1+T5ZW+FUXmc2qtReRRa6HBPtW5DKjuQspUVz//n+WIrXSD33jGuuci72qtesmVrdNarfNjQ0vOCxNppASEJd+ZzdzSt+iRA572f8HUsmtbFYSjuJze68+L9KGleOnO+vxIeUtJX4O6QLS17nsgyNwUUHOptd3WX6W/1OJbAr/1YKjlESVzMk+Xx+7o5/cEVlzhNE5bgdPsdpGKWAsFVELM8NzQBkbbjGTbh8DOUXWMMsoWkCDQ1XHGulCTC7zRljwHLhDmdfulWa7dy4hgsMUp7AuesCTvtwATOj0ci68IAyN1A7/zyuHi5HpQUwnHeFr6+aDN8LHx33UdImWBtiW9vVH3Dehvi/jnMpmEuZdLat1X3pJCoHIamm48K6c849roalvu4F4PIsXIFZho4V3wv/X+d1KVfEYW0WgXDLRTy+LgLD4bBHGjn1jQekVGiB249rM3iyuwUiwJ/VEmpqLsISAalj4NJeXay5ywWIo77o/PLV1HunSvPv3IsV7UcfNReD++j8+E6FdQsJuzhLLwKTi0yO6pjzvbvIyxhbXjRqKbv6/NmU5HlU2m+C79ndC0PNJx7HVYtAMwcaGq441kITiNVtPB73zAFe2WJ14zrrjiysBd3wNaN91Saceu9WcCfFmSAquarcyjwYDKz0dufFsWbGqOt0Npv1XH58TUcyaeCMi0hkycRurIC6aQ8PD7vn53I6XICSjoeroMuagH6XUlrSKOOzz6QQh4tSdIFSaoYxXMYqa5Gq2XEAmZszjlTWOenqdiqaJtDQcMWxNprA1tbWksR0pbbUhcYrtyP/akQck1gu4MiRXHHUNhwx49xvbt+BQC3TzGW3MSnqXErcj2ijFszjSM44P6R4KTdB78+RhxGyPBqNbOCTI8C4Lb0n51ZzbrQ4vxb+G3Nnc3NzKe+B4Qjh09P+RqquKKoLIHPamysOex6OxHFY0Zaba4o7XgRSSq8E8MMAPgfADMCjOefvSyk9COB9AF4F4HcAfE3O+dlaW4PBoEvc4CoswLLf1ak8Svicxy8KLKtcOnD8AJ26rG24+AbOP3Apq64Nx8Zzm/yZU/1d4YuaWjibzXqTlglEXUzZc+AKZAT4PI1zd+PCz1YXNl5kXCwDLyjaXxYCq9TpOHKf+BzurzMHAmxauGShGtGsz5sjOvn3pYWRj3xvFxkncALg7+Sc/wiA1wP4xpTSawG8A8CHcs4PA/jQ4v8NDQ1rijvWBHLOTwN4evH3cymljwB4OYA3A3jD4rR3A/g5AG9f1V6oeLFS6j7uLouP1SC30rvUXEcClvIVFvcGYDmWPVAioqJdR1rqb/U6fH0XLencgXxN/S3fi44ll3Pj66vpEZJ9PB4XCS83HtzvOGdra6snUV3xDO4z9zfuSWMCuD0dKycJWXuLcTw4OFjKFeBrcnx+LcqT+6gkoXNLukhENq+c9qbXLEWxRl9X1ed8XojBlNKrAHw+gMcBPLRYIGKheFnhN29LKT2RUnqiplo2NDRcLO6aGEwpXQPw4wC+Jee8dx57HAByzo8CeBQAdnd3c2RHqURgW1FXSidteeUrrdLxW2A5EMe5ilQT4PPY5VbKK5d7Xro2f+Yi6fgcNy7uM0fK6T3z+GjxDM4xiPvb3t4GsLwtNmskOkZuGzgmNLVCr9NIXDCNBgjpefoZj7e2z5oAn6ckJxeTVfAzcG5VnX/sAufx0+xBF6HJ/XZuWtUA+BnXai0Ad7kIpJRGmC8A78k5/8Ti40+mlG7knJ9OKd0A8KlV7cxms24ThtK+c44gcvXZnI9f2yqd7yLjlBh0LDQ/GCavnCqnfePEGVcxWcGqo5sMblGM+3Bqp5aqHo/H9rM46qJxcnLS+f21qq5L9dbvo6+qOvOzcGp4oLboObOEx1HHyIUN8/kKF6vBz1jnsktpB/rkKfffLQLOc6Htcn8ujBhM85bfBeAjOefvoa8+COCRxd+PAPjAnV6joaHh4nE3msCXAPjLAP53SumXF5/9fQDfBeD9KaW3AvgYgK8+T2M5z3cTdumUQLlARYBXVi3BxZs16MrKkkb7w23UUlAdqcaSWldi5w50BCXfm5NktUiwEhkGLLssd3Z2AKA7ssrKm2tqv/kz3YKLJbdKSC6xpSQcg9Vrl4TkNAAn2fmo57s2XKQlf859Y+1U55BTwZ27jok+Z8a42Bin+tfMqQtLIMo5/zcAJT3jjXfabkNDw+ViLSIGB4MBdnd3kVLC4eEhgLOVOOwojpAr5QAA8xVZMxFrv3N2Vy0FdTQa9SLNWFK7bDx1uXEGYCC2YWOwJHN8gbPxlfxjjcNFMGoA02g06sYvjizlol3er0AzFx0x61xYrt+q7fHz4cAgJ6FLmgAHkPH5mvnJc8xxA+qq5DGt2d0ukzNwfHzcKwC7KsI1rsXaiiM+Aa85KFruQEPDFcdaaAIbGxt48YtfjO3tbdy6dQvAGdPMkkq1BOfmcTnbbF/qys1SlguDllw/wZhzH4+Pj21gTcBpB+dhyJ19V7P5OBhKj6tiyOO78Xjc4wLYLRhjxZ4A1bTC1o8jg8OG+Z7U9nWegFrWJvMEKg1LktAV9lQpG2AXdRxdiXL3XDgEWbkCtx8ka2WufVcno+bNWOUdWItFgHMHSu6SOA/whS+44IhOPvbr68NiVbE2aVj1UuKMyTROtnETCZi/HLGguUhE3TmXH7IjEvk+1Z3G967JMZG+DQC7u7vdMc7TlyRcucDZ7rtxr9xfXgTUt88mCN+v8/dHm26B0HiFnHMv3t8liZWuoe3qYsB9dO5LNcPOi5IgA+ZjxXkY0Q/+rfbDba5zKRGDDQ0Nn71YC00gVBxeiTWbq+QCjM9DE5hMJj1Vvpa95yKq3ArPGopG1HEaNGsRqg5GG9PptLeRZUrJVqIN1AKUnFajblJWLdl0CW0mogLjyP1m9VPr5TmpzL+rqaIu47KWb6G/0/Ep5QyUIu9cMFfJJZxS6saUzSU1Jd2eAS6aNZBzXtKS+HecFRr9drtMOdLXjUsJTRNoaLjiWBtNYDKZLNnySohMp9OevTMej7uVN4gqJ4nZFnfx4o5IUk2A7VwnsTWTjvuhRN9wOOwkLhNbjkjjc7R9vZYLaWao+8sFLTliKySfc1VGzgfgyc4An6NSizkVfZ5chISlufIPzl3nOAEmf1V74+xEV5PA5a5otivPJdXo2N3JKGlv3K5yTnwtJkW1jFqtZFpgbRaBIJH0IbiUUX5oGsE2Ho97LylPztKmItGPODrSLdrQl2k6nfYmDRNxOimd75ZfDvbBB7R9B35JS5FsCl3sJpOJjY2IPoYZEOPIOQ8uYUvVZd7mLMjgnZ2dLmJRiUQm5PjFZ88MsGwKBZzppLEjPAa8COiRX2o25dzmoPE7zQ9x8SdM5qqHiceTyWhXmcl9Bix7e0po5kBDwxXH2mgC6ussuU2AZVLP1SJwKbMBjbJjqc/nl9x73CcmyaK9WHU5AlC1Gad9cJ84SzL+79RBFxsf0ExARwxubW31MgVradSc5Vkjm1yUostmdPEPTpuIceF+uBh/F0WobQQcaelMIf6d+vP5N+dxM5e0D/0ta2K1LEnuq9NEtf/F3688o6Gh4QWNtdAEAi5GvWRvBs4TBcf/d662AEtWldq1OHTHZbjtogNsR3K7pUg31gTYlnQRZqUIR46WjPO3t7c7W/y+++7r2tD6AEF+HR8fd985DUzHmyMYmbtRd910Oi3az871p2MT96TPxcXPO9ewc7W5eaIao4vCZGnu5onOKw5yCihvxePBn3OBEv0snr/LQFWsxSKQc8bR0dHSw9JqwyklXLt2DcDyNlrxQsVNHxwc9MJXA65oiWPZOf7AvZhOxVVSx20TVgsfdT5+t8UWx0C4F10nQbTFpCG3ob7v8Xjci5bjCaubw3A/ddKzWcWTVCclk6KOKVezhxe0OI9jL1Rg8ELC9+TMhpJ3xcVlOG9MbU6klHoeFF6MlBR18yTuX8+L6ysJyBv1lNDMgYaGK4610AROT09x+/ZtbG5u9iReHHd2dvDAAw8AOItqOz4+7mLwWSqGGru/v9+dF0d2KQHLEVi1aDxWOx3JpOopk5bqQ+a/nYbhymm5xKG45xiX++67rxdrHjg6OurGiqWQbvLK5cUin4B33I3fcu6DxkhomTEeF4Yje10aszMNXQxI5DNoaq7Ls3AktEtMcqq0M09KZgG3W7oXnQur8mdc2rWai27OFdtceUZDQ8MLGmuhCQBngRcqMTiQQiWOszk5mMJdwxXdUK2Dd8RROJcO/81uNS1a4dKGXbSatunO39zcxIte9CIAZ5rA9evXu+sruXd8fNxJZrZVXVksDSByJKAL2AowuedILheoFXCurtB4HEnMXIPLXYjruLmjgWaOJ+Dn6VyQqjHws2buSu/XaZ21cQkw0ee2cdNndXp6utJN2DSBhoYrjrXRBNjWif8Dy3aahopOJpPuM0bJHmJpVAsDZnddLSCHJYjLSSgFLTlXVOkaeq2Qijs7O523JDiS4XDYjU2My3PPPQdgbsNH3zhzMDwLPN4ax88hqeox4EAVzo0o3Q9zKvy90xiiTReAo+eNRqPuXvS6Lry2VPqsFHDETL3TBEvFQhnOo8LBWW7PSsc/qJbKwVOqdfA+EiWszSIQKPnnT09POxWX49bdLq+qfqvvnM9n9YpdORqL7YpGsDurtgjovXG7tRwGvo5rK1ygt2/f7vqmCU9qFgDLbiRdbLmQRS0y0m0mouYPk17sW68tdgF+WV3uhfZja2uri3kIQjPGZX9/v5fOzQv9eZ6BS3d20ab8ErooSHVtspBTM8ClubObm00Al6QW1yslpnV9qn7b0NDwgsfaaAKh9pXIEV5NXe4AawTq9uLVWUkSDlRhNUwDjlxl4YBLVWVJo0E6JbeTahMub8Ih+ri1tdVJmJI2xP1hNZKJTdWc4siVcfk5hWYR7cb50+m06xvfu8vQ0xRiVpNVegJnkk5disCZ9sNZkOE+jO+cRsIBVQE3d1gCq4bh3J41QpjnvJsnbq6o9sFBS25+h6u8hKYJNDRccayNJhCrpLptXHYb255a2dYRck6Ks9R32X0ulDigJBZLj1Jed1wf8FK/Ziezbc2SJD4L6TmZTIpFJHZ2djrJGLbztWvXOgnqCmU4uPBYLpum965BPas0L/0d58MzZ6PPh0k0nUNcIZolpkpq3qFKtTanATpJzfepmhw/M8ctOU1AJbzLpeD5pP3gd6SE52NX4iGAJwD8bs75K1JKrwbwGIAHAfxPAH8551ztRcTBO3bWeQmYtdYXnfe6L1W84Xb5xeeJqA+LycNYmJg4O0+EGX9XilHnz/g+NKae2+OXz5XFBuaegPvvvx8AuviC3d1dW+Zcx5R3Gdb2T05OepF6ceT71UWPwVvMqbo8Go06ok9Tirk9Lm7ikpy03xx16Co+B5zpxC+kkoU1VZ7Hlp9/ydzj72rxJGwO6MLGAqSE58Mc+FsAPkL//24A/zTn/DCAZwG89Xm4RkNDwwXhbrcmfwWAvwDgOwH87TRfjr4UwF9anPJuAN8B4PtXtRX+ZpXALo7axWw79VQ1AF6J2UVXW7GVKHKuInbRsOuqFCfA7fM9qb+dfx+rf9wbkz2uDJWaLJw3wWShq42ouRSu1j1H8TkzIM51lXG1jY2NjaJmxK5hvheVkCcnJ73UZx5v1Q6cqr2xsdGZDi7yzhG22gb3y0Vj6jiyCazPgseK57lqyTyf7oUm8L0A/h6AuLsXA7iZc44RfArAy90PU0pvSyk9kVJ6wtnPDQ0Nl4M71gRSSl8B4FM5519MKb0hPjan2mUo5/wogEcBYHd3N5eIMbdyO1uTV0VXGz+gKyVHsPFKrGRhLVaetQmWCM59GddW3oLtYseLxDXZZemiw0pBIywpQ4vY2trq5by7MmQs6dUVxoFSPB5xVPLURd6xJqCaGu8oxePjAnciOIg3GA3os3BBOpubm708BZ4TzI1EP9wziP/X+CGehyrtnVZUi1wttRvHVcFZd2MOfAmAr0wpvQnAFoDrmGsG96eUNhbawCsAfOIurtHQ0HDBuONFIOf8TgDvBICFJvB3c85fl1L6UQBfhbmH4BEAH1jV1mw260pd10qCK0/AwRrOY1Crg88rtgtLrQXuuDhxDRbiABhXV8AF6ZxHIjhtQplhN37D4bBXQpwltXNjqXfg6Oio0ybcWKoNzOG9AdZg2I7lLeh5zBg8VvE910iIvzkgKI4qDZmvCLgxDY7APWNXcLQmdR3bz7+paUhOq+BnXHJtcnslXEScwNsBPJZS+ocAfgnAu87zo4gRdxFu+v+4wclk0puozv/KA6Qvh4vxduQYPyhdBFhNdrHjtcQTB/Wjr4JbjGox/uxmVOI1pdS9RHt7ewDOYvC5hBf3VSP6eBK7cmTqJnP3zsd4Edlk0BwJVyGar6NjyeZXYDY720fAEcJ6f/EboO9S5nFx7kA2G0uLgIuudOaD/q39vpRFIOf8cwB+bvH3bwH4wuej3YaGhovH2kQMhuro1FhgTto4CRlSykn7Wqklp2I7ItGRdWo+sAajaj6f51ALMuK+Oq3CSZpA3HNIysPDQ+s2jDTkiCLkTEuNz+eMPpbA0Z5zr+n+B3yvbh8BR8iqWXV0dNQbL84j0f67jT1dWS9uw7mmdV6xVqgarHPvlUho5xqMtnTu8Fzja5cC49xW8L37qn7b0NDwgsdaaAJs96kNxMEvumEj16sP1OxM/ttJYJVQfL4LpuG2HCFTc+eoFGJpr1KxFlbKRxfyGyG9ri9s93LGYDwL3isQmLsWOUcfmEtlJwUDmuXHQVRBMnKZeL23II35fFf6jQO2eN+++L9mKXJ/WdsrBYmx5sXPTrUDx8/w3DlPWTn+nUpxvk/WWlQTqLWvWItFIKV5OioTIVp0Iefcsb9cNScmOavyJZ8qkySrWHnnFYijW6h4swfAM8erUHtY7gVzC4+aL/y5JtRwnAD7ypkRZ5ycnPSY99FoZCs4x9GZGVqNuFbTkT1AzqzjF8FF0sVYaKLZ6elpb38H5x0IOE9HLbqSyUuX78HkqfaXyUXnSXFkpC7EjkgvoZkDDQ1XHGuhCQwGA2xvb+P09LSnygUODw87TSBcV3t7e91nvMLWNIGA8+uyaqdqHpsAujrzb50P2Wkf+p3eAx+du4lXf3YLar9ZzQ8SMGoTXrt2bUki8RE487drYRDtY6ma7dbWVnctHh+n8ehnLLlVzXdRcM6PH1oi1/tfFVOv4+G2eGNTRc1WF+fgyEInsWupxJwaXJvXtRiTEpom0NBwxbE2msDOzs7Sysp76AHehnMbTTpChqFS3PEQjsCpxXG7LC5uVyWls+FddlgtX8HdZykAJo6u+IfuWOSy5ejGF3QAACAASURBVNw+BXGdo6OjYh7/aDSydjdviBr91nJu0ZZz5TFqrkSO3nTkmz5HHovatVg7VOLOkcWqVcT3cSxJa37Gmg/D1xqNRsUK2+7eFWuzCGxtba1MdNDNMzc3N7vJ4iZNoKYanTchgyeAtsOJJOwHdhtexFHVwtoiwP2pReMxoen6qJGAg8GgKzAScBWFOaEoSMN4Fhy1Gep3eA5cLAP/HYvA5uZmLzmHx87V/dNnzC9dwL2QPB76cvCzdaSoxg7w+W7hduniLtVXTTG+tkYwusIxbkHTuVRDMwcaGq441kITAM6kmEpNTn8NScOJIloN2KmPKpEBT6Y4wk9/yyq3K3PlyEVdpUtx5SUXl8sJYFcek1GqwnMNwfiOzQGVZKxhaD+m02kvxZa3MndRa1psZTQa2W3CXD/iqH5xzhNgMjA+cySnSlQ2S1hDK917zrlX3ITNI+1/iSx18yRQIwsDTMTGeW5/B75OcxE2NDRUsTaaQEgNl3oKzO3Y2FKLo9WYCwB80QqnCayy8dXuYoJLpXgpOozb4+9Go5HNUlRCjqWWk/DhfgvEWABnAUEc96/lxVzmHW9NHueH5jWdTm3Mflw3zuex0/PZ9cvlwFRCsjTXKDh2j7L2UXrunHnH0DZ4/DRjsBSkpS485yIOOBeuywthjcBpNY5o1nGuaROKpgk0NFxxrIUmEDHSbI+GVAnpz+WxahKEWd87jcF3MeGucEYtmIeDlvR8LuHl4v5dJp0GsQDL4aVx1GuyxNEyZ9PptMsFYDdgaBGcgQj4jT2n02kxe489JIEIDItrRRulopyrymO5YC8NGnLhwKWgr4BqEyW7usTKu9+UcgGU73H36zZs5XuK+RTPjvkOlxG51Hb120vCYDDA9evXl9xBbjMKfYldUkfNrcYEVI10q7mK2Nzgl+o8lWJXvdS6Eairrxg4PT3tiFJHKGnF5cFg0Kni3H60we3GeZoHwS81u2bjujERV6miMVFjP4GTkxNb5Sf66iruuIQdfRE5bkHHiJ+7Lr5xr9yWuw+XcOTaYoGi5oMjsl26OM8N7QtXjXL1Ids2ZA0NDVWshSYwHA5x7do1nJyc4ObNmwDOioVwTTsl6ZwKDdTVfkf0aHAMk38uAs+RP+q24Wuoqsu1FDmaTLUPJ8m4H+fJkmRTyEVBalrvbDbrZURGW9euXetFGAJn0icyBgNM3MZ9himg46Obn8aRtQQOKIrfcnaiS7GNzx2Z6+C0H2A+ZjouXAnZBYQ5klO1PIbTNpzLkqMvA+5agVUl/Zsm0NBwxbEWmgBwFmYZJBQXtwTmq6RqAuwqci4/dbnp38ByLQBXhVfrGrAE5uAU57YpBYxsbGwsFbzQfqmE55Vcfxf9jWNpByK9vrahUlT/jvN1bGtZkEBfovJ58awdeeV4Bd1hiM8bj8fVkFnlK7iUGPe7lHvh7Hku5qF9zTn3yF+gnmVaIxldOTK+ZolLc+HRirVYBGazebnoyWTSDUhM9pgowPLLDyynVbpYABdtFeC4eN2Zl1lcfZk4ao4TVFRFY9+0Y5q14i77uXmSaVtu4XGxCa6CjVOFdVFkc0f7c/v27d7GHvwyherP3zmTRUlL3g1YX3guc87pxVogJaXUu76L8mS1WseUfxNw5CxHqbqNZeI6mvzDnhSdtzpGOn4cp+LI6pr5WhJG3fWq3zY0NLzgsTaawN7e3hKZEgQSR6PV4q5Zyjk1Vs+Pv7e2tnpkF6vVLtOM+w0sS0NWqzkugOEkJLub9J5c/Dy7uFgL0f6yWaNmzGw267klOcNRtQku9cXjo3s+sJbjog81h4ELh8Q1uTAImw1xjhKI/L3eO48Vj7uLt1fzxWlqHKWomgBDNR52B7JWqJocmwca4+KyKl2+B2txq8yBpgk0NFxxrIUmcHp6iv39/aU4aicVnaTRIBOWZK7qrErPra2tns3kiJxYWV38PBfscHnfjhNgPiE+cxl3wFzaqduQiVInhVyko3ICTIBqleK4Lo+Vq5vAefOaQ8ARjJy5qIVD2A2oROnR0dESERjXdhqPunr12TGcBObnyNeK79Qd7bQ31j7cBqYl253PY83EBYm5QDoXJanXLOGuFoGU0v0AfhDA5wHIAL4BwEcBvA/AqwD8DoCvyTk/W2snXgpWLTVtmEkYl4LqEkS0zp6LwBqPx/bB8KSN68f/NV2TXw5WGV1aZ/xfJyC/yMoWu3uZzWbWZNEFhE0RNaP4O/Z3q9qrBCGwvDDo4szkof52Op32wr+BM4JPffG86PKz1hLivMCrGeM8NUymscmiqem8KDkTLuCS0BzL78ZSF2cePw0lZgGiC7ieF21ctDnwfQD+Y875DwP4YwA+AuAdAD6Uc34YwIcW/29oaFhT3LEmkFK6DuBPAfirAJBzngKYppTeDOANi9PejfkehW9f1V6oz7qS8f+5oAZQTtekPi5951Z/F+HniD42MZwm4IpQqObC57iU0oDzFzvy0kmVaI9V8vhcNyTlKrzsmlU1VmsT8j0Nh8NOrdeNTjY2NnoaFZsDrNayxsfX5nP42WoBE5fizc/WlWJTEi3n3Kvlx1LZmV21CEQXp6Juyel02pPUrDnq+aXrqNta51wNd6MJvAbApwH8q5TSL6WUfjCltAvgoZzz04vOPQ3gZe7HKaW3pZSeSCk9oTZfQ0PD5eFuOIENAK8D8M0558dTSt+Hz0D1zzk/CuBRALh+/XqeTCZLEWlq77oAHl4BWSvQFd7tOsOSRstMsY3f3ayJ8a/tfsOSQ0lA/ru2YnOmnpPiCtYO9NpbW1s9baUUqRfnafVg5mXCFmcehz8DlglN5lScbV0KOGL+hIlT1RTZ3enyCtz4qrZXciVGmyrZmZNy3I6Cz9eAsLgvbqsWSMTX4rmvbkmXzt1rs/ptHU8BeCrn/Pji/z+G+aLwyZTSDQBYHD91F9doaGi4YNyxJpBz/n8ppY+nlD435/xRAG8E8GuLf48A+K7F8QPnaS9cXioheZV2edQqxQHvKQDKee4akOGyE52bKa7NocTODq0VnODrlHIeOItQ8xz4PC58qTbidDrtMe/uWrNZfwPQWnAMM/XOQ6KBQaw1cR9ceLGOmZNuLndBXYWsIbH0170OOMDLFfjQ58PPTMH9dqHbNS9BgN3AAZfD4PJZarkoiruNE/hmAO9JKY0B/BaAr8dcu3h/SumtAD4G4KtXNZLSfEPS4XDYqTOqirKKzi+OplA6kiTg1CKeDC7JpeT+YrCvnN03SkLW1E73IrgYglobLpqM23RxFnq//OLqS8KJKudZHFn1d9GBTuV3ppmOB88Fl/SlR2eaTafTXlyDu5Yz4QLOtHHuXf5OF13uty48fJ/OvezaDfAiuarG4F0tAjnnXwbwBearN95Nuw0NDZeHtYgYDE0g59yrHuwkGbuRdFciPs+pna7AgnNHlWLCS9F5Tt0sSSaWtk66OWKo5jLj71ST4nM06pCj7GpwLkjWlEo5D6yGK2Gl96f3ye0r+VZqoxR1WHL5llxzfA/cH1eUxZGFcU7tXvg653Hjcb9VS3Gmh4tOLaHlDjQ0XHGshSYAnJEbTkICy+4pjtNX+3IwGPRW/Rohdx5JyOc5+4tt9pqbkVdwxwmoXexWcw4GCaKP7egIunFuMicRdBej09OzfQRUM+Hz+DnpvbvyZU4Cs8TW587Xq4WGB7iP6iLma3HItJLQnOXnOBjVSFwOg+tjrQ3Hy7hgIT7WApTc+auwFovAbDbDwcEBUko9koYnmPvMqWPnIdiYyFMVilN3daLwtdzkcXCRY9oGP1w9h1+c0suin7l8CI055xeB/fQa0eeIRB2fUp+0P67AB7PbNULOxYxwXoiOqSPfuDiMi0moRea5eeWKeMTv3Rg56DMtlbXXtpz5EuDxXnX9Zg40NFxxrI0mcHR0tORzLqVoAt4HyjHeuqLWiks4KeT86DWp74gZFx3G11btwGkJrDa7fQ0cUeW2E9f2VbpwH7mwS8C50LivTkrFd3ot9xzdeLAkc2XUuNyWtuekud4Tk6hMNJci//jenWnjfPxOa3MmYm2Mas+Kz9F+8O+aJtDQ0FDFWmgCQN8l6NxOjvRwgSGl2GuX7eeCkLgftRhydx0mJZXAqbknWSPRVd3Zp4404kxB7Q9fO845PT3t7Uo0Ho97bsCIHDw+PrbSSvvL7lIXjenG2z2X6KtqAtyG3hO3UYvsY5vdcQfumTmXnwvm0fYdHNGnGgxre6XfAp6g5LYuup5AQ0PDZznWQhMIFxhLQw4zBZYlNts7zi4uSVInQRwLPZvNeu4uV2aK6xtoSSuWIMquc6l0x4artD0v0+x4CM7T135vb2/jvvvuA3DmKnRjw6XenUQtudX4eTI0fJlrAWj4NUuymjuXA5NcmKyWSuf7Ys3IeYP4GvqdahNOaytxRqU2AszsO02qttsQa2Cr3OBrsQjMZvN9Bziyy0WYBVgNci+6I2ni6FRAVTeZHFP/Mn/HLkVXj09JS+6H85WXCKKaK4jhVO3ow9HRUa/QiEtQ4d/oojsajXqf8eLi6vOpeVLqt778rj6gKwnnNlBxC49+5iI0ObVaI0adSu3mmlPzeQF3MSCl8XZuYwd2izt346pn0MyBhoYrjrXSBDijzxGDNTcTr9yqOtfqvrMLiqVQKe6/pJ6qys/qaS0YpBaNx206s0cxm51Vy9U0Zo7xZ1OHC3Vqv5XUc2nSHFilmoDbQty5WJlsrQVd8b24jVFVk2JtQjUjTot26cUaWFULXip9plrEYHC2nwXPTReJCMw1Hj3fRVc6rYZdxU0TaGhoqGItNAHn/tNyWm6F5Zj6AK/6jrByKzb/FljmDoKYZMmqEpulJ7dVIqrYpeOChLTQJwfH1MJ0WSKoFsL3FDg5OemVBCttdAoshxkHkehCcuM6bremWgAPg7Umd+9Om9DzXN6Je8bcVk1qOhd1jcdR6cxS2RGfOuedJsDzm+dVaV/K83ACa7MIxLZjOinj85RSz2NQqiakL0KAJywPqouyUrKLTQzH4q7yxXL7bNrwguaqEgPeV+7ILyaIAky46cuZcz91ezQadbED8aKzqaAbgbp4AVeJiPuoz2AwGPReGDZ/tK4i94NR8iLwC+mIPjd31ETUe4ij8+5of2I8x+Nx9TzdK4K9TkxM6watvACqB2NVQRGgmQMNDVcea6EJDAYD7OzsYDbrb6PMBJBKZZdKzESVEltcpZbJlfNkcfFR1Xx2tbkYBr7PuLYjwrQ2Hv/f7URTy2HQ/pckn0oK3qo9tDBXtdcV8+DCLsCyFOV+q+rsxpu1It19aXNzs0f6uh2fXPSm0wBrfvSa67kGNmNiXFLql/riupCa9zEej3sEaEqpR/oOBoOlZ8TfsclXQtMEGhquONZCEwD6RT4c2aQr5WAw6NlsbM+rDcznOynBbbisPcDHhDuCku1cDRBhVx7ftwskif+7oCgXoRfQ/pdIOiWjdnZ2sLOzA2BZgkUbSlRxsFAt85M/0/vk8Xf7K6o2w9KR6yDoODB3VHKpMtw5zD3oWLImqloNcyul3JJo0/0WWA6KYiLR7cmhmlFgVQ4DsCaLQMQJAH0yhZnTIKMi1JVVHVbRtYos4zzppm6yc191cWGTgh8yJ8Ho+brdlv6t13SRabVoSRd9xu3p+Tx+LvQUWCa2ov3JZNKNt94vE7E86UuMuvaDj6XxYLXXxQLwEVgWCC5E2PVJ++0EAoeQA/MXmYVV6T6dOcibs/I4x/mu/mZAvQ8tYrChoWEl1kYTiKIimtQR5NTm5mavpt5kMln6G5i7GHVLqwCTei52wLl+nLtJYxlcnIBL4WSJrb7hVf5/BUuygJNoLLVqWkGozrdv3+7FRrAZpgQUaw7Rbjwzp7az+cAakrp1eX8AlXyTyaSo/sa96lE1P1a13Vxw0lPnBD/fGJeQ4kz48YYxauq5xDgXjRlt8PbpvIenPisev1XmQNMEGhquONZCEwDOiD+XHgmUs+fUFufoMGcTansu4o2hEsRJWyb1+H5Ycum1XRahtss2q9sHQW1rRxYGXAqqC5KZTCbdParUB/pjyeOnrkV2GbJ25tJ6azwBFzXh/zNc3gmPgc6T0WhkA2xc4JiCn13cIwcEAcu2eIzH1tZWj1PhPjqOR3kixw85zoNJxlo+BnCXmkBK6VtTSr+aUvpwSum9KaWtlNKrU0qPp5R+M6X0vjTfoqyhoWFNcceaQErp5QD+JoDX5pwPU0rvB/AWAG8C8E9zzo+llH4AwFsBfP+KtjAcDpdipdVe4y3EeVXX+gOTyaS440rJZq6xvo71d8yxW7FLLO7GxkYvzp2lkHOTRvuuKGYtZLrk/eAjn89ait4nu8S4jxzEE/cXbZXctfwZS3G9NmuEXBhUg8TYTavam+METk5Oelon7yRV87w4TYAZ/WhLx5s5L5dXou5A9m654DIeK/Xo8Jy76NyBDQDbKaVjADsAngbwpQD+0uL7dwP4DqxYBELtce4p56cNHB4e4uDgAACW3II68fjh6eAzWcgEjbqmlLzR75xbSomhAKuK7mXVazg3j1sE3GLgVEH3nfutuul4oWJVn/3mwPIOxDqObPLxYq2LvksY437oM2bzy+UH6OI4nU57AseRrTy27L+PMdAFMBaD4XDYGz9+qdkEcYldcdT7ZKLUzWvNBTlPUZc7Ngdyzr8L4J9gvvPw0wBuAfhFADdzzvF0nwLwcvf7lNLbUkpPpJSeWMVeNjQ0XBzuxhx4AMCbAbwawE0APwrgy82plpXIOT8K4FEAGI/HOQgjjaPmbbJ0RT04OOhIIlYFdQXm1VwlAks3/k774VKaa+p4SqnrexBDAZZanA+h2YMaEMP9YbW9RiSxGuwIpZoW4VRo16caUeXcXqpW18iuo6OjHjHI0pOfu2oRNbcrmw+uUjEHSAFzSa+uT85wjGfNmZfRPmur+mxZW1LTs6QJqLnrittwWxdJDP4ZAL+dc/50zvkYwE8A+GIA96eUYlRfAeATd3GNhoaGC8bdcAIfA/D6lNIOgEMAbwTwBICfBfBVAB4D8AiAD5ynsSCHNFiIV7Swd8LmPDg46K2Kzl4MuJBLxwm4eGsnMVlaqPaRc7baQ1xbuQmWhmqzl6Sc4yFK0tC5y1ZxAnrPnM3IGonLJ9BrsrSNvx3RF4hnvL+/30nSGlHGQTd6ZDdmwIXwcl5DPLuQ/teuXevC1ZnPUc2hRuA5sphdsqqBuSzM4+PjnsvZZc7ynF61NfkdLwI558dTSj8G4H8COAHwS5ir9/8ewGMppX+4+Oxd52nPFZpgsGrkCCVXUVhfbn756D66v2uVd5m9dtGEqvbytUKddWy1u5aqb5xAxEc1j1hdd6SXM4W0/zW/OBOUTIRFwpEuArwo8cKgpKhTiWOiHxwcdHkl/LLqgum8PHxUr4OLmuTkHFXvt7e3ex4AfsaxaDFBrX3kRYbnle4C7Uw4F6fCHqOSyl+qKM24K+9AzvnbAXy7fPxbAL7wbtptaGi4PKxNxCDgV+faJh4bGxtV1fY80VksQQKsQmmUHcec88YoTt10Kn+pj3yeRtSNRiOrbjpJo1pKLYbAuadqLsW4Rmk8AlwUQyUlg6WikmLOHeie8Xk9S87UchpjLWJQx4aj8VTrc6q802BOT0975fJq7mJ2DZdMGv1/yyJsaGioYi00gZxzt9mlrmSBkq2qAUH8t2tLSa+4Nn/nIswcycQ1/nUHIgdXKMNJJnVdDQaDzgZ3EWZhqzJx5iSIcg5ce8Ht+ONsa2339PR0aRyAOYkGzDUB3deAOQEXnFUiuPReXGSkCxIqjYf7zhVvCSk9Ho+XcvS1TY3y4xwWDtxxmoPLLVHwOKj7kvkeV77ss6LaMENVHUdq8ITRPeadzzngGOTSAqELiVPfa8STY47dvbhFwIWnKmvNixEvGqqyKjGn7Wu0n4s14Pt0qdVcqRhY9q2ricPqKROI8SLokUkvF+9Rmvx6nwGX9MXVjJ1XKo7OvON4Br6n09PT3hzjZ8CflaJRHYHMEZc8PxzZG/emtR8VzRxoaLjiWAtNIOKyV7lyArxKO02gJr1L/n8GS1kXMeiqCDuJpO467oOT9tpvJ/U58k6llYuIXLUHgJbFYglWizRzppZKRZZkbudcNkF0/GpmiTMDXdw/j4WOLVcDZk1BVX4eR+eSLW2cW9JgXZxKaE4uTsBpk7UEMAUnq5XQNIGGhiuOtdEEXJVZoJzuCngXIa+eLujGRZXV4uedG0lXXY4E48xBrTbLbjJXWVbdPM49VXNZnRcsPXXc2T2qUo4lmZYUA/rSaDKZ9HbV4fZZ44nIPHdPSty6vAnmcZxW5qI21X7moCXVkJh/YnteXcnsunSFT1UL4vR51d54bjoy0kHHb1W0INA0gYaGK4+10ASAs5BN3VvAheYyK66S7OjoyO4fGOcoG+7a5bh8F8rr+ArHZZQyEE9OTnphqWw/q73NOeRafIP/dq4gl9HHfdVcAKAf+8+2u8sLCGj/S33Tz9g1V9O8WEI6745zDcb5zkOj88M9T/5O+8YegwDnt8R37HGI8Qut8OTkpMfe8725DUmdhqveAdbePmtchIPBYKmard786elpb/I437CLkV8VHahwqiWrmI7Uc8UldENP9uuG+hvf8eYZNdKtNHal+wuUJoIrUMHjoJ/VXj6Nu9/e3u69mC63gxcXjsLU6zh3Kj+/80QPukrBzjzSRDYmbmumIVf5DfDcVJKOC+nUNozhcVdzhwuYOFPVRWsymjnQ0HDFsRaawGAwwO7u7pJa5FZYJ7U1ss8RVbWgHnaJOY1B1UgXnz8ejzvpFxl1Ozs7S6WmuD+j0aj7jt1OJWJwd3e3t/q7bDJWTzVAhb9jLUfVR5Y+TvMKsJsqzovnF2PAxVS45JhmzR0fH3fpwpExyOqse7b6fJxUZujcKWUiapGQ0N547ji3oc4TJnq5r9rudDrt5aew6q95BW5zWp4LjmAvmUnd99VvGxoaXvBYG01ge3t7qSSTCyhxIcWO6HOhsIB3LbErh6Wt0wCiDUew8b0A85U+JKH2h4N/SiGjwPKq7rLm4m8uLqm7L9X2XihlS7qMxTi67dOZtwHOpD7fk6uSzPa/tsvPx9nxjqQraYUl4tFJyFLYNZN6XNla5wmHIDveQuECn1xNCn4+riK3zkXm1i60nsDziXgxSrsSM2kT5/CD54mnD8SxyjzgOun5e8f664NnYtCVnHYLj2PZSwsPg4kr9XO7qEA3Abkfrky3VgNm8lDJK0dUBTY3N7vzmDjlTTv1d1xRCJgXFaltLMux+iUTjoUF9zX+5nuKew6zJNT3wWCwZLoFdE6ySediE9grEP0peUZ4zgccec51G/k8YD7eqxaBZg40NFxxrI0mACyv5rEC84qpqyLXZ3NulYD6mfl8F3XIq7OL2df9BHZ2dnD9+nUAZ2m0m5ubVgOI+2QJBvgMM1Wz+ZqsKgZY4rlYBiUBWXNgKVRLo1bCkdtnDS3+rxWXr1271v2GzZjob3zGY8ckbnzmtkMLuLiPmkruTENXqVqJQZb2zvVcy2IMsMbooJoGpwazxuMyCwFf9EXRNIGGhiuOtdIEuDhjLffeRWzFare1tVWUVm6bKW63lFEILJfM4msBcxdeVKIN9xjneGvs+/HxcY9k4siumguIORCnYWjQCEtzJQ1dgUp2tSlZx4FYTHIyGcZjllLqNIDQlHZ2dnqRdM4N6DgKtqe1H+ySc/ekUpmDdlwAVCkbk8HPWH9X0jyU/HOSmjUpPZ81TEdQKt/CbZSwNotATGAXehrfu+SIuEHeBy8+i8Hkyjuqhjv/MrPxOmk44YP3otdrsmkT14qX7/DwsBdZNpvNuntQFdNFyPHEjs+m06klnrQN97K4RVG/K53j1O/4v5pOPOmdX59DpYF5GHiQdJGizJ/FmLIqXyPfOKFJFxLujy5o3AZDf1uL3uR5xaakEqTRf15gNeqUrzWbzbrPY05y8tqqRaCZAw0NVxxroQnMZrMudl4JmQCnefJqq+7Ara2tnqRhtUmrwToijIkWVYnZReO2nOIEH04YArAk0ZSQ45h6LcThfPHOh3x8fNyTggGXAMXqrDPD1HV13tRW3e2Xr8/RlXyeJoyxdhPPjDeduX37NoBlTSDG99atWwB85V9HFroiHnoPjtyrpZ47bZI/Y81E8wnYDewke4CLkWjEKpsFzUXY0NBQxdpoAgcHB0s2mboDXYEIltgu9djBxZAr2NWmUn97e7tnp3EWF0sEJZVYorlMOrUvmQxS8g1Aj4dwwT8BF5TELj/WglQas4aiLiiWhro70fb2dieZWKIFSbi7u9uNcSlQajKZ9DILJ5NJF0zE937z5s2lfoRGwKnYLuuwllvi5hNLeHUp8u8d4ajaAWuA+uzc3OT8BtYEXJFX4HnSBFJKP5RS+lRK6cP02YMppZ9JKf3m4vjA4vOUUvpnKaUnU0q/klJ63ar2Gxoa7i3Oown8awD/HMAP02fvAPChnPN3pZTesfj/2zHfmvzhxb8/AeD7F8cqcs44OjpaKqLgXC0aLsnsM9vMnFXHx9ls1rPJnPbBocQuR17tYJf3z54Ix1bX8hpUMrAtyYjVP75jl59yDi5kmnkLtmVLmoD+NsZHpU9I/+vXr+PBBx/s/o5xjPPDrbq5udnLlitpLjpWAZaooSWE5lXKLnRx+W4+Act8QU2yOk8K5/OX3MCuX9PptBcoxV4Nfv6qhcUYs/ZWwspFIOf8X1NKr5KP3wzgDYu/3w3g5zBfBN4M4IfzvJc/n1K6P6V0I+f89IprdDHoTk0GluOoXeosEy36wjDZyBF3wLJqzhO7pPY6Qo6vx754V6OPr1NqI8B9VMKU+1gjqpyL0Km/DkoCutwONskiJuD+++8HADzwwAPdy8/uTyZqA87lB/hYELdB62Aw6BYVrWG4t7fXW1y4PiCPmS58/aZ5cwAAFkNJREFUbq6xH780fkw88lx2BKKSkHzNWECYxNR9HjgfQ58PL44l3Ckx+FC82IvjyxafvxzAx+m8pxaf9ZBSeltK6YmU0hOrOtnQ0HBxeL6JQReVYEOncs6PYr6VOUajUS6twC6Vk1d1XYl5804Xz63fsXsvMJvNeuRfgDWHpRs3gSEBlULu96WgpfhOC3G4+2RT6DzBVu5aHOSi9+Ri1FmbCE3gRS96EYC5CaDur5xzJ92CyEspVTWB6DcTjloGbDwe48UvfjGAvptsd3e3R87u7e111+S+uSCrgCvo4oJ4Ampycq6G27ZOTbOUktXeVLPkWpsxfmp21HCnmsAnU0o3Fp26AeBTi8+fAvBKOu8VAD5xh9doaGi4BNypJvBBAI8A+K7F8QP0+TellB7DnBC8tYoPAM6kibO3Xew7r7Caucb5B2qTu8AgV3UY6BeJcCG857Gn47p8LSeRWavR6sRMDLKkcuNRCvll8L3XQn0D3C8tmZZS6pXM4v67bDyVypPJpCsvpryJGyt2e3HYeGgicX12LUb7TNK6Ap8aiu20NtYINLgofs/x/A5O2mslaSZu1QXI5x8fH3fBU6H9sIuwpgEC51gEUkrvxZwEfElK6SkA3475y//+lNJbAXwMwFcvTv8pAG8C8CSAAwBfv6r9riMbG0spti4xwxEtAfYhuxc8zuFad9Guqk4ussvFu7sFhKP3HEOv9+JKQ+siwCojM/eqnro6jC4SkCePRkS6mHfeiENLpfOGl7qByHQ6tQsK51AA8wjAeEkDXJVJ23Ami3uZ+NnpwsMvMCch6eLizAL+TnNAdCMRHj/+LUOFBC8CzlOjROLp6elSrgWAbjxXbUEGnM878LWFr95ozs0AvnHlVRsaGtYGaxExGCs7qz8qxXnVDbD6q4RS7VrAsmR32VuOeFK46DyOTdC+8IqvUshlqbFkqxUJcRWFA+xWK+VUxPfRD3Vbsqak7kCuEq1prCyduc8hrcKff3h4WKyN7zZqTSn13Iyz2awjBNXs4nnFz9URbCqV+eiKrajJySXNHClaSzXWWpTD4bAX6zKdTnukNWsMMY7PPfdcdx2nfTBa7kBDwxXHWmgCbN8GlBjilZM/01z9k5OTYqFOboPtKV1tJ5NJz/XDdrlKC9YmtIQXX5clt7PVXRFPbYvvSd1HzGW4ICqNUnPBK/yZ40o0QImzAlk7iN87Dsbl8auWEr87PDzsEb3T6dRuPxcuQg2c4T5y3QkdX977QfvNWaz8/EvlyJiQYwmvGoMjOR0nwNJf5yYXE9WoSa5rUULTBBoarjjWRhMIe6zEbrNtw9KulgtQc8kxXO0AlQgsgbWmv2PU3XWdy8+5gxSc7ee0Aqf5KEPO8eUhVdgl6/rhJLx+5oKWWALHGHBVIC235rwxLFl1XNwW4hxaHeC2lD2fTCa9cms8HqoJcHn7WlWqwGw2s8Ft6jHicdB+D4dDm++h3NhwOFzaKYmvyaHZJazFIgCc+Y9VharFufOEZWLLqevxnZoXriIuD7QjHJUM4lRfdv3pi87/r6ny+uD53l2tO5dPoOm33Aa7/Nwk0/TpBx54oDvyGPER8JuPaBLVc88911tYS2m3wLIL0rm7ODaBXZPAmUp88+ZN/P7v/373N4AuYQ1YJnPVVHEvfCm2pHRP7sXn3zuBEMdSHgy3x/kBaha4vSgUzRxoaLjiWAtNIKXUubBUpWONwElxJe7YbeOi7Fzgjgs+ciRQXMeRgLXoMOfec0E0LkAqrqmqtgteYZXfpWSrxHHBRfy9/n97e7vL1GOtRfMV4v+8exC74+IzR47VIjlVsgLLG4YqKRuawP7+fi+9mDU13nFJny3PFx2/mplZ0hxcungpSpGrY3O72haPKe/lEO03TaChoaGKtdAEgOUAHcC7S1QTYHuRSy1pnXpnT/OKGp9HqCXbYiqxOaCEJZNKTy45pjHhnA/vNB0XbnyecmuhUcXYMNj95UKIA1xYQ/vDmgPfi+YMhDRi+59dZ0pasttLY/DZrRZt8Z4LcR4XK1GSmF3Jzj7nOeZ4gmhDx5b7pmASkElOd3+OA4r+qI0/Ho97GgwHYGkWJJ9XwlosAkFacUEQfdGAfh3BUpx7idzhc/lvfVg8sZWscUUaOJrR1d6rFcXQWoZAvyIOxyHwYuPURzU9dILx+Qzn9w9/P1cFcoRgQBduvj824dw2WrrA8/joC8m+b14gYhHXDVUnk0lv7wK3sPLYuEVAow6dN8vlhygJrPeu1+b/u99yVGIc47y4vwAvGiU0c6Ch4YpjbTSBIIdUdWfJoK4iV/KJ/67V8WNtIdpgCVVaPVmKuyxGVv1VdXZ713MbWs+QpYAzT7TQiNNuasSfK1E2Ho87DeChhx4CALz0pS9d6heDiVjV3liz43NUQvLGsjpmjGiLs/045oEzBIGzLMWbN29ib28PALqUW3YNu2et48L9ZtJQx56fpzP5PlPoWJUIStV6nUZaQtMEGhquONZCEwi42HReaTVKzLn3XBw/S1RXODTAK6ySYmx/aYQe2258VBuSIxP1/KOjo57GEJLKFZfgvHzmI9TOdQFK/DslLV2OhAtGYu5BSTd+Thp5x1oNR+epa5i1JhcZqXUQ+PpxfyH19/f3O74gtAPmIVj7ZI2Ir+kyP2tBQ6V6Ai4SVbWeWgAZk5E8ptpvJklLGZqBtVgEmFhzKnwcHVlTY9n1RWP2PMAPgCe2ElV85E0ttf+sttUWgUBMWCZ0Qh3n7bq4smx8Fn+7qEBngsTYsv9ckfNZPcPf+73fW2rrxo0bXSVhfkk0nTteOK64zGpqPLO4ZxcKy2SnEmEuUtSRkQGXJMamJJ9femZuM1snQPiophj3wxGl2gb3jRdOTgEH5vNEi73UwqkVzRxoaLjiWBtNQKP8zkOmMCHCmkMpkovPdy5IVb3ceSU1X80XbsPdi8tvcKof4JN0mCjlNp07MtpQuCQWjpqLNjgJRclZdoU5t13Ane8KtaiJ49TlcCdzfzY2Nmy0XPxfnxn3g7UQtwFpnO/IXJXeTvV3KecBt4cCt+W0IOdejpwOLfHGqfUlNE2goeGKYy00gQDb7Co9eTVnIsdlrpUCWnglZpeVcxuWAj2c/eqi1ZyWwtqOSkvmN9TdyFKI7VK1//g3LkCKI+7iXLWVgX7VYOUouB/MfTjpqeMymUx6EW8uBZrJNxdcFO1eu3ataz84hiitFcfj4+Put6E5OMnuxpmfnX7Gc001Vv6OtbMaJ6USfjQa9biJ8Xjc46SYZ9F2XaFWRdMEGhquONZCEwgNgFczV2jU5Zxz/HR8VwrljGsB3k5n6aMaAIcRq7RghpeZXpdFFtD2ebVWXsFl+7EEYS+BxuW7IqgsiR1rrhI9vtvb2+u0iOjvzs5OV29A3ZM5515dA+YmWPNye0RGm+r2Go1Gvc/cPXMBERfq7TJKdZz5Gep3PHfOEyLsuCMX6s32v5YSc1uNT6fTHh9TC3NXrMUiAJz5j5XUqUW8cUVXXgxcFd64hhJlLrmE1WSXkqsvKVeAdZFuzg+sKi6bNm4y60LC1Y/YZeQqBcXnTDTGdfgeYkzjN9HurVu3AMxfJs0rGA6H3SIU24/xs4jfxgvJz4AJSBcHEf93m5soKcquMCX82FXJL6suAnzvARfjX8vHcPEB/CyU/CstDDou/Dt94U9OTnqxAC4vo4RmDjQ0XHGshSbgVLOAU2XcausyqtTVxqjtuMOSWgM5HBFWygRz32sbvOo71TyOLuU3oBKe+8tkZC0un/dXUMKRI9Q4ZTvaclFqcdRxnEwmvZwHJsziM1aDYz8Bvg4H8QBzEvDZZ5/t/gbOgpYODg5szUUn0RW1vBOG0wQU7prcbsC5nrUd4Ow58ia5mgezsbFx95pASumHUkqfSil9mD77xymlX08p/UpK6d+mlO6n796ZUnoypfTRlNKfX9V+Q0PDvcV5NIF/DeCfA/hh+uxnALwz53ySUvpuAO8E8PaU0msBvAXAHwXwBwD855TSH8o5V+MWYzV0sdhu9WVXmu4Bx3aXs4trdjTbZBqbzm1piCZLMkcI1VyQzH0o1+BsZnYfau44S169J+eG4/0SOfBEaymw/apSn/PVNXafSUa29VWrYQJM7ejNzc2lfQz0XuI53r59u9MEopho5A64IiQu6IbtZ+c+VO2N+SQX3qv2P5PKTvtQvmpzc7OqpTDhqKHEgdlstrLi8Hn2IvyvKaVXyWf/if778wC+avH3mwE8lnOeAPjtlNKTAL4QwH9fdZ2IbS/FOTv/v3txHPnCRJRLKtKJV1LXAf8icDUergRTqzJbi3TU42w265kUzkQo5UFEHzUikReNAJtTGstwfHzce5lu3rzZRalF8ZGASz3e3d21qr9LTIrvdGE7PDy0m5pqHIQbd15g3aKiHh0WDBydGOOjbHyAF3qX3xBwZDWr8o4E1IW7RG7H/axKIHo+iMFvAPAfFn+/HMDH6bunFp/1kFJ6W0rpiZTSE6tcGA0NDReHuyIGU0rfBuAEwHviI3OaZSVyzo8CeBQANjc3c/j3SwQhq9yOiKuRUbGC8yYUrMaxmy6OJZ8w+/852k+vxZqAk9jqJuN7cWNQStflz3jFV03GST6Wsq46raasAv0MxIODg84sednLXgbgbJ+C3d3dnoQfj8edK9H52VVbYddpSP+bN2925F/c8+3bt3v1DHm+uH0E9FouD8LF5ztNQOFKg7nSahzX4ua3PgN+Fqy5xr1r/gRrmCXc8SKQUnoEwFcAeGM+u8pTAF5Jp70CwCfu9BoNDQ0XjztaBFJKXwbg7QD+dM75gL76IIAfSSl9D+bE4MMA/sd52oxgllK0X2k1c7akkmG8/ZYWoSzZaSX7/PT0tLND47fb29tLMelxdCXHtH2+/xIppefFUQlHDghyhFKNQ2BtQrkM1nj0t6PRqIvfj3EJl97Ozk7Hm7CmobHvrDUFWPpHuyH9b9261ZF/XEBUK+5qjoK2rwQsawdKaG5tbXXP2El2bYs1gQDzSVwiTzNoWQtyZKvjjHSnJ66V4Ob4Ur+q38479l4AbwDwkpTSUwC+HXNvwCaAn1l0/Odzzn895/yrKaX3A/g1zM2Eb1zlGQgwobe4bu/oFgSt1suhsPqisX+Zz4kHp6GrfB6rZdEuh6zqC+OIQW7Xqf5KWjrCisdLSTTnpeAJ68gjJcI4zNT1W8eUQ3jjWvFi8u9iEvMiw+OjJlBM6oODg471j63E9vb2emYJt6sLIadiu4IgfG1dOHQcuY+rQnJVCLm0aBcnwOapK1ajwsUR3jUvhOI83oGvNR+/q3L+dwL4zpVXbmhoWAusRcQgk3m68jkpx4jfcVSZkigugYOlkap0vHo6dUyTYbg2foBXbAWrdKyKlggip/ozeFxK9RXdhp3cZ5YupchFllhx73t7e931Y7zDNcjVhl2BEXZ/6VjF+awJPPPMMwDmJKA+YyZK3XN0Kb9OC3JkYZyrGib325kFalpwvoczrVRqs3bI80SlvdMEuI+rtIGWO9DQcMWxFppAgINutH66I3cGg8FSzDswJ+nCleNSedVOc1II6NuC0SZH2fG5zlXkIhYBLwWcVGHJq3Yux4SzRHUu0BgfdQc6ycdEldMYVDIdHh52tnoQhKGVbW1t9QJgOGWcyT9tlzNBIwcgiMH9/X2bCxBw9vB5bGW2zzWC0WkCrKEF3HeOLOZnp3OeNTB1+fE9MCdQ0qD5WiU0TaCh4YpjLTSBnOclu1lacXAJsGxPBTjYhe1iV3YZmEsV1Sgmk4mVKi7YJtpytmFttXVSR8/n+6vVGnCbtnKbKklLrHa0G3D5BNoGe3A4P18lUzD2m5ubS1pBtKUeAOYh1MaPuaGf6Rg4tytrOep+TSZ7lEODNX+C3a8Mp1lGW+qR4Ou556kepqOjIzs39RmzFum4tFWcwFosAoGNjY1O7dYNODimPcDqZviIeeBc8o+60DgCy8XZ64Pkdvn/zsev/XU5DJ9pNWA+6sNNKfXGjdVxfflYjXRuQ4234DgLV/QjwIuLqsscy+4IPFcD0vWjBjcuznWqZokzL93Cw/1xQgjwQsuh5jbme68JErco1kwWRTMHGhquONIq0uBSOpHSpwHsA3jmXvcFwEvQ+sFo/VjGZ3M//mDO+aX64VosAgCQUnoi5/wFrR+tH60fl9uPZg40NFxxtEWgoeGKY50WgUfvdQcWaP1YRuvHMl5w/VgbTqChoeHeYJ00gYaGhnuAtgg0NFxxrMUikFL6sjTfp+DJlNI7Lumar0wp/WxK6SMppV9NKf2txecPppR+JqX0m4vjA5fUn2FK6ZdSSj+5+P+rU0qPL/rxvpTSeFUbz0Mf7k8p/Via7ynxkZTSF92L8UgpfevimXw4pfTelNLWZY1H8vts2DFIc/yzxbz9lZTS6y64Hxez30fEVd+rfwCGAP4PgNcAGAP4XwBeewnXvQHgdYu/7wPwGwBeC+AfAXjH4vN3APjuSxqHvw3gRwD85OL/7wfwlsXfPwDgb1xCH94N4K8t/h4DuP+yxwPz6tS/DWCbxuGvXtZ4APhTAF4H4MP0mR0DAG/CvNJ2AvB6AI9fcD/+HICNxd/fTf147eK92QTw6sX7NDz3tS56Yp3jZr8IwE/T/9+J+cYml92PDwD4swA+CuDG4rMbAD56Cdd+BYAPAfhSAD+5mFTP0ANfGqML6sP1xcuX5PNLHQ+cla1/EPPclp8E8OcvczwAvEpePjsGAP4lgK91511EP+S7vwjgPYu/l94ZAD8N4IvOe511MAfOvVfBRSHNN1f5fACPA3go5/w0ACyOL7uELnwvgL8HILJJXgzgZs45MnEuY0xeA+DTAP7Vwiz5wZTSLi55PHLOvwvgnwD4GICnAdwC8Iu4/PFglMbgXs7dO9rvw2EdFoFz71VwIRdP6RqAHwfwLTnnvcu6Ll3/KwB8Kuf8i/yxOfWix2QDc/Xz+3POn495Lsel8DOMhb39ZszV2j8AYBfAl5tT18G3fU/mbrqL/T4c1mERuGd7FaSURpgvAO/JOf/E4uNPppRuLL6/AeBTF9yNLwHwlSml3wHwGOYmwfcCuD+lFDnLlzEmTwF4Kuf8+OL/P4b5onDZ4/FnAPx2zvnTOedjAD8B4Itx+ePBKI3Bpc/ddLbfx9flhe5/t/1Yh0XgFwA8vGB/x5hvaPrBi75omidevwvAR3LO30NffRDAI4u/H8GcK7gw5JzfmXN+Rc75VZjf+3/JOX8dgJ/F2R6Pl9GP/wfg4ymlz1189EbMS8df6nhgbga8PqW0s3hG0Y9LHQ9BaQw+COCvLLwErwdwK8yGi0A62+/jK3N/v4+3pJQ2U0qvxmew3weAe08MLhazN2HOzv8fAN92Sdf8k5irTL8C4JcX/96EuT3+IQC/uTg+eInj8AaceQdes3iQTwL4UQCbl3D9Pw7gicWY/DsAD9yL8QDwDwD8OoAPA/g3mLPelzIeAN6LORdxjLmEfWtpDDBXw//FYt7+bwBfcMH9eBJz2z/m6w/Q+d+26MdHAXz5Z3KtFjbc0HDFsQ7mQENDwz1EWwQaGq442iLQ0HDF0RaBhoYrjrYINDRccbRFoKHhiqMtAg0NVxz/H/i1hqnqZEePAAAAAElFTkSuQmCC", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "plt.imshow(avg_image, cmap=plt.cm.gray)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "There are multiple ways to perform the segementation. To keep it simple, we just detect the cells by setting up the threshold on the average image." - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAASdUlEQVR4nO3dfYxc1XnH8e+vBuNAYhlDQH5TMZJJSxExyOIlVC3CSQ0U4VSCCooSN6GyWtGWJK2CXf6glRoptBGhkVrSFZA4FeE1tFiIdktdUFSpOKzBMQbHsIEUFjsYKiBVXLkmefrH3BXj9d3d2bkvc++c30da7cydO3PPnJn7nOece+ceRQRmlq5fGHQBzGywHATMEucgYJY4BwGzxDkImCXOQcAscZUFAUmXStoraVzSpqq2Y2bFqIrzBCTNA14EPgFMAE8D10bEC6VvzMwKOaai1z0PGI+IlwEk3QesB3KDwHwdFws4oaKimM3sjLMPDroItdix69BbEfHhqcurCgLLgNe67k8A53evIGkjsBFgAcdzvtZWVBSzmY2O7hx0EWoxb8n4f+Utr2pMQDnLjuh3RMRIRKyJiDXHclxFxTCb3bqlq1m3dPWgizEwVQWBCWBF1/3lwL6KtmVmBVQVBJ4GVklaKWk+cA2wtaJtmVkBlYwJRMR7kv4QGAXmAXdHxPNVbMvMiqlqYJCIeAx4rKrXN7NyVBYEzNpm3dLVjO5r95GCmQc4x3OX+rRhs8Q5EzDrMrUlLTMzyGulu19/UIcpHQTMZjC5Y+YFg5l22tF9O3vaqZtwfoK7A2aJcyZg1oMmtNhVcSZgljgHAbMKtClzcBAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS5yBgljgHAbPEOQiYJc5BwCxxDgJmiXMQMEtc30FA0gpJT0jaI+l5STdmyxdLelzSS9n/E8srrpmVrUgm8B7wJxHxy8AFwA2SzgQ2AdsiYhWwLbtvZg3VdxCIiP0R8Ux2+3+APcAyYD2wJVttC/DJooU0s+qUMiYg6TTgHGA7cGpE7IdOoABOmeY5GyWNSRo7zKEyimFmfSg874CkDwLfAT4XET+R1NPzImIEGAFYqMVRtBxWvemm5GrTlXXtaIWCgKRj6QSAeyLi4WzxG5KWRMR+SUuAA0ULaYM123x8k487GLRTkaMDAu4C9kTEbV0PbQU2ZLc3AI/0Xzwzq1qRTOAi4FPAc5Imm4o/A74MPCDpeuBV4OpiRbRBmeuMvM4I2qnvIBAR/wFMNwCwtt/XNbN6eUJSO8pcM4BJzgDayacNmyWuEUHgjLMP9t36WLn6+RzWLV3tLKDFGtUdGN2301+mlvDnNDwakQmY2eA0KhOwwem1G+AMYPg4EzBLXKOCgFuZ5hvdt9ODuEOmEUHgxV3HOwCYDUgjgoCZDY4HBhPn1N6cCZglrlFBwK1S8/nswOHTqCDgL1e9+gm6PjowfBoVBMysfq0aGJypBXIWMXvLPrWO1i1d7VbdnAmYpa5VmYDl66U1z8uU2pAFdJfR2V41nAmYJa7VmUDqLUO/GUBbdV/IdOp7H6b3WbdWBYHJD7oNaWwdUqiPXrsx7jb0z90Bs8S1KhOY5Eh/pO76qOva/01OzZtWnqZzJmCWuFZmAja9Xlq9MsYQ8rZTV/Zh5SpjVuJ5wBjwekRcIWklcB+wGHgG+FRE/F/R7VhxZe5E3d0B7/ztVkZ34EZgT9f9W4GvRsQq4G3g+hK2YWYVKRQEJC0HfhO4M7sv4BLgoWyVLcAni2zDmq2JrfRkduKfPfemaCZwO/BF4OfZ/ZOAdyLivez+BLAs74mSNkoakzR2mEMFi2Fm/ep7TEDSFcCBiNgh6eLJxTmrRt7zI2IEGAFYqMW561i52npy0UytuadDL67IwOBFwJWSLgcWAAvpZAaLJB2TZQPLgX3Fi2lmVek7CETEZmAzQJYJ/GlEXCfpQeAqOkcINgCPlFBOK0HbMoBeOAMororzBG4C7pP0l8CzwF0VbMMGZOpON4yBJTWlBIGIeBJ4Mrv9MnBeGa9rZtXzGYMJKWNgsOh5+Z5+vnn82wGzxCli8EfnFmpxnK+1s67nw0Hlqqo/n/erxl7Xt+r8Wzy0IyLWTF3equ6AvyzlqHowz4OF7eLugFniGpcJTNeKOAswq4YzAbPENSIInHH2wVnnuHM/c3j5sx2sRgQBq1cTf2LriU4Hx0HALHGtCQJNa7msGs4I6teaIGBm1WjcIcKpnAGYVcuZgFniHAQS1uQsy2MD9XEQMEucg4BZ4hwErNHcJaieg4BZ4hp7iLDJg1bDwq2sQYMzAY8OG7gxqENjg4CZ1aOx3QGr3rqlq3OvHlx3BubWfrCcCZglzplAIvKu9z/bpdzmmhF44tB2KhQEJC0C7gTOojP78GeBvcD9wGnAj4Dfjoi35/ra/rKUo3tHrmKnLuO1bLCKdgf+BviXiPgl4KPAHmATsC0iVgHbsvtm1lB9Tz4iaSHwfeD06HoRSXuBiyNiv6QlwJMR8ZGZXmvNRxfE90ZXHLHMLUf5em29XffDabrJR4pkAqcDbwLfkPSspDslnQCcGhH7AbL/p+Q9WdJGSWOSxt78758VKIaZFVEkCBwDnAvcERHnAD9lDql/RIxExJqIWPP26x9y62M2IEWCwAQwERHbs/sP0QkKb2TdALL/B4oV0cyq1HcQiIgfA69JmuzvrwVeALYCG7JlG4BHCpXQStPES43b4BU9T+CPgHskzQdeBj5DJ7A8IOl64FXg6tle5IyzDzI66t8JNIGDRHoKBYGI2AkcNdpIJyswsxbwGYMJ6m7t/UtN828HzBLXuEzAfdJ69fs7ARsejQgCL+463jv/gPiHPebugFniHATMEucgYJa4RowJ2OB4LMCcCZglzkHALHEOAmaJcxAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS16qfEuddB88/hbVhU/f33JmAWeJakwn4arjt0/2ZpZyxlfHdrfKCsIUyAUmfl/S8pN2S7pW0QNJKSdslvSTp/myKMjNrqL6DgKRlwB8DayLiLGAecA1wK/DViFgFvA1cX0ZB8yJgyq2LNd/ovp2tyGCLdgeOAT4g6TBwPLAfuAT4nezxLcCfA3cU3A7gnb5tUvy8ytzp66q/IlOTvw58hc7Mw/uBd4EdwDsR8V622gSwLO/5kjZKGpM0dphD/RbDzArqOxOQdCKwHlgJvAM8CFyWs2rkPT8iRoARgIVanLuOWVsUyQAGnTEVGRj8OPBKRLwZEYeBh4GPAYskTQaX5cC+gmU0swoVGRN4FbhA0vHA/wJrgTHgCeAq4D5gA/BI0UKaNUEVg3yDzgKg2JjAduAh4Bnguey1RoCbgC9IGgdOAu4qoZxmVpFCRwci4hbglimLXwbOK/K6Zlaf1pwxaNYm06X5TTxvwL8dMEucMwGrzOi+nY0Y+KrTbO+3ifXhIGCl6055q/zhS5O0+f25O2CWOGcCVrrJVnHYugPd76v7fts5EzBLnDMBq8ywtJRTDdv7ciZgljgHAbPEOQgMWFuuPmPDy0HALHEeGByAVFp+X224HZwJmCXOmUCFymrxp3udpreuTS+fdTgINNhsQWTYzlyz8uR9d+YtyV/X3QGzxDkTKNlcuwAzteJ5j7V9UHG28jur6U+R74UzAbPEORMoyVwicZHWrq0tZa/143GO+jkI1Mhf7N7lBQ3XXzXnXrg7YJY4ZwIlmRqVfbZc+YZ1UHGmLtBM77msQWJnAmaJcyZQkba2SmWr85Dm1G215TMYdDlnzQQk3S3pgKTdXcsWS3pc0kvZ/xOz5ZL0NUnjknZJOrfKwptZcb10B74JXDpl2SZgW0SsArZl96EzNfmq7G8jcEc5xbQ2avuJTU2wbunqyjOFWbsDEfFdSadNWbweuDi7vQV4ks5EpOuBb0VEAE9JWiRpSUTsL6vA1nxN2fmHaXC2u/xl12+/A4OnTu7Y2f9TsuXLgNe61pvIlh1F0kZJY5LGDnOoz2KYWVFlDwwqZ1nkrRgRI3SmMmehFueuY+3SlAwgj08+ml6/mcAbkpYAZP8PZMsngBVd6y0H9vVfPDOrWr9BYCuwIbu9AXika/mns6MEFwDvejzAmqhtWcDkBWmryGhm7Q5IupfOIODJkiaAW4AvAw9Iuh54Fbg6W/0x4HJgHDgIfKZQ6awVmtwNGBZTp0ArUy9HB66d5qG1OesGcEPRQplZfXzGoPXFrf9g5B0qLPpZ+LcDZolzELAkDUMmU9bZhA4CZonzmIDNyTC0oNC+Q4QzWbd09Yyfy/vvdTz3cQcBm9Ww7PgwXDt/L3p5v+4OmCXOQcBsCBQZJHQQMEucxwRsVlWeslqXVMYC+nmfDgI2qzbv/DY7dwfMEudMwKY1LBlAKl2BfjkTMEucg4BZ4twdaJkyU/TZ0uRhOCqQmn4mYHEmYJY4BwFrlDom2xhG011/sJcszkHALHEeE2iZMvrpTW5pp07TPazvs0mcCZglzkGgpYa9lZuuj2v5phtL6eV74u5Ai/XyAc80WDTsgSRF/XymzgTMEudMYMjltQzDnmY7w5mbWTMBSXdLOiBpd9eyv5b0A0m7JP2jpEVdj22WNC5pr6R1VRXczMrRS3fgm8ClU5Y9DpwVEWcDLwKbASSdCVwD/Er2nL+TNK+00lophvWEnGF9X1XrZS7C70o6bcqyf+26+xRwVXZ7PXBfRBwCXpE0DpwH/GcppbWByJv6yoZHGQODnwX+Obu9DHit67GJbNlRJG2UNCZp7DCHSiiGmfWj0MCgpJuB94B7JhflrBZ5z42IEWAEYKEW565jzTM13W5KZuBuQP/6DgKSNgBXAGuzKcmh0/Kv6FptObCv/+KZWdX6CgKSLgVuAn49Ig52PbQV+Lak24ClwCrge4VLaTaFW/7yzBoEJN0LXAycLGkCuIXO0YDjgMclATwVEb8fEc9LegB4gU434YaI+FlVhbc0OQCUq5ejA9fmLL5rhvW/BHypSKHMrD4+bdgKqbNV9nkA1XAQMEucfztghVV52NAtf/WcCZglbigzgelaonVLV/d1SWabm15/uei6b4ZWBwF/sdrDn0tzuTtglji9f8bvAAshvQn8FHhr0GUBTsbl6OZyHKnN5fjFiPjw1IWNCAIAksYiYo3L4XK4HPWWw90Bs8Q5CJglrklBYGTQBci4HEdyOY40dOVozJiAmQ1GkzIBMxsABwGzxDUiCEi6NJunYFzSppq2uULSE5L2SHpe0o3Z8sWSHpf0Uvb/xJrKM0/Ss5Ieze6vlLQ9K8f9kubXUIZFkh7K5pTYI+nCQdSHpM9nn8luSfdKWlBXfUwzz0ZuHajja9n3dpekcysuRzXzfUTEQP+AecAPgdOB+cD3gTNr2O4S4Nzs9ofozJ9wJvBXwKZs+Sbg1prq4QvAt4FHs/sPANdkt78O/EENZdgC/F52ez6wqO76oHN16leAD3TVw+/WVR/ArwHnAru7luXWAXA5nSttC7gA2F5xOX4DOCa7fWtXOc7M9pvjgJXZ/jSv521V/cXq4c1eCIx23d8MbB5AOR4BPgHsBZZky5YAe2vY9nJgG3AJ8Gj2pXqr6wM/oo4qKsPCbOfTlOW11gfvX7Z+MZ3ftjwKrKuzPoDTpux8uXUA/D1wbd56VZRjymO/BdyT3T5inwFGgQt73U4TugM9z1VQlWxylXOA7cCpEbEfIPt/Sg1FuB34IvDz7P5JwDsR8V52v446OR14E/hG1i25U9IJ1FwfEfE68BXgVWA/8C6wg/rro9t0dTDI725f833kaUIQ6Hmugko2Ln0Q+A7wuYj4SV3b7dr+FcCBiNjRvThn1arr5Bg66ecdEXEOnd9y1DI+0y3rb6+nk9YuBU4ALstZtQnHtgfy3S0y30eeJgSBgc1VIOlYOgHgnoh4OFv8hqQl2eNLgAMVF+Mi4EpJPwLuo9MluB1YJGnyp9511MkEMBER27P7D9EJCnXXx8eBVyLizYg4DDwMfIz666PbdHVQ+3e3a76P6yLL/YuWowlB4GlgVTb6O5/OhKZbq96oOtdKvwvYExG3dT20FdiQ3d5AZ6ygMhGxOSKWR8RpdN77v0fEdcATvD/HYx3l+DHwmqSPZIvW0rl0fK31QacbcIGk47PPaLIctdbHFNPVwVbg09lRgguAdye7DVXomu/jyjh6vo9rJB0naSVzne+jykGeOQyAXE5ndP6HwM01bfNX6aRMu4Cd2d/ldPrj24CXsv+La6yHi3n/6MDp2Qc5DjwIHFfD9lcDY1md/BNw4iDqA/gL4AfAbuAf6Ix611IfwL10xiIO02lhr5+uDuik4X+bfW+fA9ZUXI5xOn3/ye/r17vWvzkrx17gsrlsy6cNmyWuCd0BMxsgBwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeL+HzgSnDgghdkpAAAAAElFTkSuQmCC", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "threshold = 50\n", - "mask = avg_image > threshold\n", - "plt.imshow(mask)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The outcome is different across different threholds we set. Therefore, this threshold is a parameter we could potentially tweak." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQhUlEQVR4nO3df4wc5X3H8fen/hmTItsQkH+pGMlJS6vGoBMxoYoQTmpCEaYSSEYocVNXVivakqRSsMsfqFIjhTZKaNSK1AISp6IQ6tDaQrRX4xBFlYrDAa4xGPAFWjjsYFCBREFy7ObbP+a5eO/Y8+3N7Mzu7fN5Sdbuzs7ufD13+5nv8+zejiICM8vXL/W6ADPrLYeAWeYcAmaZcwiYZc4hYJY5h4BZ5moLAUlXSXpB0qikbXVtx8yqUR2fE5A0B3gR+AQwBjwB3BgRz3V9Y2ZWydyanvdSYDQiXgKQ9ACwEWgbAvO1IBZyVk2lmHXmg7/5bq9LqMWLBxcB8BPeejMiPjD5/rpCYAXwasvtMeAjrStI2gpsBVjIIj6i9TWVYtaZ4eEDvS6hFhuWrwXg0dj1P+3ur2tOQG2WTRh3RMSOiBiKiKF5LKipDLPOjb9YBkkn/6e6OoExYFXL7ZXA0Zq2ZWZJmSCrqxN4AlgjabWk+cAmYE9N2zKzCmrpBCLilKQ/BoaBOcC9EfFsHdsyG2TjR/bho2eer6gylKlrOEBEPAI8Utfzm1l31BYCZrPR5CPqdEfgTp+nqjonLf2xYbPMuRMwO4NBfNtwMncCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa50CEhaJekxSYclPSvplrR8qaS9ko6kyyXdK9fMuq1KJ3AK+LOI+DVgHXCzpIuAbcC+iFgD7Eu3zaxPlQ6BiDgWEU+l6z8BDgMrgI3AzrTaTuC6qkWaWX26Micg6QLgYmA/cH5EHIMiKIDzpnjMVkkjkkZOcqIbZZhZCZVDQNL7ge8An42IH3f6uIjYERFDETE0jwVVyzCzkiqFgKR5FAFwX0Q8lBa/LmlZun8ZcLxaiWZWpyrvDgi4BzgcEV9puWsPsDld3wzsLl+emdWtylmJLwc+BTwjafwk7n8OfAl4UNIW4BXghmolWj8aPlr8yHM4a++gKx0CEfEfgKa4e33Z5zWzZlXpBCxj7gAGhz82bJY5h4BZ5hwCZplzCJhlzhODNsH4W3/gyb9cuBMwy5w7AZvAR//8uBOwaQ0fPTBhmGCDxSFgljmHgFnmHAJmmfPEoAG0HfOPTxJ6snCwuROwKXlCMA8OAbPMeThg0/KnCAebOwGzzLkTGCB1H7HdBQwmdwJmmXMnkDnP/ptDYIC4XbcyPBwwy5xDwDrWz0OH8Q829XON/cohYJY5h4BZ5ipPDEqaA4wAr0XENZJWAw8AS4GngE9FxM+qbse6ZxBbZk+KlteNTuAW4HDL7TuAr0bEGuAtYEsXtmFmNal6avKVwO8Ad6fbAq4EdqVVdgLXVdmGmdWraidwJ/AF4Ofp9jnA2xFxKt0eA1a0e6CkrZJGJI2c5ETFMsysrNIhIOka4HhEPNm6uM2q0e7xEbEjIoYiYmgeC8qWYSVsWL7WY2j7hSoTg5cD10q6GlgInE3RGSyWNDd1AyuBo9XLNLO6lO4EImJ7RKyMiAuATcB3I+Im4DHg+rTaZmB35SqtL7h7GEx1/O3ArcADkv4SeBq4p4ZtWAUzfYvQL/7B1pUQiIjvAd9L118CLu3G85pZ/fxXhBlqPbJ30hWMr+OOYDD5Y8NmmXMnkLnxo/tMOoLWx9ns5xDI3CD+HYHNjIcDZplzJ5CxJrqAM53ezPqDOwGzzLkTsFI8STg4HAIZ27B8bVeGBJOfw6Ewu3g4YJY5h4B13XTf+utvBe4vDgGzzDkErDbTHe3dEfQHh4BZ5hTR9tu/GjX04YXxg+FVE5Z5hrlZvT4i++ddv0dj15MRMTR5ud8iNGBmf0jU7nFlHtvKf67cOx4OmGXOnYBNcKYj8VRH+m4OJdwRNM+dgFnm+i4E/J34/cs/l8Hk4YDNSLsgmEk4TDV0cMD0Tt91AmbWLHcC1igf8fuPOwGzzPVFJ/DiwUU+Qpj1SKVOQNJiSbskPS/psKTLJC2VtFfSkXS5pFvFmln3VR0O/A3wbxHxq8CHgcPANmBfRKwB9qXbZtanSoeApLOBj5FOOBoRP4uIt4GNwM602k7guqpFmll9qnQCFwJvAN+Q9LSkuyWdBZwfEccA0uV57R4saaukEUkjJzlRoQwzq6JKCMwFLgHuioiLgZ8yg9Y/InZExFBEDM1jQYUyzKyKKiEwBoxFxP50exdFKLwuaRlAujxerUQzq1PpEIiIHwGvSvpQWrQeeA7YA2xOyzYDuytVaGa1qvo5gT8B7pM0H3gJ+AxFsDwoaQvwCnBDxW2YWY0qhUBEHADe83VFFF2Bmc0C/tiwWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYqhYCkz0l6VtIhSfdLWihptaT9ko5I+nY6RZmZ9anSISBpBfCnwFBE/AYwB9gE3AF8NSLWAG8BW7pRqJnVo+pwYC7wPklzgUXAMeBKitOUA+wErqu4DTOrUZVTk78GfJnizMPHgHeAJ4G3I+JUWm0MWNHu8ZK2ShqRNHKSE2XLMLOKqgwHlgAbgdXAcuAs4JNtVo12j4+IHRExFBFD81hQtgwzq6jKcODjwMsR8UZEnAQeAj4KLE7DA4CVwNGKNZpZjaqEwCvAOkmLJAlYDzwHPAZcn9bZDOyuVqKZ1anKnMB+ignAp4Bn0nPtAG4FPi9pFDgHuKcLdZpZTeZOv8rUIuJ24PZJi18CLq3yvGbWHH9i0CxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMlfpOwbNZmr46IFfXN+wfG0PK7FxDgHrmdZAmMwBUU27fTtnWft1PRwwy5w7AetLZ+oSwJ3CVKbbb+24EzDLnDsBa0SZI1TZ58uxS6iyf6ftBCTdK+m4pEMty5ZK2ivpSLpckpZL0tckjUo6KOmS0pWZWSM6GQ58E7hq0rJtwL6IWAPsS7ehODX5mvRvK3BXd8q02Wr46IGudwE2UdX9O20IRMT3gf+dtHgjsDNd3wlc17L8W1F4nOI05VO8MWGDrlcv/tyCp+rwp+zE4PkRcQwgXZ6Xlq8AXm1Zbywtew9JWyWNSBo5yYmSZZhZVd2eGFSbZdFuxYjYQXEqc87W0rbr2OzUL0fhdnUM6qTh+P+rybcIXx9v89Pl8bR8DFjVst5K4GjJbZhZA8qGwB5gc7q+GdjdsvzT6V2CdcA748MGM+tP0w4HJN0PXAGcK2kMuB34EvCgpC3AK8ANafVHgKuBUeBd4DM11GwD5kwter8MLWaLMsOCaUMgIm6c4q71bdYN4OaOt25mPedPDFrXzPSo3ckkXes67go6N5P95r8dMMucOwGrrKkjtLuCck7vt9G297sTMMucOwGrpFdH5MnzCe4MynMI2IxUfbF1+xN7ndYzqJ8U7AYPB8wy5xCwjs3WlttdwJk5BMwy5xCwWW3D8rU+0lfkEDDLnEPALHN+i9Cm5QnBweZOwCxz7gRs4LgDmBl3Ajatbs7A13ESktk6XOkXDgGzzHk4YI0bP3JX6S589O8edwJmmXMnYNOq66hbtiNwF9Bd7gTMMucQsGnV/fn8To/sfiegHg4B61jdQTDVC9wv/no5BMwy54lBm5EqJ74sYybb8ScFy5m2E5B0r6Tjkg61LPtrSc9LOijpnyUtbrlvu6RRSS9I2lBX4WbWHZ10At8E/hb4VsuyvcD2iDgl6Q5gO3CrpIuATcCvA8uBRyV9MCL+r7tlW690owM40zcFl31+dwHldXIuwu9LumDSsn9vufk4cH26vhF4ICJOAC9LGgUuBf6zK9Vaz3VjOOBJvv7SjYnB3wf+NV1fAbzact9YWvYekrZKGpE0cpITXSjDzMqoNDEo6TbgFHDf+KI2q0W7x0bEDmAHwNla2nYd61+9PiWY2//uKR0CkjYD1wDr0ynJoTjyr2pZbSVwtHx5Zla3UsMBSVcBtwLXRsS7LXftATZJWiBpNbAG+EH1Ms1OcxfQXdN2ApLuB64AzpU0BtxO8W7AAmCvJIDHI+IPI+JZSQ8Cz1EME272OwPWLX7x16OTdwdubLP4njOs/0Xgi1WKMrPm+BODVlmdnyL00b9+/tsBs8y5E7Cu6fSo3Y2vF7PucSdgljl3AtY4dwD9xZ2AWeYcAmaZ0+lP/PawCOkN4KfAm72uBTgX19HKdUw0m+v4lYj4wOSFfRECAJJGImLIdbgO19FsHR4OmGXOIWCWuX4KgR29LiBxHRO5jokGro6+mRMws97op07AzHrAIWCWub4IAUlXpfMUjEra1tA2V0l6TNJhSc9KuiUtXyppr6Qj6XJJQ/XMkfS0pIfT7dWS9qc6vi1pfgM1LJa0K51T4rCky3qxPyR9Lv1MDkm6X9LCpvbHFOfZaLsPVPha+r09KOmSmuuo53wfEdHTf8Ac4IfAhcB84L+AixrY7jLgknT9l4EXgYuAvwK2peXbgDsa2g+fB/4ReDjdfhDYlK5/HfijBmrYCfxBuj4fWNz0/qD4duqXgfe17Iffa2p/AB8DLgEOtSxruw+Aqym+aVvAOmB/zXX8NjA3Xb+jpY6L0utmAbA6vZ7mdLytun+xOvjPXgYMt9zeTnFik6br2A18AngBWJaWLQNeaGDbK4F9wJXAw+mX6s2WH/iEfVRTDWenF58mLW90f3D6a+uXUvyB28PAhib3B3DBpBdf230A/D1wY7v16qhj0n2/C9yXrk94zQDDwGWdbqcfhgMdn6ugLunkKhcD+4HzI+IYQLo8r4ES7gS+APw83T4HeDsiTqXbTeyTC4E3gG+kYcndks6i4f0REa8BXwZeAY4B7wBP0vz+aDXVPujl726p83200w8h0PG5CmrZuPR+4DvAZyPix01tt2X71wDHI+LJ1sVtVq17n8ylaD/vioiLKf6Wo5H5mVZpvL2Roq1dDpwFfLLNqv3w3nZPfnernO+jnX4IgZ6dq0DSPIoAuC8iHkqLX5e0LN2/DDhecxmXA9dK+m/gAYohwZ3AYknj3/fQxD4ZA8YiYn+6vYsiFJreHx8HXo6INyLiJPAQ8FGa3x+tptoHjf/utpzv46ZIvX/VOvohBJ4A1qTZ3/kUJzTdU/dGVXxX+j3A4Yj4Sstde4DN6fpmirmC2kTE9ohYGREXUPzfvxsRNwGPcfocj03U8SPgVUkfSovWU3x1fKP7g2IYsE7SovQzGq+j0f0xyVT7YA/w6fQuwTrgnfFhQx1qO99HnZM8M5gAuZpidv6HwG0NbfO3KFqmg8CB9O9qivH4PuBIulza4H64gtPvDlyYfpCjwD8BCxrY/lpgJO2TfwGW9GJ/AH8BPA8cAv6BYta7kf0B3E8xF3GS4gi7Zap9QNGG/136vX0GGKq5jlGKsf/47+vXW9a/LdXxAvDJmWzLHxs2y1w/DAfMrIccAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhl7v8B+wWmT91sIhwAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "threshold = 60\n", - "mask = avg_image > threshold\n", - "plt.imshow(mask)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we could use scipy.ndimage to detect the blobs from this binary mask\n", - "\n", - "For the detailed tutorial, please refer to https://scipy-lectures.org/advanced/image_processing/index.html#segmentation" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [], - "source": [ - "from scipy import ndimage" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The method `label` marks each blob with different number, `label_im` is the image with different blobs marked with different number and `nb_labels` is the number of blobs that are detected" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "10\n" - ] - }, - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQzUlEQVR4nO3df4wc9X3G8fdTn30Ep8g2BGR8qLaDm9aKUqBXMFAhhJPwowhTCSRTRK6pK6sVbQmpFNvlD1SpUeI2CjRSRWoBiRM5EOrQ2kK0FAxRVDV2OcA1BkN8mBQOOxhUICmRjJ18+sd+D+8de769nZ3Zvf0+L+m0O7OzOx+Pd5/5fGd/jCICM8vXr3S6ADPrLIeAWeYcAmaZcwiYZc4hYJY5h4BZ5koLAUlXSHpR0oik9WWtx8yKURmfE5A0C/gR8ClgFHgSuCEinm/7ysyskL6SHvd8YCQiDgBIuh9YBTQMgTnqj5OYW1IpZs1R/5xOl1CKOPIeAD/jrTcj4iMTby8rBBYBr9ZNjwIX1C8gaS2wFuAkTuYCrSypFLPm9A0s7nQJpTh24McAPBZb/6fR7WUdE1CDeePGHRGxKSIGI2JwNv0llWHWvLEXSy9p5t9UVicwCpxVNz0AHCxpXWaWtBJkZXUCTwLLJC2RNAdYDWwvaV1mVkApnUBEHJP0Z8AjwCzg3oh4rox1mfWysT1739LFTS3XirKGA0TEw8DDZT2+mbVHaSFgNhNN3KNOtQdu9nGKKvOgpT82bJY5dwJmJ9CLbxtO5E7ALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzLUcApLOkvSEpH2SnpN0S5q/QNKjkvany/ntK9fM2q1IJ3AM+MuI+E1gBXCzpOXAemBHRCwDdqRpM+tSLYdARByKiKfT9Z8B+4BFwCpgc1psM3Bt0SLNrDxtOSYgaTFwLrALOCMiDkEtKIDTJ7nPWknDkoaPcqQdZZhZCwqHgKQPA98DPhcRP232fhGxKSIGI2JwNv1FyzCzFhUKAUmzqQXAloh4MM1+XdLCdPtC4HCxEs2sTEXeHRBwD7AvIr5ad9N2YChdHwK2tV6emZWtyFmJLwZuAp6VtDvN+yvgy8ADktYArwDXFyvRutGxy34bgL7Hn+pwJVZUyyEQEf8BaJKbV7b6uGZWrSKdgGXMHUDv8MeGzTLnEDDLnEPALHMOAbPM+cCgjfP2TRe+f33et3/YwUqsKu4EzDLnTsDG8d4/P+4EbEqvrbuI19Zd1OkyrCQOAbPMOQTMMucQMMucDwwaACN3rPjAvLNv3QnAoo3/WXU5ViF3AjapkTtWNAwH6y0OAbPMeThgU6rvBsaGCNY73AmYZc6dQA858J1z3r++9A92n2DJ1rgL6E3uBMwy504gcz76bw6BHlLGEMB6n4cDZplzCFjTunrosGPg+J9Ni0PALHMOAbPMFT4wKGkWMAy8FhFXS1oC3A8sAJ4GboqI94qux9qnq9v6Vq0c7XQFM1Y7OoFbgH110xuBOyJiGfAWsKYN6zCzkhQ9NfkA8HvA3WlawGXA1rTIZuDaIusws3IV7QTuBL4A/DJNnwq8HRHH0vQosKjRHSWtlTQsafgoRwqWYWatajkEJF0NHI6I+jNTNjpLcTS6f0RsiojBiBicTX+rZVgLzr51p78HYO8rcmDwYuAaSVcBJwGnUOsM5knqS93AAHCweJlmVpaWO4GI2BARAxGxGFgNPB4RNwJPANelxYaAbYWrtK7g7qE3lfHdgXXA/ZL+BngGuKeEdVgB032L0C/+3taWEIiI7wPfT9cPAOe343HNrHz+FmGG6vfszXQFY8u4I+hN/tiwWebcCWRubO8+nY6g/n428zkEMteT3yOwafFwwCxz7gQyVkUXcKLTm1l3cCdgljl3AtYSHyTsHQ6BjJ196862DAkmPoZDYWbxcMAscw4Ba7upTmnuU553F4eAWeYcAlaaqfb27gi6g0PALHNdEQJzlwe/s/sX4/6sGt3wU2PuBjrLbxEaML0vEjW6Xyv3reevK3dOV3QCZtY57gRsnBPtiSfb07eznXdHUD13AmaZ67oQePKcWTx5zqxOl2ENeO/cmzwcsGlpFATTCYfJhg4OmM7puk7AzKrlTsAq5T1+93EnYJa5rugE3n1ePhho1iGFOgFJ8yRtlfSCpH2SLpS0QNKjkvany/ntKtbM2q/ocODvgX+LiN8AfgvYB6wHdkTEMmBHmjazLtVyCEg6BbiEdMLRiHgvIt4GVgGb02KbgWuLFmlm5SnSCSwF3gC+IekZSXdLmgucERGHANLl6Y3uLGmtpGFJw0c5UqAMMyuiSAj0AecBd0XEucC7TKP1j4hNETEYEYOz6S9QhpkVUSQERoHRiNiVprdSC4XXJS0ESJeHi5VoZmVqOQQi4ifAq5I+lmatBJ4HtgNDad4QsK1QhWZWqqKfE/hzYIukOcAB4LPUguUBSWuAV4DrC67DzEpUKAQiYjcw2OCmlUUe18yq448Nm2XOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuUAhIulXSc5L2SrpP0kmSlkjaJWm/pO+mU5SZWZdqOQQkLQL+AhiMiI8Ds4DVwEbgjohYBrwFrGlHoWZWjqLDgT7gQ5L6gJOBQ8Bl1E5TDrAZuLbgOsysREVOTf4a8BVqZx4+BLwDPAW8HRHH0mKjwKJG95e0VtKwpOGjHGm1DDMrqMhwYD6wClgCnAnMBa5ssGg0un9EbIqIwYgYnE1/q2WYWUFFhgOfBF6OiDci4ijwIHARMC8NDwAGgIMFazSzEhUJgVeAFZJOliRgJfA88ARwXVpmCNhWrEQzK1ORYwK7qB0AfBp4Nj3WJmAd8HlJI8CpwD1tqNPMStI39SKTi4jbgdsnzD4AnF/kcc2sOv7EoFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa7QbwyaTdeGl/a8f/1LH/1EByuxMQ4B65j6QJjIAVHMIwd3f2DerIWNl/VwwCxz7gSsK52oSwB3CpNp1AFMxZ2AWebcCVglptqzt/PxcuwSWukAxkzZCUi6V9JhSXvr5i2Q9Kik/elyfpovSV+TNCJpj6TzWq7MzCrRzHDgm8AVE+atB3ZExDJgR5qG2qnJl6W/tcBd7SnTZqoNL+1pexdg4xXpAqCJEIiIHwD/O2H2KmBzur4ZuLZu/reiZie105RP8saE9bpOvfhzC57Lzzyn0P1bPTB4RkQcAkiXp6f5i4BX65YbTfM+QNJaScOSho9ypMUyzKyodh8YVIN50WjBiNhE7VTmnKIFDZexmalb9sKN6ujVg4Zj3UCVbxG+Ptbmp8vDaf4ocFbdcgPAwRbXYWYVaDUEtgND6foQsK1u/mfSuwQrgHfGhg1m1p2mHA5Iug+4FDhN0ihwO/Bl4AFJa4BXgOvT4g8DVwEjwM+Bz5ZQs/WYE7Xo3TK0mClaGRZMGQIRccMkN61ssGwANze9djPrOH9i0NpmunvtZg7S1S/jrqB59W8bTtUV+LsDZplzJ2CFVbWHdlfQmuNdwUjD290JmGXOnYAV0qk98sTjCe4MWucQsGkp+mJr9yf2mq2nVz8p2A4eDphlziFgTZupLbe7gBNzCJhlziFgM9qXPvoJ7+kLcgiYZc4hYJY5v0VoU/IBwd7mTsAsc+4ErOe4A5gedwI2pXYegS/jJCQzdbjSLRwCZpnzcMAqN7bnLtJdeO/fPu4EzDLnTsCmVNZet9WOwF1Ae7kTMMucQ8CmVPbn85vds/udgHI4BKxpZQfBZC9wv/jL5RAwy5wPDNq0jHUDVe2Zp7Mef1KwNVN2ApLulXRY0t66eX8n6QVJeyT9s6R5dbdtkDQi6UVJl5dVuJm1h2pnDjvBAtIlwP8B34qIj6d5nwYej4hjkjYCRMQ6ScuB+4DzgTOBx4Bfj4hfnGgdp2hBXKAPnNXMulA7OoAyfinYXcDUHoutT0XE4MT5zZyL8AeSFk+Y9+91kzuB69L1VcD9EXEEeFnSCLVA+GGLdVuXacdwwAf5uks7Dgz+EfCv6foi4NW620bTvA+QtFbSsKThoxxpQxlm1opCBwYl3QYcA7aMzWqwWMPxRkRsAjZBbThQpA6rXqdPCeb2v31aDgFJQ8DVwMo4fmBhFDirbrEB4GDr5ZlZ2VoaDki6AlgHXBMRP6+7aTuwWlK/pCXAMuC/ipdpdpy7gPaashOQdB9wKXCapFHgdmAD0A88KglgZ0T8SUQ8J+kB4Hlqw4Sbp3pnwKxZfvGXo5l3B25oMPueEyz/ReCLRYoys+r4E4NWWJmfIvTev3z+7oBZ5twJWNs0u9dux8+LWfu4EzDLnDsBq5w7gO7iTsAscw4Bs8xN+VXiSoqQ3gDeBd7sdC3AabiOeq5jvJlcx69FxEcmzuyKEACQNNzou86uw3W4jnLr8HDALHMOAbPMdVMIbOp0AYnrGM91jNdzdXTNMQEz64xu6gTMrAMcAmaZ64oQkHRFOk/BiKT1Fa3zLElPSNon6TlJt6T5CyQ9Kml/upxfUT2zJD0j6aE0vUTSrlTHdyXNqaCGeZK2pnNK7JN0YSe2h6Rb0//JXkn3STqpqu0xyXk2Gm4D1XwtPW/3SDqv5DrKOd9HRHT0D5gFvAQsBeYA/w0sr2C9C4Hz0vVfBX4ELAf+Flif5q8HNla0HT4PfAd4KE0/AKxO178O/GkFNWwG/jhdnwPMq3p7UPt16peBD9Vthz+sansAlwDnAXvr5jXcBsBV1H5pW8AKYFfJdXwa6EvXN9bVsTy9bvqBJen1NKvpdZX9xGriH3sh8Ejd9AZgQwfq2AZ8CngRWJjmLQRerGDdA8AO4DLgofSkerPuP3zcNiqphlPSi08T5le6PTj+s/ULqH3B7SHg8iq3B7B4wouv4TYA/hG4odFyZdQx4bbfB7ak6+NeM8AjwIXNrqcbhgNNn6ugLOnkKucCu4AzIuIQQLo8vYIS7gS+APwyTZ8KvB0Rx9J0FdtkKfAG8I00LLlb0lwq3h4R8RrwFeAV4BDwDvAU1W+PepNtg04+d1s630cj3RACTZ+roJSVSx8Gvgd8LiJ+WtV669Z/NXA4Ip6qn91g0bK3SR+19vOuiDiX2nc5Kjk+Uy+Nt1dRa2vPBOYCVzZYtBve2+7Ic7fI+T4a6YYQ6Ni5CiTNphYAWyLiwTT7dUkL0+0LgcMll3ExcI2kHwP3UxsS3AnMkzT2ew9VbJNRYDQidqXprdRCoert8Ung5Yh4IyKOAg8CF1H99qg32Tao/Llbd76PGyP1/kXr6IYQeBJYlo7+zgFWUzt/QalU+630e4B9EfHVupu2A0Pp+hC1YwWliYgNETEQEYup/dsfj4gbgSc4fo7HKur4CfCqpI+lWSup/XR8pduD2jBghaST0//RWB2Vbo8JJtsG24HPpHcJVgDvjA0bylDa+T7KPMgzjQMgV1E7Ov8ScFtF6/xdai3THmB3+ruK2nh8B7A/XS6ocDtcyvF3B5am/8gR4J+A/grWfw4wnLbJvwDzO7E9gL8GXgD2At+mdtS7ku1B7azah4Cj1PawaybbBtTa8H9Iz9tngcGS6xihNvYfe75+vW7521IdLwJXTmdd/tiwWea6YThgZh3kEDDLnEPALHMOAbPMOQTMMucQMMucQ8Asc/8PbqC4fI92GrEAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "label_im, nb_labels = ndimage.label(mask)\n", - "print(nb_labels)\n", - "plt.imshow(label_im) " - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], dtype=int32)" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "np.unique(label_im)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "0 marks the background and 1 to 10 mark the detected blobs. Some of these are too small to be cells. We could set a cutoff on size to filter out the small blobs. Here the cutoff is another parameter that could be tweaked." - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQEElEQVR4nO3de4xU93nG8e9TLktMigA7trip4ArS0kq1rZWD4yqyTFJfahkq2RKWlWxTLNTKbZ2kUgz1H1alRgptlLhRK6cIOyGVC6HELchySwkhiirV1Gub2mBswHYLa4jBqu1EiUQgefvH/DYM61l2ds5lZvf3fCQ0c86cmfNyduY57+/M5SgiMLN8/VK3CzCz7nIImGXOIWCWOYeAWeYcAmaZcwiYZa6yEJB0q6RXJR2TtL6q9ZhZMaricwKSpgBHgE8AQ8CzwD0R8XLpKzOzQqZW9LjXA8ci4nUASduAVUDLEJiuvpjBzIpKMWuP+vq6XUIl4uxZAH7EO29HxIdG3l5VCCwATjRNDwEfaV5A0jpgHcAMLuMjWllRKWbtmbL4V7tdQiV+duQ1AL4TO/631e1VHRNQi3kXjTsiYlNE9EdE/zQmZwLbxDL8YplM2vk/VdUJDAGLmqYXAicrWpeZJZ0EWVWdwLPAUklLJE0H1gC7KlqXmRVQSScQEecl/TGwG5gCPB4Rh6pYl9lkNrxnn7Ls0scrigxlqhoOEBFPA09X9fhmVo7KQsBsIhq5Rx1rD9zu4xRV5UFLf2zYLHPuBMwuYTK+bTiSOwGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy13EISFokaZ+kw5IOSXogzZ8raY+ko+lyTnnlmlnZinQC54E/i4hfB1YA90taDqwH9kbEUmBvmjazHtVxCETEqYh4Pl3/EXAYWACsArakxbYAq4sWaWbVKeWYgKTFwLXAfuCqiDgFjaAArhzlPuskDUoaPMfZMsowsw4UDgFJHwS+DXwmIn7Y7v0iYlNE9EdE/zT6ipZhZh0qFAKSptEIgCci4sk0+y1J89Lt84DTxUo0syoVeXdAwGPA4Yj4ctNNu4CBdH0A2Nl5eWZWtSJnJb4R+CTwkqQDad6fA18EtktaCxwH7i5WoplVqeMQiIj/ADTKzSs7fVwzq5c/MWiWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuyLcIbRI5srn/ffOW3TfYhUqsbu4EbFRHNve3DAebXBwCZpnzcMDG1NwNeIgw+bgTMMucQ8Da5i5gcnIImGXOxwQy56P/5k7ALHMOAbPMOQSsbR46TE4OAbPMOQTMMlf43QFJU4BB4M2IuEPSEmAbMBd4HvhkRPy06HqsPG7rrVkZncADwOGm6Y3AVyJiKfAOsLaEdZhZRYqemnwh8LvA5jQt4GZgR1pkC7C6yDrMrFpFO4FHgM8DP0/TlwPvRsT5ND0ELGh1R0nrJA1KGjzH2YJlmFmnOg4BSXcApyPiuebZLRaNVvePiE0R0R8R/dPo67QM68Cy+wb9PQD7hSIHBm8E7pR0OzADmEWjM5gtaWrqBhYCJ4uXaWZV6bgTiIgNEbEwIhYDa4DvRsS9wD7grrTYALCzcJXWE9w9TE5VfIHoQWCbpL8EXgAeq2AdVsB43yL0i39yKyUEIuJ7wPfS9deB68t4XDOrnr9KnKHmPXs7XcHwMu4IJid/bNgsc+4EMje8dx9PR9B8P5v4HAKZ8/cIzMMBs8y5E8hYHV2AT2/W+9wJmGXOnYB1xAcJJw+HQMaW3TdYypBg5GM4FCYWDwfMMucQsNKNdUpzn/K8tzgEzDLnELDKjLW3d0fQGxwCZplzCGSuF35qzN1Ad/ktQgPG90WiVvfr5L7N/HXl7nEnYJY5dwJ2kUvtiUfb05fZzrsjqJ87AbPMOQSsbd47T04eDti4tAqC8YTDaEMHB0z3uBMwy5w7AauV9/i9x52AWeYcAmaZKxQCkmZL2iHpFUmHJd0gaa6kPZKOpss5ZRVrZuUr2gn8DfBvEfFrwG8Bh4H1wN6IWArsTdNm1qM6DgFJs4CPkU44GhE/jYh3gVXAlrTYFmB10SLNrDpFOoGrgTPA1yW9IGmzpJnAVRFxCiBdXtnqzpLWSRqUNHiOswXKMLMiioTAVOA64NGIuBb4MeNo/SNiU0T0R0T/NPoKlGFmRRQJgSFgKCL2p+kdNELhLUnzANLl6WIlmlmVOg6BiPgBcELSh9OslcDLwC5gIM0bAHYWqtDMKlX0E4N/AjwhaTrwOvBpGsGyXdJa4Dhwd8F1mFmFCoVARBwAWn0jZGWRxzWz+vgTg2aZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZKxQCkj4r6ZCkg5K2SpohaYmk/ZKOSvpWOkWZmfWojkNA0gLgT4H+iPhNYAqwBtgIfCUilgLvAGvLKNTMqlF0ODAV+ICkqcBlwCngZhqnKQfYAqwuuA4zq1CRU5O/CXyJxpmHTwHvAc8B70bE+bTYELCg1f0lrZM0KGnwHGc7LcPMCioyHJgDrAKWAPOBmcBtLRaNVvePiE0R0R8R/dPo67QMMyuoyHDg48AbEXEmIs4BTwIfBWan4QHAQuBkwRrNrEJFQuA4sELSZZIErAReBvYBd6VlBoCdxUo0syoVOSawn8YBwOeBl9JjbQIeBD4n6RhwOfBYCXWaWUWmjr3I6CLiYeDhEbNfB64v8rhmVh9/YtAscw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDJX6DcGzcZr98kDv7h+y/xruliJDXMIWNc0B8JIDoj6eDhgljl3AtaTLtUlgDuFMrkTMMucOwGrxVh79jIfz13C+IzZCUh6XNJpSQeb5s2VtEfS0XQ5J82XpK9KOibpRUnXVVm8mRXXznDgG8CtI+atB/ZGxFJgb5qGxqnJl6Z/64BHyynTJqrdJw+U3gVYucYMgYj4PvB/I2avArak61uA1U3zvxkNz9A4Tfm8soq1iaVbL34Hz/h0emDwqog4BZAur0zzFwAnmpYbSvPeR9I6SYOSBs9xtsMyzKyosg8MqsW8aLVgRGyicSpzZmluy2VsYuqVvXCrOnzQ8P067QTeGm7z0+XpNH8IWNS03ELgZOflmVnVOg2BXcBAuj4A7Gya/6n0LsEK4L3hYYOZ9aYxhwOStgI3AVdIGgIeBr4IbJe0FjgO3J0Wfxq4HTgG/AT4dAU12yRzqRa9V4YWk9mYIRAR94xy08oWywZwf9GizKw+/sSglWa8e+12DtI1L+OuoBr+7oBZ5twJWGF17aHdFVTDnYBZ5twJWCHd2iOPPJ7gzqBzDgEbl6IvtrI/sdduPf6k4Og8HDDLnEPA2jZRW253AZfmEDDLnEPAJrRb5l/jPX1BDgGzzDkEzDLntwhtTD4gOLm5EzDLnDsBm3TcAYyPOwEbU5lH4Ks4CclEHa70CoeAWeY8HLDaDe+5i3QX3vuXx52AWebcCdiYqtrrdtoRuAsolzsBs8w5BGxMVX8+v909u98JqIZDwNpWdRCM9gL3i79aDgGzzPnAoI3LcDdQ1555POvxJwU7M2YnIOlxSaclHWya99eSXpH0oqR/ljS76bYNko5JelXSLVUVbmblaKcT+Abwt8A3m+btATZExHlJG4ENwIOSlgNrgN8A5gPfkbQsIn5WbtnWLWV0AJf6peBOH99dQOfaORfh9yUtHjHv35smnwHuStdXAdsi4izwhqRjwPXAf5ZSrXVdGcMBH+TrLWUcGPwD4F/T9QXAiabbhtK895G0TtKgpMFznC2hDDPrRKEDg5IeAs4DTwzParFYtLpvRGwCNgHM0tyWy1jv6vYpwdz+l6fjEJA0ANwBrEynJIfGnn9R02ILgZOdl2dmVetoOCDpVuBB4M6I+EnTTbuANZL6JC0BlgL/VbxMswvcBZRrzE5A0lbgJuAKSUPAwzTeDegD9kgCeCYi/jAiDknaDrxMY5hwv98ZsLL4xV+Ndt4duKfF7McusfwXgC8UKcrM6uNPDFphVX6K0Hv/6vm7A2aZcydgpWl3r13Gz4tZedwJmGXOnYDVzh1Ab3EnYJY5h4BZ5nThE79dLEI6A/wYeLvbtQBX4DqauY6LTeQ6fiUiPjRyZk+EAICkwYjodx2uw3XUW4eHA2aZcwiYZa6XQmBTtwtIXMfFXMfFJl0dPXNMwMy6o5c6ATPrAoeAWeZ6IgQk3ZrOU3BM0vqa1rlI0j5JhyUdkvRAmj9X0h5JR9PlnJrqmSLpBUlPpeklkvanOr4laXoNNcyWtCOdU+KwpBu6sT0kfTb9TQ5K2ippRl3bY5TzbLTcBmr4anrevijpuorrqOZ8HxHR1X/AFOA14GpgOvDfwPIa1jsPuC5d/2XgCLAc+CtgfZq/HthY03b4HPCPwFNpejuwJl3/GvBHNdSwBbgvXZ8OzK57e9D4deo3gA80bYffr2t7AB8DrgMONs1ruQ2A22n80raAFcD+iuv4HWBqur6xqY7l6XXTByxJr6cpba+r6idWG//ZG4DdTdMbaJzYpO46dgKfAF4F5qV584BXa1j3QmAvcDPwVHpSvd30B79oG1VUw6z04tOI+bVuDy78bP1cGl9wewq4pc7tASwe8eJruQ2AvwfuabVcFXWMuO33gCfS9YteM8Bu4IZ219MLw4G2z1VQlXRylWuB/cBVEXEKIF1eWUMJjwCfB36epi8H3o2I82m6jm1yNXAG+HoalmyWNJOat0dEvAl8CTgOnALeA56j/u3RbLRt0M3nbkfn+2ilF0Kg7XMVVLJy6YPAt4HPRMQP61pv0/rvAE5HxHPNs1ssWvU2mUqj/Xw0Iq6l8V2OWo7PNEvj7VU02tr5wEzgthaL9sJ721157hY530crvRACXTtXgaRpNALgiYh4Ms1+S9K8dPs84HTFZdwI3Cnpf4BtNIYEjwCzJQ3/3kMd22QIGIqI/Wl6B41QqHt7fBx4IyLORMQ54Engo9S/PZqNtg1qf+42ne/j3ki9f9E6eiEEngWWpqO/02mc0HRX1StV47fSHwMOR8SXm27aBQyk6wM0jhVUJiI2RMTCiFhM4//+3Yi4F9jHhXM81lHHD4ATkj6cZq2k8dPxtW4PGsOAFZIuS3+j4Tpq3R4jjLYNdgGfSu8SrADeGx42VKGy831UeZBnHAdAbqdxdP414KGa1vnbNFqmF4ED6d/tNMbje4Gj6XJujdvhJi68O3B1+kMeA/4J6Kth/dcAg2mb/AswpxvbA/gL4BXgIPAPNI5617I9gK00jkWco7GHXTvaNqDRhv9det6+BPRXXMcxGmP/4efr15qWfyjV8Spw23jW5Y8Nm2WuF4YDZtZFDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMvf/gBt0F9GgVmYAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "size_cutoff = 50\n", - "sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", - "\n", - "small_size_filter = sizes < size_cutoff\n", - "pixel_to_remove = small_size_filter[label_im]\n", - "\n", - "label_im[pixel_to_remove] = 0\n", - "plt.imshow(label_im)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now there are only two blobs left." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next let's separate out each mask." - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "metadata": {}, - "outputs": [], - "source": [ - "rois = []\n", - "for i in np.unique(label_im)[1:]: # 0 is the background\n", - " rois.append(label_im == i)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "There are two special notes about the analyses we performed above.\n", - "1. The segmentation results are dependent on the parameters `threshold` and `size_cutoff`. Rather than fixing the value of the threshold, we might want to try different values and see what works well.\n", - "2. For the segemtation analyses of each `AverageFrame`, the result is a number of ROIs instead of one. We need to perform the analyses on the level of each `AverageFrame`, but save the result on the granularity of each `Roi`.\n", - "\n", - "Next we would like to introduce two DataJoint table tiers to help handling these two needs." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Parameter `Lookup` table" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We would like to perform the segmentation for a **combination** of `AverageFrame`s and different set of paremeters of `threshold` and `size_cutoff` values. To do this while still taking advantage of the `make` and `populate` logic, you would want to define a table to house parameters for segmentation in a `Lookup` table!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's define `Param` table to hold different parameter configuration for our spike detection algorithm. We are going to define this table as a `Lookup` table, rather than a `Manual` table. By now, you know that `Lookup` must be yet another **table tier** in DataJoint. `Lookup` tables are depicted by gray boxes in the Diagram.\n", - "\n", - "This tier indicates that the table will contain information:\n", - "* that will be referenced by other tables\n", - "* that doesn't change much - usually contains a few pre-known entries" - ] - }, - { - "cell_type": "code", - "execution_count": 34, - "metadata": {}, - "outputs": [], - "source": [ - "@schema\n", - "class SegmentationParam(dj.Lookup):\n", - " definition = \"\"\"\n", - " seg_param_id : int # unique id for cell segmentation parameter set\n", - " ---\n", - " threshold : float\n", - " size_cutoff : float\n", - " \"\"\"" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": "\n\n%3\n\n\n\nMouse\n\n\nMouse\n\n\n\n\n\nSession\n\n\nSession\n\n\n\n\n\nMouse->Session\n\n\n\n\nScan\n\n\nScan\n\n\n\n\n\nSession->Scan\n\n\n\n\nSegmentationParam\n\n\nSegmentationParam\n\n\n\n\n\nAverageFrame\n\n\nAverageFrame\n\n\n\n\n\nScan->AverageFrame\n\n\n\n", - "text/plain": [ - "" - ] - }, - "execution_count": 25, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dj.Diagram(schema)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So far the `SegmentationParam` is an extra table that did not relate with any of the existing, but the `Segmentation` will depend on this table." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Master `Computed` table `Segmentation` and `Part` table for `Roi`" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As mentioned above, we performed the segmentation processing on each `AverageFrame`, but the product is the masks of each ROI. In this case, we could create a `Computed` table `Segmentation` containing the `make` method to drive the processing, while using a `Part` table to save results into `Roi`.\n", - "\n", - "`Computed` table and `Part` table are another two table tiers like `Manual`, `Lookup`, and `Imported` we introduced previously. \n", - "\n", - "`Computed` table is very similar to the `Imported` table, which also supports the definition of `make` function and `populate`. The only difference is that the computation in an `Imported` table is dependent on external data, while the computation in a `Computed` table only depends on data inside the database.\n", - "\n", - "Contents in a `Part` table is dependent on its **master** table and the master table could be any type of table. This current example is a very typical usage of Part table. The master `Computed` serves as the driver for computation and the major results with a smaller granularity are saved in the `Part` table." - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": {}, - "outputs": [], - "source": [ - "@schema\n", - "class Segmentation(dj.Computed):\n", - " definition = \"\"\"\n", - " -> AverageFrame\n", - " -> SegmentationParam\n", - " ---\n", - " segmented_masks : longblob # overview of segmented masks\n", - " \"\"\"\n", - " class Roi(dj.Part):\n", - " definition = \"\"\"\n", - " -> master\n", - " roi_idx : int # index of an roi\n", - " ---\n", - " mask : longblob # mask of this roi\n", - " \"\"\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Notice that in the definition of `Roi`, apart from inheriting the primary from its master table `Segmentation`, the Roi has another primary key attribute `roi_idx`. The relationship between `Segmentation` and `Roi` is **one-to-many**." - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": "\n\n%3\n\n\n\nMouse\n\n\nMouse\n\n\n\n\n\nSession\n\n\nSession\n\n\n\n\n\nMouse->Session\n\n\n\n\nScan\n\n\nScan\n\n\n\n\n\nSession->Scan\n\n\n\n\nSegmentation\n\n\nSegmentation\n\n\n\n\n\nSegmentation.Roi\n\n\nSegmentation.Roi\n\n\n\n\n\nSegmentation->Segmentation.Roi\n\n\n\n\nSegmentationParam\n\n\nSegmentationParam\n\n\n\n\n\nSegmentationParam->Segmentation\n\n\n\n\nAverageFrame\n\n\nAverageFrame\n\n\n\n\n\nAverageFrame->Segmentation\n\n\n\n\nScan->AverageFrame\n\n\n\n", - "text/plain": [ - "" - ] - }, - "execution_count": 27, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dj.Diagram(schema)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `Computed` table is labeled as a pink oval and the `Part` table is bare text. We see that `Segmentation` is a `Computed` table that depends on **both AverageFrame and SegmentationParam**. Finally, let's go ahead and implement the `make` method for the `Segmenation` table. " - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": {}, - "outputs": [], - "source": [ - "@schema\n", - "class Segmentation(dj.Computed):\n", - " definition = \"\"\"\n", - " -> AverageFrame\n", - " -> SegmentationParam\n", - " ---\n", - " segmented_masks : longblob # overview of segmented masks\n", - " \"\"\"\n", - " class Roi(dj.Part):\n", - " definition = \"\"\"\n", - " -> master\n", - " roi_idx : int # index of an roi\n", - " ---\n", - " mask : longblob # mask of this roi\n", - " \"\"\"\n", - " \n", - " def make(self, key): # key is one of the primary keys of the join product of AverageFrame and ParameterSet\n", - " \n", - " print('Populating for: ', key)\n", - " \n", - " # fetch average image from the previous table AverageFrame\n", - " avg_image = (AverageFrame & key).fetch1('average_frame')\n", - " \n", - " # fetch the parameters threshold and size_cutoff\n", - " threshold, size_cutoff = (SegmentationParam & key).fetch1(\n", - " 'threshold', 'size_cutoff')\n", - " \n", - " # perform the thresholding and blob detection\n", - " mask = avg_image > threshold\n", - " label_im, nb_labels = ndimage.label(mask)\n", - " sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", - "\n", - " small_size_filter = sizes < size_cutoff\n", - " pixel_to_remove = small_size_filter[label_im]\n", - "\n", - " label_im[pixel_to_remove] = 0\n", - " \n", - " rois = []\n", - " for i in np.unique(label_im)[1:]: # 0 is the background\n", - " rois.append(\n", - " dict(**key, # inherit primary key from master table\n", - " roi_idx=i, \n", - " mask=label_im==i))\n", - " \n", - " # insert into the master table first\n", - " self.insert1(\n", - " dict(**key, segmented_masks=label_im)\n", - " )\n", - " print('Detected {} ROIs!\\n'.format(len(rois)))\n", - " # then insert into the part table\n", - " self.Roi.insert(rois)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The implementation of the segmentation is pretty much what we had above, except that we now fetch the value of `threshold` and `size_cutoff` from the `SegmentationParam` table.\n", - "\n", - "**Important note: always insert into the master table first and then insert the corresponding entries in the part table.** If a master table entry does not exist, its corresponding entries would not be valid." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Looking at the `Segmentation` table, we see that it indeed inherits the primary key attributes from **both AverageFrame (`mouse_id`, `session_date`, `scan_idx`) and SegmentationParam (`seg_param_id`)**." - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 29, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And for the part table `Segmenation.Roi`, there was an additional primary key attribute `roi_idx`:`" - ] - }, - { - "cell_type": "code", - "execution_count": 30, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

roi_idx

\n", - " index of an roi\n", - "
\n", - "

mask

\n", - " mask of this roi\n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id *roi_idx mask \n", - "+----------+ +------------+ +----------+ +------------+ +---------+ +--------+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation.Roi()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Populating `Segmentation` table" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We are now ready to populate! When we call `populate` on `Segmentation`, DataJoint will automatically call `make` on **every valid combination of the parent tables - AverageFrame and SegmentationParam**." - ] - }, - { - "cell_type": "code", - "execution_count": 31, - "metadata": {}, - "outputs": [], - "source": [ - "# ENTER YOUR CODE! - populate the Segmentation table\n", - "Segmentation.populate()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Hm... `populate` doesn't seem to be doing anything... What could be the cause?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Looking at `SegmentationParam` reveals the issue:" - ] - }, - { - "cell_type": "code", - "execution_count": 32, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 32, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "That's right! We have not added a parameter set yet. Let's go ahead and add one." - ] - }, - { - "cell_type": "code", - "execution_count": 33, - "metadata": {}, - "outputs": [], - "source": [ - "SegmentationParam.insert1((0, 50, 50))" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
050.050.0
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "0 50.0 50.0 \n", - " (Total: 1)" - ] - }, - "execution_count": 35, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we should really be ready to perform the computation..." - ] - }, - { - "cell_type": "code", - "execution_count": 36, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 0}\n", - "Detected 6 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 0}\n", - "Detected 6 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 0}\n", - "Detected 9 ROIs!\n", - "\n" - ] - } - ], - "source": [ - "# ENTER YOUR CODE! - populate the Segmenation table for real!\n", - "Segmentation.populate()" - ] - }, - { - "cell_type": "code", - "execution_count": 37, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "0 2017-05-15 1 0 =BLOB= \n", - "0 2017-05-15 2 0 =BLOB= \n", - "100 2017-05-25 1 0 =BLOB= \n", - " (Total: 3)" - ] - }, - "execution_count": 37, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "...and we now have spike detection running!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Trying out other parameter values" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's see how different thresholds affect the results." - ] - }, - { - "cell_type": "code", - "execution_count": 38, - "metadata": {}, - "outputs": [], - "source": [ - "SegmentationParam.insert1((1, 60, 50)) # add another threshold and size cutoff" - ] - }, - { - "cell_type": "code", - "execution_count": 39, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
050.050.0
160.050.0
\n", - " \n", - "

Total: 2

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "0 50.0 50.0 \n", - "1 60.0 50.0 \n", - " (Total: 2)" - ] - }, - "execution_count": 39, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam()" - ] - }, - { - "cell_type": "code", - "execution_count": 40, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", - "Detected 3 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", - "Detected 2 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n", - "Detected 3 ROIs!\n", - "\n" - ] - } - ], - "source": [ - "# ENTER YOUR CODE! - populate the \"missing\" entry in Segmentation table\n", - "Segmentation.populate()" - ] - }, - { - "cell_type": "code", - "execution_count": 43, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", - " \n", - "

Total: 6

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "0 2017-05-15 1 0 =BLOB= \n", - "0 2017-05-15 2 0 =BLOB= \n", - "100 2017-05-25 1 0 =BLOB= \n", - "0 2017-05-15 1 1 =BLOB= \n", - "0 2017-05-15 2 1 =BLOB= \n", - "100 2017-05-25 1 1 =BLOB= \n", - " (Total: 6)" - ] - }, - "execution_count": 43, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can see that the results of segmentation under different parameter settings can live happily next to each other, without any confusion as to what is what." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Deleting entries from \"upstream\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now let's say that we decided that we don't like the first threshold of `50`. While there is really nothing wrong keeping those results around, you might decide that you'd rather delete all computations performed with that threshold to keep your tables clean." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "While you can restrict `Segmentation` table to the specific parameter id (i.e. `seg_param_id = 0`) and delete the entries:" - ] - }, - { - "cell_type": "code", - "execution_count": 42, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "About to delete:\n", - "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", - "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", - "Proceed? [yes, No]: No\n", - "Cancelled deletes.\n" - ] - } - ], - "source": [ - "# Select 'No' when it pops up\n", - "(Segmentation & 'seg_param_id = 0').delete()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We can simply delete the unwanted paramter from the `SegmentationParam` table, and let DataJoint cascade the deletion:" - ] - }, - { - "cell_type": "code", - "execution_count": 44, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
050.050.0
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "0 50.0 50.0 \n", - " (Total: 1)" - ] - }, - "execution_count": 44, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam & 'seg_param_id = 0'" - ] - }, - { - "cell_type": "code", - "execution_count": 45, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "About to delete:\n", - "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", - "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", - "`shan_tutorial_pipeline`.`#segmentation_param`: 1 items\n", - "Proceed? [yes, No]: yes\n", - "Committed.\n" - ] - } - ], - "source": [ - "(SegmentationParam() & 'seg_param_id = 0').delete()" - ] - }, - { - "cell_type": "code", - "execution_count": 46, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "0 2017-05-15 1 1 =BLOB= \n", - "0 2017-05-15 2 1 =BLOB= \n", - "100 2017-05-25 1 1 =BLOB= \n", - " (Total: 3)" - ] - }, - "execution_count": 46, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Visualize ROIs" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we have all ROI masks saved in the table `Segmentation.Roi`, let's quickly look at an example by fetching the `mask` from the table." - ] - }, - { - "cell_type": "code", - "execution_count": 51, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 51, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPGklEQVR4nO3df+xddX3H8edrLS2CI1AVUloyStK5sSUT0iDoYozVgcwISzSBmNk5lmaL2/yxRMr8g+wPE9mMOrNF14haF0QZstEQN4YVY5bMzi/KEKjYCht8pVLMBI0mrJ3v/XFP5VK/pd/ec8/9fuHzfCTNvedzz73n3c/3e1/nc8493/tJVSGpXb+w1AVIWlqGgNQ4Q0BqnCEgNc4QkBpnCEiNGywEklyS5IEk+5JsG2o7kvrJENcJJFkBfBt4HTAPfA24sqrun/rGJPWycqDXvQDYV1UPAiT5LHAZsGAIrMrqOpGTBypFEsCP+MH3q+olR7YPFQLrgEfGlueBl4+vkGQrsBXgRE7i5dk8UCmSAL5YN//3Qu1DnRPIAm3POO6oqu1VtamqNp3A6oHKkHQsQ4XAPHDW2PJ64NGBtiWph6FC4GvAxiQbkqwCrgB2DrQtST0Mck6gqg4l+WPgdmAF8Imqum+IbUnqZ6gTg1TVF4AvDPX6kqbDKwalxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxk0cAknOSnJnkj1J7kvyjq59TZI7kuztbk+bXrmSpq3PSOAQ8GdV9avAhcDbk5wLbAN2VdVGYFe3LGmZmjgEqmp/VX29u/8jYA+wDrgM2NGttgO4vG+RkoYzlXMCSc4GzgN2A2dU1X4YBQVw+lGeszXJXJK5gzw1jTIkTaB3CCR5IfB54J1V9cPFPq+qtlfVpqradAKr+5YhaUK9QiDJCYwC4IaquqVrfizJ2u7xtcCBfiVKGlKfTwcCXA/sqaoPjj20E9jS3d8C3Dp5eZKGtrLHc18J/C7wzSR3d21/DrwfuCnJVcDDwJv7lShpSBOHQFX9G5CjPLx50teVNFteMSg1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1bhqzEq9I8o0kt3XLG5LsTrI3yeeSrOpfpqShTGMk8A5gz9jydcCHqmoj8APgqilsQ9JA+k5Nvh74beDj3XKA1wA3d6vsAC7vsw1Jw+o7Evgw8B7gp93yi4AnqupQtzwPrFvoiUm2JplLMneQp3qWIWlSE4dAkjcAB6rqrvHmBVathZ5fVduralNVbTqB1ZOWIamniacmB14JvDHJpcCJwCmMRganJlnZjQbWA4/2L1PSUCYeCVTVNVW1vqrOBq4AvlRVbwHuBN7UrbYFuLV3lZIGM8R1AlcD706yj9E5gusH2IakKelzOPAzVfVl4Mvd/QeBC6bxupKG5xWDUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuN6hUCSU5PcnORbSfYkuSjJmiR3JNnb3Z42rWIlTV/fkcBfA/9SVb8C/AawB9gG7KqqjcCublnSMjVxCCQ5BXgV3YSjVfW/VfUEcBmwo1ttB3B53yIlDafPSOAc4HHgk0m+keTjSU4Gzqiq/QDd7ekLPTnJ1iRzSeYO8lSPMiT10ScEVgLnAx+tqvOAH3McQ/+q2l5Vm6pq0wms7lGGpD76hMA8MF9Vu7vlmxmFwmNJ1gJ0twf6lShpSBOHQFV9D3gkyUu7ps3A/cBOYEvXtgW4tVeFkga1sufz/wS4Ickq4EHgbYyC5aYkVwEPA2/uuQ1JA+oVAlV1N7BpgYc293ldSbPjFYNS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS43qFQJJ3Jbkvyb1JbkxyYpINSXYn2Zvkc90UZZKWqYlDIMk64E+BTVX168AK4ArgOuBDVbUR+AFw1TQKlTSMvocDK4EXJFkJnATsB17DaJpygB3A5T23IWlAfaYm/y7wAUYzD+8HngTuAp6oqkPdavPAuoWen2Rrkrkkcwd5atIyJPXU53DgNOAyYANwJnAy8PoFVq2Fnl9V26tqU1VtOoHVk5Yhqac+hwOvBR6qqser6iBwC/AK4NTu8ABgPfBozxolDahPCDwMXJjkpCQBNgP3A3cCb+rW2QLc2q9ESUPqc05gN6MTgF8Hvtm91nbgauDdSfYBLwKun0Kdkgay8tirHF1VXQtce0Tzg8AFfV5X0ux4xaDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNa7XdwxKx+v2R+/+2f2Lz3zZElaiwwwBLZnxQDiSATE7Hg5IjXMkoGXp2UYJ4EhhmhwJSI1zJKCZONaefZqv5yjh+BxzJJDkE0kOJLl3rG1NkjuS7O1uT+vak+QjSfYluSfJ+UMWL6m/xRwOfAq45Ii2bcCuqtoI7OqWYTQ1+cbu31bgo9MpU89Vtz9699RHAZquY4ZAVX0F+J8jmi8DdnT3dwCXj7V/uka+ymia8rXTKlbPLUv15jd4js+kJwbPqKr9AN3t6V37OuCRsfXmu7afk2Rrkrkkcwd5asIyJPU17RODWaCtFlqxqrYzmsqcU7JmwXX03LRc9sIL1eFJw5836UjgscPD/O72QNc+D5w1tt564NHJy5M0tElDYCewpbu/Bbh1rP2t3acEFwJPHj5skLQ8HfNwIMmNwKuBFyeZB64F3g/clOQq4GHgzd3qXwAuBfYBPwHeNkDNep55tiH6cjm0eD47ZghU1ZVHeWjzAusW8Pa+RUmaHa8Y1NQc7157MSfpxtdxVDAM/3ZAapwjAfU2qz20o4JhOBKQGudIQL0s1R75yPMJjgwmZwjouPR9s037ir3F1uOVgkfn4YDUOENAi/ZcHXI7Cnh2hoDUOENAz2kXn/ky9/Q9GQJS4wwBqXF+RKhj8oTg85sjAalxjgT0vOMI4Pg4EtAxTfMM/BCTkDxXD1eWC0NAapyHA5q5w3vuPqML9/7T40hAapwjAR3TUHvdSUcEjgKmy5GA1DhDQMc09PX5i92z+0nAMAwBLdrQQXC0N7hv/mEZAlLjPDGo43J4NDCrPfPxbMcrBSdzzJFAkk8kOZDk3rG2v0ryrST3JPnHJKeOPXZNkn1JHkhy8VCFS5qOxYwEPgX8DfDpsbY7gGuq6lCS64BrgKuTnAtcAfwacCbwxSS/XFX/N92ytVSmMQJ4tm8KnvT1HQVMbjFzEX4lydlHtP3r2OJXgTd19y8DPltVTwEPJdkHXAD8+1Sq1ZKbxuGAJ/mWl2mcGPx94J+7++uAR8Yem+/afk6SrUnmkswd5KkplCFpEr1ODCZ5L3AIuOFw0wKr1ULPrartwHaAU7JmwXW0fC31lGAO/6dn4hBIsgV4A7C5m5IcRnv+s8ZWWw88Onl5koY20eFAkkuAq4E3VtVPxh7aCVyRZHWSDcBG4D/6lyk9zVHAdB1zJJDkRuDVwIuTzAPXMvo0YDVwRxKAr1bVH1bVfUluAu5ndJjwdj8Z0LT45h/GYj4duHKB5uufZf33Ae/rU5Sk2fGKQfU25FWE7v2H598OSI1zJKCpWexeexpfL6bpcSQgNc6RgGbOEcDy4khAapwhIDUuT1/xu4RFJI8DPwa+v9S1AC/GOsZZxzM9l+v4pap6yZGNyyIEAJLMVdUm67AO65htHR4OSI0zBKTGLacQ2L7UBXSs45ms45med3Usm3MCkpbGchoJSFoChoDUuGURAkku6eYp2Jdk24y2eVaSO5PsSXJfknd07WuS3JFkb3d72ozqWZHkG0lu65Y3JNnd1fG5JKtmUMOpSW7u5pTYk+SipeiPJO/qfib3JrkxyYmz6o+jzLOxYB9k5CPd7+09Sc4fuI5h5vuoqiX9B6wAvgOcA6wC/hM4dwbbXQuc393/ReDbwLnAXwLbuvZtwHUz6od3A58BbuuWbwKu6O5/DPijGdSwA/iD7v4q4NRZ9wejb6d+CHjBWD/83qz6A3gVcD5w71jbgn0AXMrom7YDXAjsHriO3wJWdvevG6vj3O59sxrY0L2fVix6W0P/Yi3iP3sRcPvY8jWMJjaZdR23Aq8DHgDWdm1rgQdmsO31wC7gNcBt3S/V98d+4M/oo4FqOKV78+WI9pn2B09/bf0aRn/gdhtw8Sz7Azj7iDffgn0A/B1w5ULrDVHHEY/9DnBDd/8Z7xngduCixW5nORwOLHqugqF0k6ucB+wGzqiq/QDd7ekzKOHDwHuAn3bLLwKeqKpD3fIs+uQc4HHgk91hyceTnMyM+6Oqvgt8AHgY2A88CdzF7Ptj3NH6YCl/dyea72MhyyEEFj1XwSAbT14IfB54Z1X9cFbbHdv+G4ADVXXXePMCqw7dJysZDT8/WlXnMfpbjpmcnxnXHW9fxmhYeyZwMvD6BVZdDp9tL8nvbp/5PhayHEJgyeYqSHICowC4oapu6ZofS7K2e3wtcGDgMl4JvDHJfwGfZXRI8GHg1CSHv+9hFn0yD8xX1e5u+WZGoTDr/ngt8FBVPV5VB4FbgFcw+/4Yd7Q+mPnv7th8H2+pbuzft47lEAJfAzZ2Z39XMZrQdOfQG83ou9KvB/ZU1QfHHtoJbOnub2F0rmAwVXVNVa2vqrMZ/d+/VFVvAe7k6TkeZ1HH94BHkry0a9rM6KvjZ9ofjA4DLkxyUvczOlzHTPvjCEfrg53AW7tPCS4Enjx82DCEweb7GPIkz3GcALmU0dn57wDvndE2f5PRkOke4O7u36WMjsd3AXu72zUz7IdX8/SnA+d0P8h9wD8Aq2ew/ZcBc12f/BNw2lL0B/AXwLeAe4G/Z3TWeyb9AdzI6FzEQUZ72KuO1geMhuF/2/3efhPYNHAd+xgd+x/+ff3Y2Prv7ep4AHj98WzLy4alxi2HwwFJS8gQkBpnCEiNMwSkxhkCUuMMAalxhoDUuP8Hh33yejtUZzUAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "# show one example ROI\n", - "masks = (Segmentation.Roi).fetch('mask')\n", - "plt.imshow(masks[2])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Fluorescence trace of each segmented ROI" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we got masks of ROIs in the table `Segmetation.Roi` obtained with different parameter combinations. We would like to extract the fluorescence trace of each segmentation. \n", - "\n", - "The table design is similar to the `Segmentation` and `Roi`. The master table `Fluorescence` is the driver for the computation, with a secondary attribute `time` shared across traces of all ROIs. The part table `Trace` saves the extracted trace for each ROI over time (frame). " - ] - }, - { - "cell_type": "code", - "execution_count": 57, - "metadata": {}, - "outputs": [], - "source": [ - "from skimage import io\n", - "import os\n", - "@schema\n", - "class Fluorescence(dj.Imported): # imported table because it also rely on the external tiff file.\n", - " definition = \"\"\"\n", - " -> Segmentation\n", - " ---\n", - " time : longblob # time for each frame\n", - " \"\"\"\n", - " \n", - " class Trace(dj.Part):\n", - " definition = \"\"\"\n", - " -> master\n", - " -> Segmentation.Roi\n", - " ---\n", - " trace : longblob # fluorescence trace of each ROI\n", - " \"\"\"\n", - " \n", - " # the master table is mainly to perform the computation, while the part table contains the result\n", - " def make(self, key):\n", - " \n", - " print('Populating: {}'.format(key))\n", - " # fetch data directory from table Session\n", - " data_path = '../01-Calcium_Imaging/' + (Session & key).fetch1('data_path')\n", - " \n", - " # fetch data file name from table Scan\n", - " file_name = (Scan & key).fetch1('file_name')\n", - " \n", - " # load the file\n", - " im = io.imread(os.path.join(data_path, file_name))\n", - " \n", - " # get dimensions of the image and reshape\n", - " n, w, h = np.shape(im)\n", - " im_reshaped = np.reshape(im, [n, w*h])\n", - " \n", - " # get frames per second to compute time\n", - " fps = (Scan & key).fetch1('fps')\n", - " \n", - " # insert into master table first\n", - " self.insert1(dict(**key, time=np.array(range(n))/fps))\n", - " \n", - " \n", - " # extract traces\n", - " roi_keys, masks = (Segmentation.Roi & key).fetch('KEY', 'mask')\n", - " \n", - " traces = []\n", - " for roi_key, mask in zip(roi_keys, masks):\n", - " \n", - " # reshape mask\n", - " mask_reshaped = np.reshape(mask, [w*h])\n", - " trace = np.mean(im_reshaped[:, mask_reshaped], axis=1)\n", - " \n", - " traces.append(dict(**roi_key, trace=trace))\n", - " \n", - " self.Trace.insert(traces)" - ] - }, - { - "cell_type": "code", - "execution_count": 58, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", - "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", - "Populating: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n" - ] - } - ], - "source": [ - "# ENTER YOUR CODE! - populate the Fluorescence table\n", - "Fluorescence.populate()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we could plot the traces of an example scan" - ] - }, - { - "cell_type": "code", - "execution_count": 61, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Text(0, 0.5, 'Fluorescence')" - ] - }, - "execution_count": 61, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEJCAYAAAB7UTvrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOydd3zU9f34n++7JJe9B2QPVgaEvYcyFCmCAxUp7lGrrdZWu6zV/r611tbWFkcrtiourOJCcCFDRPYmCSNkkr33zr1/f3xyIeOSXJK7DHw/H497JPcZ78/rIPd5fV5bSClRKBQKhQJAN9gCKBQKhWLooJSCQqFQKFpRSkGhUCgUrSiloFAoFIpWlFJQKBQKRStKKSgUCoWiFZspBSHEq0KIAiFEQoftPxVCnBVCJAoh/tJm+2+EEOdb9l1pK7kUCoVC0TV2Nlz7deAF4A3TBiHE5cBKYIKUsl4I4d+yPQZYDcQCgcDXQogxUspmG8qnUCgUig7YTClIKXcLIcI7bP4x8GcpZX3LMQUt21cC77ZsTxNCnAemA/u6u4avr68MD+94CYVCoVB0x5EjR4qklH7m9tnSUjDHGGCeEOIpoA54REp5CAgC9rc5LqtlWyeEEPcC9wKEhoZy+PBh20qsUCgUlxhCiIyu9g10oNkO8AJmAo8C7wkhBCDMHGu2/4aUcr2UcqqUcqqfn1lFp1AoFIo+MtBKIQv4UGocBIyAb8v2kDbHBQM5AyybQqFQfO8ZaKXwMbAQQAgxBnAAioDNwGohhEEIEQGMBg4OsGwKhULxvcdmMQUhxEbgMsBXCJEFPAG8CrzakqbaANwmtTatiUKI94AkoAl4QGUeKRQKxcAjhnPr7KlTp0oVaFYoFIreIYQ4IqWcam6fqmhWKBQKRStKKSgUCoWiFaUUFL2mpK6ED5M/ZDi7HhUKhXkGunhNcQnw7KFn+TT1UwKcA5gTNGewxVEoFFZEWQqKXpFcmsyW1C0AvH367UGWRqFQWBulFBS94oVjL+Bi78KacWv4NvtbMiq6rJZXKBTDEKUUFBZzsvAkOy7s4LbY27hnwj3Y6ezYeGbjYIulUCisiFIKCotZd3Qd3o7e3BJzC75OviwNX8rH5z+murF6sEVTKBRWQikFhUUcyD3AgbwD3D3+blzsXQBYM24N1Y3VfHL+k0GWTqFQWAuVfaSwiK2pW3FzcOPGsTe2bhvvN54JvhN4I+kNUstTSShKILc6lw1LNxDuET54wioUij6jLAVFj0gp2Ze7j5kjZ2LQG9rtuzX2VrKrstmSugWD3kBJXQnf5Xw3SJIqFIr+oiwFRY+kVaSRV53HvRPu7bTvyvArmew/GR8nHwSCRe8v4lTRqUGQUqFQWAOlFBQ9si9Hm4o6a+Qss/v9nC8OOxrvO55ThUopKBTDFeU+UvTIvpx9hLqFEuwW3OOxE/wmkFmZSVld2QBIplAorI1SCopuaWxu5GDeQWYFmrcSOjLBbwKAciEpFMMUpRQU3XKi8AS1TbUWK4VYn1h0QqeUgkIxTFFKQdEte3P2ohd6po+YbtHxzvbORHlGcbLopI0lUygUtkApBUW37M/dz3jf8bg5uFl8zgTfCSQUJajW2grFMEQpBUWXlNeXk1CUwOzA2b06b7zveMrry8mszLSRZAqFwlYopaDokgO5B5BIi+MJJsb7jQe0BnoKhWJ4oZSCokuO5B/Byc6JON+4Xp0X5RGFs52zUgoKxTBEKQVFl6SVpxHpEYmdrnc1jnqdnljfWJWBpFAMQ5RSUHRJekU6Ye5hfTp3vO94zpaepb653spSKRQKW6KUgsIstU215Fbn9rnb6QTfCTQZmzhdfNq6gikUCpuilILCLJkVWuZQhHtEn843xSESixOtJpNCobA9NlMKQohXhRAFQogEM/seEUJIIYRvy3shhFgnhDgvhDgphJhsK7kUlpFekQ7QZ0shwCUAPyc/Eoo6/fcrFIohjC0thdeBpR03CiFCgCVA2yT2q4DRLa97gX/ZUC6FBaSXpwMQ6hba5zVifWOVpaBQDDNsphSklLuBEjO7ngN+CbQtd10JvCE19gOeQoiRtpJN0TPpFemMcBmBs71zn9eI9YklvTydqoYqK0qmUChsyYDGFIQQK4BsKeWJDruCgAtt3me1bDO3xr1CiMNCiMOFhYU2klSRXp5OuHt4v9aI841DIkkqTrKOUAqFwuYMmFIQQjgDjwG/N7fbzDazjXOklOullFOllFP9/PzMHaLoJ1JK0iv6rxRifWIBFWxWKIYTAzl5LQqIAE4IIQCCgaNCiOlolkFIm2ODgZwBlE3RhuK6Yqoaq/ocZDbh5ehFkGuQCjYrFMOIAbMUpJSnpJT+UspwKWU4miKYLKXMAzYDt7ZkIc0EyqWUuQMlm6I9aeVpAP22FABifGKUpaBQDCNsmZK6EdgHjBVCZAkh7urm8M+AVOA88Apwv63kUvRMRkUG0Pd01LbE+caRXZVNaV1pv9dSKBS2x2buIynlzT3sD2/zuwQesJUsit6RXp6OQW9gpEv/E8DifLQitqTiJOYEzen3egqFwraoimZFJ9Ir0gl1D0Un+v/nEe0TDaDiCgrFMEEpBUUnrJF5ZMLNwY1w93ASipVSUCiGA0opKNrR2NxIVmWW1ZQCaJXNSUWqVkGhGA4opaBox4WqCzTLZiI8+tYIzxxxPnEU1BZQUFNgtTUVCoVtUEpB0Q5Tz6O+zlEwh6ljqoorKBRDH6UUFO3ob3dUc4zxGgNAcmmy1dZUKBS2QSkFRTsyKjLwdvTG3cHdams62zsz0mUkaRVpVltToVDYBqUUFO3IrMi0quvIRIRHBKllqVZfV6FQWBelFBTtyKzMJMQtpOcDe0mkRyTpFekYpdHqaysUCuuhlIKildqmWgpqCvo1WKcrIjwiqG2qJb863+prKxQK66GUgqKVrMosAELdbaMUAFLLlQtJoRjKKKWgaCWzUpuQagtLIdIjErjYgVWhUAxNlFJQtHKhQht+F+wWbPW1TRlNylJQKIY2SikoWsmszMTT4ImHwcPqawshiPSI7KQU1h1dx4HcA1a/nkKh6BtKKShayazMtInryESkZ2Q791FWZRavnHqF5448Z7NrKhSK3qGUgqKVCxUXCHG3fjqqiQj3CErqSiivLwdgd9ZuQJvhnFSsGuYpFEMBpRQUADQ0N5BbnWuTGgUTkZ7tg827s3Yz0mUkBr2BD859YLPrKhQKy1FKQQFAdlU2EmlT91HbtNSaxhoO5h1kSdgSrgy/kq1pW6lprLHZtRUKhWUopaAA4EKllnlkS0sh0CUQB50DqWWp7MvdR6OxkQXBC1g1ZhXVjdV8kf6Fza6tUCgsw2YzmhXDi8yKlhoFGxSumdDr9IR7hJNWkUZlYyWu9q5MCpiEnbBjlOcoNp3bxHWjr7PZ9RUKRc8oS0EBaJlHrvaueBm8bHqdSI9IUspS+DbrW+YEzcFeZ48QglVjVnGq6BRnSs7Y9PoKhaJ7lFJQABcb4QkhbHqdCI8IsquyKawtZEHwgtbtyyOXY9Ab+Cj5I5teX6FQdI9SCgpAS0e1pevIhKndhUAwN2hu63YPgwezAmfxTdY3SCltLodCoTCPUgoKGo2N5FTl2DTzyIQpAyneLx4vx/auqnlB88iuylbDeBSKQUQpBQV5VXk0ySabZh6ZCPcIx93BnSvDr+y0z2Q57MnaY3M5BpMDqcU88M5RGpvVbAnF0MNmSkEI8aoQokAIkdBm21+FEGeEECeFEB8JITzb7PuNEOK8EOKsEKLzHUNhM1q7ow6A+8igN/Dl9V+yJnpNp32BroFEeUTxbfa3NpdjMHnrQCZbT+by2ancwRZFoeiELS2F14GlHbZtA+KklBOAc8BvAIQQMcBqILblnJeEEHobyqZogy1bZpvD1cEVnTD/pzcveB5H8o9csoVsRqNkT3IhAK99lz64wigUZrCZUpBS7gZKOmz7SkrZ1PJ2P2Dq0bwSeFdKWS+lTAPOA9NtJZuiPZkVmTjZOeHr5DvYojA3aC6NxsZLtnNqQk45pTWNTA3z4viFMo5mlg62SApFOwYzpnAn8HnL70HAhTb7slq2dUIIca8Q4rAQ4nBhYaGNRfx+kFWVRZBrkM3TUS1hsv9knO2c2ZN9acYVdp/T/mb/fuNE3BztlLWgGHIMilIQQjwGNAFvmzaZOcxsXqKUcr2UcqqUcqqfn5+tRPxeUVRTRIBzwGCLAYC93p6ZI2eyJ3vPJZmauju5iNhAd0J9nLlpagifn8olt7x2sMVSKFoZcKUghLgNWA78UF781mcBbVNfgoGcgZbt+0pRXRE+Tj6DLUYrc4PnklOdc8lNaausa+RoRinzx2gPM7fNDscoJW/uyxhkyRSKiwyoUhBCLAV+BayQUraNJG4GVgshDEKICGA0cHAgZfu+IqWkqLZoSMQTTMwLmgdwybmQ9qUU02SUzB+tKYUQb2eWxATwzsFM6hqbB1k6hULDlimpG4F9wFghRJYQ4i7gBcAN2CaEOC6E+DeAlDIReA9IAr4AHpBSqm/JAFBeX06TsWlIKYURLiMY7TWarzO+HmxRrMru5EKcHfRMCbtYtLdmRhhlNY3sTSkaRMkUiovYMvvoZinlSCmlvZQyWEr5XynlKClliJRyYsvrvjbHPyWljJJSjpVSft7d2grrUVSr3YyGklIAWBm1kuOFxzlfen6wRbEa3yYXMSvSBwe7i1+7mZHeuDjo2ZZUMIiSKRQXURXN33OK6oamUrg66mrsdHZ8kHxpTGTLKK4mo7imNZ5gwmCnZ/4YP3acyb8kA+uK4YdSCt9zhqql4O3ozeLQxWxO2Ux9c/1gi9Nvdidr/84dlQLA4ugA8ivqSciuGGixFIpOKKXwPae4thgYekoB4Pox11PRUMG2jG2DLUq/ScqpwNvFgXAf5077Lh/nj07AttP5gyCZQtEepRS+5xTVFmHQG3C1dx1sUToxfcR0gl2D+eDc8HchZZXWEOLlZLZA0NvFgSlhXmxXSkExBLBIKQiNtUKI37e8DxVCqDYUlwCmdNShUM3cEZ3Qcf2Y6zmcf5i08uHdTju7tJZgr85WgolF0QEk5lSoQjbFoGOppfASMAu4ueV9JfCiTSRSDChFtUOrcK0j14y6Bjthx/vn3h9sUfqM0SjJKqsl2Mupy2MWR/sD8PVplYWkGFwsVQozpJQPAHUAUspSwMFmUikGjKLaInwdh148wYSvky9LI5byzul32J21e7DF6RNF1fU0NBm7VQpRfq6E+zgrF5Ji0LFUKTS2tLKWAEIIP0BNCLkEKK4tHpJB5rY8PvNxxnqP5ZFvHuFU4anBFqfXZJVqLqGgbpSCEIJF0QHsPV9MdX1Tl8cpFLbGUqWwDvgI8BdCPAXsAf5kM6kUA0KjsZHS+tIhrxSc7Z15cdGLeDt688D2B8isyBxskXqFSSl0F1MAWDTOn4ZmI/tTiwdCLIXCLBYpBSnl28AvgaeBXOAaKeXwdfIqACip1cZdDOWYgglfJ19eXvIyAD/f9fNhVeiVVaq1+Qry7NpSAJgS7oWjvY5vk1XLC8XgYWn20UwgW0r5opTyBSBLCDHDtqIpbM1QrWbuijD3MH425WecLT3L8cLjgy2OxWSV1uLt4oCLwa7b4wx2eqZH+PDdeaUUFIOHpe6jfwFVbd5Xt2xTDGOGcuFaVywNX4qLvQubzm0abFEsJru0tkcrwcTcUT4kF1SRV15nY6kUCvNYqhREm9kHSCmNQPePPYohz1BtcdEdzvbOLItYxlfpX1HRMDzaQmSV1nSbedSWOaO0/wtlLSgGC0uVQqoQ4kEhhH3L6yHg0pqA8j3EpBSGQ0yhLavGrKKuuY6tqVsHW5QekVKSVdp9jUJboke44+PiwB6lFBSDhKVK4T5gNpCNNiVtBnCvrYRSDAxFtUW4O7hj0BsGW5ReEeMTQ7R3NJvObbJ6wNlolDy1NYmE7HKrrFdU1UB9k7HHzCMTOp1g9ihf9pwvGlbBdMWlg6XZRwVSytVSSn8pZYCUco2UUpVeDnOG2sS13rBqzCrOlZ4joSjBquvmVtTxyrdp/OjNI5TXNvZ7PUszj9oyb5QvhZX1nMuv6vlghcLKWJp95CeE+K0QYr0Q4lXTy9bCKWzLcFYKyyKW4WTnZPV5CzllWk1Bdlktv/nwZL+f1rNb1gv2tlwpzBmt/Z8oF5JiMLDUffQJ4AF8DWxt81JYifyKOjYezORkVhnNxoFxGwz1vkfd4ergylURV7E1dWtrFpU1MCmFVVOC+exUHu8eutCv9VqrmXthKQR5OhHp68Ke5MJ+XVuh6AuWZhA5Syl/ZVNJvsfklNVy0/p9XCjRbiBujnYsGOPH326Mx2Cnt9l1h7OlAHB77O18lPwRbyS9wcNTHrbKmqab+JMrYsmvqOMPnyYyNcyL0QFufVyvBk9ne9wc7Xt13pxRvnxwNIuGJmO78Z0Kha2x9K9tixBimU0l+Z6SV17Hza/sp6y6kTfunM66mycxf7QfW07mcji91GbXrWmsobapdlgrhQiPCJZGLGXjmY3sSE7liU8SMPbTysopq8XL2R5Xgx1/uzEee52Ol3dbnmj31v4M/rvnYpvvrF7UKLRl7mhfahqaOZRe0utzFYr+YKlSeAhNMdQJISqEEJVCiOGRJD6EKajUFEJxVQMb7prO/DF+rIgP5M/Xj0cn4GCa7W4Iw7FGwRz3jr+XuqY6/rb/P2zYl8HX/ewymlNWS2DLTdzfzZGF0f7sOFNgkUvvxZ3n+d3HCfzps9OtAebsXqSjtmX+aD/cDHZ8cDSr1+cqFP3B0uwjNymlTkrpKKV0b3nvbmvhLnX++sVZcstr2XDnNCaHerVud3O0J3qku02fEluVwhBum20Jo7xGsSRsCZlNX4Guhpd2pfQrOJxTVteqFACuiBlBSXUDRzK6t9pe/iaFv355lsXRAQjgte/S29QoWJaO2hYnBz3L4wP5/FQeVaprqmIA6e3ktcdb3oeoyWv9o7ymkU9P5nDd5GCmhHl32j8t3JtjmWU0NtumQ/lwLVwzx1XBt4CunrCIwxy/UMa+fnQZzSlr7+5ZMNYPB72ObUl5XZ7z1v4Mnv78DMsnjOTfayfzgwkjefdgJunFNdQ2NvfJUgC4YWowtY3NbD2Z06fzFYq+0NvJa2ta3lehJq/1iw+PZVHXaGTN9FCz+6dHeFPb2Gy1IqqOXCruI4DsAk8aK2Kpc96Nr5vgpZ0pfVqnoq6RyvomAj0dW7e5GuyYPcqHr5Lyu7RAXt2TxpQwL567aSJ2eh33zIukuqGZZ788C/TcMrsrJoV4EuXnwvuHlQtJMXCoyWuDgJSStw9kEh/iSVyQh9ljpoVr1oOt4gpFtUXohR5Pg2fXBzU1QPXQz5Xfl1KMR+MCapqquHxSMXvOF3Eyq6zX65jSUQM7BIaXxASQUVxDckHnYrLq+ibSiquZP9oPe732dYoL8mBWpA9bT+UCvUtHbYsQghumhnA4o5TUQlXIphgYbDZ5raXArUAIkdBmm7cQYpsQIrnlp1fLdiGEWCeEOC+EOCmEmNzHzzMsOJReyvmCKn44w7yVAODnZiDC18VmcYXiumK8Hb3R67pJed35FPxjAuQcs4kM1sBolOxPK2Zu8Ex8HH2oMxzG3dGuT9ZCl0ohOgCAbUmdg9incyuQEuKC2ofY7pkf0fp7dxPXeuK6SUHodYJNR5S1oBgYbDl57XVgaYdtvwa2SylHA9tb3gNcBYxued3LJd6W++0DGbg52nH1hMBuj5sW7sWh9NJ+p1maw6IaheSvoLEa3rkJyjpMOzPnSsk5DpsfhJqBS6M8k1dJWU0js6P8uDL8Svbm7GH1DD++SMwjp6yW/bn7ufqjq8mv7jkrKbtMa1fd8cne392RiSGefJXYOa5gcu/FBra3+C4b488of1fcHe3wcOpdjULHay8Y48eHR7MHrKhR8f3GZpPXpJS7gY53h5XAhpbfNwDXtNn+htTYD3gKIUZa9hGGF8VV9Xx+Ko/rJwfj5NB9Ydr0CB/KaxvNui36S251Lv7O/l0fUFUABUkQvwYa6+DtG7Sb/ZnP4NWl8Ed/2PoIVOSC0Qh7n4f/LIajG+DERqvL2xWmoPKsKB+uiriKBmMD/iOSAfgiIZtnDj5DekU6bya92eNaOWW12OsFfq6dGwQuiQngRFZ5pzkHiTkV+Lg4EODe/hydTvDsDfH88drxff1ordwwJZi8ijq+OafajSlsz0BPXguQUuYCtPw03ZWCgLb9BLJatpmT5V4hxGEhxOHCwuHXBuDDo9k0NBtZ043ryMR0U1zByi6kZmMzGeUZRHhEdH1Q2u4WIe6G1W9BcQo8Fwvv3gzl2RB9NRx5DdZNhPUL4KvfwZgrwW8cJH3SvQBVBZoisQL7UooJ93Em0NOJeL94glyDOFy0k9H+rvzvzCecLztPsGsw7597n/L67oP2OWW1jPBwRKcTnfZdGdviQupQB5GQU0FMoDtCdD5nYognK+K7twYtYVF0AP5uBjbszej3WgpFTwyVyWudv1Et8YtOG6VcL6WcKqWc6ufnZ0URBoZ9qcWM9ndljAVtE0K8nQhwN3DIysHm7KpsGowNRHpEdn1Q6i4weMDIiRAxH657GYKmwPX/hQePwapX4SeHIe56KL8Ay5+Dm96CuFVw4QBUdJFGWVMC/5wIW3/e78/RbJQcSCtmVpSWViuEYGn4Uvbn7mdetB1ZfES0VyzPXf4cNU01vHf2vW7XyymrJdDDvP8/ys+VCF+XdnGF+qZmkvMru0wWsBYOdjrWzgzjm3OFnLeB1ahQtGWgJ6/lm9xCLT9N9nAWENLmuGDgkkzOTswpt/gmIoRgWrg3B9NKrNpbP7Vca9vQo6UQPhdMgei46+H2LTB+Fehb/uu9I+Cal+BX6TD1ThACYlZo+05vMb9u0sdanOLIa10fYyFJORVU1jUxM/JircVVEVfRLJs5WPMsOvtyprqvZZz3OOYEzuGt029R19T1mMucsrouM4WEEFwRE8C+lCIq67SW2sn5VTQZJbGBtq/jXDMjFAe9jg17021+LcX3m4GevLYZuK3l99vQuq+att/akoU0Eyg3uZkuJYqq6smvqO/VTWR6hDd5FXWtjdqsgUkpRHp2YSmUpkNZBkQu6P3ifmO7dyGd2gQ+o2BkPGz+qRaTACg4A29eB/tesvhS+1K1dNlZbZTCGK8xRHlEkVF1Hn1dNGfSNbfPnXF3UlJXwuaUzWbXamo2kldR122m0JKYABqbJbvOam7LroLMtsDX1cCKiYF8cDTLKnMeFIqusNnkNSHERmAfMFYIkSWEuAv4M7BECJEMLGl5D/AZmpI5D7wC3N/LzzEsSMzR2kXF9EIpmNpfHLvQ+7z7rkgtS8XXyRd3hy7kSP1G+xnRB6UAELMSMvdqsYO2lGdBxncw4SbNDdVYCx/fB7ufhZfnQcp22P7/LiqKHjiVXUGwlxP+7heLzYQQLI9ajk7oWBhwO3uSi6iqb2LaiGnE+cTxeuLrNBubO61VUFlPs1F2Skdty6RQL3xcHFpdSIk5Fbga7Ajz7ltxWm+5fXY4NQ3NvNfPdt4KRXfYbPKalPJmKeVIKaW9lDJYSvlfKWWxlHKRlHJ0y8+SlmOllPIBKWWUlHK8lPKwNT7cUCMxp+XJcqTlT5ZjR7hhsNNx0opKIa08rft4Qto34DpCe+rvCzErQRrhTAf3UELLQJzxq8B3NCx9Wotd7Pg/GLsM7t4Oxib45s+dljQrZlEVUX6unbbfFnsbm6/ZzOr4mTQ0G9l5pgAhBHfE3cGFygt8l/Ndp3O6qlFoi14nWBwdwM4zBTQ0GUnMKSdmpLvZwLQtiAvyYHq4Nxv2pav0VIXNsDT76C9CCPcW19F2IUSREGKtrYW71EjM0Z5sPZwtz1u31+uIDXTnRB8qdM0hpSS1PLXreIKUWjwhYr4WI+gL/jHgHdXZhXTqfQiaCt4tCmnK7bD4D3DT23DjBgieClPvgKNvQlFyj58jrbCaCF+XTvvsdfaEuYcxJcwLX1cHvmypL7g85HI8DZ5sSekcyzBNSAtq0+LCHEtiAqisb2JvShGncyt7ZfVZgzvmhJNVWssW1Q9JYSMsdR9dIaWsAJajuY/GAI/aTKpLlKScCmJG9v4mMiHYk4TsCpqs0ByvqLaIqsaqri2FgiSoLuxbPMGEEJq1kPYtVLc0pys4A3mnYPwN7Y+b+zOIXn5x2/xHwc4Rdvyx20vkV9RT3dBMlF9npWBCrxMsidGe7Osam7HX23Nl+JXsvLCT6sbq1uOajc1sz9oKuhpGdpF9ZGLuaF+c7PW8/E0qtY3NNs886siSmADigtz5w6dJFFR2HTQfaGobmq2aDKEYPCxVCqZH22XARpPbR2E5VfVNpBVV9ykoGR/iQW1jM+et0P+mxyBzazxhfv8uFHe95kL67xJI2QEJm0DoIPba7s9z9YfZP9GylLKPdHlYapH2bxFpxn3UlitiR1Dd0MzeFC0ovTxyOXXNdWzP3N56zCcpn7Cz+Hncgz/FxdB9Up2jvZ75Y3xbi+YGIvOoLXZ6Hc/dOJHq+iZ+/cGp1htxYWU9P3/vONv7OU+iL6QWVhH35Jcs+OsuntycyHfni5SCGMZYqhQ+FUKcAaYC21t6Hw2dx5RhwJlcLcjcl5tIfLDWtO6EFeIKrUqhK0sh6WPwGQ2ePRfXdcuIOLjlQ0DCm9fCd+u0wLVbQM/nzvoJOPvAa8u0SuqDr3RqnZFaqD3pm3MftWV2lA+uBju+StRulqYCN5MLqa6pjpeOv4SQdkiXYxzNP9qjeEtiRgBa/cAo/+6Vki0YHeDGr68ax44zBWw8eIF9KcUsW/ctHx7N5o9bT9ukLUp37E8todkoCfZyYuPBTH74nwO8czCz5xMVQxJLA82/RmudPVVK2QjUoLWmUFiIKfMoNqj3SiHcxwU3RztOZPW/jXZqWSqu9q74OZkp/Ms+ohWeTbur39cBIGoh/HgfXP4Y6OwsX9fRHW7bosUcipLhs0dg4+p2h6QVVeNkr2eEe/cxAIOdnsvH+bMtKZ9mo9SykyKXcyDvAAU1Bbx75l3ya/LxrroHB7x5+uDTZrOT2rJonIVVRC0AACAASURBVD86AeNGuLV2Rh1obpsVztxRvvzh00R++J/9uBnseGjRaNKKqtl5dmDbYZzMKsPT2Z63757B8d9fwcQQT9bvTlXB8GGKpYFmZ+ABLlYxB6JZDQoLScwpx9vFocebmDl0OkF8sKdVLAVT5pG5tgzs/zc4uMHEH/b7Oq3YO8KCX8JvW1pjWEpADFz1jFY9vfB3mrIqvdjmIbWwinBfF4syf66MDaC4zfS0H0T+AKM08t7Z93jl1CvMCZxDcVEUE51v4UzJGT5I/qDb9bxcHLhzTgQ3Tg3p9jhbotMJ/nrDBLycHbg6PpDNP53LTxeOItDDsd2M6IHg+IUyJgR7IoTAyUHPvfMjySiuMdtVVjH0sfQx5zWgAa1WAbRgc/eRQEU7EnMqiO2iR44lTAj24GxeJXWNXTzFlmVCc89jG1PLUwn3CO+8oyIXEj+CSWu1J3Vr09dMJiG01hnQLsU1taiayG6CzG1ZMEabnmbqchrhEUGcTxzrT66noqGCu+MeoLKuiWl+lzE1YCrPH3u+xz5Jv1sew9qZYX37TFZipIcT+36zkH+unoSrwQ47vY5bZ4ezN6WY07kDM0K9pqGJ5IIqJgZfjJVdERNAsJcT/93Tl/pWxWBjqVKIklL+BWgEkFLWYr5fkcIMDU1GzuX3L31xQrAnTUZJkrkve1kmrJsMm27vttFcZUMlhbWF5uMJh/+r1QjM6LYmcXDwjoAR4yFJq0ZuaDJyoaSGqB7iCSbcHO2ZM8qHL5PyWgOgP4j8ARLJVeFX4dCsPfGH+rjw6+m/prKhkgd3PEhlQ6VtPo8V6fiQcfO0UJzs9bw6QNZCYk4FzUbJhOCLw5rs9DrunBPBofRSjluxvkYxMFiqFBqEEE5cHLITBdTbTKpLjOSCShqbZb/aIUwM6SbYnLQZjI1w+lPY2bUB12WQubEODr8GY6+6WEMw1IheobmQKvPILKnGKCHCQksBtCykCyW1nM7VbvRXR13N8sjl/GzKz0jI1hRtXKAHY73H8uf5f+Zk4Unu+vIuSutKbfJxbIWHsz2rpgTzyfEciqps/xU1/T1OCGn/t33jtBDcHO34z7fKWhhuWKoUngC+AEKEEG+jDcj5pc2kusRoDTL3w1IY4eGIv5uBk+aCzUmfQMB4LTD77d/guPl5BqllXaSjJmyCmiKYcV+f5bM50SsACWe2tGYeRfpanvmzODoAIeCrJM2F5GHw4Ol5TxPoGsip7HLcHe0I8dZqFJaGL+WfC/9Jankqd3xxB4U1w6tF++1zwmloNvLOAdtnAJ3MKifQwxF/t/axMleDHWumh/J5Qh5ZpTU2l0NhPXpUCkKzT88A1wG3AxvRspB22VSyS4jE7HKcHfRE+Fj+ZGuOCcGenSuby7Mh6yDEroRlz2r1BZt/ChcOdTo/rTwNe509Qa4dRlWceBd8x/a/NsGW+I3VUmVPf0pqUUs6ai8sBT83A1PDvPgysXPw09S5tq0rZn7wfP61+F/kVOfw1IGn+i//ABLl58qkUE92n7O9MjuRVdbOddSW22aHY5SSD45k21wOhfXoUSm0tMz+uKVv0VYp5RYp5dCf5j5EKKtp4KNj2cyO8u13j5yJIR6kFla375J5+lPtZ8w1oLeHG98AF1+tn1AHUstTCXMPw07XpkCrsQ6yDsHoJX0PBg8Eprbcad+Sl5uNr6sBd8fejbm8ImYEp3MruFBy8cm1sdnImVzzMxGmjZjGHXF3sD1zO6cKT/X7Iwwkk0O9OJVdTqMVquC7oqymgYziGuJDzCuFQE8nYgPdWwsHFcMDS91H+4UQ02wqySXKS7tSqKxv4hdXjOn3WqYnspNtrYWkT7ReQ76jtfdOXjDjR1pTu/zEdudnVJiZtpZ9BJrqIGxOv+WzOdFXg2zGL2enxZlHbVkcoxXO7WqTx38uv5KGZmOX7SpujbkVb0dv/nnsn32TeZCYGOJJfZOm8GyFqW4mPrjrWNmsSB+OZZZ1nTWnGHJYqhQuB/YJIVKEECeFEKeEECdtKdilQHZZLa/vTefaSUFE96HnUUcmh3lhpxPsTWnpJ1SZD5n7tD5D7Q68DeycYH/74XhFtUUEOHeoKM74DhAQNqvf8tmckRPBI5TJFduJ9Ol9u+pwH2dCvZ35po1bJbE1yGz+/8fF3oW7x9/NgdwD7M/d3ze5B4FJodoDxPELpXye9jl/3P9H/nXiX7x/7n1yq6wzquTkhTItY7gbpTA7ypeGZmNrjYhi6GPp9LSrbCrFJcrfvjoLwC+u6GML6g64GuyYFOrJd+dbzPEznwKys1Jw9ob41XD8HVj8JLj4UttUS1VjFT5OPu2PTd+jtaRw8rKKjDZFCOrib2PW7v+Duo+A+F6eLlgwxo8PjmbR0GTEwU5HQk45rgY7wruJ99w49kbeSHqDdUfXMWPZjD7XmgwkQZ5O+LkZ2J6+j6PnnsbRzpHappb24C6BbL1ua3s3Yh84kVVOpK9Lt268aRHe6HWCvSlFzBnl26/rKQYGS9tcZACewNUtL8+WbYouSMqp4KNj2dwxO7zLEY99Yc4oX05ll1NW06C5jnzHaJPOOjLjPmiu18ZeAsW1mnXh49hGKTQ1wIWDEDbXavLZmtNRd7CleQYzz/+jtW6hN8wf40dNQzOHM7ReSqeyy4kJ7H4mgkFv4P74+zlVdIodmTv6LPtAIoQgLtiBY7X/IsQthF037uLo2qM8M+8Zcqpz+Drj636tL6XkRFZZl/EEE64GO+KDPS5at4ohj6VtLh4C3gb8W15vCSF+akvBhiN/+eIMC5/dxeT/28bVL+zB3dGe+y8bZdVrzB3li5RwOOm89pQfs9J8gNh/HEQtgoP/gaYGiutalEJbSyHnKDTVQnjv4gkHUot55P0Tg+InTi2q5ReNP6Y+YBJ8eG+3nVTNMSvKB3u94JtzhTQ1GzmdW8F4C9pfXx11NeHu4bx04qVh0wG0zPldmnXl/Hba/+Fs74y93p6lEUsJdQtlQ+KGfn2OvIo6CivrW5s1dsfsKF9OZpVTVd9zxb1i8LE0pnAXMENK+Xsp5e+BmcA9thNraFPT0NTpC7XzTAEv7UohwN2Rq+JG8KP5kWy4c3qvBupYQnyIJy4OeopPfqG1ph7TjWdv5v1QlQfbHqcoQ5s25uvUxoRP36P9DJ1t5mTznM2r5O4Nh9l0JIvPEwZ+jHZaUTVNOgP6H76rtdl+dy00WJ4H72qwY2qYN9+cLSSlsJq6RiNxFjQptNPZce+EezlXeo5dF3b14xMMDFtTt3K+5lsaChfRUBPcul0ndNwScwsJxQkcLzze5/WPZWrJDj1ZCqAp4maj5FCa6rg/HLBUKQig7WNhM9/TNhfFVfXMe2Yn97xxuDXdr7ahmd9vTmCUvysb7pzOU9eO55dLx7VWIVsTe72OmZE+uOXsAUdPCJzY9cFRCzVr4cC/Kf5Gy7X3OfT6xf0Z32mZSy4+5s/vQH5FHXe8dhAnBz1Bnk68dyirH5+kd2SV1vD4xwms/zZV607qHgDX/hsqc+Dg+l6ttWCsH2fyKtl+RqtZiLOw0vyqiKsIdg3m5ZMvD3lr4fljzxPrM56mkstbb+AmVkStwMPgwYbEDX1e/2BaCc4OeosKMqeEeeGg16nU1GFCbxriHRBCPCmEeBLYD/zXZlINYf7xdTKlNQ18fbqAR98/gdEoeWFnMhdKavnjNXE42Nm+lfKcKB8mNh6jJngu6PRdH6jTaTMNHkmmOP5GALz3vwwn34PmRsg8YHEqanV9E3dtOERZbSOv3j6Nm6eHsC+1mIzi6p5P7gd1jc08uTmRy/66i3cPZXLdpCBevmWKtjNsNoxaAnuegzrL24ovGKO1DX91TxpO9voeB/WYsNPZcff4u0ksTjQ753moUNlQSXZVNovDFjImwKNT/yFne2duHHMjOzJ3cKHiQp+usT+1mClhXha1Dne01zM5zFPFFYYJlgaa/w7cAZQApcAdUsp/2FKwoUhyfiXvHMxk7cwwHr1yLB8fz+HBd4+xfncq108OZmakZU/c/eVyv3ICRQmJTpMtO8HVn2JXHzwNntiHzoZPH4ITG6Gx2uJ4woZ96SRkV/DimsnEBXlw/ZRgdALeP2w7a+FcfiUrX/iO1/emc9O0EL559HL+fP0Egr3apKMu/B3UlcHeFyxed9wIN/zdDBRVNRAT6I6+F0WFK6JWMNJlJC+fGLrWgqnHVZRHFJNCPTmeWdpp8M7qcavR6/S8efrNXq9fVtPA2fxKpod7W3zO7ChfknIrtAQJxZDG0kBzKFAEfAx8BBS3bPte8dRnp3F20POzxWO4/7Io7pobwZaTuTg72PHbZWYygGxEeJmWL7+1Ktric4pqi7R4wg2vgYMrbH5Q22Fh5tHJC+VE+Lpw+Th/QGvbPH+MH5uOZNlkmMonx7O5+vk9FFfXt7rkAs1lcQVO1Kq5978E1Za5J4QQzG+xFiwJMrfFXm/PnXF3crzwOAfzDvbq3IEipSwFgFGeo5gU4kVFXVNraxAT/s7+rIhawftn3+dkYfuSo2ZjM0W1Xf9bHk4vRUqYHmG5UpgV5YOU2pQ2xdDGUl/HVmBLy2s7kAp8biuhhiLfnCtk19lCHlw4Gm8XB4QQPLYsml8uHcvzN0/Cx9UwYLKI1F0U2gfxaaa9xaMXi2uLtXRUtxGw6lUtY8l3LLiamcBmhtN5FUSPdGu37aapIeRV1LE72bo9duqbmnn84wRiAt35/KH5re6eLln4O2is0dxIFmJasy9NCq8dfS3+zv78cvcv2ZO9p9fn25rzZecx6A0EugYysbWIrXN33V9M/UXr56ho0Ir4ahpruO/r+1j8/mL+ceQf1DV1nrp7ML0EBzudRUFmE/HBWoLEjjNq8M5Qx1L30Xgp5YSW12hgOjD0vg02QkrJn7aeJszHmVtnXxysotMJ7r9sVOtT54DQ3Ajpe6gInEtxdQNn8ixrY1BUW3QxHTViHqx6Da78k0XnVtY1klFcQ0yHquxF0QF4uzjw/uG++aW7Yve5Iirqmnhw0Wj83CxQtr6jYeIaLeCcddiiayyJCeDhxWNYGjei1/IZ9AbWL1mPj5MPP/76x/zl0F9oaB46bpHUslQiPSLR6/SM8nPFzWDH0czOFcXuDu78ZcFfyK/O58m9T1JeX8492+7hUN4hZgfO5r8J/2XVp6s4lNe+ueKBtBImhnjiaN9NPKsDDnY6rowbwecJearlxRCnT1FRKeVR4HvTCymzpIaz+ZXcPTcCg53lXwSbkHUIGqrwmnAlwMXq5h4orituX6MQew2MXmzRuSbF07FVh4OdjmsnBbEtKZ9iK/bu/+R4Nt4uDsztTQXskv8Dt5Hwv7Va+48ecLTX89Di0bj1sqmeiSjPKN5Z9g43j7uZN5Pe5Fe7f9WndWzB+bLzRHlGAdqDy/QIb3adKTBrVcb7xfPTyT9lW8Y2Vn68ktPFp/nbgr/x0uKXeOWKV2g2NnPPV/dwpuQMAFX1TSRklzOjF64jE9dOCqKyromdZwZ2hrSid1gaU/h5m9cjQoh3gD77DIQQDwshEoUQCUKIjUIIRyFEhBDigBAiWQjxPyGEQ1/XtzZJLfMQemMu24yUnSB0eMcsIsLXhX2pPWd01DTWUNtU265G4XB6CUcyLPPvmkY7mpscd9O0EBqbJR8etU575Or6Jr4+nc+y8SMsymxpxdkbVr+jZSG9d6tWrW1jHO0c+e2M33LP+Hv4OvNr0soHdjayOaoaqsivyW9VCgArJgaSU17HwXTz/9+3x97O3KC51DTV8MKiF1gUtgiAmSNn8u7yd/EwePDU/qcwSiNHM0ppNspexRNMzI7yxc/NwEfHVCvtoYyl3zq3Ni8DWoxhZbdndIEQIgh4EG0mQxygB1YDzwDPtbinStEK5mxGb1IpE3Mq0OsEYwLcej7Y1qTsgKAp4OTJzEgfDqaV0NRDe+SOLS6MRskD7xzl5vUHLFIMSTkVeDnbM8LdsdO+MQFuTA71ZOOhTKtk42xLyqeu0cjKiUE9H9yREXGw8kW4sB++GLgn9zXRa7DX2fP26bcH7JpdkVKuBZmjPC4qhSUxATg76Pm4i5uxTuhYt3AdX17/JbMD2xcyehg8eHjKwxwvPM7mlM0cTCtBrxNMDu19ryy9TrAiPpCdZwtUFtIQxtKYwh/avJ6SUr4tpewcgbIcO8BJCGEHOAO5wEJgU8v+DcA1/Vi/Wz44ksVlz+7irIX++MScckb5ufbKh2p1Guvg/Ndaa4qohQDMjvLRzPmc7oe0F9VpLiaTpXDsQin5FfXodHDvG0fazRcwx+ncCqJHunfZCG719FBSC6s5lN7/TpifHM8myNOJKX246QAQdx3M+gkcfhWKkvstjyX4OvmyLGIZm1M2U15veb2ELWibeWTC2cGOpbEj2Hoqt50/v6HJ2Dqbw15nj5ej+X/zFVErmOg3keeOPMe+tAuMD/LAxdC3ZnrXTgqisVny2am8Pp2vsD3dKgUhxKdCiM1dvfpyQSllNvAskImmDMqBI0CZlNLUHCULMPuoKIS4VwhxWAhxuLCwbx6sRdH+ONvreWnXeYuOT8qt6NcozX5RVQgbb4a/RMBb14O9C8ReB9BaF7Gvh6Igk6VgUgpbT+bhYKfj/R/NpskoufP1Q1TUNZo9t6nZyJm8yk5B5rYsnzASN4Md7x7q3/jHkuoGvk0u4ur4wP4NJJr1gPYz6eN+ydMb1saspbaplo+SPxqwa5ojpSylNfOoLde0+PNNsySajZLbXztI/B++Yuk/dvPk5kQOdeFe0gkdj818jLL6Mk7Xv9+neIKJ2EB3Rvm7dmm1DBZSSp7+/DSbT+QMtiiDTk+WwrPA37p59RohhBea6ykCCARcMN+a26wvQkq5Xko5VUo51c+vb1k/ns4OrJ0Zxqcnckgv6t6NVFRVT35FvVl/+oCQ/BWc/QzG3wBr3oNHzmrN7tBGTI72d+0xrmDKOfdx8sFolHyekMv80X6MD/bgX2snk1ZUzSPvnTB7blpRNfVNxm7nQTg72LFiYiCfncptPxWul3x2Kpcmo2RFfGDPB3eHeyCEzIDET/q3Ti8Y5z2OqQFT2XhmI03GwWv8llKW0pp51JbZUT74uhr4+Jh20/vn9mT2phRz09QQ/NwM/O/QBW5ev5+EbPOWzjjvccwfsRKdxz58fft+QxdCcM3EQA6mlwyp2c3ZZbW8/E0qD248xiPvn6Cm4fvbvK8npZAmpfymq1cfr7m4Zd1CKWUj8CEwG/BscScBBAM2Vdl3zYvATq/jX7tSuj3OFGQeNKWQn6gNzFn+HIy5Ehza9/2fHeXD4fQSGpq6jisU1RahEzq8DF6cyCojt7yOZeNHtJzvy8NLxvBVUr7Z+EJSN0HmtqyeFkpdo5FPjvf9hrH5eA6j/V071UP0iZiVkH8Kirv//7Uma6PXklOdw84LOwfsmh1JKU8h0jOy03Y7vY4V8YHsOFPAZ6dyeX5HMqumBPPMqgm8edcM9v1mIb6uBh7+3/EuU0arcpcgmnzYmP40pXV9dxWa4kWfHB86T+UJLcOWro4P5IOjWSx/fg8phVWDLNXg0JNSaLW/hRAfWOmamcBMIYSz0JzUi4AkYCewquWY2wCbPub5uzmyeloIHx7LIqestsvjEluUQuzI3lW+Wo2CRM0y6KLH0awoH2oamtuP6OxAcV0xngZP9Do9nyfkYa8XLIq+OIHtjjnh+Lo68LevznU6Nym3Anu9IKqH/kDjgz2IDXRn48ELfQo4F1fVcyijhGXjR1pniE30Cu3nALqQLgu5jCDXIP594t9UN9q2J5Q5qhqqyKvOaxdPaMs1kwJpaDbyk3eOMsrPlf+3MrZ1n6ezA8+smkByQVXrcKi2nMoqZ+fpCq4N+jVl9aU8tucxjLJv859DvJ2ZGOLJV4lDJ66QkF2OXif466oJvH33DEqqG3hyc2LPJ16C9KQU2n47Oz9+9AEp5QG0gPJR4FSLDOuBXwE/F0KcB3wYgIZ7P1oQhZSwfndql8ck5pQT5Olk9RbYFpOfCP6xXe6eEeGDEN3HFUwtLqSUfHYql7mjfPFwuvh5nB3s+PFlo9ibUtxpndO5lYz2d7Oo0d/q6aGczq0wWz3bEzvPFiIlLI4O6PlgS/AMgaCp2iCiAUKv0/PbGb8lpSyFn2z/Seuks4Gibc8jc4wP8iDSzwUHOx0v/nAyzg7tg8ULxvixdmYo/9mTxv4OLsm/bzuLp7M9j16+kEenPcq32d/yeuLrfZZ1cbQ/J7LKKajsT76K9UjIKWe0v5ZMMjvKl7vmRPBtctH30lro6Zsuu/i9X0gpn5BSjpNSxkkpb5FS1kspU6WU06WUo6SUN0gprVcN1QVBnk5cOymIjQczya8w/8c5uEHmAqguhICYLg/xcnFg3Aj3buMKJbUl+Dj6kJBdQVZpLVeNH9npmB/OCCXA3cDft51t96SflFNh8XzpayYG4mqw4/W96RYd35btp/MJcDdYNNvAYmKvgdwTUDJw9QPzg+fzp7l/4kj+ER7e9fCAVjqbMo/a1ii0RQjBy2unsOm+2V2mV/92WTSh3s48/L/jrdbn0cxSdp4t5N75kbg52rN67GqWhC1h3dF1rdfsLSZLdSgUskkpScguJ7ZNC/XV00Nx0Ot4c9/3b8BkT0ohXghRIYSoBCa0/F4hhKgUQnSfBzlM+MlCzdR+dNPJThWf1fVNpBVVD248ASCga0sBtLjCkYxS6pvM+4JNlsLWU7nY6QRXxHR+Gne01/OTy0dxKL2Ub5O1wHRBZR1FVZYH2d0c7blhajBbT+Z2qWTN0dBkZPe5QhaOC7Du/ONWF9LAWQsAyyKX8cSsJ/gu+zse/+7xAbuuqedRkGvXNR6jA9yI66YJoLODHS+u0brvXvfSXv61K4Xntp3Dx8WB22aFA5py+d3M3yGEYNO5TV2u1R3jRrgR6OHI16cHXykUVNZTVNXA+DYPJH5uBn4wYSSbjmR97ybGdasUpJR6KaW7lNJNSmnX8rvp/SDdKa1LmI8Lv1sew+5zhWzYl95u35m8CqSk3RPEgFKQpP3sxn0EMCvSh/omY6dhKqA9BZlaXGxLymNWlA+ezuaLxW+cFkKQpxO/+fAUb+xLb+1o2V06akdunx1Os5S8tf/iE1ZdYzN/3JLEizvPsy+luFNmx4G0Yqobmlkc7W/xdSzCKwwCJw9oXMHE9WOu5774+/gs7TOOF/R9wllvSClPIcIjolPmUW+JC/Lg84fmcUVsAM98cYZvk4u4b0FUu9oEb0dvLg+5nC2pW/pkDQmhxbX2JBcNei8kU8ZVR2V566wwquqb+PDowA2TGgrYfiLMMGDtjFAWR/vz9OdnOJN30QAyZR4NmvsoPwlc/HrsZDo90huDnY4PjnT+461qrKK+uR5nvScphdXMjuq6n5DBTs/fbozHw8me33+SyIMbjwG9UwphPi4sGufP2wcyqWtsRkrJLzed5D970vjrl2e5+ZX9THjyKzYevFjTsP10AQY7Xbey9ZmYlZBzbECzkEzcEXsHXgYv/n3i3za/VpOxiXMl57p0HfUWT2cHXlwzmb9cP4EfTBjJ2plhnY65bvR1lNWX9Xk86aJof2obmy1q1WJLTmWXI0Tn3l6TQr2ID/Zgw950pJQUVtazbnsyG/amt05dvBRRSgHtqeWZ6yfg4WTPgxuPtZqLiTkVeDrbM9Kjc3uHASE/oUfXEYC7oz1rZ4bx4bFsUjsExkyFa2WV2meYEtZ9pfDMSB8+e2geW346l9tnh3PbrLBeB9nvmBNBSXUDm4/n8NKuFDafyOHRK8dy7PElvHb7NKaGe/Hk5kSS8yuRUvL16XzmjvLFycEGFePxq0FnDwdsf2PuiLO9M7fH3c53Od/Z3Fp4K+ktCmsLWRxqWZNDSxBCcOO0EF5cM9ns/82skbMIcA7gw/Mf9mn9mZE+ODvo2THILqSE7AoifV3MVmnfOiuclMJq7n3zCHOf2cHft53jic2JLF+3p8tiv+GOUgot+LgaePaGeJILqljy92/4/FQuiTlakNmqfm5LMTZD4ZkeXUcm7lsQhYNexz+3t2/tYCpcyyu1x14vmBBsmSssLsiDJ1fE8oeVcb2TGy3GMTbAjb9+dZa/fnmWlRMDuf+yKLxcHLh8nD/rbp6Ei8GOh987TmKOFvxeZK2so464jdAK/469DbX9b8PRW1aPXW1zayGjIoMXjr/AwpCFLApdZLPrdESv07Ny1Er2Zu8lr7r36aWO9nrmjvJl++n8QZ1il5hT3uWwpeXxI/F1NfDN2UKunRTEjl8sYP0tU6iqb+KGf+/juW2d07gHgk+OZ9us+E8phTYsGOPHpvtm4enswI/fPsqp7PJeuU6sSkkqNNVZZCmAFhi7dXYYm0/kcC7/Yk+n4jrNUkjNE8QFeQxI/yYhBHfMCaewsp74YA+euX5CO8Xq7+bIn64dT0J2BT9++wiguRJsxqz7tdGjR/o+qL6v2NpaMEojT+x9AgedA4/NfGzAH2CuGXUNEskn5/sWzF8U7U9OeR2ncy3rQ2ZtiqrqyS2v6zL4brDTs/knc9jza20UbKSfK1fEjmDbz+ezNHYE/9qVQmn1wDb3K6is4xfvneC179Jtsr5SCh2YEubNpz+ZwxNXxxDo4cjCcTZ6gu2J1syjrtNRO/Kj+VE42+v5x9cXn15MlsLZHPreZK4PXDs5iMeXx/DKbVPNKqKlcSNYNSWYCyW1jA/yIMBMB1arMWI8RMzXhvA0970NR1+xpbWw6dwmjuQf4dFpj+LvbEPF2gUhbiFMHzGdj89/3KdiNtN418GayNZanNpNMkmgpxP+bu3/Pp0d7Hho8Wgamo18OMB9nN49eIEmozQb57EGSimYwU6v4445Eez9zSJmRfn0fIItKEgCoQM/y2c/e7s4cOfcCD47lUdijpZRUVxbjE7oaah3ZGr4wCkFg52eu+ZG7zuYVgAAIABJREFUdPoyteWJq2OIDXTnxqnBthdo5gNQkT3g6amgWQtrotfwXc53fXKzdEV1YzXPHXmOmSNncs0omzUV7pFrR19LVlUWR/KP9PpcfzdH4oM92DFI9QqmzKPYPtTHRI90Jz7Ek/9ZqW28JTQ1G3nnQCbzRvsS4evS8wl9QCmFoUp+InhHgb2ZYfXdcPe8SNwMdry0U8u2Ka4rxknnDuiY3EOQeaBxc7Rn64PzuKUl/92mjL4CfEbBvhdhEPzXyyKWAfBV+ldWW3NbxjaqGqt4YOIDgxP3amFhyEIcdA7syNzRp/MXjPXn+IWyQZmxkJBdTriPM+59nMB387QQzuVXcdRMOrgt+Pp0PnkVddxqw++MUgpDlfzEXrmOTHg42fPDmWF8npBLRnG1ln3U7E6ot3O3T+2XPDodzPyxNo8i2Xo3ZksJdQ9lnPc4tmVs6/KYqoYqPj7/MZtTNrM9YzuJRd333tmSsoVQt1Di/eItF6Q0w+rpuc72zswMnMnOCzv79MR82Vg/jBJ2J1s2WtaaJOSUE9tNMV9PLI8PxNlBz//62TbeUt7Yl0GQpxMLx9nOVaiUwlCkvgpK0yzOPOrIHXPC0esE//k2jaLaImpqnZg6xKyEQWHSLeAfA58+NCiZSEvClnC88LhZF1JicSI3brmRx797nMf2PMbPdv2M1VtXsyV1i9m18qrzOJh3kOWRyy23Esqz4JWFsP5yqyuGy0MuJ7sqm3Olvc/GiQ/2xMvZvnXWw0BRUdfIhZJa4vpRnOpqsGNFfCCfnsilsouZJNbifEEle1OKWTMjFH1/5o30gFIKQ5FCbUi6pZlHHQlwd+TaSUG8f+QCuVUFNNQ7DznX0aBgZ4BrXtJ6Sn3x2wG//BVhVwCwPXN76zYpJW+ffpu1n62lobmB9UvWs/Xarby3/D2ivaN54dgLNJoJjm9J3YJEsjxqufmL5Z2CsgsX3zfWwf/WQlO9ZjW9+0Pt4cNKXBZyGQLRp7bhep1g3mg/dp8r7NRqxpZkl2oNC8N8nPu1zk3TQqhtbObTE7nWEKtL3tqfiYNex03TQmx6HaUUhiJpu7WfI8b3eYl750dS19hESV0xxib3AQ0yD2kCJ8G8X8CJd+Ds59q24hQ48S7U2zYtMtwjnNFeo9vFFV44/gJ/Pvhn5gbOZdPVm5gVOItQ91CifaJ5aPJDZFdl8/6599utI6Xk05RPmeQ/iRC3DjcIYzPsegZeng/PT4GdT0NjLWz9uVbZfd16WPUaFJ2FT+63WnzF18mX8X7j+zxL4rKxfhRVNbTO7xgIcss1pdDf4tSJIZ6MG+HGOwczbBZwrm1o5oMjWSwbPwJfV4NNrmFCKYWhRnMjHPoPhM/Tevf0kVH+biyIdgJhxIAno/2tMLjmUmH+oxAQBx/fD+smwfOT4aMfwcc/tnkQ+oqwKzhWcIyCmgK2Z2xn/cn1XDvqWtYtXIeno2e7Y2cHzmbaiGm8fPJlahovFir9//bOPK6qanvg38UoIA4oIioOOKDmiDibOablnA2OWVk2WGn9Kptfs/Z6r/GV9nLIyrLUV+ZQauac8zxPOAOCoghcBIH9+2NfcGLmXg7E/n4+fO495+yzz+LCPeustdewL3Yf4XHh9A2+wUpIiIbv7oKV70GTu6FRP1g1CT66BXbMgttehIZ3Qt2u0OMNHYm1+l8O+926BnVl3/l9BYqw6txAl3IpShdSZJwu2hhYPn/BHDciIgxvW5M9Zy6x7aRz3JJ/HogmPjmVe8OcayWAUQrFj/0LdOhku8cLPVXfUN0Yp07F6k71QZY43Dxg0BTwKKsjku74AG59Tn/2W6Y79dK317odheKrXV/x8tqXaVq5abZJZyLCuNBxxF6O5dt932buX3h0Ie4u7vSq3evq4LRUmN4LTm6A/p/ZLYJpMGohlK+h+3rfNuHq+A5P60zvFe/AwmcgtfCRP92CugEUqBZS5bKeNKtRnpUHC9Z3vSBEXryMq4vg71v4J++7QmvgW8bNaQllC3ZG4O/rSdtg54fI31zsw2AtG6dAxdrQoHehp/KvoJ+EBjTNe65DqaFqU3hm99Xt9HTtXlnyMtRsr/NDds2GlROhZge4YxJ4Fd4FF1whmHoV6jH74Gz8yvjxYZcP8XTN/qbU3L853YK6MWPvDJJSk7iUconfj/9Ol6AulPe8ZoH0yB86C/6embqPRAZ1boVHV988sQgM+lL3s173CUTvh3u/gbIFj2qpU74OtcrVYsWpFQxpOCTf59/WwJ/PVxwhznalSJpaRcZdJsDX0yEPTD6ebgxpHcT0dceJjEsqtPVxLfGXr/DnwWiGtXHuAnMGxlIoTpzZCqc2QptHs22/mR9ikvRTV5/GRinkiouLth48y8GcUTC1m3YnufvA7jnwRXs4/IdDLtU3uC9u4sa/b/s3VX2q5jr+6dCnSVfpzNw3k+Unl+Pv5c/IxiOvH7T9W11Rt2GfvAvi4go934LB0yBiB0y5FQ4szudvcxURoWtQVzZFbSI+Jf/rMxmhqWuOFI21EHUpiaoOLHZ5f/vaKKUc3phn6d6zpKSm0695NYfOmx1GKRQnNkwBD19oOcIh00XbohGESl4WZWWXNMpWgbu+hHOHIf4s3PUVPP4XPLIcypSHWYNh/ReFvsxDTR5i+b3LCasalqfxdSvUZf3Q9WwbsY1V961i/sD5tKzS8uqAhBg49LuuCOtagCfspnfDw8vApzLMHgpzH4LEguUMdA3qSmp6KqtOr8r3uS2CKlLey73IXEiRFy8TWMFxT/RBft70bBzAD5tOOrRHxIJdEVSv4EVozQq5D3YARikUF+KjYO/P0HI4lHFMEb4YWwyVvCrh7mJRf+mSSN1uMHYjPLUFmt2rLYhqLWHMKqjXE1a8q2/ChUBE8Cvjl69zXF1cs89H2PUjpKdCi0I8TFRtCo+sgK6vwL5f4YN6MKkmfNwMfhgGKYl5mqZFlRYE+QYx5+Cc3AffgKuL0LmBPysPRjs9NFUpRWTcZQIdXHPrwY51uGC7wvwdjqmHFJuYwtrD5+jXvFqRZa0bpeAsjq2Bhc/CrHvhiw76NTY8+/HrPtVf7DZjHCbCWdtZ/L1ybtBjyAL/EPC4oa6MexnobQ/vXP2BNXJlhVLadVSjNVQppJvQzQNuewEeWwtdXoRmQ6B6Kzi4CDZ+macpXMSF+0LuY1v0tgIlsvVoVIVzCSnsOO3cshFxSVdIupLmUEsBoG0dPxpW9eXLVeGcT8h7m/mT522kpN5cUPD3PVGkpiv6Nb+5r7qzMErBWSwcDzt/gPgIqBAEJ9fD5I76y5V+wx///FFdwbPlCKjkmM5ZoC2FAG+Lqrz+HalcH0JH6gil2GNWS6M5s00nOzrI5Qho5dLlRbjzn3DPDKjfC9Z9DEl5u1EPqDsAT1dPfjzwY74v3aVBFVxdhOX7nVs19Wo4qmMtBRHhlT6NOHMxiUFf/MWR6NwTBP/Yd5bOH6yg2ZtLGPbVBj5cdogFOyPYdfoiv2w/Q7C/T5GW8DdKwRnEhsP5IzoW/LG1MOxHeGID1OoAv70A3w26PlHqj3+Aqwd0e9WhYkTbovH3NpaCQ7ntRXBxgxXvWSfDqc0Qvko/TGydDm5eOuTUWXR7FS7HwV+f5Wl4hTIV6F27NwvCF5CQkr+s6fLe7rSuXZHlTu7G5qjEtay4tb4/s8e0w5aSyl1frOOnzaeYsuooT36/jdFfb76pR/m0tceoWq4MQ9vUJC7pCp/9eZinfthO//+sY9PxWPo2KzrXEZiQVOeQEaVS75rWiOWrw/C5sG2mdit9dzcMn6PLEexfAF1f1V3CHERKWgoXki9YUmP/b025QJ1DsvZD6PAUBDYr2uvv+F5HRV1L86EOW4fKksBm0GQwbJgMbR/NU9jqkIZDmH90Pr8e/ZVhjYbl63I9GgXwzqL9nIq1EeRXuBIU2eGoxLXsaFmzIj8/0ZGHvt7MC/N2AVC9ghdnLiYxY91xxnatB8DBqHjWh59nQu+GPN5FewlsKamcjLVx4ryN6EuX6d+iulNkzA5jKTiDw0t12esbXUEi0OoBuHs6nN4M3w2GJS9BuerQfqxDRcgIRzVKwQl0HKdzForaWojcqRPNat8KoxbAwCnQ402HW5hZ0vUV3Qlwzb/zNLxJ5SY0qdSEHw/+mO/SDxmtWZ3pQnJk4lp2BPl5M//Jjsx9rD3bX+vJuhe70aNRAFNWHc0sEz5z/XE83VwYck09I28PNxpWLUevW6oysn1tynsVbaCIUQqOJsUGx9fo+v3ZcctA7auN2Ka/6D3eAA/HPhHF2IxScBpeFaDtY3DoN4g5WDTXtMXCjyPBy0/XLqrTGVoMhU7jdcays6lUV69bbJ6mrZU83Ojva3gf4XHhbI7anK9L1ansQ7C/D8ud2HjHkYlrOeHt4UZYbT8q+ngA8HyvEBKSU5m86ihxtiv8vO0MA1pUyzxeHLBEKYhIBRGZKyIHRGS/iLQXET8RWSYih+2vJbOC2/G1+omqfo+cxzUeAENn66fOJnc7XIyzNv2UZaKPnETrh8GtTJ797Pkm7gxM7Qmz7oHFL+gKp5ci7FnHFv1Nu7+uo5x+eRy+vw8u5VwVtHft3lT0rMg3+77J96V6NgpgQ/h5p5WjdnTiWl4JqerLoBbV+XrdcT778zBJV9IY1aF2kcuRE1ZZCp8AvyulGgLNgf3Ai8BypVR9YLl9u+RxZJle+KvVKfex9XvqjFIXx/8ZMiwFE33kJHwqQ4vhOkcg3nEtNjPZ8T2c3qRvvDtm6ZpGd/4Tglo7/lp5xacyPLAIek/SlXw/bwM/P6bzay7H3TS8jFsZhjYayqrTqzh84XC+LtW9UQBX0hSrDzmn8Y6jE9fywzM9G5CuFFPXHqNNbb8c+0NbQZErBREpB3QGpgEopVKUUheBAcBM+7CZgHVNZwuKUno9Ifg2HdduIdG2aDxcPK6vj2NwLO3H6qq2eYzhz0QpSDyf85g983TNpcfXwkun4eUzEPZQwWV1FBkd7B5fByF36PLjcx6AD+pD+Mqbhg8NGYqXmxdf7/06X5cJrVmBCt7ufL/pBDtOXeRK2s0x/AXFWYlreSXIz5thbWoCFDsrAayxFIKBGGCGiGwXkaki4gMEKKUiAeyvWTrDRWSMiGwRkS0xMQXMLD13GOY/qRuOOJLzR+DC8eujjiwiOkmHo1rZu/dvT6W60KgvbJmWv4Y1S16GD4LhszBY/PzV/hkZnN0HMfuhiT3MVCTfvbqdTqW6uhLr80fhwd91Ls6vT92U+VyhTAXuqn8Xi8MXE5mQ9yY0bq4ujGhbi3VHzjPw83U0fWMJ42ZvJzm18OUjMhLXrHAfZfB/vUKYeFdTejdxXMSho7BCKbgBocBkpVRLIJF8uIqUUv9VSoUppcL8/QvoW714UmeBrv+8YOdnx2F7/936PR07bwGItkUb11FR0GGcdp1s/zb3sQAHFsGGL6DBHboa7rZvYWY/nXeQwd7/gbjodafijqsb1GoP/T7R36uVE68eUwpObeL+enehUPleW3iuVwgbX+7O58NCGRxag/k7Injmxx2kFbIERkY4ajWL3EcA5cq4M7SIqp7mFyuUwmngtFJqo317LlpJnBWRQAD7q/NCD+p1h4Z9dbmCOMfUKOHyJV1quXKI/rJbTIwtxiSuFQVBrbWbZ82HWfrVr+PiKd3YJ7AF3DsTRsyFF45C+Zqw9BXdNU0p7Tqq07lQZayLnNqdIPR+/aAVsUNnP88bDdN6Um3WMO4I7Mi8w/OIS87lM7qBgHJl6NMskHcHNeWVOxuxeHcUr83fU6gOZxmJa1ZaCsWZIlcKSqko4JSIhNh3dQf2Ab8Co+z7RgHznSpIr3dBpcNSB8R4nz8K03pC1B7o/Fzh5yskSinO2s6acNSiotc7kBgDq/6Z/Zi0VJj3sL7x3zND94sGXWOpxz90EuPOHyByh86Id2aGsrPo+RZ4V9Zd7KZ0gr2/QNvHIekCD2xfSFJqEj8c+KHA0z/SOZjHbqvL9xtP8tGy/NdVyiDTUnBS4prTuHxJW5RrP4afRsGOgn+WOWFV9NFTwCwR2QW0AN4DJgE9ReQw0NO+7Twq1oZOz2hT/diags2Rkqj/8b/qBglnYeTPurKmxSReSSQpNYkqXkYpFAnVW+kY/o1Tss9b+OsTOLUB+n0MfsHXH2syWId6Ln9bu5Nc3HQrzZKGV0UdIRVzQJfwHr1MNyd6dBUhFerQ2ZbEdzsmY1s1SUdTpeU/3HRC7xDuCwvi0z+PMGfLqQKJWRSJa5nYYuHUpsLPc3YffNwEvumvy+JEbIdk5/SztkQpKKV22NcFmimlBiqlLiilziuluiul6ttfY50uSMdxUKGmXuzL6z+oUrqi6dQeurTwnFHgG6jLDgffViAxlFL8dPAnHvj9AbZHby/QHNcSbdOeN2MpFCHd/6Eb8vw24ebErqSLsPYTCOmjexfciAj0mggJUXrRum538M5fae1iQ+OBeuH50TVQo5XeV74GPPgbj9TsTRzpzNn6H9069KMmOis8Hy5cEeGdQU3oWK8SL/+8m43huURxZUFk3GWqODtxTSnYNQf+E6a9CAufLXhgiy0WfhiiQ92Hz4UXjsH4XbrkiBMo3RnN7l7Q+30d6ZHH9H1W/ROWvaZdTx2ehuHzYMxK8KtTIBGupF3hrQ1v8faGt9l7bi+jfhvF+5vev65Re37JTFwzawpFR1l/6PoyhK+AAwuvP7ZhMiTHQdeXsj8/qLW2GOBq1FFJREQvPHuWvX6/uxct+vyHtlXb8nX1uiQPnqprKq36J3zcVOc75JIMlzmVqwtfDGtFUEVvHvtuKyfO563XQwZRl5KcUggvkwsn4Pt74X8Pa49Em0e1sp92u45OzA9pqTrkNz4S7vtOB7E4+YHBFMRreKeuG7/qfd1gJahN9mN3/ggr34Pmw2DgF/oLUAguXL7AuBXj2B69nUeaPsJDTR7i0+2f8t3+71h5aiUfd/2YEL+Q3Ce6gYy6Ryb6qIhp/fDVgocBt2g3UdIFHW3UqJ9uZJMTvd7TVmej/kUjrwWMaTaG0UtH87N7GkOGz9E3yU1f6dLx+36F256Hdk9cXXPJhvLe7kx7oDUDP1/H0P9uoGvDKjSuVo42tf2oH+Cb47mRFy/TyBmlqG2x+uFy03+1C7DXRP007+IKwV3gl8fg83b6fb3u2u148aTuj207Dx2f1p6LDJTSa57HVsGAL4oscVEKs4pvNWFhYWrLli2Fn+jyJb0wBrrU9Y0VJ9PT4cRa+PYuqNkORvxPNyQpJJ9u+5Tpe6Yz6dZJ9K7TO3P/lqgtTFgzgfiUeCZ2mkj3Wt3zNe/U3VP5ZNsnbBq+CS+3EraYVtKJOQjTe4OnLzy0RD8hrv4AHlsHVZtYLZ3lKKUY+dtIYmwxLLxr4dWugLHhsORV3dCnXA1o/4SOZvLM+Qa/9UQsHyw5yN6IS8RfTsVFYPaY9rSpk/XTtFKKxq8vYXjbmrzat3Hhf6G0VN1X/eBivR6UEg8thkGXl26uSXXhuC6LcngZXLy2j7PoNRivirrMfrWWOu9l0bM6Y77t43ptxoGIyFalVJb9YI1SyODkRpjRW9chqt9TZ2qGr9AKQ9kTZiqHwOiluiCaA3hh1QvsOb+HxXfd3Cw9xhbD+BXj2XVuF2NbjOXRZo/mORHtvY3vsTB8IX8N/cshchryyZmtMLM/lA+CuNNQr5uuWWQAYPXp1YxdPpa3OrzFoPqDrj949E9Y/S84sQ48y+sbbMM7oWb7HPtPK6U4GWtj2FcbKevpxsKnO+HuerN3/KIthRZvLePVPo14+NbgLGbKheQErQQid+jQ2+NrtDXo4g4NeulqsgG5KBultBKM2gUV60DlBlpJzLpHWww93oTNX+lk2C4vwa3/p60NB5KTUjDuowxqtoXOz2s30u6fwKcKhNypexy4uOuyFc2GOEwhAEQkRlDNp1qWx/y9/Zneezpv/PUGn+/4nNrla9O7du8sx95ItC3aRB5ZSfVWMGSW/pKnXdGNeQyZ3Fr9Vhr5NeLLXV/SN7gv7tfe7Ot20z+nt8Jfn2pLa+NkKFNeu217vZvlDVJEqFXJh3/0a8yYb7cyY90xxnS+uYthRvOeYH+fm47lysWT8HUf/Qr6hl6/ly73Ubdb3ntaiOiM8GtL61dpBA8v12sRvz0PZQPg/vk6X6WIMUrhWjq/oH16/g2hWqhTCtVdS2RiJO0D22d73NPVk3c6vsPBCwf5dNundK/Z/aq5nQMxthgTeWQ1wV1gxDx9A8ntybGUISI8Hfo0j//xOPMOz2NIwyE3D6rRSif4JSdoi33ffK0cUpOg78f6xqqUdsds+q+O7ElP5Xa3Mnxf+Rbm/nELkSFPEBhwdV0tLukKE3/bT/OgCnRpkM/vR9wZnXl+OQ6G/KC7KDrwAREA3wB4cLEuhth4gGXJi0YpXIurm2N73ebAlbQrxNhiqFY2a0shUyQXV8aHjmfs8rHMPTSXoQ2H5jr3WdtZ2ga2dZSohoJiwVNeSaFjtY60CmjFl7u+pH/d/ni7Z9NPxLOsXqRv1E+749Z+qBPkOj+nay3tnqM/Z79gvbhri6Xt0RV0cFlG6uT/QIOe0OweaHAHnyw9jGviWd4f3AKX7MJRk+Nh+3ew9WvwLKetgJrtdK00Wyzc/4u2BJ2Fhw+0ecR58+cBoxQs4qztLApFoE9grmNvrX4rrau2ZsrOKfSv2x8f9+xN3/iUeM4lnaOqT/ErtGUwZCAijAsdx/2/3c8PB35gdNPRuZ/U/XXtc1/zL9j1E8Sdgm6vaZ/7NettrmmpzFvwC+e3/I97j26kwqHfUOLG6yqV1z2BH9Hlv9td09ZUKZ0zsXGKTgqr0QbSUmD5m/q4R1mdnOpMhVBMMErBIiITdUx2YNnclYKI8EzoMwxbPIyZe2fyRIsnsh27OHwxaSqNbkHdHCarweAMWlZpSecanZm+Zzr3hNxDOY9cfPIi0Pcj7cI5shyGfK8XoW/E1Y2B/QfzhW9z2q04REfXA/Rw38OFVHce7NkaryOL4I839cJwRnb5+s9h9T91OHCn8Vdv/pci4MgfULUZVGvh2A+gmFK6k9csJFMp5MFSAGjq35Tba93O13u/Zu+5vVmOUUox9/BcGvo1pHEl48c2FH+ebvk0l1IuMX339Lyd4OIK93wNzx3KWiHYcXURnupen9/HdyE5qBMvxd+Nf5/X8OrwiM4xcnWHX5+2V3LdrEtHNOqno8SutQbKVdOhsaVEIYBRCpYRkRABkC83z/jQ8Xi5eTFk0RBeXPNi5hwZ7Du/jwOxBxhcf7Dpo2AoEYT4hdAvuB/f7PuGE5dO5H4CaIshjz3Na1f24dvRbVj1fBfuCQvSO8tVg9vf1uGk6z6BuQ/pff3/U+iE1L8DRilYRFRiFJXKVMLTNe9FuYLKBbFw0EJGNxnNHyf+oN/P/Vh+Ynnm8bmH51LGtQx9gvs4Q2SDwSk8G/Ysnq6eTNw4sVAlsbMjI1z1OkJH6QXqP/6hS0jc/bXjo4lKKEYpWEREQkSukUdZ4evhy/hW41kwcAEN/RoyYc0EdsXswnbFxuLwxfSq3Qtfj5yzQA2G4kRlr8o82fJJ1kWsY/nJ5bmf4AhEoN+nUK469J54tXifwSgFq4hMjMzzekJWBJYN5LPun+Hv5c9Tfz7F1N1TsaXauLtBFlU4DYZizn0h9xFSMYT3NxeuGGS+8KsDz+y1PAS0uGGUggUopQqtFAD8yvgxucdk0lQaX+3+irrl69Lcv7mDpDQYig43FzdeafcKUYlRfLT1I6e4kbLErCHchFEKFhB7OZbktOQ8haPmRu3ytfm066d4uXkxsvFIs8BsKLG0rNKS4Y2GM/vgbF7/63WupOe/CY+h8Jg8BQuISowC8h6OmhuhAaGsHbIWD9fCV241GKxkQusJ+Hr4MmXnFKISo/iwy4dmjayIMZaCBUQk6lDSgiw0Z4dRCIa/AyLC2Ba6guqWqC2MWDyC8IvhVotVqjBKwQIiE/KXuGYwlDYG1R/ElJ5TuJh8kSGLhrA4/Oby8gbnYJSCBUQmRuLt5p17Wr/BUIppG9iWn/r+lBl6PWnTpKJbgC7FGKVgARmRR2ZR2GDImQCfAKb1msbwRsOZtX8Wn23/zGqR/vaYhWYLiEiIcEjkkcFQGnB3cWdC6wlcTr3MV7u/oqpPVe4Nuddqsf62GKVgAVGJUTSpbPr1Ggx5RUR4td2rxCTF8O7Gd/F09aSZfzO83LxIV+kciztGeFw4ZxPP4uPuQ1mPspT3LE+gTyA1ytagincVXB3c0vLvilEKRYztio0LyRccGnlkMJQG3Fzc+KDzB4xeMppX172a5RhPV0+S05Jv2u/l5sX/tfo/7g25t1i6bZPTkolPiSfxSiJp6WnUKV/HMjmNUihiomyOzVEwGEoT3u7eTOs1jY2RG7Gl2khKTQKgVrlaBJcPxq+MH+kqnYQrCVxMvsiZhDOcSTjDsuPLeGfjO+w6t4vX2r3GxeSLzNgzgwVHF9C+WntebPMi/t7+TpFZKcWxuGNsiNxAREIE7au1p03VNri7urP33F5m7J3BshPLSFfpmed0rN6RN9u/SYBPQOYcRy4eoZxHOap4V3GqwhCrVvNFxBXYApxRSvUVkTrAbMAP2AaMVEql5DRHWFiY2rJli/OFdSDrzqzjsT8eY2bvmYQGhFotjsFQKkhX6UzZOYXJOydTvWx1ztrOgtI33/UR6/F09WR8q/Hc3eBuXCR/8TdKKZYcX8LmqM2kqTRS01NJSUsh/ko88SnxnEk4w7mkc4C2dlLTU/H18KWmb032nt+Lr7svA+sPpJZvLbzdvYm2RfPlri9xc3HjmVbPEJsUy4LwBZmlxf3K+NHQryED6g7gzuDse0rkhIhsVUqFZXXMSkthHLAfyIjLfB/4SCk1W0SmAKOByUUhiO2KLfsesQ5YaDKgAAAJZElEQVQmo7mOcR8ZDEWHi7jwRIsnaFK5Ce9vep/B9QfzUJOHqFa2GsfjjvP2hrd5e8Pb/Hr0V15r9xohfiEApKWnsS16G5W8KhFcPvimefee38ukjZPYEbMDXw9fvFy9cHVxxcPVg7LuZfH18KVdYDvCAsJoG9gWf29/NkRsYNmJZRy6cIjnwp5jcP3BlPUoe928PWr14NW1r/LW+rcACAsIY9Qto0hJS+FA7AEOxB4gJinGKZ+VJZaCiNQAZgLvAs8C/YAYoKpSKlVE2gNvKKV65TRPYSyF5LRklh5fyk8Hf2JHzA5aBbRiWMNhdKvZDTeXwunKPef28O2+b1FKMaLxCJr5NwMg8Uoib294m9+P/c6WEVsKfR2DweAYlFL8evRX/r3l31xKucTQhkNxd3Vn0dFFRCdFA9AlqAsPN32YAO8A1p5Zy+rTq1l5aiUVy1RkXOg4BtQd4NDF7LT0NDZEbqB2+dpUL1vdYfNCzpaCVUphLjAR8AWeAx4ANiil6tmPBwG/KaVuCtERkTHAGICaNWu2OnEij92armHlqZW8vu51LiRfoFa5WnSu0Zk/T/7JmYQz+Hv5U6tcLXw9fLXmd/PC280bL3cvutfsToOKDbKcMz4lnvUR65l9cDabozbj6+4LoveHBYRRvWx1lp5YSlJqEh2rdWRKzyn5lttgMDiXuOQ4Ptn2CXMPzcVVXOlUvRN96vYh/GI43x/4nrjkuMyxVX2qcmedO3m46cMlrj5TsVIKItIXuFMp9YSIdEErhQeB9TcohcVKqaY5zVVQS+FY3DE+3vox9zW8j3aB7XARF9LS01h9ejWLji3ifNJ54lO0PzBjMSs5LRlB6Fe3H2NbjKWCZwV2ndvFtrPb2BC5gV0xu0hTaVTxrsL9je/PbIk579A8vtn3DfEp8fSu05tB9QbR3L95sYyAMBgMmlPxp/B286aSV6XMfbYrNuYfnU9yajKdqneiboW6JfZ7XNyUwkRgJJAKlEGvKfwM9KII3Uf5JS45jmm7pzFr/yzSSUcpRZpKQxAaV2pMh2od6Fi9I838m+Hu4n7duWnpaaSTftN+g8FgsIJipRSuu7jdUrBHH80B5l2z0LxLKfVFTudbEX0UlRjFt/u+xdPVk9CAUJr7Ny9xpqPBYCjdFNfooxuZAMwWkXeA7cA0i+XJkqo+VXm+9fNWi2EwGAxOwVKloJRaCay0vw8H2lgpj8FgMJR2TJVUg8FgMGRilILBYDAYMjFKwWAwGAyZGKVgMBgMhkyMUjAYDAZDJkYpGAwGgyEToxQMBoPBkImlGc2FRURigPxXxNNUBs45UJyipKTKbuQuWozcRUtJkruWUirLrkIlWikUBhHZkl2ad3GnpMpu5C5ajNxFS0mV+0aM+8hgMBgMmRilYDAYDIZMSrNS+K/VAhSCkiq7kbtoMXIXLSVV7usotWsKBoPBYLiZ0mwpGAwGg+EGjFIwGAwGQyalUimISG8ROSgiR0TkRavlyQsiMl1EokVkj9Wy5AcRCRKRFSKyX0T2isg4q2XKCyJSRkQ2ichOu9xvWi1TfhARVxHZLiILrZYlP4jIcRHZLSI7RKRo2yoWAhGpICJzReSA/X+9vdUyFZRSt6YgIq7AIaAncBrYDAxVSu2zVLBcEJHOQALwjVKqidXy5BURCQQClVLbRMQX2AoMLAGftwA+SqkEEXEH1gLjlFIbLBYtT4jIs0AYUE4p1ddqefKKiBwHwpRSJSUJDAARmQmsUUpNFREPwFspddFquQpCabQU2gBHlFLhSqkUYDYwwGKZckUptRqItVqO/KKUilRKbbO/jwf2A9WtlSp3lCbBvulu/ykRT1AiUgPoA0y1WpbSgIiUAzpjbyGslEopqQoBSqdSqA6cumb7NCXgJvV3QERqAy2BjdZKkjfsLpgdQDSwTClVIuQGPgZeANKtFqQAKGCpiGwVkTFWC5NHgoEYYIbdZTdVRHysFqqglEalIFnsKxFPgCUZESkLzAPGK6UuWS1PXlBKpSmlWgA1gDYiUuzddiLSF4hWSm21WpYC0lEpFQrcAYy1u02LO25AKDBZKdUSSARKxFplVpRGpXAaCLpmuwYQYZEspQK7T34eMEsp9T+r5ckvdlfASqC3xaLkhY5Af7tvfjbQTUS+s1akvKOUirC/RgM/o929xZ3TwOlrLMm5aCVRIimNSmEzUF9E6tgXhIYAv1os098W+4LtNGC/UupDq+XJKyLiLyIV7O+9gB7AAWulyh2l1EtKqRpKqdro/+0/lVIjLBYrT4iIjz0YAbv75Xag2EfbKaWigFMiEmLf1R0o1oEUOeFmtQBFjVIqVUSeBJYArsB0pdRei8XKFRH5AegCVBaR08A/lFLTrJUqT3QERgK77f55gJeVUostlCkvBAIz7dFqLsBPSqkSFd5ZAgkAftbPEbgB3yulfrdWpDzzFDDL/qAZDjxosTwFptSFpBoMBoMhe0qj+8hgMBgM2WCUgsFgMBgyMUrBYDAYDJkYpWAwGAyGTIxSMBgMBkMmRikYDICIVLJX5twhIlEicuaa7b+cdM2WIpJtfSJ7rkRJCck0/E0odXkKBkNWKKXOAy0AROQNIEEp9S8nX/Zl4J0cZIoRkUgR6aiUWudkWQwGwFgKBkOuiEiC/bWLiKwSkZ9E5JCITBKR4fa+C7tFpK59nL+IzBORzfafjlnM6Qs0U0rttG/fdo1lsj0jsxf4BRheRL+qwWCUgsGQT5oD44Cm6EztBkqpNugy1U/Zx3wCfKSUag0MJusS1mFcX8LhOWCsvQDfrUCSff8W+7bBUCQY95HBkD82K6UiAUTkKLDUvn830NX+vgfQ2F6uAaCciPja+0lkEIgut5zBOuBDEZkF/E8pddq+Pxqo5vhfw2DIGqMUDIb8kXzN+/RrttO5+n1yAdorpZLIniSgTMaGUmqSiCwC7gQ2iEgPpdQB+5ic5jEYHIpxHxkMjmcp8GTGhoi0yGLMfqDeNWPqKqV2K6XeR7uMGtoPNaAEVAo1/H0wSsFgcDxPA2EisktE9gGP3TjAbgWUv2ZBebyI7BGRnWjL4Df7/q7AoqIQ2mAAUyXVYLAMEXkGiFdK5ZSrsBoYoJS6UHSSGUozxlIwGKxjMtevUVyHiPgDHxqFYChKjKVgMBgMhkyMpWAwGAyGTIxSMBgMBkMmRikYDAaDIROjFAwGg8GQiVEKBoPBYMjk/wFURZjo5gIJGgAAAABJRU5ErkJggg==", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "key = dict(mouse_id=0, session_number=1, scan_idx=1, seg_param_id=1)\n", - "\n", - "# ENTER YOUR CODE! - fetch 'time' from the Fluorescence table using fetch1()\n", - "time = (Fluorescence & key).fetch1('time')\n", - "\n", - "# ENTER YOUR CODE! - fetch 'trace' from the Fluorescence.Trace table using fetch()\n", - "traces = (Fluorescence.Trace & key).fetch('trace')\n", - "\n", - "plt.plot(time, np.vstack(traces).T)\n", - "plt.xlabel('Time (s)')\n", - "plt.ylabel('Fluorescence')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Summary" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Congratulations! You have successfully extended your pipeline with all types of tables. This pipeline is simple, but reflecting a typical preprocessin pipeline of calcium imaging. We have all the table tiers and major dependencies between DataJoint tables.\n", - "\n", - "**Table tiers**: \n", - "Manual table: green box \n", - "Lookup table: gray box \n", - "Imported table: blue oval \n", - "Computed table: red circle \n", - "Part table: plain text\n", - "\n", - "**Dependencies**: \n", - "One-to-one primary: thick solid line, share the exact same primary key \n", - "One-to-many primary: thin solid line, inherit the primary key from the parent table, but have additional field(s) as part of the primary key as well" - ] - }, - { - "cell_type": "code", - "execution_count": 62, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": "\n\n%3\n\n\n\nFluorescence.Trace\n\n\nFluorescence.Trace\n\n\n\n\n\nFluorescence\n\n\nFluorescence\n\n\n\n\n\nFluorescence->Fluorescence.Trace\n\n\n\n\nMouse\n\n\nMouse\n\n\n\n\n\nSession\n\n\nSession\n\n\n\n\n\nMouse->Session\n\n\n\n\nScan\n\n\nScan\n\n\n\n\n\nSession->Scan\n\n\n\n\nSegmentation\n\n\nSegmentation\n\n\n\n\n\nSegmentation->Fluorescence\n\n\n\n\nSegmentation.Roi\n\n\nSegmentation.Roi\n\n\n\n\n\nSegmentation->Segmentation.Roi\n\n\n\n\nSegmentationParam\n\n\nSegmentationParam\n\n\n\n\n\nSegmentationParam->Segmentation\n\n\n\n\nSegmentation.Roi->Fluorescence.Trace\n\n\n\n\nAverageFrame\n\n\nAverageFrame\n\n\n\n\n\nAverageFrame->Segmentation\n\n\n\n\nScan->AverageFrame\n\n\n\n", - "text/plain": [ - "" - ] - }, - "execution_count": 62, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dj.Diagram(schema)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We have covered most of the building elements of data pipeline design. Using these elements, we could design more sophiscated pipelines that facillitates your experimental recordings and data analyses." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.16" - }, - "vscode": { - "interpreter": { - "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" - } - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} From ad996f51b90ea04ff09385c51ef1671a3a209261 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:31:22 +0200 Subject: [PATCH 45/70] Revert "Rename including a space" This reverts commit 800e3d621f22bb70c7dde23dee14badcd276d389. --- .../03-Calcium ImagingComputed Tables.ipynb | 2691 +++++++++++++++++ 1 file changed, 2691 insertions(+) create mode 100644 completed_tutorials/03-Calcium ImagingComputed Tables.ipynb diff --git a/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb b/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb new file mode 100644 index 0000000..4436773 --- /dev/null +++ b/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb @@ -0,0 +1,2691 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Working with automated computations: Computed tables" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example. \n", + "\n", + "In this session, we will learn to:\n", + "\n", + "* compute various statistics for each neuron by defining a `Computed` table\n", + "* define a `Lookup` table to store parameters for computation\n", + "* define another `Computed` table to perform spike detection and store the detected spikes\n", + "* automatically trigger computations for all missing entries with `populate`\n", + "* define a `Part` table to save the results computed with the master `Computed` table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's import `datajoint` again." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import datajoint as dj" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we are going to perform some computations, let's go ahead and import NumPy and Matplotlib." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Similarly as before, to continue working with the tables we defined in the previous notebook, we can either redefine the classes for each table `Mouse`, `Session`, `Scan`, `AverageFrame` and populate them. Or, again for your convenience, we can import them from the `tutorial_pipeline.imaging` module. " + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Connecting shan@workshop-db.datajoint.io:3306\n" + ] + } + ], + "source": [ + "from tutorial_pipeline.imaging import schema, Mouse, Session, Scan, AverageFrame" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " Experiment session\n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

experiment_setup

\n", + " experiment setup ID\n", + "
\n", + "

experimenter

\n", + " experimenter name\n", + "
\n", + "

data_path

\n", + " \n", + "
02017-05-150Edgar Y. Walkerdata
02017-05-190Edgar Y. Walkerdata
52017-01-051Fabian Sinzdata
1002017-05-25100Jacob Reimerdata
\n", + " \n", + "

Total: 4

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date experiment_set experimenter data_path \n", + "+----------+ +------------+ +------------+ +------------+ +-----------+\n", + "0 2017-05-15 0 Edgar Y. Walke data \n", + "0 2017-05-19 0 Edgar Y. Walke data \n", + "5 2017-01-05 1 Fabian Sinz data \n", + "100 2017-05-25 100 Jacob Reimer data \n", + " (Total: 4)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Session()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

depth

\n", + " depth of this scan\n", + "
\n", + "

wavelength

\n", + " wavelength used\n", + "
\n", + "

laser_power

\n", + " power of the laser used\n", + "
\n", + "

fps

\n", + " frames per second\n", + "
\n", + "

file_name

\n", + " name of the tif file\n", + "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
1002017-05-251150.0920.025.015.0example_scan_03.tif
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", + "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", + "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", + "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", + "100 2017-05-25 1 150.0 920.0 25.0 15.0 example_scan_0\n", + " (Total: 3)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Scan()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

average_frame

\n", + " average fluorescence across frames\n", + "
02017-05-151=BLOB=
02017-05-152=BLOB=
1002017-05-251=BLOB=
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx average_fr\n", + "+----------+ +------------+ +----------+ +--------+\n", + "0 2017-05-15 1 =BLOB= \n", + "0 2017-05-15 2 =BLOB= \n", + "100 2017-05-25 1 =BLOB= \n", + " (Total: 3)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "AverageFrame()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `imaging.py` also fill each table by inserting manually and loading data from the external tiff files." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Computations in data pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now it's time to perform more complicated analyses. \n", + "\n", + "When you perform computations in the DataJoint data pipeline, you focus and design tables in terms of **what** is it that you are computing rather than the **how**. You should think in terms of the \"things\" that you are computing!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we would like to detect cells from the average image. The final product we would like is the binary **mask** for each individual cell, with the 1 in the region of interest (ROI) and 0 in other places." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So the new \"thing\" or entity here is `Roi`, where each entry corresponds the mask of one ROI. Let's start designing the table, paying special attention to the dependencies." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Detect cells from the average fluorescence image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's perform the segmentation to isolate ROIs. Let's start by taking a look at one average fluoresence image." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "keys = AverageFrame.fetch('KEY')\n", + "\n", + "# pick one key\n", + "key = keys[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

average_frame

\n", + " average fluorescence across frames\n", + "
02017-05-151=BLOB=
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx average_fr\n", + "+----------+ +------------+ +----------+ +--------+\n", + "0 2017-05-15 1 =BLOB= \n", + " (Total: 1)" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# ENTER YOUR CODE! - preview an AverageFrame for a particular key\n", + "AverageFrame & key" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "avg_image = AverageFrame.fetch('average_frame')" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", + " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", + " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", + " ...,\n", + " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", + " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", + " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]]),\n", + " array([[ 29.53, 30.8 , 30.96, ..., 230.09, 230.78, 228.19],\n", + " [ 29.61, 29.69, 30.71, ..., 224.86, 227.21, 227.22],\n", + " [ 29.61, 28.28, 30.19, ..., 220.51, 224.43, 224.85],\n", + " ...,\n", + " [ 23.67, 23.05, 24.06, ..., 32.54, 32.07, 35.28],\n", + " [ 24.03, 22.45, 22.4 , ..., 31.67, 30.75, 32.72],\n", + " [ 25.78, 23.27, 21.92, ..., 31.12, 31.29, 32.94]]),\n", + " array([[41.55, 40.2 , 38.3 , ..., 55.67, 57.17, 58.65],\n", + " [43.66, 42.2 , 38.59, ..., 50.35, 54.83, 57.9 ],\n", + " [42.07, 41.46, 38.3 , ..., 49.46, 52.67, 55.61],\n", + " ...,\n", + " [30.04, 31.05, 30.92, ..., 45.04, 44.93, 44.81],\n", + " [28.88, 29.52, 29.93, ..., 46.5 , 45.65, 45.13],\n", + " [27.84, 27.34, 28.5 , ..., 42.75, 43.28, 45.17]])], dtype=object)" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "avg_image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It's a bit subtle, but `fetch` returns a NumPy array of the attribute, even if the attribute contains a NumPy array. So here, we actually got a NumPy array of NumPy array. We can of course just index into it," + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", + " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", + " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", + " ...,\n", + " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", + " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", + " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "avg_image[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "but if we knew that there was only one item, we can use `fetch1` instead to save some trouble" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "avg_image = (AverageFrame & key).fetch1('average_frame')" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", + " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", + " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", + " ...,\n", + " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", + " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", + " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "avg_image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's plot to take a quick look:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO19fayt2V3Ws/Y+e5+vO7cz04/h2ja2TUa0kigNwQJGG+oHVEJjAqRIdISaRgMoqLGt/AEmkoAaBKMBJxQtpnRaPrQNoogVYkxkZBDEQimMgO3Qse3A3Hvmno+9zzl7+cfev/c8+3mftfbpvXPO3Z2znuTmPXfvd693vetd7/r9fs/vY6WcMxoaGq4uBve6Aw0NDfcWbRFoaLjiaItAQ8MVR1sEGhquONoi0NBwxdEWgYaGK44LWwRSSl+WUvpoSunJlNI7Luo6DQ0Nd4d0EXECKaUhgN8A8GcBPAXgFwB8bc751573izU0NNwVNi6o3S8E8GTO+bcAIKX0GIA3A7CLwHA4zBsbG+AFKaW0dMw5wy1Y8f1gMOiO8dl5wG3G33wtPerfpf6UrlE7Tz9z13aI71NK1TbOc83ZbHau+9PjZ4Lab2vjd6fjzvce8ySlhOFwCADY2Ji/Bpubm9je3u7+BoDRaNT9Ls6PNgDg5OQEAHB6enquvs5ms6Xv3XyNtk5OTrq/43ez2ax3rfjOXT/n3H3/7LPPPpNzfqn26aIWgZcD+Dj9/ykAf4JPSCm9DcDbAGA4HOLGjRs4OTnpBpgHH1gekDgCwHg8BgBcu3YNALCzs9P9Rh+Emyiz2ax7kHycTqcAgOPj46XvZrPZ0gOJ68Q1Y0INBoPu+jpRUkpdv7lPPEH52ny/fL5OkI2Nja7dwGQy6c6NScxtRX+jr5PJpLt3vi4wf07xXKKt0WjU/a2TkhclfhZxzTgOh8Ole+Bjzrm7hxhHJyyGw2Hv+cZ9nJycdP2Ol3xra6ubMy95yUsAAA8//DA+7/M+DwDwmte8BgDw0EMPAQCuX7+O++67D8B8jsX4PPvsswCAvb293jV1/s1ms969jMdjbG1tLY3b7du3AQDPPPNM1/7BwUF3jO8PDw+7a8az0jkzmUxwdHQEAHjf+973f2FwUYuAEw9LI5JzfhTAowAwHo/z8fExjo+Pey8wr7q9BmmV4xc4HrhOQF5FefLE+W6R0TbiOvyd+577rQ/ISZXT09PeQqJtu/4wTk5OelI2xuf09LTrW1w7pbQ0afX+9PosUePIi7PTyhQlKar3zM81rs8LrFuI475U2mp/gfmLFi9HPBfg7CX+5Cc/CQD4nM/5HADAAw880C0g/Nzj5bx582bXLoCubeBMqxgOh70Fdjwed/Mv+hu/vXXrFp577rmlPuace0Ll5OSk+z6O/Ny5Lw4XtQg8BeCV9P9XAPhE6eScM05PT5ekbMCp+fwCqXTjF50nexx10eCJHZ8dHx/3+uHMEu6HUwNLWg2rojwGKuni6BYlp/rHGAJ+IVGtxo3HbDbrLVDupeJr6v2GNhL3zW2U4DSAgKrtvAjES8UanS4QvPC4MY1xOTo6wq1btwCcvbj8YurLyr+Nlz9e2tu3b3d9i/HY2trqveiz2cxqvQCwv7/fLTLxXfSLr80LvAoyXqRLuCjvwC8AeDil9OqU0hjAWwB88IKu1dDQcBe4EE0g53ySUvomAD8NYAjgh3LOv1o6P6XUSXWVgryqq7q8sbFRVddVNXfqKa+UvLLq6lkjX5xd71T+EicR/VCJy+frd2wD18hLRxC59lXL0nuO35VUf3efg8GgahKopsbfsUbjTBwn3VQVdn0MbGxsdFKV+Zn4bbQVdvdgMOiZjY47ivP39/c7aR/z9fj4uBt7tud1Xkcfjo6OetrEeDxeMiGBM96HP4vfMV9QwkWZA8g5/xSAn7qo9hsaGp4fXNgi8JlCV8EASztd0UKDAJZJJmWrAyXJ5K7pbEdgLr2czazSh6+vNi1LcdZcVEKzVqF2LrfrGHLXfyd51W520pDvQ8ePORX3O7bjdfy4H8qH8HfKbzhNzUnqAHMw7vlEf/f29nrSNWzy7e3trh9xzuHh4RKpyPfBUjx+d/v2besxUu2H+Y44P+6Nr8fEprYb3zkvhaKFDTc0XHGshSbAnIBKGrZ/1AfKDDm7FmMFPk9wEUtPlti6Kjs7N9qfTqfVIBDHjKtd59xvztZnDiEkTJzP9rP2Z2Njw3IjTvKqdON7dx6XANvK0Qf1VnBMBWsf+lt+Fuq2dM91MBj0PAtubHnc1We/v7/feQdCA9jf3wcwZ+XVBbm/v9/91nljlNdi3ke9IdwP7re28eyzz/bmJrcbbcQ7cB6sxSIQpFUtJgDov8C8CDgVV3/HJgWruEG68PVLkXH8cq+K1HOTPaCBPnyePuTT09Pey8rkmDMb9D5dZBovuqwaqxmj/eN+cKDPecdF70+vw22xKl8juGazWe9Z8aKgrjN2p/KLHPcY6jT7/d3iHwsJRx3G/+NaLh7BBUrpYsdRtLxgOVey3p9b3Eto5kBDwxXHWmgCITE46EZXx1LAjFNxVaPg8zUyjkmmOI9ND12dmchhOGJT++bUcD5Xg12cOlsjefheVDo7d6BToTl0W+/DSXHXJ1blnfngXISqTteIRO6ji4hkKRvnKHHGpgr3V/sRUpzNRjbDoh8cDRpHdde5sZ1MJr3n7gKmnKuX29Ux0vuooWkCDQ1XHGujCYxGoyVJrUTRqoAHbivgEjhUUvKKzaSKS94JqMuvFNDibPtSH13oZ+2emDRyocrKQ7iAHO4bcwcu6EevqQFWfD5rRWqbjsdjyx2oW5dtXHXJOlcoj7cLNKvNJxcgxRpA9DHuKxJ+RqNR91lI5SAUOQgt2uLAINY6Vdt0AWc8b+NazBOECzH65p5xCWuxCABnE0H91szmqtrEGXO1F4xfQuf314k3nU6XYtL5GL8BluO4HSuvE5tj9h0J6BaouJ7LjHRxBbXoSCWJImmL731VKrFem5npmufA5V7wfZbMQE6K4qg5zZZkj47eOy8QfJ/uhdQxYhPDEaW8eALLKrrz2SvcYu7mHJ/vxkjP5byNVeRgMwcaGq441kYTAHw6La+OrphCbZVTYoZXetYwnBsrUCPCWOqqpB4Ohz0/fi1ewLXBWYjaljuPXVzafz4/JMhkMlmqN1Dqm4MbFxfpWMubqJkl3JaT4qp9ODejA2tXrq6Bi+6M3+lnTCDrs2VNoCTRow29dza1XFatmycuGxTwbtremFS/bWhoeMFjLTSBsHPYTtMjr9y1QAiOfef6AAEXh66uGUdUsQ3vuIbSPXE/WXq5CDO36gdU4vHqz+eUog1ZMoX0Pz4+7mlX3N8aD8HQiMuwR0suQgXbxQrmQ3gcNWCGn6PLftQ+uiCxjY2NjliL75wGFnCBOw48l2vagYPLBtXPWNq781dhLRYB4EzdL3kBeILzy+VIFRcVGCjFEPD5rPoxacR95fM5oi/AE0QnoIu8cyG5vJhpIomLi2A/t6r+0+m0I6iYRHVmV6m/zkziz1S9Zq+JMwt4fHScuX2NK+BFoEaiBbh9Zz5E+0w41rwrfM+uZFuco3NiOp1W1fXzRJGyCcJ90EQt7k9LIGpoaKhibTQBYFmlcySTfsaqOfuVnW864PziToVS6RNwPnn+3Em6WsQgH/X+uB31c/N5rLqWNB0XGcmEI6cB8/c6BkrIuRqALG2dRC3lN8T1+T5ZW+FUXmc2qtReRRa6HBPtW5DKjuQspUVz//n+WIrXSD33jGuuci72qtesmVrdNarfNjQ0vOCxNppASEJd+ZzdzSt+iRA572f8HUsmtbFYSjuJze68+L9KGleOnO+vxIeUtJX4O6QLS17nsgyNwUUHOptd3WX6W/1OJbAr/1YKjlESVzMk+Xx+7o5/cEVlzhNE5bgdPsdpGKWAsFVELM8NzQBkbbjGTbh8DOUXWMMsoWkCDQ1XHGulCTC7zRljwHLhDmdfulWa7dy4hgsMUp7AuesCTvtwATOj0ci68IAyN1A7/zyuHi5HpQUwnHeFr6+aDN8LHx33UdImWBtiW9vVH3Dehvi/jnMpmEuZdLat1X3pJCoHIamm48K6c849roalvu4F4PIsXIFZho4V3wv/X+d1KVfEYW0WgXDLRTy+LgLD4bBHGjn1jQekVGiB249rM3iyuwUiwJ/VEmpqLsISAalj4NJeXay5ywWIo77o/PLV1HunSvPv3IsV7UcfNReD++j8+E6FdQsJuzhLLwKTi0yO6pjzvbvIyxhbXjRqKbv6/NmU5HlU2m+C79ndC0PNJx7HVYtAMwcaGq441kITiNVtPB73zAFe2WJ14zrrjiysBd3wNaN91Saceu9WcCfFmSAquarcyjwYDKz0dufFsWbGqOt0Npv1XH58TUcyaeCMi0hkycRurIC6aQ8PD7vn53I6XICSjoeroMuagH6XUlrSKOOzz6QQh4tSdIFSaoYxXMYqa5Gq2XEAmZszjlTWOenqdiqaJtDQcMWxNprA1tbWksR0pbbUhcYrtyP/akQck1gu4MiRXHHUNhwx49xvbt+BQC3TzGW3MSnqXErcj2ijFszjSM44P6R4KTdB78+RhxGyPBqNbOCTI8C4Lb0n51ZzbrQ4vxb+G3Nnc3NzKe+B4Qjh09P+RqquKKoLIHPamysOex6OxHFY0Zaba4o7XgRSSq8E8MMAPgfADMCjOefvSyk9COB9AF4F4HcAfE3O+dlaW4PBoEvc4CoswLLf1ak8Svicxy8KLKtcOnD8AJ26rG24+AbOP3Apq64Nx8Zzm/yZU/1d4YuaWjibzXqTlglEXUzZc+AKZAT4PI1zd+PCz1YXNl5kXCwDLyjaXxYCq9TpOHKf+BzurzMHAmxauGShGtGsz5sjOvn3pYWRj3xvFxkncALg7+Sc/wiA1wP4xpTSawG8A8CHcs4PA/jQ4v8NDQ1rijvWBHLOTwN4evH3cymljwB4OYA3A3jD4rR3A/g5AG9f1V6oeLFS6j7uLouP1SC30rvUXEcClvIVFvcGYDmWPVAioqJdR1rqb/U6fH0XLencgXxN/S3fi44ll3Pj66vpEZJ9PB4XCS83HtzvOGdra6snUV3xDO4z9zfuSWMCuD0dKycJWXuLcTw4OFjKFeBrcnx+LcqT+6gkoXNLukhENq+c9qbXLEWxRl9X1ed8XojBlNKrAHw+gMcBPLRYIGKheFnhN29LKT2RUnqiplo2NDRcLO6aGEwpXQPw4wC+Jee8dx57HAByzo8CeBQAdnd3c2RHqURgW1FXSidteeUrrdLxW2A5EMe5ilQT4PPY5VbKK5d7Xro2f+Yi6fgcNy7uM0fK6T3z+GjxDM4xiPvb3t4GsLwtNmskOkZuGzgmNLVCr9NIXDCNBgjpefoZj7e2z5oAn6ckJxeTVfAzcG5VnX/sAufx0+xBF6HJ/XZuWtUA+BnXai0Ad7kIpJRGmC8A78k5/8Ti40+mlG7knJ9OKd0A8KlV7cxms24ThtK+c44gcvXZnI9f2yqd7yLjlBh0LDQ/GCavnCqnfePEGVcxWcGqo5sMblGM+3Bqp5aqHo/H9rM46qJxcnLS+f21qq5L9dbvo6+qOvOzcGp4oLboObOEx1HHyIUN8/kKF6vBz1jnsktpB/rkKfffLQLOc6Htcn8ujBhM85bfBeAjOefvoa8+COCRxd+PAPjAnV6joaHh4nE3msCXAPjLAP53SumXF5/9fQDfBeD9KaW3AvgYgK8+T2M5z3cTdumUQLlARYBXVi3BxZs16MrKkkb7w23UUlAdqcaSWldi5w50BCXfm5NktUiwEhkGLLssd3Z2AKA7ssrKm2tqv/kz3YKLJbdKSC6xpSQcg9Vrl4TkNAAn2fmo57s2XKQlf859Y+1U55BTwZ27jok+Z8a42Bin+tfMqQtLIMo5/zcAJT3jjXfabkNDw+ViLSIGB4MBdnd3kVLC4eEhgLOVOOwojpAr5QAA8xVZMxFrv3N2Vy0FdTQa9SLNWFK7bDx1uXEGYCC2YWOwJHN8gbPxlfxjjcNFMGoA02g06sYvjizlol3er0AzFx0x61xYrt+q7fHz4cAgJ6FLmgAHkPH5mvnJc8xxA+qq5DGt2d0ukzNwfHzcKwC7KsI1rsXaiiM+Aa85KFruQEPDFcdaaAIbGxt48YtfjO3tbdy6dQvAGdPMkkq1BOfmcTnbbF/qys1SlguDllw/wZhzH4+Pj21gTcBpB+dhyJ19V7P5OBhKj6tiyOO78Xjc4wLYLRhjxZ4A1bTC1o8jg8OG+Z7U9nWegFrWJvMEKg1LktAV9lQpG2AXdRxdiXL3XDgEWbkCtx8ka2WufVcno+bNWOUdWItFgHMHSu6SOA/whS+44IhOPvbr68NiVbE2aVj1UuKMyTROtnETCZi/HLGguUhE3TmXH7IjEvk+1Z3G967JMZG+DQC7u7vdMc7TlyRcucDZ7rtxr9xfXgTUt88mCN+v8/dHm26B0HiFnHMv3t8liZWuoe3qYsB9dO5LNcPOi5IgA+ZjxXkY0Q/+rfbDba5zKRGDDQ0Nn71YC00gVBxeiTWbq+QCjM9DE5hMJj1Vvpa95yKq3ArPGopG1HEaNGsRqg5GG9PptLeRZUrJVqIN1AKUnFajblJWLdl0CW0mogLjyP1m9VPr5TmpzL+rqaIu47KWb6G/0/Ep5QyUIu9cMFfJJZxS6saUzSU1Jd2eAS6aNZBzXtKS+HecFRr9drtMOdLXjUsJTRNoaLjiWBtNYDKZLNnySohMp9OevTMej7uVN4gqJ4nZFnfx4o5IUk2A7VwnsTWTjvuhRN9wOOwkLhNbjkjjc7R9vZYLaWao+8sFLTliKySfc1VGzgfgyc4An6NSizkVfZ5chISlufIPzl3nOAEmf1V74+xEV5PA5a5otivPJdXo2N3JKGlv3K5yTnwtJkW1jFqtZFpgbRaBIJH0IbiUUX5oGsE2Ho97LylPztKmItGPODrSLdrQl2k6nfYmDRNxOimd75ZfDvbBB7R9B35JS5FsCl3sJpOJjY2IPoYZEOPIOQ8uYUvVZd7mLMjgnZ2dLmJRiUQm5PjFZ88MsGwKBZzppLEjPAa8COiRX2o25dzmoPE7zQ9x8SdM5qqHiceTyWhXmcl9Bix7e0po5kBDwxXH2mgC6ussuU2AZVLP1SJwKbMBjbJjqc/nl9x73CcmyaK9WHU5AlC1Gad9cJ84SzL+79RBFxsf0ExARwxubW31MgVradSc5Vkjm1yUostmdPEPTpuIceF+uBh/F0WobQQcaelMIf6d+vP5N+dxM5e0D/0ta2K1LEnuq9NEtf/F3688o6Gh4QWNtdAEAi5GvWRvBs4TBcf/d662AEtWldq1OHTHZbjtogNsR3K7pUg31gTYlnQRZqUIR46WjPO3t7c7W/y+++7r2tD6AEF+HR8fd985DUzHmyMYmbtRd910Oi3az871p2MT96TPxcXPO9ewc7W5eaIao4vCZGnu5onOKw5yCihvxePBn3OBEv0snr/LQFWsxSKQc8bR0dHSw9JqwyklXLt2DcDyNlrxQsVNHxwc9MJXA65oiWPZOf7AvZhOxVVSx20TVgsfdT5+t8UWx0C4F10nQbTFpCG3ob7v8Xjci5bjCaubw3A/ddKzWcWTVCclk6KOKVezhxe0OI9jL1Rg8ELC9+TMhpJ3xcVlOG9MbU6klHoeFF6MlBR18yTuX8+L6ysJyBv1lNDMgYaGK4610AROT09x+/ZtbG5u9iReHHd2dvDAAw8AOItqOz4+7mLwWSqGGru/v9+dF0d2KQHLEVi1aDxWOx3JpOopk5bqQ+a/nYbhymm5xKG45xiX++67rxdrHjg6OurGiqWQbvLK5cUin4B33I3fcu6DxkhomTEeF4Yje10aszMNXQxI5DNoaq7Ls3AktEtMcqq0M09KZgG3W7oXnQur8mdc2rWai27OFdtceUZDQ8MLGmuhCQBngRcqMTiQQiWOszk5mMJdwxXdUK2Dd8RROJcO/81uNS1a4dKGXbSatunO39zcxIte9CIAZ5rA9evXu+sruXd8fNxJZrZVXVksDSByJKAL2AowuedILheoFXCurtB4HEnMXIPLXYjruLmjgWaOJ+Dn6VyQqjHws2buSu/XaZ21cQkw0ee2cdNndXp6utJN2DSBhoYrjrXRBNjWif8Dy3aahopOJpPuM0bJHmJpVAsDZnddLSCHJYjLSSgFLTlXVOkaeq2Qijs7O523JDiS4XDYjU2My3PPPQdgbsNH3zhzMDwLPN4ax88hqeox4EAVzo0o3Q9zKvy90xiiTReAo+eNRqPuXvS6Lry2VPqsFHDETL3TBEvFQhnOo8LBWW7PSsc/qJbKwVOqdfA+EiWszSIQKPnnT09POxWX49bdLq+qfqvvnM9n9YpdORqL7YpGsDurtgjovXG7tRwGvo5rK1ygt2/f7vqmCU9qFgDLbiRdbLmQRS0y0m0mouYPk17sW68tdgF+WV3uhfZja2uri3kIQjPGZX9/v5fOzQv9eZ6BS3d20ab8ErooSHVtspBTM8ClubObm00Al6QW1yslpnV9qn7b0NDwgsfaaAKh9pXIEV5NXe4AawTq9uLVWUkSDlRhNUwDjlxl4YBLVWVJo0E6JbeTahMub8Ih+ri1tdVJmJI2xP1hNZKJTdWc4siVcfk5hWYR7cb50+m06xvfu8vQ0xRiVpNVegJnkk5disCZ9sNZkOE+jO+cRsIBVQE3d1gCq4bh3J41QpjnvJsnbq6o9sFBS25+h6u8hKYJNDRccayNJhCrpLptXHYb255a2dYRck6Ks9R32X0ulDigJBZLj1Jed1wf8FK/Ziezbc2SJD4L6TmZTIpFJHZ2djrJGLbztWvXOgnqCmU4uPBYLpum965BPas0L/0d58MzZ6PPh0k0nUNcIZolpkpq3qFKtTanATpJzfepmhw/M8ctOU1AJbzLpeD5pP3gd6SE52NX4iGAJwD8bs75K1JKrwbwGIAHAfxPAH8551ztRcTBO3bWeQmYtdYXnfe6L1W84Xb5xeeJqA+LycNYmJg4O0+EGX9XilHnz/g+NKae2+OXz5XFBuaegPvvvx8AuviC3d1dW+Zcx5R3Gdb2T05OepF6ceT71UWPwVvMqbo8Go06ok9Tirk9Lm7ikpy03xx16Co+B5zpxC+kkoU1VZ7Hlp9/ydzj72rxJGwO6MLGAqSE58Mc+FsAPkL//24A/zTn/DCAZwG89Xm4RkNDwwXhbrcmfwWAvwDgOwH87TRfjr4UwF9anPJuAN8B4PtXtRX+ZpXALo7axWw79VQ1AF6J2UVXW7GVKHKuInbRsOuqFCfA7fM9qb+dfx+rf9wbkz2uDJWaLJw3wWShq42ouRSu1j1H8TkzIM51lXG1jY2NjaJmxK5hvheVkCcnJ73UZx5v1Q6cqr2xsdGZDi7yzhG22gb3y0Vj6jiyCazPgseK57lqyTyf7oUm8L0A/h6AuLsXA7iZc44RfArAy90PU0pvSyk9kVJ6wtnPDQ0Nl4M71gRSSl8B4FM5519MKb0hPjan2mUo5/wogEcBYHd3N5eIMbdyO1uTV0VXGz+gKyVHsPFKrGRhLVaetQmWCM59GddW3oLtYseLxDXZZemiw0pBIywpQ4vY2trq5by7MmQs6dUVxoFSPB5xVPLURd6xJqCaGu8oxePjAnciOIg3GA3os3BBOpubm708BZ4TzI1EP9wziP/X+CGehyrtnVZUi1wttRvHVcFZd2MOfAmAr0wpvQnAFoDrmGsG96eUNhbawCsAfOIurtHQ0HDBuONFIOf8TgDvBICFJvB3c85fl1L6UQBfhbmH4BEAH1jV1mw260pd10qCK0/AwRrOY1Crg88rtgtLrQXuuDhxDRbiABhXV8AF6ZxHIjhtQplhN37D4bBXQpwltXNjqXfg6Oio0ybcWKoNzOG9AdZg2I7lLeh5zBg8VvE910iIvzkgKI4qDZmvCLgxDY7APWNXcLQmdR3bz7+paUhOq+BnXHJtcnslXEScwNsBPJZS+ocAfgnAu87zo4gRdxFu+v+4wclk0puozv/KA6Qvh4vxduQYPyhdBFhNdrHjtcQTB/Wjr4JbjGox/uxmVOI1pdS9RHt7ewDOYvC5hBf3VSP6eBK7cmTqJnP3zsd4Edlk0BwJVyGar6NjyeZXYDY720fAEcJ6f/EboO9S5nFx7kA2G0uLgIuudOaD/q39vpRFIOf8cwB+bvH3bwH4wuej3YaGhovH2kQMhuro1FhgTto4CRlSykn7Wqklp2I7ItGRdWo+sAajaj6f51ALMuK+Oq3CSZpA3HNIysPDQ+s2jDTkiCLkTEuNz+eMPpbA0Z5zr+n+B3yvbh8BR8iqWXV0dNQbL84j0f67jT1dWS9uw7mmdV6xVqgarHPvlUho5xqMtnTu8Fzja5cC49xW8L37qn7b0NDwgsdaaAJs96kNxMEvumEj16sP1OxM/ttJYJVQfL4LpuG2HCFTc+eoFGJpr1KxFlbKRxfyGyG9ri9s93LGYDwL3isQmLsWOUcfmEtlJwUDmuXHQVRBMnKZeL23II35fFf6jQO2eN+++L9mKXJ/WdsrBYmx5sXPTrUDx8/w3DlPWTn+nUpxvk/WWlQTqLWvWItFIKV5OioTIVp0Iefcsb9cNScmOavyJZ8qkySrWHnnFYijW6h4swfAM8erUHtY7gVzC4+aL/y5JtRwnAD7ypkRZ5ycnPSY99FoZCs4x9GZGVqNuFbTkT1AzqzjF8FF0sVYaKLZ6elpb38H5x0IOE9HLbqSyUuX78HkqfaXyUXnSXFkpC7EjkgvoZkDDQ1XHGuhCQwGA2xvb+P09LSnygUODw87TSBcV3t7e91nvMLWNIGA8+uyaqdqHpsAujrzb50P2Wkf+p3eAx+du4lXf3YLar9ZzQ8SMGoTXrt2bUki8RE487drYRDtY6ma7dbWVnctHh+n8ehnLLlVzXdRcM6PH1oi1/tfFVOv4+G2eGNTRc1WF+fgyEInsWupxJwaXJvXtRiTEpom0NBwxbE2msDOzs7Sysp76AHehnMbTTpChqFS3PEQjsCpxXG7LC5uVyWls+FddlgtX8HdZykAJo6u+IfuWOSy5ejGF3QAACAASURBVNw+BXGdo6OjYh7/aDSydjdviBr91nJu0ZZz5TFqrkSO3nTkmz5HHovatVg7VOLOkcWqVcT3cSxJa37Gmg/D1xqNRsUK2+7eFWuzCGxtba1MdNDNMzc3N7vJ4iZNoKYanTchgyeAtsOJJOwHdhtexFHVwtoiwP2pReMxoen6qJGAg8GgKzAScBWFOaEoSMN4Fhy1Gep3eA5cLAP/HYvA5uZmLzmHx87V/dNnzC9dwL2QPB76cvCzdaSoxg7w+W7hduniLtVXTTG+tkYwusIxbkHTuVRDMwcaGq441kITAM6kmEpNTn8NScOJIloN2KmPKpEBT6Y4wk9/yyq3K3PlyEVdpUtx5SUXl8sJYFcek1GqwnMNwfiOzQGVZKxhaD+m02kvxZa3MndRa1psZTQa2W3CXD/iqH5xzhNgMjA+cySnSlQ2S1hDK917zrlX3ITNI+1/iSx18yRQIwsDTMTGeW5/B75OcxE2NDRUsTaaQEgNl3oKzO3Y2FKLo9WYCwB80QqnCayy8dXuYoJLpXgpOozb4+9Go5HNUlRCjqWWk/DhfgvEWABnAUEc96/lxVzmHW9NHueH5jWdTm3Mflw3zuex0/PZ9cvlwFRCsjTXKDh2j7L2UXrunHnH0DZ4/DRjsBSkpS485yIOOBeuywthjcBpNY5o1nGuaROKpgk0NFxxrIUmEDHSbI+GVAnpz+WxahKEWd87jcF3MeGucEYtmIeDlvR8LuHl4v5dJp0GsQDL4aVx1GuyxNEyZ9PptMsFYDdgaBGcgQj4jT2n02kxe489JIEIDItrRRulopyrymO5YC8NGnLhwKWgr4BqEyW7usTKu9+UcgGU73H36zZs5XuK+RTPjvkOlxG51Hb120vCYDDA9evXl9xBbjMKfYldUkfNrcYEVI10q7mK2Nzgl+o8lWJXvdS6Eairrxg4PT3tiFJHKGnF5cFg0Kni3H60we3GeZoHwS81u2bjujERV6miMVFjP4GTkxNb5Sf66iruuIQdfRE5bkHHiJ+7Lr5xr9yWuw+XcOTaYoGi5oMjsl26OM8N7QtXjXL1Ids2ZA0NDVWshSYwHA5x7do1nJyc4ObNmwDOioVwTTsl6ZwKDdTVfkf0aHAMk38uAs+RP+q24Wuoqsu1FDmaTLUPJ8m4H+fJkmRTyEVBalrvbDbrZURGW9euXetFGAJn0icyBgNM3MZ9himg46Obn8aRtQQOKIrfcnaiS7GNzx2Z6+C0H2A+ZjouXAnZBYQ5klO1PIbTNpzLkqMvA+5agVUl/Zsm0NBwxbEWmgBwFmYZJBQXtwTmq6RqAuwqci4/dbnp38ByLQBXhVfrGrAE5uAU57YpBYxsbGwsFbzQfqmE55Vcfxf9jWNpByK9vrahUlT/jvN1bGtZkEBfovJ58awdeeV4Bd1hiM8bj8fVkFnlK7iUGPe7lHvh7Hku5qF9zTn3yF+gnmVaIxldOTK+ZolLc+HRirVYBGazebnoyWTSDUhM9pgowPLLDyynVbpYABdtFeC4eN2Zl1lcfZk4ao4TVFRFY9+0Y5q14i77uXmSaVtu4XGxCa6CjVOFdVFkc0f7c/v27d7GHvwyherP3zmTRUlL3g1YX3guc87pxVogJaXUu76L8mS1WseUfxNw5CxHqbqNZeI6mvzDnhSdtzpGOn4cp+LI6pr5WhJG3fWq3zY0NLzgsTaawN7e3hKZEgQSR6PV4q5Zyjk1Vs+Pv7e2tnpkF6vVLtOM+w0sS0NWqzkugOEkJLub9J5c/Dy7uFgL0f6yWaNmzGw267klOcNRtQku9cXjo3s+sJbjog81h4ELh8Q1uTAImw1xjhKI/L3eO48Vj7uLt1fzxWlqHKWomgBDNR52B7JWqJocmwca4+KyKl2+B2txq8yBpgk0NFxxrIUmcHp6iv39/aU4aicVnaTRIBOWZK7qrErPra2tns3kiJxYWV38PBfscHnfjhNgPiE+cxl3wFzaqduQiVInhVyko3ICTIBqleK4Lo+Vq5vAefOaQ8ARjJy5qIVD2A2oROnR0dESERjXdhqPunr12TGcBObnyNeK79Qd7bQ31j7cBqYl253PY83EBYm5QDoXJanXLOGuFoGU0v0AfhDA5wHIAL4BwEcBvA/AqwD8DoCvyTk/W2snXgpWLTVtmEkYl4LqEkS0zp6LwBqPx/bB8KSN68f/NV2TXw5WGV1aZ/xfJyC/yMoWu3uZzWbWZNEFhE0RNaP4O/Z3q9qrBCGwvDDo4szkof52Op32wr+BM4JPffG86PKz1hLivMCrGeM8NUymscmiqem8KDkTLuCS0BzL78ZSF2cePw0lZgGiC7ieF21ctDnwfQD+Y875DwP4YwA+AuAdAD6Uc34YwIcW/29oaFhT3LEmkFK6DuBPAfirAJBzngKYppTeDOANi9PejfkehW9f1V6oz7qS8f+5oAZQTtekPi5951Z/F+HniD42MZwm4IpQqObC57iU0oDzFzvy0kmVaI9V8vhcNyTlKrzsmlU1VmsT8j0Nh8NOrdeNTjY2NnoaFZsDrNayxsfX5nP42WoBE5fizc/WlWJTEi3n3Kvlx1LZmV21CEQXp6Juyel02pPUrDnq+aXrqNta51wNd6MJvAbApwH8q5TSL6WUfjCltAvgoZzz04vOPQ3gZe7HKaW3pZSeSCk9oTZfQ0PD5eFuOIENAK8D8M0558dTSt+Hz0D1zzk/CuBRALh+/XqeTCZLEWlq77oAHl4BWSvQFd7tOsOSRstMsY3f3ayJ8a/tfsOSQ0lA/ru2YnOmnpPiCtYO9NpbW1s9baUUqRfnafVg5mXCFmcehz8DlglN5lScbV0KOGL+hIlT1RTZ3enyCtz4qrZXciVGmyrZmZNy3I6Cz9eAsLgvbqsWSMTX4rmvbkmXzt1rs/ptHU8BeCrn/Pji/z+G+aLwyZTSDQBYHD91F9doaGi4YNyxJpBz/n8ppY+nlD435/xRAG8E8GuLf48A+K7F8QPnaS9cXioheZV2edQqxQHvKQDKee4akOGyE52bKa7NocTODq0VnODrlHIeOItQ8xz4PC58qTbidDrtMe/uWrNZfwPQWnAMM/XOQ6KBQaw1cR9ceLGOmZNuLndBXYWsIbH0170OOMDLFfjQ58PPTMH9dqHbNS9BgN3AAZfD4PJZarkoiruNE/hmAO9JKY0B/BaAr8dcu3h/SumtAD4G4KtXNZLSfEPS4XDYqTOqirKKzi+OplA6kiTg1CKeDC7JpeT+YrCvnN03SkLW1E73IrgYglobLpqM23RxFnq//OLqS8KJKudZHFn1d9GBTuV3ppmOB88Fl/SlR2eaTafTXlyDu5Yz4QLOtHHuXf5OF13uty48fJ/OvezaDfAiuarG4F0tAjnnXwbwBearN95Nuw0NDZeHtYgYDE0g59yrHuwkGbuRdFciPs+pna7AgnNHlWLCS9F5Tt0sSSaWtk66OWKo5jLj71ST4nM06pCj7GpwLkjWlEo5D6yGK2Gl96f3ye0r+VZqoxR1WHL5llxzfA/cH1eUxZGFcU7tXvg653Hjcb9VS3Gmh4tOLaHlDjQ0XHGshSYAnJEbTkICy+4pjtNX+3IwGPRW/Rohdx5JyOc5+4tt9pqbkVdwxwmoXexWcw4GCaKP7egIunFuMicRdBej09OzfQRUM+Hz+DnpvbvyZU4Cs8TW587Xq4WGB7iP6iLma3HItJLQnOXnOBjVSFwOg+tjrQ3Hy7hgIT7WApTc+auwFovAbDbDwcEBUko9koYnmPvMqWPnIdiYyFMVilN3daLwtdzkcXCRY9oGP1w9h1+c0suin7l8CI055xeB/fQa0eeIRB2fUp+0P67AB7PbNULOxYxwXoiOqSPfuDiMi0moRea5eeWKeMTv3Rg56DMtlbXXtpz5EuDxXnX9Zg40NFxxrI0mcHR0tORzLqVoAt4HyjHeuqLWiks4KeT86DWp74gZFx3G11btwGkJrDa7fQ0cUeW2E9f2VbpwH7mwS8C50LivTkrFd3ot9xzdeLAkc2XUuNyWtuekud4Tk6hMNJci//jenWnjfPxOa3MmYm2Mas+Kz9F+8O+aJtDQ0FDFWmgCQN8l6NxOjvRwgSGl2GuX7eeCkLgftRhydx0mJZXAqbknWSPRVd3Zp4404kxB7Q9fO845PT3t7Uo0Ho97bsCIHDw+PrbSSvvL7lIXjenG2z2X6KtqAtyG3hO3UYvsY5vdcQfumTmXnwvm0fYdHNGnGgxre6XfAp6g5LYuup5AQ0PDZznWQhMIFxhLQw4zBZYlNts7zi4uSVInQRwLPZvNeu4uV2aK6xtoSSuWIMquc6l0x4artD0v0+x4CM7T135vb2/jvvvuA3DmKnRjw6XenUQtudX4eTI0fJlrAWj4NUuymjuXA5NcmKyWSuf7Ys3IeYP4GvqdahNOaytxRqU2AszsO02qttsQa2Cr3OBrsQjMZvN9Bziyy0WYBVgNci+6I2ni6FRAVTeZHFP/Mn/HLkVXj09JS+6H85WXCKKaK4jhVO3ow9HRUa/QiEtQ4d/oojsajXqf8eLi6vOpeVLqt778rj6gKwnnNlBxC49+5iI0ObVaI0adSu3mmlPzeQF3MSCl8XZuYwd2izt346pn0MyBhoYrjrXSBDijzxGDNTcTr9yqOtfqvrMLiqVQKe6/pJ6qys/qaS0YpBaNx206s0cxm51Vy9U0Zo7xZ1OHC3Vqv5XUc2nSHFilmoDbQty5WJlsrQVd8b24jVFVk2JtQjUjTot26cUaWFULXip9plrEYHC2nwXPTReJCMw1Hj3fRVc6rYZdxU0TaGhoqGItNAHn/tNyWm6F5Zj6AK/6jrByKzb/FljmDoKYZMmqEpulJ7dVIqrYpeOChLTQJwfH1MJ0WSKoFsL3FDg5OemVBCttdAoshxkHkehCcuM6bremWgAPg7Umd+9Om9DzXN6Je8bcVk1qOhd1jcdR6cxS2RGfOuedJsDzm+dVaV/K83ACa7MIxLZjOinj85RSz2NQqiakL0KAJywPqouyUrKLTQzH4q7yxXL7bNrwguaqEgPeV+7ILyaIAky46cuZcz91ezQadbED8aKzqaAbgbp4AVeJiPuoz2AwGPReGDZ/tK4i94NR8iLwC+mIPjd31ETUe4ij8+5of2I8x+Nx9TzdK4K9TkxM6watvACqB2NVQRGgmQMNDVcea6EJDAYD7OzsYDbrb6PMBJBKZZdKzESVEltcpZbJlfNkcfFR1Xx2tbkYBr7PuLYjwrQ2Hv/f7URTy2HQ/pckn0oK3qo9tDBXtdcV8+DCLsCyFOV+q+rsxpu1It19aXNzs0f6uh2fXPSm0wBrfvSa67kGNmNiXFLql/riupCa9zEej3sEaEqpR/oOBoOlZ8TfsclXQtMEGhquONZCEwD6RT4c2aQr5WAw6NlsbM+rDcznOynBbbisPcDHhDuCku1cDRBhVx7ftwskif+7oCgXoRfQ/pdIOiWjdnZ2sLOzA2BZgkUbSlRxsFAt85M/0/vk8Xf7K6o2w9KR6yDoODB3VHKpMtw5zD3oWLImqloNcyul3JJo0/0WWA6KYiLR7cmhmlFgVQ4DsCaLQMQJAH0yhZnTIKMi1JVVHVbRtYos4zzppm6yc191cWGTgh8yJ8Ho+brdlv6t13SRabVoSRd9xu3p+Tx+LvQUWCa2ov3JZNKNt94vE7E86UuMuvaDj6XxYLXXxQLwEVgWCC5E2PVJ++0EAoeQA/MXmYVV6T6dOcibs/I4x/mu/mZAvQ8tYrChoWEl1kYTiKIimtQR5NTm5mavpt5kMln6G5i7GHVLqwCTei52wLl+nLtJYxlcnIBL4WSJrb7hVf5/BUuygJNoLLVqWkGozrdv3+7FRrAZpgQUaw7Rbjwzp7az+cAakrp1eX8AlXyTyaSo/sa96lE1P1a13Vxw0lPnBD/fGJeQ4kz48YYxauq5xDgXjRlt8PbpvIenPisev1XmQNMEGhquONZCEwDOiD+XHgmUs+fUFufoMGcTansu4o2hEsRJWyb1+H5Ycum1XRahtss2q9sHQW1rRxYGXAqqC5KZTCbdParUB/pjyeOnrkV2GbJ25tJ6azwBFzXh/zNc3gmPgc6T0WhkA2xc4JiCn13cIwcEAcu2eIzH1tZWj1PhPjqOR3kixw85zoNJxlo+BnCXmkBK6VtTSr+aUvpwSum9KaWtlNKrU0qPp5R+M6X0vjTfoqyhoWFNcceaQErp5QD+JoDX5pwPU0rvB/AWAG8C8E9zzo+llH4AwFsBfP+KtjAcDpdipdVe4y3EeVXX+gOTyaS440rJZq6xvo71d8yxW7FLLO7GxkYvzp2lkHOTRvuuKGYtZLrk/eAjn89ait4nu8S4jxzEE/cXbZXctfwZS3G9NmuEXBhUg8TYTavam+METk5Oelon7yRV87w4TYAZ/WhLx5s5L5dXou5A9m654DIeK/Xo8Jy76NyBDQDbKaVjADsAngbwpQD+0uL7dwP4DqxYBELtce4p56cNHB4e4uDgAACW3II68fjh6eAzWcgEjbqmlLzR75xbSomhAKuK7mXVazg3j1sE3GLgVEH3nfutuul4oWJVn/3mwPIOxDqObPLxYq2LvksY437oM2bzy+UH6OI4nU57AseRrTy27L+PMdAFMBaD4XDYGz9+qdkEcYldcdT7ZKLUzWvNBTlPUZc7Ngdyzr8L4J9gvvPw0wBuAfhFADdzzvF0nwLwcvf7lNLbUkpPpJSeWMVeNjQ0XBzuxhx4AMCbAbwawE0APwrgy82plpXIOT8K4FEAGI/HOQgjjaPmbbJ0RT04OOhIIlYFdQXm1VwlAks3/k774VKaa+p4SqnrexBDAZZanA+h2YMaEMP9YbW9RiSxGuwIpZoW4VRo16caUeXcXqpW18iuo6OjHjHI0pOfu2oRNbcrmw+uUjEHSAFzSa+uT85wjGfNmZfRPmur+mxZW1LTs6QJqLnrittwWxdJDP4ZAL+dc/50zvkYwE8A+GIA96eUYlRfAeATd3GNhoaGC8bdcAIfA/D6lNIOgEMAbwTwBICfBfBVAB4D8AiAD5ynsSCHNFiIV7Swd8LmPDg46K2Kzl4MuJBLxwm4eGsnMVlaqPaRc7baQ1xbuQmWhmqzl6Sc4yFK0tC5y1ZxAnrPnM3IGonLJ9BrsrSNvx3RF4hnvL+/30nSGlHGQTd6ZDdmwIXwcl5DPLuQ/teuXevC1ZnPUc2hRuA5sphdsqqBuSzM4+PjnsvZZc7ynF61NfkdLwI558dTSj8G4H8COAHwS5ir9/8ewGMppX+4+Oxd52nPFZpgsGrkCCVXUVhfbn756D66v2uVd5m9dtGEqvbytUKddWy1u5aqb5xAxEc1j1hdd6SXM4W0/zW/OBOUTIRFwpEuArwo8cKgpKhTiWOiHxwcdHkl/LLqgum8PHxUr4OLmuTkHFXvt7e3ex4AfsaxaDFBrX3kRYbnle4C7Uw4F6fCHqOSyl+qKM24K+9AzvnbAXy7fPxbAL7wbtptaGi4PKxNxCDgV+faJh4bGxtV1fY80VksQQKsQmmUHcec88YoTt10Kn+pj3yeRtSNRiOrbjpJo1pKLYbAuadqLsW4Rmk8AlwUQyUlg6WikmLOHeie8Xk9S87UchpjLWJQx4aj8VTrc6q802BOT0975fJq7mJ2DZdMGv1/yyJsaGioYi00gZxzt9mlrmSBkq2qAUH8t2tLSa+4Nn/nIswcycQ1/nUHIgdXKMNJJnVdDQaDzgZ3EWZhqzJx5iSIcg5ce8Ht+ONsa2339PR0aRyAOYkGzDUB3deAOQEXnFUiuPReXGSkCxIqjYf7zhVvCSk9Ho+XcvS1TY3y4xwWDtxxmoPLLVHwOKj7kvkeV77ss6LaMENVHUdq8ITRPeadzzngGOTSAqELiVPfa8STY47dvbhFwIWnKmvNixEvGqqyKjGn7Wu0n4s14Pt0qdVcqRhY9q2ricPqKROI8SLokUkvF+9Rmvx6nwGX9MXVjJ1XKo7OvON4Br6n09PT3hzjZ8CflaJRHYHMEZc8PxzZG/emtR8VzRxoaLjiWAtNIOKyV7lyArxKO02gJr1L/n8GS1kXMeiqCDuJpO467oOT9tpvJ/U58k6llYuIXLUHgJbFYglWizRzppZKRZZkbudcNkF0/GpmiTMDXdw/j4WOLVcDZk1BVX4eR+eSLW2cW9JgXZxKaE4uTsBpk7UEMAUnq5XQNIGGhiuOtdEEXJVZoJzuCngXIa+eLujGRZXV4uedG0lXXY4E48xBrTbLbjJXWVbdPM49VXNZnRcsPXXc2T2qUo4lmZYUA/rSaDKZ9HbV4fZZ44nIPHdPSty6vAnmcZxW5qI21X7moCXVkJh/YnteXcnsunSFT1UL4vR51d54bjoy0kHHb1W0INA0gYaGK4+10ASAs5BN3VvAheYyK66S7OjoyO4fGOcoG+7a5bh8F8rr+ArHZZQyEE9OTnphqWw/q73NOeRafIP/dq4gl9HHfdVcAKAf+8+2u8sLCGj/S33Tz9g1V9O8WEI6745zDcb5zkOj88M9T/5O+8YegwDnt8R37HGI8Qut8OTkpMfe8725DUmdhqveAdbePmtchIPBYKmard786elpb/I437CLkV8VHahwqiWrmI7Uc8UldENP9uuG+hvf8eYZNdKtNHal+wuUJoIrUMHjoJ/VXj6Nu9/e3u69mC63gxcXjsLU6zh3Kj+/80QPukrBzjzSRDYmbmumIVf5DfDcVJKOC+nUNozhcVdzhwuYOFPVRWsymjnQ0HDFsRaawGAwwO7u7pJa5FZYJ7U1ss8RVbWgHnaJOY1B1UgXnz8ejzvpFxl1Ozs7S6WmuD+j0aj7jt1OJWJwd3e3t/q7bDJWTzVAhb9jLUfVR5Y+TvMKsJsqzovnF2PAxVS45JhmzR0fH3fpwpExyOqse7b6fJxUZujcKWUiapGQ0N547ji3oc4TJnq5r9rudDrt5aew6q95BW5zWp4LjmAvmUnd99VvGxoaXvBYG01ge3t7qSSTCyhxIcWO6HOhsIB3LbErh6Wt0wCiDUew8b0A85U+JKH2h4N/SiGjwPKq7rLm4m8uLqm7L9X2XihlS7qMxTi67dOZtwHOpD7fk6uSzPa/tsvPx9nxjqQraYUl4tFJyFLYNZN6XNla5wmHIDveQuECn1xNCn4+riK3zkXm1i60nsDziXgxSrsSM2kT5/CD54mnD8SxyjzgOun5e8f664NnYtCVnHYLj2PZSwsPg4kr9XO7qEA3Abkfrky3VgNm8lDJK0dUBTY3N7vzmDjlTTv1d1xRCJgXFaltLMux+iUTjoUF9zX+5nuKew6zJNT3wWCwZLoFdE6ySediE9grEP0peUZ4zgccec51G/k8YD7eqxaBZg40NFxxrI0mACyv5rEC84qpqyLXZ3NulYD6mfl8F3XIq7OL2df9BHZ2dnD9+nUAZ2m0m5ubVgOI+2QJBvgMM1Wz+ZqsKgZY4rlYBiUBWXNgKVRLo1bCkdtnDS3+rxWXr1271v2GzZjob3zGY8ckbnzmtkMLuLiPmkruTENXqVqJQZb2zvVcy2IMsMbooJoGpwazxuMyCwFf9EXRNIGGhiuOtdIEuDhjLffeRWzFare1tVWUVm6bKW63lFEILJfM4msBcxdeVKIN9xjneGvs+/HxcY9k4siumguIORCnYWjQCEtzJQ1dgUp2tSlZx4FYTHIyGcZjllLqNIDQlHZ2dnqRdM4N6DgKtqe1H+ySc/ekUpmDdlwAVCkbk8HPWH9X0jyU/HOSmjUpPZ81TEdQKt/CbZSwNotATGAXehrfu+SIuEHeBy8+i8Hkyjuqhjv/MrPxOmk44YP3otdrsmkT14qX7/DwsBdZNpvNuntQFdNFyPHEjs+m06klnrQN97K4RVG/K53j1O/4v5pOPOmdX59DpYF5GHiQdJGizJ/FmLIqXyPfOKFJFxLujy5o3AZDf1uL3uR5xaakEqTRf15gNeqUrzWbzbrPY05y8tqqRaCZAw0NVxxroQnMZrMudl4JmQCnefJqq+7Ara2tnqRhtUmrwToijIkWVYnZReO2nOIEH04YArAk0ZSQ45h6LcThfPHOh3x8fNyTggGXAMXqrDPD1HV13tRW3e2Xr8/RlXyeJoyxdhPPjDeduX37NoBlTSDG99atWwB85V9HFroiHnoPjtyrpZ47bZI/Y81E8wnYDewke4CLkWjEKpsFzUXY0NBQxdpoAgcHB0s2mboDXYEIltgu9djBxZAr2NWmUn97e7tnp3EWF0sEJZVYorlMOrUvmQxS8g1Aj4dwwT8BF5TELj/WglQas4aiLiiWhro70fb2dieZWKIFSbi7u9uNcSlQajKZ9DILJ5NJF0zE937z5s2lfoRGwKnYLuuwllvi5hNLeHUp8u8d4ajaAWuA+uzc3OT8BtYEXJFX4HnSBFJKP5RS+lRK6cP02YMppZ9JKf3m4vjA4vOUUvpnKaUnU0q/klJ63ar2Gxoa7i3Oown8awD/HMAP02fvAPChnPN3pZTesfj/2zHfmvzhxb8/AeD7F8cqcs44OjpaKqLgXC0aLsnsM9vMnFXHx9ls1rPJnPbBocQuR17tYJf3z54Ix1bX8hpUMrAtyYjVP75jl59yDi5kmnkLtmVLmoD+NsZHpU9I/+vXr+PBBx/s/o5xjPPDrbq5udnLlitpLjpWAZaooSWE5lXKLnRx+W4+Act8QU2yOk8K5/OX3MCuX9PptBcoxV4Nfv6qhcUYs/ZWwspFIOf8X1NKr5KP3wzgDYu/3w3g5zBfBN4M4IfzvJc/n1K6P6V0I+f89IprdDHoTk0GluOoXeosEy36wjDZyBF3wLJqzhO7pPY6Qo6vx754V6OPr1NqI8B9VMKU+1gjqpyL0Km/DkoCutwONskiJuD+++8HADzwwAPdy8/uTyZqA87lB/hYELdB62Aw6BYVrWG4t7fXW1y4PiCPmS58/aZ5cwAAFkNJREFUbq6xH780fkw88lx2BKKSkHzNWECYxNR9HjgfQ58PL44l3Ckx+FC82IvjyxafvxzAx+m8pxaf9ZBSeltK6YmU0hOrOtnQ0HBxeL6JQReVYEOncs6PYr6VOUajUS6twC6Vk1d1XYl5804Xz63fsXsvMJvNeuRfgDWHpRs3gSEBlULu96WgpfhOC3G4+2RT6DzBVu5aHOSi9+Ri1FmbCE3gRS96EYC5CaDur5xzJ92CyEspVTWB6DcTjloGbDwe48UvfjGAvptsd3e3R87u7e111+S+uSCrgCvo4oJ4Ampycq6G27ZOTbOUktXeVLPkWpsxfmp21HCnmsAnU0o3Fp26AeBTi8+fAvBKOu8VAD5xh9doaGi4BNypJvBBAI8A+K7F8QP0+TellB7DnBC8tYoPAM6kibO3Xew7r7Caucb5B2qTu8AgV3UY6BeJcCG857Gn47p8LSeRWavR6sRMDLKkcuNRCvll8L3XQn0D3C8tmZZS6pXM4v67bDyVypPJpCsvpryJGyt2e3HYeGgicX12LUb7TNK6Ap8aiu20NtYINLgofs/x/A5O2mslaSZu1QXI5x8fH3fBU6H9sIuwpgEC51gEUkrvxZwEfElK6SkA3475y//+lNJbAXwMwFcvTv8pAG8C8CSAAwBfv6r9riMbG0spti4xwxEtAfYhuxc8zuFad9Guqk4ussvFu7sFhKP3HEOv9+JKQ+siwCojM/eqnro6jC4SkCePRkS6mHfeiENLpfOGl7qByHQ6tQsK51AA8wjAeEkDXJVJ23Ami3uZ+NnpwsMvMCch6eLizAL+TnNAdCMRHj/+LUOFBC8CzlOjROLp6elSrgWAbjxXbUEGnM878LWFr95ozs0AvnHlVRsaGtYGaxExGCs7qz8qxXnVDbD6q4RS7VrAsmR32VuOeFK46DyOTdC+8IqvUshlqbFkqxUJcRWFA+xWK+VUxPfRD3Vbsqak7kCuEq1prCyduc8hrcKff3h4WKyN7zZqTSn13Iyz2awjBNXs4nnFz9URbCqV+eiKrajJySXNHClaSzXWWpTD4bAX6zKdTnukNWsMMY7PPfdcdx2nfTBa7kBDwxXHWmgCbN8GlBjilZM/01z9k5OTYqFOboPtKV1tJ5NJz/XDdrlKC9YmtIQXX5clt7PVXRFPbYvvSd1HzGW4ICqNUnPBK/yZ40o0QImzAlk7iN87Dsbl8auWEr87PDzsEb3T6dRuPxcuQg2c4T5y3QkdX977QfvNWaz8/EvlyJiQYwmvGoMjOR0nwNJf5yYXE9WoSa5rUULTBBoarjjWRhMIe6zEbrNtw9KulgtQc8kxXO0AlQgsgbWmv2PU3XWdy8+5gxSc7ee0Aqf5KEPO8eUhVdgl6/rhJLx+5oKWWALHGHBVIC235rwxLFl1XNwW4hxaHeC2lD2fTCa9cms8HqoJcHn7WlWqwGw2s8Ft6jHicdB+D4dDm++h3NhwOFzaKYmvyaHZJazFIgCc+Y9VharFufOEZWLLqevxnZoXriIuD7QjHJUM4lRfdv3pi87/r6ny+uD53l2tO5dPoOm33Aa7/Nwk0/TpBx54oDvyGPER8JuPaBLVc88911tYS2m3wLIL0rm7ODaBXZPAmUp88+ZN/P7v/373N4AuYQ1YJnPVVHEvfCm2pHRP7sXn3zuBEMdSHgy3x/kBaha4vSgUzRxoaLjiWAtNIKXUubBUpWONwElxJe7YbeOi7Fzgjgs+ciRQXMeRgLXoMOfec0E0LkAqrqmqtgteYZXfpWSrxHHBRfy9/n97e7vL1GOtRfMV4v+8exC74+IzR47VIjlVsgLLG4YqKRuawP7+fi+9mDU13nFJny3PFx2/mplZ0hxcungpSpGrY3O72haPKe/lEO03TaChoaGKtdAEgOUAHcC7S1QTYHuRSy1pnXpnT/OKGp9HqCXbYiqxOaCEJZNKTy45pjHhnA/vNB0XbnyecmuhUcXYMNj95UKIA1xYQ/vDmgPfi+YMhDRi+59dZ0pasttLY/DZrRZt8Z4LcR4XK1GSmF3Jzj7nOeZ4gmhDx5b7pmASkElOd3+OA4r+qI0/Ho97GgwHYGkWJJ9XwlosAkFacUEQfdGAfh3BUpx7idzhc/lvfVg8sZWscUUaOJrR1d6rFcXQWoZAvyIOxyHwYuPURzU9dILx+Qzn9w9/P1cFcoRgQBduvj824dw2WrrA8/joC8m+b14gYhHXDVUnk0lv7wK3sPLYuEVAow6dN8vlhygJrPeu1+b/u99yVGIc47y4vwAvGiU0c6Ch4YpjbTSBIIdUdWfJoK4iV/KJ/67V8WNtIdpgCVVaPVmKuyxGVv1VdXZ713MbWs+QpYAzT7TQiNNuasSfK1E2Ho87DeChhx4CALz0pS9d6heDiVjV3liz43NUQvLGsjpmjGiLs/045oEzBIGzLMWbN29ib28PALqUW3YNu2et48L9ZtJQx56fpzP5PlPoWJUIStV6nUZaQtMEGhquONZCEwi42HReaTVKzLn3XBw/S1RXODTAK6ySYmx/aYQe2258VBuSIxP1/KOjo57GEJLKFZfgvHzmI9TOdQFK/DslLV2OhAtGYu5BSTd+Thp5x1oNR+epa5i1JhcZqXUQ+PpxfyH19/f3O74gtAPmIVj7ZI2Ir+kyP2tBQ6V6Ai4SVbWeWgAZk5E8ptpvJklLGZqBtVgEmFhzKnwcHVlTY9n1RWP2PMAPgCe2ElV85E0ttf+sttUWgUBMWCZ0Qh3n7bq4smx8Fn+7qEBngsTYsv9ckfNZPcPf+73fW2rrxo0bXSVhfkk0nTteOK64zGpqPLO4ZxcKy2SnEmEuUtSRkQGXJMamJJ9femZuM1snQPiophj3wxGl2gb3jRdOTgEH5vNEi73UwqkVzRxoaLjiWBtNQKP8zkOmMCHCmkMpkovPdy5IVb3ceSU1X80XbsPdi8tvcKof4JN0mCjlNp07MtpQuCQWjpqLNjgJRclZdoU5t13Ane8KtaiJ49TlcCdzfzY2Nmy0XPxfnxn3g7UQtwFpnO/IXJXeTvV3KecBt4cCt+W0IOdejpwOLfHGqfUlNE2goeGKYy00gQDb7Co9eTVnIsdlrpUCWnglZpeVcxuWAj2c/eqi1ZyWwtqOSkvmN9TdyFKI7VK1//g3LkCKI+7iXLWVgX7VYOUouB/MfTjpqeMymUx6EW8uBZrJNxdcFO1eu3ataz84hiitFcfj4+Put6E5OMnuxpmfnX7Gc001Vv6OtbMaJ6USfjQa9biJ8Xjc46SYZ9F2XaFWRdMEGhquONZCEwgNgFczV2jU5Zxz/HR8VwrljGsB3k5n6aMaAIcRq7RghpeZXpdFFtD2ebVWXsFl+7EEYS+BxuW7IqgsiR1rrhI9vtvb2+u0iOjvzs5OV29A3ZM5515dA+YmWPNye0RGm+r2Go1Gvc/cPXMBERfq7TJKdZz5Gep3PHfOEyLsuCMX6s32v5YSc1uNT6fTHh9TC3NXrMUiAJz5j5XUqUW8cUVXXgxcFd64hhJlLrmE1WSXkqsvKVeAdZFuzg+sKi6bNm4y60LC1Y/YZeQqBcXnTDTGdfgeYkzjN9HurVu3AMxfJs0rGA6H3SIU24/xs4jfxgvJz4AJSBcHEf93m5soKcquMCX82FXJL6suAnzvARfjX8vHcPEB/CyU/CstDDou/Dt94U9OTnqxAC4vo4RmDjQ0XHGshSbgVLOAU2XcausyqtTVxqjtuMOSWgM5HBFWygRz32sbvOo71TyOLuU3oBKe+8tkZC0un/dXUMKRI9Q4ZTvaclFqcdRxnEwmvZwHJsziM1aDYz8Bvg4H8QBzEvDZZ5/t/gbOgpYODg5szUUn0RW1vBOG0wQU7prcbsC5nrUd4Ow58ia5mgezsbFx95pASumHUkqfSil9mD77xymlX08p/UpK6d+mlO6n796ZUnoypfTRlNKfX9V+Q0PDvcV5NIF/DeCfA/hh+uxnALwz53ySUvpuAO8E8PaU0msBvAXAHwXwBwD855TSH8o5V+MWYzV0sdhu9WVXmu4Bx3aXs4trdjTbZBqbzm1piCZLMkcI1VyQzH0o1+BsZnYfau44S169J+eG4/0SOfBEaymw/apSn/PVNXafSUa29VWrYQJM7ejNzc2lfQz0XuI53r59u9MEopho5A64IiQu6IbtZ+c+VO2N+SQX3qv2P5PKTvtQvmpzc7OqpTDhqKHEgdlstrLi8Hn2IvyvKaVXyWf/if778wC+avH3mwE8lnOeAPjtlNKTAL4QwH9fdZ2IbS/FOTv/v3txHPnCRJRLKtKJV1LXAf8icDUergRTqzJbi3TU42w265kUzkQo5UFEHzUikReNAJtTGstwfHzce5lu3rzZRalF8ZGASz3e3d21qr9LTIrvdGE7PDy0m5pqHIQbd15g3aKiHh0WDBydGOOjbHyAF3qX3xBwZDWr8o4E1IW7RG7H/axKIHo+iMFvAPAfFn+/HMDH6bunFp/1kFJ6W0rpiZTSE6tcGA0NDReHuyIGU0rfBuAEwHviI3OaZSVyzo8CeBQANjc3c/j3SwQhq9yOiKuRUbGC8yYUrMaxmy6OJZ8w+/852k+vxZqAk9jqJuN7cWNQStflz3jFV03GST6Wsq46raasAv0MxIODg84sednLXgbgbJ+C3d3dnoQfj8edK9H52VVbYddpSP+bN2925F/c8+3bt3v1DHm+uH0E9FouD8LF5ztNQOFKg7nSahzX4ua3PgN+Fqy5xr1r/gRrmCXc8SKQUnoEwFcAeGM+u8pTAF5Jp70CwCfu9BoNDQ0XjztaBFJKXwbg7QD+dM75gL76IIAfSSl9D+bE4MMA/sd52oxgllK0X2k1c7akkmG8/ZYWoSzZaSX7/PT0tLND47fb29tLMelxdCXHtH2+/xIppefFUQlHDghyhFKNQ2BtQrkM1nj0t6PRqIvfj3EJl97Ozk7Hm7CmobHvrDUFWPpHuyH9b9261ZF/XEBUK+5qjoK2rwQsawdKaG5tbXXP2El2bYs1gQDzSVwiTzNoWQtyZKvjjHSnJ66V4Ob4Ur+q38479l4AbwDwkpTSUwC+HXNvwCaAn1l0/Odzzn895/yrKaX3A/g1zM2Eb1zlGQgwobe4bu/oFgSt1suhsPqisX+Zz4kHp6GrfB6rZdEuh6zqC+OIQW7Xqf5KWjrCisdLSTTnpeAJ68gjJcI4zNT1W8eUQ3jjWvFi8u9iEvMiw+OjJlBM6oODg471j63E9vb2emYJt6sLIadiu4IgfG1dOHQcuY+rQnJVCLm0aBcnwOapK1ajwsUR3jUvhOI83oGvNR+/q3L+dwL4zpVXbmhoWAusRcQgk3m68jkpx4jfcVSZkigugYOlkap0vHo6dUyTYbg2foBXbAWrdKyKlggip/ozeFxK9RXdhp3cZ5YupchFllhx73t7e931Y7zDNcjVhl2BEXZ/6VjF+awJPPPMMwDmJKA+YyZK3XN0Kb9OC3JkYZyrGib325kFalpwvoczrVRqs3bI80SlvdMEuI+rtIGWO9DQcMWxFppAgINutH66I3cGg8FSzDswJ+nCleNSedVOc1II6NuC0SZH2fG5zlXkIhYBLwWcVGHJq3Yux4SzRHUu0BgfdQc6ycdEldMYVDIdHh52tnoQhKGVbW1t9QJgOGWcyT9tlzNBIwcgiMH9/X2bCxBw9vB5bGW2zzWC0WkCrKEF3HeOLOZnp3OeNTB1+fE9MCdQ0qD5WiU0TaCh4YpjLTSBnOclu1lacXAJsGxPBTjYhe1iV3YZmEsV1Sgmk4mVKi7YJtpytmFttXVSR8/n+6vVGnCbtnKbKklLrHa0G3D5BNoGe3A4P18lUzD2m5ubS1pBtKUeAOYh1MaPuaGf6Rg4tytrOep+TSZ7lEODNX+C3a8Mp1lGW+qR4Ou556kepqOjIzs39RmzFum4tFWcwFosAoGNjY1O7dYNODimPcDqZviIeeBc8o+60DgCy8XZ64Pkdvn/zsev/XU5DJ9pNWA+6sNNKfXGjdVxfflYjXRuQ4234DgLV/QjwIuLqsscy+4IPFcD0vWjBjcuznWqZokzL93Cw/1xQgjwQsuh5jbme68JErco1kwWRTMHGhquONIq0uBSOpHSpwHsA3jmXvcFwEvQ+sFo/VjGZ3M//mDO+aX64VosAgCQUnoi5/wFrR+tH60fl9uPZg40NFxxtEWgoeGKY50WgUfvdQcWaP1YRuvHMl5w/VgbTqChoeHeYJ00gYaGhnuAtgg0NFxxrMUikFL6sjTfp+DJlNI7Lumar0wp/WxK6SMppV9NKf2txecPppR+JqX0m4vjA5fUn2FK6ZdSSj+5+P+rU0qPL/rxvpTSeFUbz0Mf7k8p/Via7ynxkZTSF92L8UgpfevimXw4pfTelNLWZY1H8vts2DFIc/yzxbz9lZTS6y64Hxez30fEVd+rfwCGAP4PgNcAGAP4XwBeewnXvQHgdYu/7wPwGwBeC+AfAXjH4vN3APjuSxqHvw3gRwD85OL/7wfwlsXfPwDgb1xCH94N4K8t/h4DuP+yxwPz6tS/DWCbxuGvXtZ4APhTAF4H4MP0mR0DAG/CvNJ2AvB6AI9fcD/+HICNxd/fTf147eK92QTw6sX7NDz3tS56Yp3jZr8IwE/T/9+J+cYml92PDwD4swA+CuDG4rMbAD56Cdd+BYAPAfhSAD+5mFTP0ANfGqML6sP1xcuX5PNLHQ+cla1/EPPclp8E8OcvczwAvEpePjsGAP4lgK91511EP+S7vwjgPYu/l94ZAD8N4IvOe511MAfOvVfBRSHNN1f5fACPA3go5/w0ACyOL7uELnwvgL8HILJJXgzgZs45MnEuY0xeA+DTAP7Vwiz5wZTSLi55PHLOvwvgnwD4GICnAdwC8Iu4/PFglMbgXs7dO9rvw2EdFoFz71VwIRdP6RqAHwfwLTnnvcu6Ll3/KwB8Kuf8i/yxOfWix2QDc/Xz+3POn495Lsel8DOMhb39ZszV2j8AYBfAl5tT18G3fU/mbrqL/T4c1mERuGd7FaSURpgvAO/JOf/E4uNPppRuLL6/AeBTF9yNLwHwlSml3wHwGOYmwfcCuD+lFDnLlzEmTwF4Kuf8+OL/P4b5onDZ4/FnAPx2zvnTOedjAD8B4Itx+ePBKI3Bpc/ddLbfx9flhe5/t/1Yh0XgFwA8vGB/x5hvaPrBi75omidevwvAR3LO30NffRDAI4u/H8GcK7gw5JzfmXN+Rc75VZjf+3/JOX8dgJ/F2R6Pl9GP/wfg4ymlz1189EbMS8df6nhgbga8PqW0s3hG0Y9LHQ9BaQw+COCvLLwErwdwK8yGi0A62+/jK3N/v4+3pJQ2U0qvxmew3weAe08MLhazN2HOzv8fAN92Sdf8k5irTL8C4JcX/96EuT3+IQC/uTg+eInj8AaceQdes3iQTwL4UQCbl3D9Pw7gicWY/DsAD9yL8QDwDwD8OoAPA/g3mLPelzIeAN6LORdxjLmEfWtpDDBXw//FYt7+bwBfcMH9eBJz2z/m6w/Q+d+26MdHAXz5Z3KtFjbc0HDFsQ7mQENDwz1EWwQaGq442iLQ0HDF0RaBhoYrjrYINDRccbRFoKHhiqMtAg0NVxz/H/i1hqnqZEePAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.imshow(avg_image, cmap=plt.cm.gray)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are multiple ways to perform the segementation. To keep it simple, we just detect the cells by setting up the threshold on the average image." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAASdUlEQVR4nO3dfYxc1XnH8e+vBuNAYhlDQH5TMZJJSxExyOIlVC3CSQ0U4VSCCooSN6GyWtGWJK2CXf6glRoptBGhkVrSFZA4FeE1tFiIdktdUFSpOKzBMQbHsIEUFjsYKiBVXLkmefrH3BXj9d3d2bkvc++c30da7cydO3PPnJn7nOece+ceRQRmlq5fGHQBzGywHATMEucgYJY4BwGzxDkImCXOQcAscZUFAUmXStoraVzSpqq2Y2bFqIrzBCTNA14EPgFMAE8D10bEC6VvzMwKOaai1z0PGI+IlwEk3QesB3KDwHwdFws4oaKimM3sjLMPDroItdix69BbEfHhqcurCgLLgNe67k8A53evIGkjsBFgAcdzvtZWVBSzmY2O7hx0EWoxb8n4f+Utr2pMQDnLjuh3RMRIRKyJiDXHclxFxTCb3bqlq1m3dPWgizEwVQWBCWBF1/3lwL6KtmVmBVQVBJ4GVklaKWk+cA2wtaJtmVkBlYwJRMR7kv4QGAXmAXdHxPNVbMvMiqlqYJCIeAx4rKrXN7NyVBYEzNpm3dLVjO5r95GCmQc4x3OX+rRhs8Q5EzDrMrUlLTMzyGulu19/UIcpHQTMZjC5Y+YFg5l22tF9O3vaqZtwfoK7A2aJcyZg1oMmtNhVcSZgljgHAbMKtClzcBAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS5yBgljgHAbPEOQiYJc5BwCxxDgJmiXMQMEtc30FA0gpJT0jaI+l5STdmyxdLelzSS9n/E8srrpmVrUgm8B7wJxHxy8AFwA2SzgQ2AdsiYhWwLbtvZg3VdxCIiP0R8Ux2+3+APcAyYD2wJVttC/DJooU0s+qUMiYg6TTgHGA7cGpE7IdOoABOmeY5GyWNSRo7zKEyimFmfSg874CkDwLfAT4XET+R1NPzImIEGAFYqMVRtBxWvemm5GrTlXXtaIWCgKRj6QSAeyLi4WzxG5KWRMR+SUuAA0ULaYM123x8k487GLRTkaMDAu4C9kTEbV0PbQU2ZLc3AI/0Xzwzq1qRTOAi4FPAc5Imm4o/A74MPCDpeuBV4OpiRbRBmeuMvM4I2qnvIBAR/wFMNwCwtt/XNbN6eUJSO8pcM4BJzgDayacNmyWuEUHgjLMP9t36WLn6+RzWLV3tLKDFGtUdGN2301+mlvDnNDwakQmY2eA0KhOwwem1G+AMYPg4EzBLXKOCgFuZ5hvdt9ODuEOmEUHgxV3HOwCYDUgjgoCZDY4HBhPn1N6cCZglrlFBwK1S8/nswOHTqCDgL1e9+gm6PjowfBoVBMysfq0aGJypBXIWMXvLPrWO1i1d7VbdnAmYpa5VmYDl66U1z8uU2pAFdJfR2V41nAmYJa7VmUDqLUO/GUBbdV/IdOp7H6b3WbdWBYHJD7oNaWwdUqiPXrsx7jb0z90Bs8S1KhOY5Eh/pO76qOva/01OzZtWnqZzJmCWuFZmAja9Xlq9MsYQ8rZTV/Zh5SpjVuJ5wBjwekRcIWklcB+wGHgG+FRE/F/R7VhxZe5E3d0B7/ztVkZ34EZgT9f9W4GvRsQq4G3g+hK2YWYVKRQEJC0HfhO4M7sv4BLgoWyVLcAni2zDmq2JrfRkduKfPfemaCZwO/BF4OfZ/ZOAdyLivez+BLAs74mSNkoakzR2mEMFi2Fm/ep7TEDSFcCBiNgh6eLJxTmrRt7zI2IEGAFYqMW561i52npy0UytuadDL67IwOBFwJWSLgcWAAvpZAaLJB2TZQPLgX3Fi2lmVek7CETEZmAzQJYJ/GlEXCfpQeAqOkcINgCPlFBOK0HbMoBeOAMororzBG4C7pP0l8CzwF0VbMMGZOpON4yBJTWlBIGIeBJ4Mrv9MnBeGa9rZtXzGYMJKWNgsOh5+Z5+vnn82wGzxCli8EfnFmpxnK+1s67nw0Hlqqo/n/erxl7Xt+r8Wzy0IyLWTF3equ6AvyzlqHowz4OF7eLugFniGpcJTNeKOAswq4YzAbPENSIInHH2wVnnuHM/c3j5sx2sRgQBq1cTf2LriU4Hx0HALHGtCQJNa7msGs4I6teaIGBm1WjcIcKpnAGYVcuZgFniHAQS1uQsy2MD9XEQMEucg4BZ4hwErNHcJaieg4BZ4hp7iLDJg1bDwq2sQYMzAY8OG7gxqENjg4CZ1aOx3QGr3rqlq3OvHlx3BubWfrCcCZglzplAIvKu9z/bpdzmmhF44tB2KhQEJC0C7gTOojP78GeBvcD9wGnAj4Dfjoi35/ra/rKUo3tHrmKnLuO1bLCKdgf+BviXiPgl4KPAHmATsC0iVgHbsvtm1lB9Tz4iaSHwfeD06HoRSXuBiyNiv6QlwJMR8ZGZXmvNRxfE90ZXHLHMLUf5em29XffDabrJR4pkAqcDbwLfkPSspDslnQCcGhH7AbL/p+Q9WdJGSWOSxt78758VKIaZFVEkCBwDnAvcERHnAD9lDql/RIxExJqIWPP26x9y62M2IEWCwAQwERHbs/sP0QkKb2TdALL/B4oV0cyq1HcQiIgfA69JmuzvrwVeALYCG7JlG4BHCpXQStPES43b4BU9T+CPgHskzQdeBj5DJ7A8IOl64FXg6tle5IyzDzI66t8JNIGDRHoKBYGI2AkcNdpIJyswsxbwGYMJ6m7t/UtN828HzBLXuEzAfdJ69fs7ARsejQgCL+463jv/gPiHPebugFniHATMEucgYJa4RowJ2OB4LMCcCZglzkHALHEOAmaJcxAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS16qfEuddB88/hbVhU/f33JmAWeJakwn4arjt0/2ZpZyxlfHdrfKCsIUyAUmfl/S8pN2S7pW0QNJKSdslvSTp/myKMjNrqL6DgKRlwB8DayLiLGAecA1wK/DViFgFvA1cX0ZB8yJgyq2LNd/ovp2tyGCLdgeOAT4g6TBwPLAfuAT4nezxLcCfA3cU3A7gnb5tUvy8ytzp66q/IlOTvw58hc7Mw/uBd4EdwDsR8V622gSwLO/5kjZKGpM0dphD/RbDzArqOxOQdCKwHlgJvAM8CFyWs2rkPT8iRoARgIVanLuOWVsUyQAGnTEVGRj8OPBKRLwZEYeBh4GPAYskTQaX5cC+gmU0swoVGRN4FbhA0vHA/wJrgTHgCeAq4D5gA/BI0UKaNUEVg3yDzgKg2JjAduAh4Bnguey1RoCbgC9IGgdOAu4qoZxmVpFCRwci4hbglimLXwbOK/K6Zlaf1pwxaNYm06X5TTxvwL8dMEucMwGrzOi+nY0Y+KrTbO+3ifXhIGCl6055q/zhS5O0+f25O2CWOGcCVrrJVnHYugPd76v7fts5EzBLnDMBq8ywtJRTDdv7ciZgljgHAbPEOQgMWFuuPmPDy0HALHEeGByAVFp+X224HZwJmCXOmUCFymrxp3udpreuTS+fdTgINNhsQWTYzlyz8uR9d+YtyV/X3QGzxDkTKNlcuwAzteJ5j7V9UHG28jur6U+R74UzAbPEORMoyVwicZHWrq0tZa/143GO+jkI1Mhf7N7lBQ3XXzXnXrg7YJY4ZwIlmRqVfbZc+YZ1UHGmLtBM77msQWJnAmaJcyZQkba2SmWr85Dm1G215TMYdDlnzQQk3S3pgKTdXcsWS3pc0kvZ/xOz5ZL0NUnjknZJOrfKwptZcb10B74JXDpl2SZgW0SsArZl96EzNfmq7G8jcEc5xbQ2avuJTU2wbunqyjOFWbsDEfFdSadNWbweuDi7vQV4ks5EpOuBb0VEAE9JWiRpSUTsL6vA1nxN2fmHaXC2u/xl12+/A4OnTu7Y2f9TsuXLgNe61pvIlh1F0kZJY5LGDnOoz2KYWVFlDwwqZ1nkrRgRI3SmMmehFueuY+3SlAwgj08+ml6/mcAbkpYAZP8PZMsngBVd6y0H9vVfPDOrWr9BYCuwIbu9AXika/mns6MEFwDvejzAmqhtWcDkBWmryGhm7Q5IupfOIODJkiaAW4AvAw9Iuh54Fbg6W/0x4HJgHDgIfKZQ6awVmtwNGBZTp0ArUy9HB66d5qG1OesGcEPRQplZfXzGoPXFrf9g5B0qLPpZ+LcDZolzELAkDUMmU9bZhA4CZonzmIDNyTC0oNC+Q4QzWbd09Yyfy/vvdTz3cQcBm9Ww7PgwXDt/L3p5v+4OmCXOQcBsCBQZJHQQMEucxwRsVlWeslqXVMYC+nmfDgI2qzbv/DY7dwfMEudMwKY1LBlAKl2BfjkTMEucg4BZ4twdaJkyU/TZ0uRhOCqQmn4mYHEmYJY4BwFrlDom2xhG011/sJcszkHALHEeE2iZMvrpTW5pp07TPazvs0mcCZglzkGgpYa9lZuuj2v5phtL6eV74u5Ai/XyAc80WDTsgSRF/XymzgTMEudMYMjltQzDnmY7w5mbWTMBSXdLOiBpd9eyv5b0A0m7JP2jpEVdj22WNC5pr6R1VRXczMrRS3fgm8ClU5Y9DpwVEWcDLwKbASSdCVwD/Er2nL+TNK+00lophvWEnGF9X1XrZS7C70o6bcqyf+26+xRwVXZ7PXBfRBwCXpE0DpwH/GcppbWByJv6yoZHGQODnwX+Obu9DHit67GJbNlRJG2UNCZp7DCHSiiGmfWj0MCgpJuB94B7JhflrBZ5z42IEWAEYKEW565jzTM13W5KZuBuQP/6DgKSNgBXAGuzKcmh0/Kv6FptObCv/+KZWdX6CgKSLgVuAn49Ig52PbQV+Lak24ClwCrge4VLaTaFW/7yzBoEJN0LXAycLGkCuIXO0YDjgMclATwVEb8fEc9LegB4gU434YaI+FlVhbc0OQCUq5ejA9fmLL5rhvW/BHypSKHMrD4+bdgKqbNV9nkA1XAQMEucfztghVV52NAtf/WcCZglbigzgelaonVLV/d1SWabm15/uei6b4ZWBwF/sdrDn0tzuTtglji9f8bvAAshvQn8FHhr0GUBTsbl6OZyHKnN5fjFiPjw1IWNCAIAksYiYo3L4XK4HPWWw90Bs8Q5CJglrklBYGTQBci4HEdyOY40dOVozJiAmQ1GkzIBMxsABwGzxDUiCEi6NJunYFzSppq2uULSE5L2SHpe0o3Z8sWSHpf0Uvb/xJrKM0/Ss5Ieze6vlLQ9K8f9kubXUIZFkh7K5pTYI+nCQdSHpM9nn8luSfdKWlBXfUwzz0ZuHajja9n3dpekcysuRzXzfUTEQP+AecAPgdOB+cD3gTNr2O4S4Nzs9ofozJ9wJvBXwKZs+Sbg1prq4QvAt4FHs/sPANdkt78O/EENZdgC/F52ez6wqO76oHN16leAD3TVw+/WVR/ArwHnAru7luXWAXA5nSttC7gA2F5xOX4DOCa7fWtXOc7M9pvjgJXZ/jSv521V/cXq4c1eCIx23d8MbB5AOR4BPgHsBZZky5YAe2vY9nJgG3AJ8Gj2pXqr6wM/oo4qKsPCbOfTlOW11gfvX7Z+MZ3ftjwKrKuzPoDTpux8uXUA/D1wbd56VZRjymO/BdyT3T5inwFGgQt73U4TugM9z1VQlWxylXOA7cCpEbEfIPt/Sg1FuB34IvDz7P5JwDsR8V52v446OR14E/hG1i25U9IJ1FwfEfE68BXgVWA/8C6wg/rro9t0dTDI725f833kaUIQ6Hmugko2Ln0Q+A7wuYj4SV3b7dr+FcCBiNjRvThn1arr5Bg66ecdEXEOnd9y1DI+0y3rb6+nk9YuBU4ALstZtQnHtgfy3S0y30eeJgSBgc1VIOlYOgHgnoh4OFv8hqQl2eNLgAMVF+Mi4EpJPwLuo9MluB1YJGnyp9511MkEMBER27P7D9EJCnXXx8eBVyLizYg4DDwMfIz666PbdHVQ+3e3a76P6yLL/YuWowlB4GlgVTb6O5/OhKZbq96oOtdKvwvYExG3dT20FdiQ3d5AZ6ygMhGxOSKWR8RpdN77v0fEdcATvD/HYx3l+DHwmqSPZIvW0rl0fK31QacbcIGk47PPaLIctdbHFNPVwVbg09lRgguAdye7DVXomu/jyjh6vo9rJB0naSVzne+jykGeOQyAXE5ndP6HwM01bfNX6aRMu4Cd2d/ldPrj24CXsv+La6yHi3n/6MDp2Qc5DjwIHFfD9lcDY1md/BNw4iDqA/gL4AfAbuAf6Ix611IfwL10xiIO02lhr5+uDuik4X+bfW+fA9ZUXI5xOn3/ye/r17vWvzkrx17gsrlsy6cNmyWuCd0BMxsgBwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeL+HzgSnDgghdkpAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "threshold = 50\n", + "mask = avg_image > threshold\n", + "plt.imshow(mask)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The outcome is different across different threholds we set. Therefore, this threshold is a parameter we could potentially tweak." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQhUlEQVR4nO3df4wc5X3H8fen/hmTItsQkH+pGMlJS6vGoBMxoYoQTmpCEaYSSEYocVNXVivakqRSsMsfqFIjhTZKaNSK1AISp6IQ6tDaQrRX4xBFlYrDAa4xGPAFWjjsYFCBREFy7ObbP+a5eO/Y8+3N7Mzu7fN5Sdbuzs7ufD13+5nv8+zejiICM8vXL/W6ADPrLYeAWeYcAmaZcwiYZc4hYJY5h4BZ5moLAUlXSXpB0qikbXVtx8yqUR2fE5A0B3gR+AQwBjwB3BgRz3V9Y2ZWydyanvdSYDQiXgKQ9ACwEWgbAvO1IBZyVk2lmHXmg7/5bq9LqMWLBxcB8BPeejMiPjD5/rpCYAXwasvtMeAjrStI2gpsBVjIIj6i9TWVYtaZ4eEDvS6hFhuWrwXg0dj1P+3ur2tOQG2WTRh3RMSOiBiKiKF5LKipDLPOjb9YBkkn/6e6OoExYFXL7ZXA0Zq2ZWZJmSCrqxN4AlgjabWk+cAmYE9N2zKzCmrpBCLilKQ/BoaBOcC9EfFsHdsyG2TjR/bho2eer6gylKlrOEBEPAI8Utfzm1l31BYCZrPR5CPqdEfgTp+nqjonLf2xYbPMuRMwO4NBfNtwMncCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa50CEhaJekxSYclPSvplrR8qaS9ko6kyyXdK9fMuq1KJ3AK+LOI+DVgHXCzpIuAbcC+iFgD7Eu3zaxPlQ6BiDgWEU+l6z8BDgMrgI3AzrTaTuC6qkWaWX26Micg6QLgYmA/cH5EHIMiKIDzpnjMVkkjkkZOcqIbZZhZCZVDQNL7ge8An42IH3f6uIjYERFDETE0jwVVyzCzkiqFgKR5FAFwX0Q8lBa/LmlZun8ZcLxaiWZWpyrvDgi4BzgcEV9puWsPsDld3wzsLl+emdWtylmJLwc+BTwjafwk7n8OfAl4UNIW4BXghmolWj8aPlr8yHM4a++gKx0CEfEfgKa4e33Z5zWzZlXpBCxj7gAGhz82bJY5h4BZ5hwCZplzCJhlzhODNsH4W3/gyb9cuBMwy5w7AZvAR//8uBOwaQ0fPTBhmGCDxSFgljmHgFnmHAJmmfPEoAG0HfOPTxJ6snCwuROwKXlCMA8OAbPMeThg0/KnCAebOwGzzLkTGCB1H7HdBQwmdwJmmXMnkDnP/ptDYIC4XbcyPBwwy5xDwDrWz0OH8Q829XON/cohYJY5h4BZ5ipPDEqaA4wAr0XENZJWAw8AS4GngE9FxM+qbse6ZxBbZk+KlteNTuAW4HDL7TuAr0bEGuAtYEsXtmFmNal6avKVwO8Ad6fbAq4EdqVVdgLXVdmGmdWraidwJ/AF4Ofp9jnA2xFxKt0eA1a0e6CkrZJGJI2c5ETFMsysrNIhIOka4HhEPNm6uM2q0e7xEbEjIoYiYmgeC8qWYSVsWL7WY2j7hSoTg5cD10q6GlgInE3RGSyWNDd1AyuBo9XLNLO6lO4EImJ7RKyMiAuATcB3I+Im4DHg+rTaZmB35SqtL7h7GEx1/O3ArcADkv4SeBq4p4ZtWAUzfYvQL/7B1pUQiIjvAd9L118CLu3G85pZ/fxXhBlqPbJ30hWMr+OOYDD5Y8NmmXMnkLnxo/tMOoLWx9ns5xDI3CD+HYHNjIcDZplzJ5CxJrqAM53ezPqDOwGzzLkTsFI8STg4HAIZ27B8bVeGBJOfw6Ewu3g4YJY5h4B13XTf+utvBe4vDgGzzDkErDbTHe3dEfQHh4BZ5hTR9tu/GjX04YXxg+FVE5Z5hrlZvT4i++ddv0dj15MRMTR5ud8iNGBmf0jU7nFlHtvKf67cOx4OmGXOnYBNcKYj8VRH+m4OJdwRNM+dgFnm+i4E/J34/cs/l8Hk4YDNSLsgmEk4TDV0cMD0Tt91AmbWLHcC1igf8fuPOwGzzPVFJ/DiwUU+Qpj1SKVOQNJiSbskPS/psKTLJC2VtFfSkXS5pFvFmln3VR0O/A3wbxHxq8CHgcPANmBfRKwB9qXbZtanSoeApLOBj5FOOBoRP4uIt4GNwM602k7guqpFmll9qnQCFwJvAN+Q9LSkuyWdBZwfEccA0uV57R4saaukEUkjJzlRoQwzq6JKCMwFLgHuioiLgZ8yg9Y/InZExFBEDM1jQYUyzKyKKiEwBoxFxP50exdFKLwuaRlAujxerUQzq1PpEIiIHwGvSvpQWrQeeA7YA2xOyzYDuytVaGa1qvo5gT8B7pM0H3gJ+AxFsDwoaQvwCnBDxW2YWY0qhUBEHADe83VFFF2Bmc0C/tiwWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYqhYCkz0l6VtIhSfdLWihptaT9ko5I+nY6RZmZ9anSISBpBfCnwFBE/AYwB9gE3AF8NSLWAG8BW7pRqJnVo+pwYC7wPklzgUXAMeBKitOUA+wErqu4DTOrUZVTk78GfJnizMPHgHeAJ4G3I+JUWm0MWNHu8ZK2ShqRNHKSE2XLMLOKqgwHlgAbgdXAcuAs4JNtVo12j4+IHRExFBFD81hQtgwzq6jKcODjwMsR8UZEnAQeAj4KLE7DA4CVwNGKNZpZjaqEwCvAOkmLJAlYDzwHPAZcn9bZDOyuVqKZ1anKnMB+ignAp4Bn0nPtAG4FPi9pFDgHuKcLdZpZTeZOv8rUIuJ24PZJi18CLq3yvGbWHH9i0CxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMlfpOwbNZmr46IFfXN+wfG0PK7FxDgHrmdZAmMwBUU27fTtnWft1PRwwy5w7AetLZ+oSwJ3CVKbbb+24EzDLnDsBa0SZI1TZ58uxS6iyf6ftBCTdK+m4pEMty5ZK2ivpSLpckpZL0tckjUo6KOmS0pWZWSM6GQ58E7hq0rJtwL6IWAPsS7ehODX5mvRvK3BXd8q02Wr46IGudwE2UdX9O20IRMT3gf+dtHgjsDNd3wlc17L8W1F4nOI05VO8MWGDrlcv/tyCp+rwp+zE4PkRcQwgXZ6Xlq8AXm1Zbywtew9JWyWNSBo5yYmSZZhZVd2eGFSbZdFuxYjYQXEqc87W0rbr2OzUL0fhdnUM6qTh+P+rybcIXx9v89Pl8bR8DFjVst5K4GjJbZhZA8qGwB5gc7q+GdjdsvzT6V2CdcA748MGM+tP0w4HJN0PXAGcK2kMuB34EvCgpC3AK8ANafVHgKuBUeBd4DM11GwD5kwter8MLWaLMsOCaUMgIm6c4q71bdYN4OaOt25mPedPDFrXzPSo3ckkXes67go6N5P95r8dMMucOwGrrKkjtLuCck7vt9G297sTMMucOwGrpFdH5MnzCe4MynMI2IxUfbF1+xN7ndYzqJ8U7AYPB8wy5xCwjs3WlttdwJk5BMwy5xCwWW3D8rU+0lfkEDDLnEPALHN+i9Cm5QnBweZOwCxz7gRs4LgDmBl3Ajatbs7A13ESktk6XOkXDgGzzHk4YI0bP3JX6S589O8edwJmmXMnYNOq66hbtiNwF9Bd7gTMMucQsGnV/fn8To/sfiegHg4B61jdQTDVC9wv/no5BMwy54lBm5EqJ74sYybb8ScFy5m2E5B0r6Tjkg61LPtrSc9LOijpnyUtbrlvu6RRSS9I2lBX4WbWHZ10At8E/hb4VsuyvcD2iDgl6Q5gO3CrpIuATcCvA8uBRyV9MCL+r7tlW690owM40zcFl31+dwHldXIuwu9LumDSsn9vufk4cH26vhF4ICJOAC9LGgUuBf6zK9Vaz3VjOOBJvv7SjYnB3wf+NV1fAbzact9YWvYekrZKGpE0cpITXSjDzMqoNDEo6TbgFHDf+KI2q0W7x0bEDmAHwNla2nYd61+9PiWY2//uKR0CkjYD1wDr0ynJoTjyr2pZbSVwtHx5Zla3UsMBSVcBtwLXRsS7LXftATZJWiBpNbAG+EH1Ms1OcxfQXdN2ApLuB64AzpU0BtxO8W7AAmCvJIDHI+IPI+JZSQ8Cz1EME272OwPWLX7x16OTdwdubLP4njOs/0Xgi1WKMrPm+BODVlmdnyL00b9+/tsBs8y5E7Cu6fSo3Y2vF7PucSdgljl3AtY4dwD9xZ2AWeYcAmaZ0+lP/PawCOkN4KfAm72uBTgX19HKdUw0m+v4lYj4wOSFfRECAJJGImLIdbgO19FsHR4OmGXOIWCWuX4KgR29LiBxHRO5jokGro6+mRMws97op07AzHrAIWCWub4IAUlXpfMUjEra1tA2V0l6TNJhSc9KuiUtXyppr6Qj6XJJQ/XMkfS0pIfT7dWS9qc6vi1pfgM1LJa0K51T4rCky3qxPyR9Lv1MDkm6X9LCpvbHFOfZaLsPVPha+r09KOmSmuuo53wfEdHTf8Ac4IfAhcB84L+AixrY7jLgknT9l4EXgYuAvwK2peXbgDsa2g+fB/4ReDjdfhDYlK5/HfijBmrYCfxBuj4fWNz0/qD4duqXgfe17Iffa2p/AB8DLgEOtSxruw+Aqym+aVvAOmB/zXX8NjA3Xb+jpY6L0utmAbA6vZ7mdLytun+xOvjPXgYMt9zeTnFik6br2A18AngBWJaWLQNeaGDbK4F9wJXAw+mX6s2WH/iEfVRTDWenF58mLW90f3D6a+uXUvyB28PAhib3B3DBpBdf230A/D1wY7v16qhj0n2/C9yXrk94zQDDwGWdbqcfhgMdn6ugLunkKhcD+4HzI+IYQLo8r4ES7gS+APw83T4HeDsiTqXbTeyTC4E3gG+kYcndks6i4f0REa8BXwZeAY4B7wBP0vz+aDXVPujl726p83200w8h0PG5CmrZuPR+4DvAZyPix01tt2X71wDHI+LJ1sVtVq17n8ylaD/vioiLKf6Wo5H5mVZpvL2Roq1dDpwFfLLNqv3w3nZPfnernO+jnX4IgZ6dq0DSPIoAuC8iHkqLX5e0LN2/DDhecxmXA9dK+m/gAYohwZ3AYknj3/fQxD4ZA8YiYn+6vYsiFJreHx8HXo6INyLiJPAQ8FGa3x+tptoHjf/utpzv46ZIvX/VOvohBJ4A1qTZ3/kUJzTdU/dGVXxX+j3A4Yj4Sstde4DN6fpmirmC2kTE9ohYGREXUPzfvxsRNwGPcfocj03U8SPgVUkfSovWU3x1fKP7g2IYsE7SovQzGq+j0f0xyVT7YA/w6fQuwTrgnfFhQx1qO99HnZM8M5gAuZpidv6HwG0NbfO3KFqmg8CB9O9qivH4PuBIulza4H64gtPvDlyYfpCjwD8BCxrY/lpgJO2TfwGW9GJ/AH8BPA8cAv6BYta7kf0B3E8xF3GS4gi7Zap9QNGG/136vX0GGKq5jlGKsf/47+vXW9a/LdXxAvDJmWzLHxs2y1w/DAfMrIccAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhl7v8B+wWmT91sIhwAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "threshold = 60\n", + "mask = avg_image > threshold\n", + "plt.imshow(mask)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we could use scipy.ndimage to detect the blobs from this binary mask\n", + "\n", + "For the detailed tutorial, please refer to https://scipy-lectures.org/advanced/image_processing/index.html#segmentation" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "from scipy import ndimage" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The method `label` marks each blob with different number, `label_im` is the image with different blobs marked with different number and `nb_labels` is the number of blobs that are detected" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "10\n" + ] + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQzUlEQVR4nO3df4wc9X3G8fdTn30Ep8g2BGR8qLaDm9aKUqBXMFAhhJPwowhTCSRTRK6pK6sVbQmpFNvlD1SpUeI2CjRSRWoBiRM5EOrQ2kK0FAxRVDV2OcA1BkN8mBQOOxhUICmRjJ18+sd+D+8de769nZ3Zvf0+L+m0O7OzOx+Pd5/5fGd/jCICM8vXr3S6ADPrLIeAWeYcAmaZcwiYZc4hYJY5h4BZ5koLAUlXSHpR0oik9WWtx8yKURmfE5A0C/gR8ClgFHgSuCEinm/7ysyskL6SHvd8YCQiDgBIuh9YBTQMgTnqj5OYW1IpZs1R/5xOl1CKOPIeAD/jrTcj4iMTby8rBBYBr9ZNjwIX1C8gaS2wFuAkTuYCrSypFLPm9A0s7nQJpTh24McAPBZb/6fR7WUdE1CDeePGHRGxKSIGI2JwNv0llWHWvLEXSy9p5t9UVicwCpxVNz0AHCxpXWaWtBJkZXUCTwLLJC2RNAdYDWwvaV1mVkApnUBEHJP0Z8AjwCzg3oh4rox1mfWysT1739LFTS3XirKGA0TEw8DDZT2+mbVHaSFgNhNN3KNOtQdu9nGKKvOgpT82bJY5dwJmJ9CLbxtO5E7ALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzLUcApLOkvSEpH2SnpN0S5q/QNKjkvany/ntK9fM2q1IJ3AM+MuI+E1gBXCzpOXAemBHRCwDdqRpM+tSLYdARByKiKfT9Z8B+4BFwCpgc1psM3Bt0SLNrDxtOSYgaTFwLrALOCMiDkEtKIDTJ7nPWknDkoaPcqQdZZhZCwqHgKQPA98DPhcRP232fhGxKSIGI2JwNv1FyzCzFhUKAUmzqQXAloh4MM1+XdLCdPtC4HCxEs2sTEXeHRBwD7AvIr5ad9N2YChdHwK2tV6emZWtyFmJLwZuAp6VtDvN+yvgy8ADktYArwDXFyvRutGxy34bgL7Hn+pwJVZUyyEQEf8BaJKbV7b6uGZWrSKdgGXMHUDv8MeGzTLnEDDLnEPALHMOAbPM+cCgjfP2TRe+f33et3/YwUqsKu4EzDLnTsDG8d4/P+4EbEqvrbuI19Zd1OkyrCQOAbPMOQTMMucQMMucDwwaACN3rPjAvLNv3QnAoo3/WXU5ViF3AjapkTtWNAwH6y0OAbPMeThgU6rvBsaGCNY73AmYZc6dQA858J1z3r++9A92n2DJ1rgL6E3uBMwy504gcz76bw6BHlLGEMB6n4cDZplzCFjTunrosGPg+J9Ni0PALHMOAbPMFT4wKGkWMAy8FhFXS1oC3A8sAJ4GboqI94qux9qnq9v6Vq0c7XQFM1Y7OoFbgH110xuBOyJiGfAWsKYN6zCzkhQ9NfkA8HvA3WlawGXA1rTIZuDaIusws3IV7QTuBL4A/DJNnwq8HRHH0vQosKjRHSWtlTQsafgoRwqWYWatajkEJF0NHI6I+jNTNjpLcTS6f0RsiojBiBicTX+rZVgLzr51p78HYO8rcmDwYuAaSVcBJwGnUOsM5knqS93AAHCweJlmVpaWO4GI2BARAxGxGFgNPB4RNwJPANelxYaAbYWrtK7g7qE3lfHdgXXA/ZL+BngGuKeEdVgB032L0C/+3taWEIiI7wPfT9cPAOe343HNrHz+FmGG6vfszXQFY8u4I+hN/tiwWebcCWRubO8+nY6g/n428zkEMteT3yOwafFwwCxz7gQyVkUXcKLTm1l3cCdgljl3AtYSHyTsHQ6BjJ196862DAkmPoZDYWbxcMAscw4Ba7upTmnuU553F4eAWeYcAlaaqfb27gi6g0PALHNdEQJzlwe/s/sX4/6sGt3wU2PuBjrLbxEaML0vEjW6Xyv3reevK3dOV3QCZtY57gRsnBPtiSfb07eznXdHUD13AmaZ67oQePKcWTx5zqxOl2ENeO/cmzwcsGlpFATTCYfJhg4OmM7puk7AzKrlTsAq5T1+93EnYJa5rugE3n1ePhho1iGFOgFJ8yRtlfSCpH2SLpS0QNKjkvany/ntKtbM2q/ocODvgX+LiN8AfgvYB6wHdkTEMmBHmjazLtVyCEg6BbiEdMLRiHgvIt4GVgGb02KbgWuLFmlm5SnSCSwF3gC+IekZSXdLmgucERGHANLl6Y3uLGmtpGFJw0c5UqAMMyuiSAj0AecBd0XEucC7TKP1j4hNETEYEYOz6S9QhpkVUSQERoHRiNiVprdSC4XXJS0ESJeHi5VoZmVqOQQi4ifAq5I+lmatBJ4HtgNDad4QsK1QhWZWqqKfE/hzYIukOcAB4LPUguUBSWuAV4DrC67DzEpUKAQiYjcw2OCmlUUe18yq448Nm2XOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuUAhIulXSc5L2SrpP0kmSlkjaJWm/pO+mU5SZWZdqOQQkLQL+AhiMiI8Ds4DVwEbgjohYBrwFrGlHoWZWjqLDgT7gQ5L6gJOBQ8Bl1E5TDrAZuLbgOsysREVOTf4a8BVqZx4+BLwDPAW8HRHH0mKjwKJG95e0VtKwpOGjHGm1DDMrqMhwYD6wClgCnAnMBa5ssGg0un9EbIqIwYgYnE1/q2WYWUFFhgOfBF6OiDci4ijwIHARMC8NDwAGgIMFazSzEhUJgVeAFZJOliRgJfA88ARwXVpmCNhWrEQzK1ORYwK7qB0AfBp4Nj3WJmAd8HlJI8CpwD1tqNPMStI39SKTi4jbgdsnzD4AnF/kcc2sOv7EoFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa7QbwyaTdeGl/a8f/1LH/1EByuxMQ4B65j6QJjIAVHMIwd3f2DerIWNl/VwwCxz7gSsK52oSwB3CpNp1AFMxZ2AWebcCVglptqzt/PxcuwSWukAxkzZCUi6V9JhSXvr5i2Q9Kik/elyfpovSV+TNCJpj6TzWq7MzCrRzHDgm8AVE+atB3ZExDJgR5qG2qnJl6W/tcBd7SnTZqoNL+1pexdg4xXpAqCJEIiIHwD/O2H2KmBzur4ZuLZu/reiZie105RP8saE9bpOvfhzC57Lzzyn0P1bPTB4RkQcAkiXp6f5i4BX65YbTfM+QNJaScOSho9ypMUyzKyodh8YVIN50WjBiNhE7VTmnKIFDZexmalb9sKN6ujVg4Zj3UCVbxG+Ptbmp8vDaf4ocFbdcgPAwRbXYWYVaDUEtgND6foQsK1u/mfSuwQrgHfGhg1m1p2mHA5Iug+4FDhN0ihwO/Bl4AFJa4BXgOvT4g8DVwEjwM+Bz5ZQs/WYE7Xo3TK0mClaGRZMGQIRccMkN61ssGwANze9djPrOH9i0NpmunvtZg7S1S/jrqB59W8bTtUV+LsDZplzJ2CFVbWHdlfQmuNdwUjD290JmGXOnYAV0qk98sTjCe4MWucQsGkp+mJr9yf2mq2nVz8p2A4eDphlziFgTZupLbe7gBNzCJhlziFgM9qXPvoJ7+kLcgiYZc4hYJY5v0VoU/IBwd7mTsAsc+4ErOe4A5gedwI2pXYegS/jJCQzdbjSLRwCZpnzcMAqN7bnLtJdeO/fPu4EzDLnTsCmVNZet9WOwF1Ae7kTMMucQ8CmVPbn85vds/udgHI4BKxpZQfBZC9wv/jL5RAwy5wPDNq0jHUDVe2Zp7Mef1KwNVN2ApLulXRY0t66eX8n6QVJeyT9s6R5dbdtkDQi6UVJl5dVuJm1h2pnDjvBAtIlwP8B34qIj6d5nwYej4hjkjYCRMQ6ScuB+4DzgTOBx4Bfj4hfnGgdp2hBXKAPnNXMulA7OoAyfinYXcDUHoutT0XE4MT5zZyL8AeSFk+Y9+91kzuB69L1VcD9EXEEeFnSCLVA+GGLdVuXacdwwAf5uks7Dgz+EfCv6foi4NW620bTvA+QtFbSsKThoxxpQxlm1opCBwYl3QYcA7aMzWqwWMPxRkRsAjZBbThQpA6rXqdPCeb2v31aDgFJQ8DVwMo4fmBhFDirbrEB4GDr5ZlZ2VoaDki6AlgHXBMRP6+7aTuwWlK/pCXAMuC/ipdpdpy7gPaashOQdB9wKXCapFHgdmAD0A88KglgZ0T8SUQ8J+kB4Hlqw4Sbp3pnwKxZfvGXo5l3B25oMPueEyz/ReCLRYoys+r4E4NWWJmfIvTev3z+7oBZ5twJWNs0u9dux8+LWfu4EzDLnDsBq5w7gO7iTsAscw4Bs8xN+VXiSoqQ3gDeBd7sdC3AabiOeq5jvJlcx69FxEcmzuyKEACQNNzou86uw3W4jnLr8HDALHMOAbPMdVMIbOp0AYnrGM91jNdzdXTNMQEz64xu6gTMrAMcAmaZ64oQkHRFOk/BiKT1Fa3zLElPSNon6TlJt6T5CyQ9Kml/upxfUT2zJD0j6aE0vUTSrlTHdyXNqaCGeZK2pnNK7JN0YSe2h6Rb0//JXkn3STqpqu0xyXk2Gm4D1XwtPW/3SDqv5DrKOd9HRHT0D5gFvAQsBeYA/w0sr2C9C4Hz0vVfBX4ELAf+Flif5q8HNla0HT4PfAd4KE0/AKxO178O/GkFNWwG/jhdnwPMq3p7UPt16peBD9Vthz+sansAlwDnAXvr5jXcBsBV1H5pW8AKYFfJdXwa6EvXN9bVsTy9bvqBJen1NKvpdZX9xGriH3sh8Ejd9AZgQwfq2AZ8CngRWJjmLQRerGDdA8AO4DLgofSkerPuP3zcNiqphlPSi08T5le6PTj+s/ULqH3B7SHg8iq3B7B4wouv4TYA/hG4odFyZdQx4bbfB7ak6+NeM8AjwIXNrqcbhgNNn6ugLOnkKucCu4AzIuIQQLo8vYIS7gS+APwyTZ8KvB0Rx9J0FdtkKfAG8I00LLlb0lwq3h4R8RrwFeAV4BDwDvAU1W+PepNtg04+d1s630cj3RACTZ+roJSVSx8Gvgd8LiJ+WtV669Z/NXA4Ip6qn91g0bK3SR+19vOuiDiX2nc5Kjk+Uy+Nt1dRa2vPBOYCVzZYtBve2+7Ic7fI+T4a6YYQ6Ni5CiTNphYAWyLiwTT7dUkL0+0LgcMll3ExcI2kHwP3UxsS3AnMkzT2ew9VbJNRYDQidqXprdRCoert8Ung5Yh4IyKOAg8CF1H99qg32Tao/Llbd76PGyP1/kXr6IYQeBJYlo7+zgFWUzt/QalU+630e4B9EfHVupu2A0Pp+hC1YwWliYgNETEQEYup/dsfj4gbgSc4fo7HKur4CfCqpI+lWSup/XR8pduD2jBghaST0//RWB2Vbo8JJtsG24HPpHcJVgDvjA0bylDa+T7KPMgzjQMgV1E7Ov8ScFtF6/xdai3THmB3+ruK2nh8B7A/XS6ocDtcyvF3B5am/8gR4J+A/grWfw4wnLbJvwDzO7E9gL8GXgD2At+mdtS7ku1B7azah4Cj1PawaybbBtTa8H9Iz9tngcGS6xihNvYfe75+vW7521IdLwJXTmdd/tiwWea6YThgZh3kEDDLnEPALHMOAbPMOQTMMucQMMucQ8Asc/8PbqC4fI92GrEAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "label_im, nb_labels = ndimage.label(mask)\n", + "print(nb_labels)\n", + "plt.imshow(label_im) " + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], dtype=int32)" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "np.unique(label_im)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "0 marks the background and 1 to 10 mark the detected blobs. Some of these are too small to be cells. We could set a cutoff on size to filter out the small blobs. Here the cutoff is another parameter that could be tweaked." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQEElEQVR4nO3de4xU93nG8e9TLktMigA7trip4ArS0kq1rZWD4yqyTFJfahkq2RKWlWxTLNTKbZ2kUgz1H1alRgptlLhRK6cIOyGVC6HELchySwkhiirV1Gub2mBswHYLa4jBqu1EiUQgefvH/DYM61l2ds5lZvf3fCQ0c86cmfNyduY57+/M5SgiMLN8/VK3CzCz7nIImGXOIWCWOYeAWeYcAmaZcwiYZa6yEJB0q6RXJR2TtL6q9ZhZMaricwKSpgBHgE8AQ8CzwD0R8XLpKzOzQqZW9LjXA8ci4nUASduAVUDLEJiuvpjBzIpKMWuP+vq6XUIl4uxZAH7EO29HxIdG3l5VCCwATjRNDwEfaV5A0jpgHcAMLuMjWllRKWbtmbL4V7tdQiV+duQ1AL4TO/631e1VHRNQi3kXjTsiYlNE9EdE/zQmZwLbxDL8YplM2vk/VdUJDAGLmqYXAicrWpeZJZ0EWVWdwLPAUklLJE0H1gC7KlqXmRVQSScQEecl/TGwG5gCPB4Rh6pYl9lkNrxnn7Ls0scrigxlqhoOEBFPA09X9fhmVo7KQsBsIhq5Rx1rD9zu4xRV5UFLf2zYLHPuBMwuYTK+bTiSOwGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy13EISFokaZ+kw5IOSXogzZ8raY+ko+lyTnnlmlnZinQC54E/i4hfB1YA90taDqwH9kbEUmBvmjazHtVxCETEqYh4Pl3/EXAYWACsArakxbYAq4sWaWbVKeWYgKTFwLXAfuCqiDgFjaAArhzlPuskDUoaPMfZMsowsw4UDgFJHwS+DXwmIn7Y7v0iYlNE9EdE/zT6ipZhZh0qFAKSptEIgCci4sk0+y1J89Lt84DTxUo0syoVeXdAwGPA4Yj4ctNNu4CBdH0A2Nl5eWZWtSJnJb4R+CTwkqQDad6fA18EtktaCxwH7i5WoplVqeMQiIj/ADTKzSs7fVwzq5c/MWiWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuyLcIbRI5srn/ffOW3TfYhUqsbu4EbFRHNve3DAebXBwCZpnzcMDG1NwNeIgw+bgTMMucQ8Da5i5gcnIImGXOxwQy56P/5k7ALHMOAbPMOQSsbR46TE4OAbPMOQTMMlf43QFJU4BB4M2IuEPSEmAbMBd4HvhkRPy06HqsPG7rrVkZncADwOGm6Y3AVyJiKfAOsLaEdZhZRYqemnwh8LvA5jQt4GZgR1pkC7C6yDrMrFpFO4FHgM8DP0/TlwPvRsT5ND0ELGh1R0nrJA1KGjzH2YJlmFmnOg4BSXcApyPiuebZLRaNVvePiE0R0R8R/dPo67QM68Cy+wb9PQD7hSIHBm8E7pR0OzADmEWjM5gtaWrqBhYCJ4uXaWZV6bgTiIgNEbEwIhYDa4DvRsS9wD7grrTYALCzcJXWE9w9TE5VfIHoQWCbpL8EXgAeq2AdVsB43yL0i39yKyUEIuJ7wPfS9deB68t4XDOrnr9KnKHmPXs7XcHwMu4IJid/bNgsc+4EMje8dx9PR9B8P5v4HAKZ8/cIzMMBs8y5E8hYHV2AT2/W+9wJmGXOnYB1xAcJJw+HQMaW3TdYypBg5GM4FCYWDwfMMucQsNKNdUpzn/K8tzgEzDLnELDKjLW3d0fQGxwCZplzCGSuF35qzN1Ad/ktQgPG90WiVvfr5L7N/HXl7nEnYJY5dwJ2kUvtiUfb05fZzrsjqJ87AbPMOQSsbd47T04eDti4tAqC8YTDaEMHB0z3uBMwy5w7AauV9/i9x52AWeYcAmaZKxQCkmZL2iHpFUmHJd0gaa6kPZKOpss5ZRVrZuUr2gn8DfBvEfFrwG8Bh4H1wN6IWArsTdNm1qM6DgFJs4CPkU44GhE/jYh3gVXAlrTYFmB10SLNrDpFOoGrgTPA1yW9IGmzpJnAVRFxCiBdXtnqzpLWSRqUNHiOswXKMLMiioTAVOA64NGIuBb4MeNo/SNiU0T0R0T/NPoKlGFmRRQJgSFgKCL2p+kdNELhLUnzANLl6WIlmlmVOg6BiPgBcELSh9OslcDLwC5gIM0bAHYWqtDMKlX0E4N/AjwhaTrwOvBpGsGyXdJa4Dhwd8F1mFmFCoVARBwAWn0jZGWRxzWz+vgTg2aZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZKxQCkj4r6ZCkg5K2SpohaYmk/ZKOSvpWOkWZmfWojkNA0gLgT4H+iPhNYAqwBtgIfCUilgLvAGvLKNTMqlF0ODAV+ICkqcBlwCngZhqnKQfYAqwuuA4zq1CRU5O/CXyJxpmHTwHvAc8B70bE+bTYELCg1f0lrZM0KGnwHGc7LcPMCioyHJgDrAKWAPOBmcBtLRaNVvePiE0R0R8R/dPo67QMMyuoyHDg48AbEXEmIs4BTwIfBWan4QHAQuBkwRrNrEJFQuA4sELSZZIErAReBvYBd6VlBoCdxUo0syoVOSawn8YBwOeBl9JjbQIeBD4n6RhwOfBYCXWaWUWmjr3I6CLiYeDhEbNfB64v8rhmVh9/YtAscw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDJX6DcGzcZr98kDv7h+y/xruliJDXMIWNc0B8JIDoj6eDhgljl3AtaTLtUlgDuFMrkTMMucOwGrxVh79jIfz13C+IzZCUh6XNJpSQeb5s2VtEfS0XQ5J82XpK9KOibpRUnXVVm8mRXXznDgG8CtI+atB/ZGxFJgb5qGxqnJl6Z/64BHyynTJqrdJw+U3gVYucYMgYj4PvB/I2avArak61uA1U3zvxkNz9A4Tfm8soq1iaVbL34Hz/h0emDwqog4BZAur0zzFwAnmpYbSvPeR9I6SYOSBs9xtsMyzKyosg8MqsW8aLVgRGyicSpzZmluy2VsYuqVvXCrOnzQ8P067QTeGm7z0+XpNH8IWNS03ELgZOflmVnVOg2BXcBAuj4A7Gya/6n0LsEK4L3hYYOZ9aYxhwOStgI3AVdIGgIeBr4IbJe0FjgO3J0Wfxq4HTgG/AT4dAU12yRzqRa9V4YWk9mYIRAR94xy08oWywZwf9GizKw+/sSglWa8e+12DtI1L+OuoBr+7oBZ5twJWGF17aHdFVTDnYBZ5twJWCHd2iOPPJ7gzqBzDgEbl6IvtrI/sdduPf6k4Og8HDDLnEPA2jZRW253AZfmEDDLnEPAJrRb5l/jPX1BDgGzzDkEzDLntwhtTD4gOLm5EzDLnDsBm3TcAYyPOwEbU5lH4Ks4CclEHa70CoeAWeY8HLDaDe+5i3QX3vuXx52AWebcCdiYqtrrdtoRuAsolzsBs8w5BGxMVX8+v909u98JqIZDwNpWdRCM9gL3i79aDgGzzPnAoI3LcDdQ1555POvxJwU7M2YnIOlxSaclHWya99eSXpH0oqR/ljS76bYNko5JelXSLVUVbmblaKcT+Abwt8A3m+btATZExHlJG4ENwIOSlgNrgN8A5gPfkbQsIn5WbtnWLWV0AJf6peBOH99dQOfaORfh9yUtHjHv35smnwHuStdXAdsi4izwhqRjwPXAf5ZSrXVdGcMBH+TrLWUcGPwD4F/T9QXAiabbhtK895G0TtKgpMFznC2hDDPrRKEDg5IeAs4DTwzParFYtLpvRGwCNgHM0tyWy1jv6vYpwdz+l6fjEJA0ANwBrEynJIfGnn9R02ILgZOdl2dmVetoOCDpVuBB4M6I+EnTTbuANZL6JC0BlgL/VbxMswvcBZRrzE5A0lbgJuAKSUPAwzTeDegD9kgCeCYi/jAiDknaDrxMY5hwv98ZsLL4xV+Ndt4duKfF7McusfwXgC8UKcrM6uNPDFphVX6K0Hv/6vm7A2aZcydgpWl3r13Gz4tZedwJmGXOnYDVzh1Ab3EnYJY5h4BZ5nThE79dLEI6A/wYeLvbtQBX4DqauY6LTeQ6fiUiPjRyZk+EAICkwYjodx2uw3XUW4eHA2aZcwiYZa6XQmBTtwtIXMfFXMfFJl0dPXNMwMy6o5c6ATPrAoeAWeZ6IgQk3ZrOU3BM0vqa1rlI0j5JhyUdkvRAmj9X0h5JR9PlnJrqmSLpBUlPpeklkvanOr4laXoNNcyWtCOdU+KwpBu6sT0kfTb9TQ5K2ippRl3bY5TzbLTcBmr4anrevijpuorrqOZ8HxHR1X/AFOA14GpgOvDfwPIa1jsPuC5d/2XgCLAc+CtgfZq/HthY03b4HPCPwFNpejuwJl3/GvBHNdSwBbgvXZ8OzK57e9D4deo3gA80bYffr2t7AB8DrgMONs1ruQ2A22n80raAFcD+iuv4HWBqur6xqY7l6XXTByxJr6cpba+r6idWG//ZG4DdTdMbaJzYpO46dgKfAF4F5qV584BXa1j3QmAvcDPwVHpSvd30B79oG1VUw6z04tOI+bVuDy78bP1cGl9wewq4pc7tASwe8eJruQ2AvwfuabVcFXWMuO33gCfS9YteM8Bu4IZ219MLw4G2z1VQlXRylWuB/cBVEXEKIF1eWUMJjwCfB36epi8H3o2I82m6jm1yNXAG+HoalmyWNJOat0dEvAl8CTgOnALeA56j/u3RbLRt0M3nbkfn+2ilF0Kg7XMVVLJy6YPAt4HPRMQP61pv0/rvAE5HxHPNs1ssWvU2mUqj/Xw0Iq6l8V2OWo7PNEvj7VU02tr5wEzgthaL9sJ721157hY530crvRACXTtXgaRpNALgiYh4Ms1+S9K8dPs84HTFZdwI3Cnpf4BtNIYEjwCzJQ3/3kMd22QIGIqI/Wl6B41QqHt7fBx4IyLORMQ54Engo9S/PZqNtg1qf+42ne/j3ki9f9E6eiEEngWWpqO/02mc0HRX1StV47fSHwMOR8SXm27aBQyk6wM0jhVUJiI2RMTCiFhM4//+3Yi4F9jHhXM81lHHD4ATkj6cZq2k8dPxtW4PGsOAFZIuS3+j4Tpq3R4jjLYNdgGfSu8SrADeGx42VKGy831UeZBnHAdAbqdxdP414KGa1vnbNFqmF4ED6d/tNMbje4Gj6XJujdvhJi68O3B1+kMeA/4J6Kth/dcAg2mb/AswpxvbA/gL4BXgIPAPNI5617I9gK00jkWco7GHXTvaNqDRhv9det6+BPRXXMcxGmP/4efr15qWfyjV8Spw23jW5Y8Nm2WuF4YDZtZFDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMvf/gBt0F9GgVmYAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "size_cutoff = 50\n", + "sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", + "\n", + "small_size_filter = sizes < size_cutoff\n", + "pixel_to_remove = small_size_filter[label_im]\n", + "\n", + "label_im[pixel_to_remove] = 0\n", + "plt.imshow(label_im)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now there are only two blobs left." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next let's separate out each mask." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "rois = []\n", + "for i in np.unique(label_im)[1:]: # 0 is the background\n", + " rois.append(label_im == i)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are two special notes about the analyses we performed above.\n", + "1. The segmentation results are dependent on the parameters `threshold` and `size_cutoff`. Rather than fixing the value of the threshold, we might want to try different values and see what works well.\n", + "2. For the segemtation analyses of each `AverageFrame`, the result is a number of ROIs instead of one. We need to perform the analyses on the level of each `AverageFrame`, but save the result on the granularity of each `Roi`.\n", + "\n", + "Next we would like to introduce two DataJoint table tiers to help handling these two needs." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Parameter `Lookup` table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We would like to perform the segmentation for a **combination** of `AverageFrame`s and different set of paremeters of `threshold` and `size_cutoff` values. To do this while still taking advantage of the `make` and `populate` logic, you would want to define a table to house parameters for segmentation in a `Lookup` table!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's define `Param` table to hold different parameter configuration for our spike detection algorithm. We are going to define this table as a `Lookup` table, rather than a `Manual` table. By now, you know that `Lookup` must be yet another **table tier** in DataJoint. `Lookup` tables are depicted by gray boxes in the Diagram.\n", + "\n", + "This tier indicates that the table will contain information:\n", + "* that will be referenced by other tables\n", + "* that doesn't change much - usually contains a few pre-known entries" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class SegmentationParam(dj.Lookup):\n", + " definition = \"\"\"\n", + " seg_param_id : int # unique id for cell segmentation parameter set\n", + " ---\n", + " threshold : float\n", + " size_cutoff : float\n", + " \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": "\n\n%3\n\n\n\nMouse\n\n\nMouse\n\n\n\n\n\nSession\n\n\nSession\n\n\n\n\n\nMouse->Session\n\n\n\n\nScan\n\n\nScan\n\n\n\n\n\nSession->Scan\n\n\n\n\nSegmentationParam\n\n\nSegmentationParam\n\n\n\n\n\nAverageFrame\n\n\nAverageFrame\n\n\n\n\n\nScan->AverageFrame\n\n\n\n", + "text/plain": [ + "" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dj.Diagram(schema)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So far the `SegmentationParam` is an extra table that did not relate with any of the existing, but the `Segmentation` will depend on this table." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Master `Computed` table `Segmentation` and `Part` table for `Roi`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As mentioned above, we performed the segmentation processing on each `AverageFrame`, but the product is the masks of each ROI. In this case, we could create a `Computed` table `Segmentation` containing the `make` method to drive the processing, while using a `Part` table to save results into `Roi`.\n", + "\n", + "`Computed` table and `Part` table are another two table tiers like `Manual`, `Lookup`, and `Imported` we introduced previously. \n", + "\n", + "`Computed` table is very similar to the `Imported` table, which also supports the definition of `make` function and `populate`. The only difference is that the computation in an `Imported` table is dependent on external data, while the computation in a `Computed` table only depends on data inside the database.\n", + "\n", + "Contents in a `Part` table is dependent on its **master** table and the master table could be any type of table. This current example is a very typical usage of Part table. The master `Computed` serves as the driver for computation and the major results with a smaller granularity are saved in the `Part` table." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class Segmentation(dj.Computed):\n", + " definition = \"\"\"\n", + " -> AverageFrame\n", + " -> SegmentationParam\n", + " ---\n", + " segmented_masks : longblob # overview of segmented masks\n", + " \"\"\"\n", + " class Roi(dj.Part):\n", + " definition = \"\"\"\n", + " -> master\n", + " roi_idx : int # index of an roi\n", + " ---\n", + " mask : longblob # mask of this roi\n", + " \"\"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that in the definition of `Roi`, apart from inheriting the primary from its master table `Segmentation`, the Roi has another primary key attribute `roi_idx`. The relationship between `Segmentation` and `Roi` is **one-to-many**." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": "\n\n%3\n\n\n\nMouse\n\n\nMouse\n\n\n\n\n\nSession\n\n\nSession\n\n\n\n\n\nMouse->Session\n\n\n\n\nScan\n\n\nScan\n\n\n\n\n\nSession->Scan\n\n\n\n\nSegmentation\n\n\nSegmentation\n\n\n\n\n\nSegmentation.Roi\n\n\nSegmentation.Roi\n\n\n\n\n\nSegmentation->Segmentation.Roi\n\n\n\n\nSegmentationParam\n\n\nSegmentationParam\n\n\n\n\n\nSegmentationParam->Segmentation\n\n\n\n\nAverageFrame\n\n\nAverageFrame\n\n\n\n\n\nAverageFrame->Segmentation\n\n\n\n\nScan->AverageFrame\n\n\n\n", + "text/plain": [ + "" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dj.Diagram(schema)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `Computed` table is labeled as a pink oval and the `Part` table is bare text. We see that `Segmentation` is a `Computed` table that depends on **both AverageFrame and SegmentationParam**. Finally, let's go ahead and implement the `make` method for the `Segmenation` table. " + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "@schema\n", + "class Segmentation(dj.Computed):\n", + " definition = \"\"\"\n", + " -> AverageFrame\n", + " -> SegmentationParam\n", + " ---\n", + " segmented_masks : longblob # overview of segmented masks\n", + " \"\"\"\n", + " class Roi(dj.Part):\n", + " definition = \"\"\"\n", + " -> master\n", + " roi_idx : int # index of an roi\n", + " ---\n", + " mask : longblob # mask of this roi\n", + " \"\"\"\n", + " \n", + " def make(self, key): # key is one of the primary keys of the join product of AverageFrame and ParameterSet\n", + " \n", + " print('Populating for: ', key)\n", + " \n", + " # fetch average image from the previous table AverageFrame\n", + " avg_image = (AverageFrame & key).fetch1('average_frame')\n", + " \n", + " # fetch the parameters threshold and size_cutoff\n", + " threshold, size_cutoff = (SegmentationParam & key).fetch1(\n", + " 'threshold', 'size_cutoff')\n", + " \n", + " # perform the thresholding and blob detection\n", + " mask = avg_image > threshold\n", + " label_im, nb_labels = ndimage.label(mask)\n", + " sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", + "\n", + " small_size_filter = sizes < size_cutoff\n", + " pixel_to_remove = small_size_filter[label_im]\n", + "\n", + " label_im[pixel_to_remove] = 0\n", + " \n", + " rois = []\n", + " for i in np.unique(label_im)[1:]: # 0 is the background\n", + " rois.append(\n", + " dict(**key, # inherit primary key from master table\n", + " roi_idx=i, \n", + " mask=label_im==i))\n", + " \n", + " # insert into the master table first\n", + " self.insert1(\n", + " dict(**key, segmented_masks=label_im)\n", + " )\n", + " print('Detected {} ROIs!\\n'.format(len(rois)))\n", + " # then insert into the part table\n", + " self.Roi.insert(rois)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The implementation of the segmentation is pretty much what we had above, except that we now fetch the value of `threshold` and `size_cutoff` from the `SegmentationParam` table.\n", + "\n", + "**Important note: always insert into the master table first and then insert the corresponding entries in the part table.** If a master table entry does not exist, its corresponding entries would not be valid." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looking at the `Segmentation` table, we see that it indeed inherits the primary key attributes from **both AverageFrame (`mouse_id`, `session_date`, `scan_idx`) and SegmentationParam (`seg_param_id`)**." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And for the part table `Segmenation.Roi`, there was an additional primary key attribute `roi_idx`:`" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

roi_idx

\n", + " index of an roi\n", + "
\n", + "

mask

\n", + " mask of this roi\n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id *roi_idx mask \n", + "+----------+ +------------+ +----------+ +------------+ +---------+ +--------+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation.Roi()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Populating `Segmentation` table" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We are now ready to populate! When we call `populate` on `Segmentation`, DataJoint will automatically call `make` on **every valid combination of the parent tables - AverageFrame and SegmentationParam**." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "# ENTER YOUR CODE! - populate the Segmentation table\n", + "Segmentation.populate()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Hm... `populate` doesn't seem to be doing anything... What could be the cause?" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looking at `SegmentationParam` reveals the issue:" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
\n", + " \n", + "

Total: 0

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "\n", + " (Total: 0)" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "That's right! We have not added a parameter set yet. Let's go ahead and add one." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "SegmentationParam.insert1((0, 50, 50))" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
050.050.0
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "0 50.0 50.0 \n", + " (Total: 1)" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we should really be ready to perform the computation..." + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 0}\n", + "Detected 6 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 0}\n", + "Detected 6 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 0}\n", + "Detected 9 ROIs!\n", + "\n" + ] + } + ], + "source": [ + "# ENTER YOUR CODE! - populate the Segmenation table for real!\n", + "Segmentation.populate()" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "0 2017-05-15 1 0 =BLOB= \n", + "0 2017-05-15 2 0 =BLOB= \n", + "100 2017-05-25 1 0 =BLOB= \n", + " (Total: 3)" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "...and we now have spike detection running!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Trying out other parameter values" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's see how different thresholds affect the results." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [], + "source": [ + "SegmentationParam.insert1((1, 60, 50)) # add another threshold and size cutoff" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
050.050.0
160.050.0
\n", + " \n", + "

Total: 2

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "0 50.0 50.0 \n", + "1 60.0 50.0 \n", + " (Total: 2)" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam()" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", + "Detected 3 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", + "Detected 2 ROIs!\n", + "\n", + "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n", + "Detected 3 ROIs!\n", + "\n" + ] + } + ], + "source": [ + "# ENTER YOUR CODE! - populate the \"missing\" entry in Segmentation table\n", + "Segmentation.populate()" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", + " \n", + "

Total: 6

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "0 2017-05-15 1 0 =BLOB= \n", + "0 2017-05-15 2 0 =BLOB= \n", + "100 2017-05-25 1 0 =BLOB= \n", + "0 2017-05-15 1 1 =BLOB= \n", + "0 2017-05-15 2 1 =BLOB= \n", + "100 2017-05-25 1 1 =BLOB= \n", + " (Total: 6)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can see that the results of segmentation under different parameter settings can live happily next to each other, without any confusion as to what is what." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Deleting entries from \"upstream\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's say that we decided that we don't like the first threshold of `50`. While there is really nothing wrong keeping those results around, you might decide that you'd rather delete all computations performed with that threshold to keep your tables clean." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "While you can restrict `Segmentation` table to the specific parameter id (i.e. `seg_param_id = 0`) and delete the entries:" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "About to delete:\n", + "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", + "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", + "Proceed? [yes, No]: No\n", + "Cancelled deletes.\n" + ] + } + ], + "source": [ + "# Select 'No' when it pops up\n", + "(Segmentation & 'seg_param_id = 0').delete()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can simply delete the unwanted paramter from the `SegmentationParam` table, and let DataJoint cascade the deletion:" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

threshold

\n", + " \n", + "
\n", + "

size_cutoff

\n", + " \n", + "
050.050.0
\n", + " \n", + "

Total: 1

\n", + " " + ], + "text/plain": [ + "*seg_param_id threshold size_cutoff \n", + "+------------+ +-----------+ +------------+\n", + "0 50.0 50.0 \n", + " (Total: 1)" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SegmentationParam & 'seg_param_id = 0'" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "About to delete:\n", + "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", + "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", + "`shan_tutorial_pipeline`.`#segmentation_param`: 1 items\n", + "Proceed? [yes, No]: yes\n", + "Committed.\n" + ] + } + ], + "source": [ + "(SegmentationParam() & 'seg_param_id = 0').delete()" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + " \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "

mouse_id

\n", + " Unique animal ID\n", + "
\n", + "

session_date

\n", + " date\n", + "
\n", + "

scan_idx

\n", + " scan index\n", + "
\n", + "

seg_param_id

\n", + " unique id for cell segmentation parameter set\n", + "
\n", + "

segmented_masks

\n", + " overview of segmented masks\n", + "
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", + " \n", + "

Total: 3

\n", + " " + ], + "text/plain": [ + "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", + "+----------+ +------------+ +----------+ +------------+ +--------+\n", + "0 2017-05-15 1 1 =BLOB= \n", + "0 2017-05-15 2 1 =BLOB= \n", + "100 2017-05-25 1 1 =BLOB= \n", + " (Total: 3)" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Segmentation()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Visualize ROIs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we have all ROI masks saved in the table `Segmentation.Roi`, let's quickly look at an example by fetching the `mask` from the table." + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPGklEQVR4nO3df+xddX3H8edrLS2CI1AVUloyStK5sSUT0iDoYozVgcwISzSBmNk5lmaL2/yxRMr8g+wPE9mMOrNF14haF0QZstEQN4YVY5bMzi/KEKjYCht8pVLMBI0mrJ3v/XFP5VK/pd/ec8/9fuHzfCTNvedzz73n3c/3e1/nc8493/tJVSGpXb+w1AVIWlqGgNQ4Q0BqnCEgNc4QkBpnCEiNGywEklyS5IEk+5JsG2o7kvrJENcJJFkBfBt4HTAPfA24sqrun/rGJPWycqDXvQDYV1UPAiT5LHAZsGAIrMrqOpGTBypFEsCP+MH3q+olR7YPFQLrgEfGlueBl4+vkGQrsBXgRE7i5dk8UCmSAL5YN//3Qu1DnRPIAm3POO6oqu1VtamqNp3A6oHKkHQsQ4XAPHDW2PJ64NGBtiWph6FC4GvAxiQbkqwCrgB2DrQtST0Mck6gqg4l+WPgdmAF8Imqum+IbUnqZ6gTg1TVF4AvDPX6kqbDKwalxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxk0cAknOSnJnkj1J7kvyjq59TZI7kuztbk+bXrmSpq3PSOAQ8GdV9avAhcDbk5wLbAN2VdVGYFe3LGmZmjgEqmp/VX29u/8jYA+wDrgM2NGttgO4vG+RkoYzlXMCSc4GzgN2A2dU1X4YBQVw+lGeszXJXJK5gzw1jTIkTaB3CCR5IfB54J1V9cPFPq+qtlfVpqradAKr+5YhaUK9QiDJCYwC4IaquqVrfizJ2u7xtcCBfiVKGlKfTwcCXA/sqaoPjj20E9jS3d8C3Dp5eZKGtrLHc18J/C7wzSR3d21/DrwfuCnJVcDDwJv7lShpSBOHQFX9G5CjPLx50teVNFteMSg1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1bhqzEq9I8o0kt3XLG5LsTrI3yeeSrOpfpqShTGMk8A5gz9jydcCHqmoj8APgqilsQ9JA+k5Nvh74beDj3XKA1wA3d6vsAC7vsw1Jw+o7Evgw8B7gp93yi4AnqupQtzwPrFvoiUm2JplLMneQp3qWIWlSE4dAkjcAB6rqrvHmBVathZ5fVduralNVbTqB1ZOWIamniacmB14JvDHJpcCJwCmMRganJlnZjQbWA4/2L1PSUCYeCVTVNVW1vqrOBq4AvlRVbwHuBN7UrbYFuLV3lZIGM8R1AlcD706yj9E5gusH2IakKelzOPAzVfVl4Mvd/QeBC6bxupKG5xWDUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuN6hUCSU5PcnORbSfYkuSjJmiR3JNnb3Z42rWIlTV/fkcBfA/9SVb8C/AawB9gG7KqqjcCublnSMjVxCCQ5BXgV3YSjVfW/VfUEcBmwo1ttB3B53yIlDafPSOAc4HHgk0m+keTjSU4Gzqiq/QDd7ekLPTnJ1iRzSeYO8lSPMiT10ScEVgLnAx+tqvOAH3McQ/+q2l5Vm6pq0wms7lGGpD76hMA8MF9Vu7vlmxmFwmNJ1gJ0twf6lShpSBOHQFV9D3gkyUu7ps3A/cBOYEvXtgW4tVeFkga1sufz/wS4Ickq4EHgbYyC5aYkVwEPA2/uuQ1JA+oVAlV1N7BpgYc293ldSbPjFYNS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS43qFQJJ3Jbkvyb1JbkxyYpINSXYn2Zvkc90UZZKWqYlDIMk64E+BTVX168AK4ArgOuBDVbUR+AFw1TQKlTSMvocDK4EXJFkJnATsB17DaJpygB3A5T23IWlAfaYm/y7wAUYzD+8HngTuAp6oqkPdavPAuoWen2Rrkrkkcwd5atIyJPXU53DgNOAyYANwJnAy8PoFVq2Fnl9V26tqU1VtOoHVk5Yhqac+hwOvBR6qqser6iBwC/AK4NTu8ABgPfBozxolDahPCDwMXJjkpCQBNgP3A3cCb+rW2QLc2q9ESUPqc05gN6MTgF8Hvtm91nbgauDdSfYBLwKun0Kdkgay8tirHF1VXQtce0Tzg8AFfV5X0ux4xaDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNa7XdwxKx+v2R+/+2f2Lz3zZElaiwwwBLZnxQDiSATE7Hg5IjXMkoGXp2UYJ4EhhmhwJSI1zJKCZONaefZqv5yjh+BxzJJDkE0kOJLl3rG1NkjuS7O1uT+vak+QjSfYluSfJ+UMWL6m/xRwOfAq45Ii2bcCuqtoI7OqWYTQ1+cbu31bgo9MpU89Vtz9699RHAZquY4ZAVX0F+J8jmi8DdnT3dwCXj7V/uka+ymia8rXTKlbPLUv15jd4js+kJwbPqKr9AN3t6V37OuCRsfXmu7afk2Rrkrkkcwd5asIyJPU17RODWaCtFlqxqrYzmsqcU7JmwXX03LRc9sIL1eFJw5836UjgscPD/O72QNc+D5w1tt564NHJy5M0tElDYCewpbu/Bbh1rP2t3acEFwJPHj5skLQ8HfNwIMmNwKuBFyeZB64F3g/clOQq4GHgzd3qXwAuBfYBPwHeNkDNep55tiH6cjm0eD47ZghU1ZVHeWjzAusW8Pa+RUmaHa8Y1NQc7157MSfpxtdxVDAM/3ZAapwjAfU2qz20o4JhOBKQGudIQL0s1R75yPMJjgwmZwjouPR9s037ir3F1uOVgkfn4YDUOENAi/ZcHXI7Cnh2hoDUOENAz2kXn/ky9/Q9GQJS4wwBqXF+RKhj8oTg85sjAalxjgT0vOMI4Pg4EtAxTfMM/BCTkDxXD1eWC0NAapyHA5q5w3vuPqML9/7T40hAapwjAR3TUHvdSUcEjgKmy5GA1DhDQMc09PX5i92z+0nAMAwBLdrQQXC0N7hv/mEZAlLjPDGo43J4NDCrPfPxbMcrBSdzzJFAkk8kOZDk3rG2v0ryrST3JPnHJKeOPXZNkn1JHkhy8VCFS5qOxYwEPgX8DfDpsbY7gGuq6lCS64BrgKuTnAtcAfwacCbwxSS/XFX/N92ytVSmMQJ4tm8KnvT1HQVMbjFzEX4lydlHtP3r2OJXgTd19y8DPltVTwEPJdkHXAD8+1Sq1ZKbxuGAJ/mWl2mcGPx94J+7++uAR8Yem+/afk6SrUnmkswd5KkplCFpEr1ODCZ5L3AIuOFw0wKr1ULPrartwHaAU7JmwXW0fC31lGAO/6dn4hBIsgV4A7C5m5IcRnv+s8ZWWw88Onl5koY20eFAkkuAq4E3VtVPxh7aCVyRZHWSDcBG4D/6lyk9zVHAdB1zJJDkRuDVwIuTzAPXMvo0YDVwRxKAr1bVH1bVfUluAu5ndJjwdj8Z0LT45h/GYj4duHKB5uufZf33Ae/rU5Sk2fGKQfU25FWE7v2H598OSI1zJKCpWexeexpfL6bpcSQgNc6RgGbOEcDy4khAapwhIDUuT1/xu4RFJI8DPwa+v9S1AC/GOsZZxzM9l+v4pap6yZGNyyIEAJLMVdUm67AO65htHR4OSI0zBKTGLacQ2L7UBXSs45ms45med3Usm3MCkpbGchoJSFoChoDUuGURAkku6eYp2Jdk24y2eVaSO5PsSXJfknd07WuS3JFkb3d72ozqWZHkG0lu65Y3JNnd1fG5JKtmUMOpSW7u5pTYk+SipeiPJO/qfib3JrkxyYmz6o+jzLOxYB9k5CPd7+09Sc4fuI5h5vuoqiX9B6wAvgOcA6wC/hM4dwbbXQuc393/ReDbwLnAXwLbuvZtwHUz6od3A58BbuuWbwKu6O5/DPijGdSwA/iD7v4q4NRZ9wejb6d+CHjBWD/83qz6A3gVcD5w71jbgn0AXMrom7YDXAjsHriO3wJWdvevG6vj3O59sxrY0L2fVix6W0P/Yi3iP3sRcPvY8jWMJjaZdR23Aq8DHgDWdm1rgQdmsO31wC7gNcBt3S/V98d+4M/oo4FqOKV78+WI9pn2B09/bf0aRn/gdhtw8Sz7Azj7iDffgn0A/B1w5ULrDVHHEY/9DnBDd/8Z7xngduCixW5nORwOLHqugqF0k6ucB+wGzqiq/QDd7ekzKOHDwHuAn3bLLwKeqKpD3fIs+uQc4HHgk91hyceTnMyM+6Oqvgt8AHgY2A88CdzF7Ptj3NH6YCl/dyea72MhyyEEFj1XwSAbT14IfB54Z1X9cFbbHdv+G4ADVXXXePMCqw7dJysZDT8/WlXnMfpbjpmcnxnXHW9fxmhYeyZwMvD6BVZdDp9tL8nvbp/5PhayHEJgyeYqSHICowC4oapu6ZofS7K2e3wtcGDgMl4JvDHJfwGfZXRI8GHg1CSHv+9hFn0yD8xX1e5u+WZGoTDr/ngt8FBVPV5VB4FbgFcw+/4Yd7Q+mPnv7th8H2+pbuzft47lEAJfAzZ2Z39XMZrQdOfQG83ou9KvB/ZU1QfHHtoJbOnub2F0rmAwVXVNVa2vqrMZ/d+/VFVvAe7k6TkeZ1HH94BHkry0a9rM6KvjZ9ofjA4DLkxyUvczOlzHTPvjCEfrg53AW7tPCS4Enjx82DCEweb7GPIkz3GcALmU0dn57wDvndE2f5PRkOke4O7u36WMjsd3AXu72zUz7IdX8/SnA+d0P8h9wD8Aq2ew/ZcBc12f/BNw2lL0B/AXwLeAe4G/Z3TWeyb9AdzI6FzEQUZ72KuO1geMhuF/2/3efhPYNHAd+xgd+x/+ff3Y2Prv7ep4AHj98WzLy4alxi2HwwFJS8gQkBpnCEiNMwSkxhkCUuMMAalxhoDUuP8Hh33yejtUZzUAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# show one example ROI\n", + "masks = (Segmentation.Roi).fetch('mask')\n", + "plt.imshow(masks[2])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Fluorescence trace of each segmented ROI" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we got masks of ROIs in the table `Segmetation.Roi` obtained with different parameter combinations. We would like to extract the fluorescence trace of each segmentation. \n", + "\n", + "The table design is similar to the `Segmentation` and `Roi`. The master table `Fluorescence` is the driver for the computation, with a secondary attribute `time` shared across traces of all ROIs. The part table `Trace` saves the extracted trace for each ROI over time (frame). " + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": {}, + "outputs": [], + "source": [ + "from skimage import io\n", + "import os\n", + "@schema\n", + "class Fluorescence(dj.Imported): # imported table because it also rely on the external tiff file.\n", + " definition = \"\"\"\n", + " -> Segmentation\n", + " ---\n", + " time : longblob # time for each frame\n", + " \"\"\"\n", + " \n", + " class Trace(dj.Part):\n", + " definition = \"\"\"\n", + " -> master\n", + " -> Segmentation.Roi\n", + " ---\n", + " trace : longblob # fluorescence trace of each ROI\n", + " \"\"\"\n", + " \n", + " # the master table is mainly to perform the computation, while the part table contains the result\n", + " def make(self, key):\n", + " \n", + " print('Populating: {}'.format(key))\n", + " # fetch data directory from table Session\n", + " data_path = '../01-Calcium_Imaging/' + (Session & key).fetch1('data_path')\n", + " \n", + " # fetch data file name from table Scan\n", + " file_name = (Scan & key).fetch1('file_name')\n", + " \n", + " # load the file\n", + " im = io.imread(os.path.join(data_path, file_name))\n", + " \n", + " # get dimensions of the image and reshape\n", + " n, w, h = np.shape(im)\n", + " im_reshaped = np.reshape(im, [n, w*h])\n", + " \n", + " # get frames per second to compute time\n", + " fps = (Scan & key).fetch1('fps')\n", + " \n", + " # insert into master table first\n", + " self.insert1(dict(**key, time=np.array(range(n))/fps))\n", + " \n", + " \n", + " # extract traces\n", + " roi_keys, masks = (Segmentation.Roi & key).fetch('KEY', 'mask')\n", + " \n", + " traces = []\n", + " for roi_key, mask in zip(roi_keys, masks):\n", + " \n", + " # reshape mask\n", + " mask_reshaped = np.reshape(mask, [w*h])\n", + " trace = np.mean(im_reshaped[:, mask_reshaped], axis=1)\n", + " \n", + " traces.append(dict(**roi_key, trace=trace))\n", + " \n", + " self.Trace.insert(traces)" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", + "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", + "Populating: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n" + ] + } + ], + "source": [ + "# ENTER YOUR CODE! - populate the Fluorescence table\n", + "Fluorescence.populate()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we could plot the traces of an example scan" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Text(0, 0.5, 'Fluorescence')" + ] + }, + "execution_count": 61, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEJCAYAAAB7UTvrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOydd3zU9f34n++7JJe9B2QPVgaEvYcyFCmCAxUp7lGrrdZWu6zV/r611tbWFkcrtiourOJCcCFDRPYmCSNkkr33zr1/f3xyIeOSXJK7DHw/H497JPcZ78/rIPd5fV5bSClRKBQKhQJAN9gCKBQKhWLooJSCQqFQKFpRSkGhUCgUrSiloFAoFIpWlFJQKBQKRStKKSgUCoWiFZspBSHEq0KIAiFEQoftPxVCnBVCJAoh/tJm+2+EEOdb9l1pK7kUCoVC0TV2Nlz7deAF4A3TBiHE5cBKYIKUsl4I4d+yPQZYDcQCgcDXQogxUspmG8qnUCgUig7YTClIKXcLIcI7bP4x8GcpZX3LMQUt21cC77ZsTxNCnAemA/u6u4avr68MD+94CYVCoVB0x5EjR4qklH7m9tnSUjDHGGCeEOIpoA54REp5CAgC9rc5LqtlWyeEEPcC9wKEhoZy+PBh20qsUCgUlxhCiIyu9g10oNkO8AJmAo8C7wkhBCDMHGu2/4aUcr2UcqqUcqqfn1lFp1AoFIo+MtBKIQv4UGocBIyAb8v2kDbHBQM5AyybQqFQfO8ZaKXwMbAQQAgxBnAAioDNwGohhEEIEQGMBg4OsGwKhULxvcdmMQUhxEbgMsBXCJEFPAG8CrzakqbaANwmtTatiUKI94AkoAl4QGUeKRQKxcAjhnPr7KlTp0oVaFYoFIreIYQ4IqWcam6fqmhWKBQKRStKKSgUCoWiFaUUFL2mpK6ED5M/ZDi7HhUKhXkGunhNcQnw7KFn+TT1UwKcA5gTNGewxVEoFFZEWQqKXpFcmsyW1C0AvH367UGWRqFQWBulFBS94oVjL+Bi78KacWv4NvtbMiq6rJZXKBTDEKUUFBZzsvAkOy7s4LbY27hnwj3Y6ezYeGbjYIulUCisiFIKCotZd3Qd3o7e3BJzC75OviwNX8rH5z+murF6sEVTKBRWQikFhUUcyD3AgbwD3D3+blzsXQBYM24N1Y3VfHL+k0GWTqFQWAuVfaSwiK2pW3FzcOPGsTe2bhvvN54JvhN4I+kNUstTSShKILc6lw1LNxDuET54wioUij6jLAVFj0gp2Ze7j5kjZ2LQG9rtuzX2VrKrstmSugWD3kBJXQnf5Xw3SJIqFIr+oiwFRY+kVaSRV53HvRPu7bTvyvArmew/GR8nHwSCRe8v4lTRqUGQUqFQWAOlFBQ9si9Hm4o6a+Qss/v9nC8OOxrvO55ThUopKBTDFeU+UvTIvpx9hLqFEuwW3OOxE/wmkFmZSVld2QBIplAorI1SCopuaWxu5GDeQWYFmrcSOjLBbwKAciEpFMMUpRQU3XKi8AS1TbUWK4VYn1h0QqeUgkIxTFFKQdEte3P2ohd6po+YbtHxzvbORHlGcbLopI0lUygUtkApBUW37M/dz3jf8bg5uFl8zgTfCSQUJajW2grFMEQpBUWXlNeXk1CUwOzA2b06b7zveMrry8mszLSRZAqFwlYopaDokgO5B5BIi+MJJsb7jQe0BnoKhWJ4oZSCokuO5B/Byc6JON+4Xp0X5RGFs52zUgoKxTBEKQVFl6SVpxHpEYmdrnc1jnqdnljfWJWBpFAMQ5RSUHRJekU6Ye5hfTp3vO94zpaepb653spSKRQKW6KUgsIstU215Fbn9rnb6QTfCTQZmzhdfNq6gikUCpuilILCLJkVWuZQhHtEn843xSESixOtJpNCobA9NlMKQohXhRAFQogEM/seEUJIIYRvy3shhFgnhDgvhDgphJhsK7kUlpFekQ7QZ0shwCUAPyc/Eoo6/fcrFIohjC0thdeBpR03CiFCgCVA2yT2q4DRLa97gX/ZUC6FBaSXpwMQ6hba5zVifWOVpaBQDDNsphSklLuBEjO7ngN+CbQtd10JvCE19gOeQoiRtpJN0TPpFemMcBmBs71zn9eI9YklvTydqoYqK0qmUChsyYDGFIQQK4BsKeWJDruCgAtt3me1bDO3xr1CiMNCiMOFhYU2klSRXp5OuHt4v9aI841DIkkqTrKOUAqFwuYMmFIQQjgDjwG/N7fbzDazjXOklOullFOllFP9/PzMHaLoJ1JK0iv6rxRifWIBFWxWKIYTAzl5LQqIAE4IIQCCgaNCiOlolkFIm2ODgZwBlE3RhuK6Yqoaq/ocZDbh5ehFkGuQCjYrFMOIAbMUpJSnpJT+UspwKWU4miKYLKXMAzYDt7ZkIc0EyqWUuQMlm6I9aeVpAP22FABifGKUpaBQDCNsmZK6EdgHjBVCZAkh7urm8M+AVOA88Apwv63kUvRMRkUG0Pd01LbE+caRXZVNaV1pv9dSKBS2x2buIynlzT3sD2/zuwQesJUsit6RXp6OQW9gpEv/E8DifLQitqTiJOYEzen3egqFwraoimZFJ9Ir0gl1D0Un+v/nEe0TDaDiCgrFMEEpBUUnrJF5ZMLNwY1w93ASipVSUCiGA0opKNrR2NxIVmWW1ZQCaJXNSUWqVkGhGA4opaBox4WqCzTLZiI8+tYIzxxxPnEU1BZQUFNgtTUVCoVtUEpB0Q5Tz6O+zlEwh6ljqoorKBRDH6UUFO3ob3dUc4zxGgNAcmmy1dZUKBS2QSkFRTsyKjLwdvTG3cHdams62zsz0mUkaRVpVltToVDYBqUUFO3IrMi0quvIRIRHBKllqVZfV6FQWBelFBTtyKzMJMQtpOcDe0mkRyTpFekYpdHqaysUCuuhlIKildqmWgpqCvo1WKcrIjwiqG2qJb863+prKxQK66GUgqKVrMosAELdbaMUAFLLlQtJoRjKKKWgaCWzUpuQagtLIdIjErjYgVWhUAxNlFJQtHKhQht+F+wWbPW1TRlNylJQKIY2SikoWsmszMTT4ImHwcPqawshiPSI7KQU1h1dx4HcA1a/nkKh6BtKKShayazMtInryESkZ2Q791FWZRavnHqF5448Z7NrKhSK3qGUgqKVCxUXCHG3fjqqiQj3CErqSiivLwdgd9ZuQJvhnFSsGuYpFEMBpRQUADQ0N5BbnWuTGgUTkZ7tg827s3Yz0mUkBr2BD859YLPrKhQKy1FKQQFAdlU2EmlT91HbtNSaxhoO5h1kSdgSrgy/kq1pW6lprLHZtRUKhWUopaAA4EKllnlkS0sh0CUQB50DqWWp7MvdR6OxkQXBC1g1ZhXVjdV8kf6Fza6tUCgsw2YzmhXDi8yKlhoFGxSumdDr9IR7hJNWkUZlYyWu9q5MCpiEnbBjlOcoNp3bxHWjr7PZ9RUKRc8oS0EBaJlHrvaueBm8bHqdSI9IUspS+DbrW+YEzcFeZ48QglVjVnGq6BRnSs7Y9PoKhaJ7lFJQABcb4QkhbHqdCI8IsquyKawtZEHwgtbtyyOXY9Ab+Cj5I5teX6FQdI9SCgpAS0e1pevIhKndhUAwN2hu63YPgwezAmfxTdY3SCltLodCoTCPUgoKGo2N5FTl2DTzyIQpAyneLx4vx/auqnlB88iuylbDeBSKQUQpBQV5VXk0ySabZh6ZCPcIx93BnSvDr+y0z2Q57MnaY3M5BpMDqcU88M5RGpvVbAnF0MNmSkEI8aoQokAIkdBm21+FEGeEECeFEB8JITzb7PuNEOK8EOKsEKLzHUNhM1q7ow6A+8igN/Dl9V+yJnpNp32BroFEeUTxbfa3NpdjMHnrQCZbT+by2ancwRZFoeiELS2F14GlHbZtA+KklBOAc8BvAIQQMcBqILblnJeEEHobyqZogy1bZpvD1cEVnTD/pzcveB5H8o9csoVsRqNkT3IhAK99lz64wigUZrCZUpBS7gZKOmz7SkrZ1PJ2P2Dq0bwSeFdKWS+lTAPOA9NtJZuiPZkVmTjZOeHr5DvYojA3aC6NxsZLtnNqQk45pTWNTA3z4viFMo5mlg62SApFOwYzpnAn8HnL70HAhTb7slq2dUIIca8Q4rAQ4nBhYaGNRfx+kFWVRZBrkM3TUS1hsv9knO2c2ZN9acYVdp/T/mb/fuNE3BztlLWgGHIMilIQQjwGNAFvmzaZOcxsXqKUcr2UcqqUcqqfn5+tRPxeUVRTRIBzwGCLAYC93p6ZI2eyJ3vPJZmauju5iNhAd0J9nLlpagifn8olt7x2sMVSKFoZcKUghLgNWA78UF781mcBbVNfgoGcgZbt+0pRXRE+Tj6DLUYrc4PnklOdc8lNaausa+RoRinzx2gPM7fNDscoJW/uyxhkyRSKiwyoUhBCLAV+BayQUraNJG4GVgshDEKICGA0cHAgZfu+IqWkqLZoSMQTTMwLmgdwybmQ9qUU02SUzB+tKYUQb2eWxATwzsFM6hqbB1k6hULDlimpG4F9wFghRJYQ4i7gBcAN2CaEOC6E+DeAlDIReA9IAr4AHpBSqm/JAFBeX06TsWlIKYURLiMY7TWarzO+HmxRrMru5EKcHfRMCbtYtLdmRhhlNY3sTSkaRMkUiovYMvvoZinlSCmlvZQyWEr5XynlKClliJRyYsvrvjbHPyWljJJSjpVSft7d2grrUVSr3YyGklIAWBm1kuOFxzlfen6wRbEa3yYXMSvSBwe7i1+7mZHeuDjo2ZZUMIiSKRQXURXN33OK6oamUrg66mrsdHZ8kHxpTGTLKK4mo7imNZ5gwmCnZ/4YP3acyb8kA+uK4YdSCt9zhqql4O3ozeLQxWxO2Ux9c/1gi9Nvdidr/84dlQLA4ugA8ivqSciuGGixFIpOKKXwPae4thgYekoB4Pox11PRUMG2jG2DLUq/ScqpwNvFgXAf5077Lh/nj07AttP5gyCZQtEepRS+5xTVFmHQG3C1dx1sUToxfcR0gl2D+eDc8HchZZXWEOLlZLZA0NvFgSlhXmxXSkExBLBIKQiNtUKI37e8DxVCqDYUlwCmdNShUM3cEZ3Qcf2Y6zmcf5i08uHdTju7tJZgr85WgolF0QEk5lSoQjbFoGOppfASMAu4ueV9JfCiTSRSDChFtUOrcK0j14y6Bjthx/vn3h9sUfqM0SjJKqsl2Mupy2MWR/sD8PVplYWkGFwsVQozpJQPAHUAUspSwMFmUikGjKLaInwdh148wYSvky9LI5byzul32J21e7DF6RNF1fU0NBm7VQpRfq6E+zgrF5Ji0LFUKTS2tLKWAEIIP0BNCLkEKK4tHpJB5rY8PvNxxnqP5ZFvHuFU4anBFqfXZJVqLqGgbpSCEIJF0QHsPV9MdX1Tl8cpFLbGUqWwDvgI8BdCPAXsAf5kM6kUA0KjsZHS+tIhrxSc7Z15cdGLeDt688D2B8isyBxskXqFSSl0F1MAWDTOn4ZmI/tTiwdCLIXCLBYpBSnl28AvgaeBXOAaKeXwdfIqACip1cZdDOWYgglfJ19eXvIyAD/f9fNhVeiVVaq1+Qry7NpSAJgS7oWjvY5vk1XLC8XgYWn20UwgW0r5opTyBSBLCDHDtqIpbM1QrWbuijD3MH425WecLT3L8cLjgy2OxWSV1uLt4oCLwa7b4wx2eqZH+PDdeaUUFIOHpe6jfwFVbd5Xt2xTDGOGcuFaVywNX4qLvQubzm0abFEsJru0tkcrwcTcUT4kF1SRV15nY6kUCvNYqhREm9kHSCmNQPePPYohz1BtcdEdzvbOLItYxlfpX1HRMDzaQmSV1nSbedSWOaO0/wtlLSgGC0uVQqoQ4kEhhH3L6yHg0pqA8j3EpBSGQ0yhLavGrKKuuY6tqVsHW5QekVKSVdp9jUJboke44+PiwB6lFBSDhKVK4T5gNpCNNiVtBnCvrYRSDAxFtUW4O7hj0BsGW5ReEeMTQ7R3NJvObbJ6wNlolDy1NYmE7HKrrFdU1UB9k7HHzCMTOp1g9ihf9pwvGlbBdMWlg6XZRwVSytVSSn8pZYCUco2UUpVeDnOG2sS13rBqzCrOlZ4joSjBquvmVtTxyrdp/OjNI5TXNvZ7PUszj9oyb5QvhZX1nMuv6vlghcLKWJp95CeE+K0QYr0Q4lXTy9bCKWzLcFYKyyKW4WTnZPV5CzllWk1Bdlktv/nwZL+f1rNb1gv2tlwpzBmt/Z8oF5JiMLDUffQJ4AF8DWxt81JYifyKOjYezORkVhnNxoFxGwz1vkfd4ergylURV7E1dWtrFpU1MCmFVVOC+exUHu8eutCv9VqrmXthKQR5OhHp68Ke5MJ+XVuh6AuWZhA5Syl/ZVNJvsfklNVy0/p9XCjRbiBujnYsGOPH326Mx2Cnt9l1h7OlAHB77O18lPwRbyS9wcNTHrbKmqab+JMrYsmvqOMPnyYyNcyL0QFufVyvBk9ne9wc7Xt13pxRvnxwNIuGJmO78Z0Kha2x9K9tixBimU0l+Z6SV17Hza/sp6y6kTfunM66mycxf7QfW07mcji91GbXrWmsobapdlgrhQiPCJZGLGXjmY3sSE7liU8SMPbTysopq8XL2R5Xgx1/uzEee52Ol3dbnmj31v4M/rvnYpvvrF7UKLRl7mhfahqaOZRe0utzFYr+YKlSeAhNMdQJISqEEJVCiOGRJD6EKajUFEJxVQMb7prO/DF+rIgP5M/Xj0cn4GCa7W4Iw7FGwRz3jr+XuqY6/rb/P2zYl8HX/ewymlNWS2DLTdzfzZGF0f7sOFNgkUvvxZ3n+d3HCfzps9OtAebsXqSjtmX+aD/cDHZ8cDSr1+cqFP3B0uwjNymlTkrpKKV0b3nvbmvhLnX++sVZcstr2XDnNCaHerVud3O0J3qku02fEluVwhBum20Jo7xGsSRsCZlNX4Guhpd2pfQrOJxTVteqFACuiBlBSXUDRzK6t9pe/iaFv355lsXRAQjgte/S29QoWJaO2hYnBz3L4wP5/FQeVaprqmIA6e3ktcdb3oeoyWv9o7ymkU9P5nDd5GCmhHl32j8t3JtjmWU0NtumQ/lwLVwzx1XBt4CunrCIwxy/UMa+fnQZzSlr7+5ZMNYPB72ObUl5XZ7z1v4Mnv78DMsnjOTfayfzgwkjefdgJunFNdQ2NvfJUgC4YWowtY3NbD2Z06fzFYq+0NvJa2ta3lehJq/1iw+PZVHXaGTN9FCz+6dHeFPb2Gy1IqqOXCruI4DsAk8aK2Kpc96Nr5vgpZ0pfVqnoq6RyvomAj0dW7e5GuyYPcqHr5Lyu7RAXt2TxpQwL567aSJ2eh33zIukuqGZZ788C/TcMrsrJoV4EuXnwvuHlQtJMXCoyWuDgJSStw9kEh/iSVyQh9ljpoVr1oOt4gpFtUXohR5Pg2fXBzU1QPXQz5Xfl1KMR+MCapqquHxSMXvOF3Eyq6zX65jSUQM7BIaXxASQUVxDckHnYrLq+ibSiquZP9oPe732dYoL8mBWpA9bT+UCvUtHbYsQghumhnA4o5TUQlXIphgYbDZ5raXArUAIkdBmm7cQYpsQIrnlp1fLdiGEWCeEOC+EOCmEmNzHzzMsOJReyvmCKn44w7yVAODnZiDC18VmcYXiumK8Hb3R67pJed35FPxjAuQcs4kM1sBolOxPK2Zu8Ex8HH2oMxzG3dGuT9ZCl0ohOgCAbUmdg9incyuQEuKC2ofY7pkf0fp7dxPXeuK6SUHodYJNR5S1oBgYbDl57XVgaYdtvwa2SylHA9tb3gNcBYxued3LJd6W++0DGbg52nH1hMBuj5sW7sWh9NJ+p1maw6IaheSvoLEa3rkJyjpMOzPnSsk5DpsfhJqBS6M8k1dJWU0js6P8uDL8Svbm7GH1DD++SMwjp6yW/bn7ufqjq8mv7jkrKbtMa1fd8cne392RiSGefJXYOa5gcu/FBra3+C4b488of1fcHe3wcOpdjULHay8Y48eHR7MHrKhR8f3GZpPXpJS7gY53h5XAhpbfNwDXtNn+htTYD3gKIUZa9hGGF8VV9Xx+Ko/rJwfj5NB9Ydr0CB/KaxvNui36S251Lv7O/l0fUFUABUkQvwYa6+DtG7Sb/ZnP4NWl8Ed/2PoIVOSC0Qh7n4f/LIajG+DERqvL2xWmoPKsKB+uiriKBmMD/iOSAfgiIZtnDj5DekU6bya92eNaOWW12OsFfq6dGwQuiQngRFZ5pzkHiTkV+Lg4EODe/hydTvDsDfH88drxff1ordwwJZi8ijq+OafajSlsz0BPXguQUuYCtPw03ZWCgLb9BLJatpmT5V4hxGEhxOHCwuHXBuDDo9k0NBtZ043ryMR0U1zByi6kZmMzGeUZRHhEdH1Q2u4WIe6G1W9BcQo8Fwvv3gzl2RB9NRx5DdZNhPUL4KvfwZgrwW8cJH3SvQBVBZoisQL7UooJ93Em0NOJeL94glyDOFy0k9H+rvzvzCecLztPsGsw7597n/L67oP2OWW1jPBwRKcTnfZdGdviQupQB5GQU0FMoDtCdD5nYognK+K7twYtYVF0AP5uBjbszej3WgpFTwyVyWudv1Et8YtOG6VcL6WcKqWc6ufnZ0URBoZ9qcWM9ndljAVtE0K8nQhwN3DIysHm7KpsGowNRHpEdn1Q6i4weMDIiRAxH657GYKmwPX/hQePwapX4SeHIe56KL8Ay5+Dm96CuFVw4QBUdJFGWVMC/5wIW3/e78/RbJQcSCtmVpSWViuEYGn4Uvbn7mdetB1ZfES0VyzPXf4cNU01vHf2vW7XyymrJdDDvP8/ys+VCF+XdnGF+qZmkvMru0wWsBYOdjrWzgzjm3OFnLeB1ahQtGWgJ6/lm9xCLT9N9nAWENLmuGDgkkzOTswpt/gmIoRgWrg3B9NKrNpbP7Vca9vQo6UQPhdMgei46+H2LTB+Fehb/uu9I+Cal+BX6TD1ThACYlZo+05vMb9u0sdanOLIa10fYyFJORVU1jUxM/JircVVEVfRLJs5WPMsOvtyprqvZZz3OOYEzuGt029R19T1mMucsrouM4WEEFwRE8C+lCIq67SW2sn5VTQZJbGBtq/jXDMjFAe9jg17021+LcX3m4GevLYZuK3l99vQuq+att/akoU0Eyg3uZkuJYqq6smvqO/VTWR6hDd5FXWtjdqsgUkpRHp2YSmUpkNZBkQu6P3ifmO7dyGd2gQ+o2BkPGz+qRaTACg4A29eB/tesvhS+1K1dNlZbZTCGK8xRHlEkVF1Hn1dNGfSNbfPnXF3UlJXwuaUzWbXamo2kldR122m0JKYABqbJbvOam7LroLMtsDX1cCKiYF8cDTLKnMeFIqusNnkNSHERmAfMFYIkSWEuAv4M7BECJEMLGl5D/AZmpI5D7wC3N/LzzEsSMzR2kXF9EIpmNpfHLvQ+7z7rkgtS8XXyRd3hy7kSP1G+xnRB6UAELMSMvdqsYO2lGdBxncw4SbNDdVYCx/fB7ufhZfnQcp22P7/LiqKHjiVXUGwlxP+7heLzYQQLI9ajk7oWBhwO3uSi6iqb2LaiGnE+cTxeuLrNBubO61VUFlPs1F2Skdty6RQL3xcHFpdSIk5Fbga7Ajz7ltxWm+5fXY4NQ3NvNfPdt4KRXfYbPKalPJmKeVIKaW9lDJYSvlfKWWxlHKRlHJ0y8+SlmOllPIBKWWUlHK8lPKwNT7cUCMxp+XJcqTlT5ZjR7hhsNNx0opKIa08rft4Qto34DpCe+rvCzErQRrhTAf3UELLQJzxq8B3NCx9Wotd7Pg/GLsM7t4Oxib45s+dljQrZlEVUX6unbbfFnsbm6/ZzOr4mTQ0G9l5pgAhBHfE3cGFygt8l/Ndp3O6qlFoi14nWBwdwM4zBTQ0GUnMKSdmpLvZwLQtiAvyYHq4Nxv2pav0VIXNsDT76C9CCPcW19F2IUSREGKtrYW71EjM0Z5sPZwtz1u31+uIDXTnRB8qdM0hpSS1PLXreIKUWjwhYr4WI+gL/jHgHdXZhXTqfQiaCt4tCmnK7bD4D3DT23DjBgieClPvgKNvQlFyj58jrbCaCF+XTvvsdfaEuYcxJcwLX1cHvmypL7g85HI8DZ5sSekcyzBNSAtq0+LCHEtiAqisb2JvShGncyt7ZfVZgzvmhJNVWssW1Q9JYSMsdR9dIaWsAJajuY/GAI/aTKpLlKScCmJG9v4mMiHYk4TsCpqs0ByvqLaIqsaqri2FgiSoLuxbPMGEEJq1kPYtVLc0pys4A3mnYPwN7Y+b+zOIXn5x2/xHwc4Rdvyx20vkV9RT3dBMlF9npWBCrxMsidGe7Osam7HX23Nl+JXsvLCT6sbq1uOajc1sz9oKuhpGdpF9ZGLuaF+c7PW8/E0qtY3NNs886siSmADigtz5w6dJFFR2HTQfaGobmq2aDKEYPCxVCqZH22XARpPbR2E5VfVNpBVV9ykoGR/iQW1jM+et0P+mxyBzazxhfv8uFHe95kL67xJI2QEJm0DoIPba7s9z9YfZP9GylLKPdHlYapH2bxFpxn3UlitiR1Dd0MzeFC0ovTxyOXXNdWzP3N56zCcpn7Cz+Hncgz/FxdB9Up2jvZ75Y3xbi+YGIvOoLXZ6Hc/dOJHq+iZ+/cGp1htxYWU9P3/vONv7OU+iL6QWVhH35Jcs+OsuntycyHfni5SCGMZYqhQ+FUKcAaYC21t6Hw2dx5RhwJlcLcjcl5tIfLDWtO6EFeIKrUqhK0sh6WPwGQ2ePRfXdcuIOLjlQ0DCm9fCd+u0wLVbQM/nzvoJOPvAa8u0SuqDr3RqnZFaqD3pm3MftWV2lA+uBju+StRulqYCN5MLqa6pjpeOv4SQdkiXYxzNP9qjeEtiRgBa/cAo/+6Vki0YHeDGr68ax44zBWw8eIF9KcUsW/ctHx7N5o9bT9ukLUp37E8todkoCfZyYuPBTH74nwO8czCz5xMVQxJLA82/RmudPVVK2QjUoLWmUFiIKfMoNqj3SiHcxwU3RztOZPW/jXZqWSqu9q74OZkp/Ms+ohWeTbur39cBIGoh/HgfXP4Y6OwsX9fRHW7bosUcipLhs0dg4+p2h6QVVeNkr2eEe/cxAIOdnsvH+bMtKZ9mo9SykyKXcyDvAAU1Bbx75l3ya/LxrroHB7x5+uDTZrOT2rJonIVVRC0AACAASURBVD86AeNGuLV2Rh1obpsVztxRvvzh00R++J/9uBnseGjRaNKKqtl5dmDbYZzMKsPT2Z63757B8d9fwcQQT9bvTlXB8GGKpYFmZ+ABLlYxB6JZDQoLScwpx9vFocebmDl0OkF8sKdVLAVT5pG5tgzs/zc4uMHEH/b7Oq3YO8KCX8JvW1pjWEpADFz1jFY9vfB3mrIqvdjmIbWwinBfF4syf66MDaC4zfS0H0T+AKM08t7Z93jl1CvMCZxDcVEUE51v4UzJGT5I/qDb9bxcHLhzTgQ3Tg3p9jhbotMJ/nrDBLycHbg6PpDNP53LTxeOItDDsd2M6IHg+IUyJgR7IoTAyUHPvfMjySiuMdtVVjH0sfQx5zWgAa1WAbRgc/eRQEU7EnMqiO2iR44lTAj24GxeJXWNXTzFlmVCc89jG1PLUwn3CO+8oyIXEj+CSWu1J3Vr09dMJiG01hnQLsU1taiayG6CzG1ZMEabnmbqchrhEUGcTxzrT66noqGCu+MeoLKuiWl+lzE1YCrPH3u+xz5Jv1sew9qZYX37TFZipIcT+36zkH+unoSrwQ47vY5bZ4ezN6WY07kDM0K9pqGJ5IIqJgZfjJVdERNAsJcT/93Tl/pWxWBjqVKIklL+BWgEkFLWYr5fkcIMDU1GzuX3L31xQrAnTUZJkrkve1kmrJsMm27vttFcZUMlhbWF5uMJh/+r1QjM6LYmcXDwjoAR4yFJq0ZuaDJyoaSGqB7iCSbcHO2ZM8qHL5PyWgOgP4j8ARLJVeFX4dCsPfGH+rjw6+m/prKhkgd3PEhlQ6VtPo8V6fiQcfO0UJzs9bw6QNZCYk4FzUbJhOCLw5rs9DrunBPBofRSjluxvkYxMFiqFBqEEE5cHLITBdTbTKpLjOSCShqbZb/aIUwM6SbYnLQZjI1w+lPY2bUB12WQubEODr8GY6+6WEMw1IheobmQKvPILKnGKCHCQksBtCykCyW1nM7VbvRXR13N8sjl/GzKz0jI1hRtXKAHY73H8uf5f+Zk4Unu+vIuSutKbfJxbIWHsz2rpgTzyfEciqps/xU1/T1OCGn/t33jtBDcHO34z7fKWhhuWKoUngC+AEKEEG+jDcj5pc2kusRoDTL3w1IY4eGIv5uBk+aCzUmfQMB4LTD77d/guPl5BqllXaSjJmyCmiKYcV+f5bM50SsACWe2tGYeRfpanvmzODoAIeCrJM2F5GHw4Ol5TxPoGsip7HLcHe0I8dZqFJaGL+WfC/9Jankqd3xxB4U1w6tF++1zwmloNvLOAdtnAJ3MKifQwxF/t/axMleDHWumh/J5Qh5ZpTU2l0NhPXpUCkKzT88A1wG3AxvRspB22VSyS4jE7HKcHfRE+Fj+ZGuOCcGenSuby7Mh6yDEroRlz2r1BZt/ChcOdTo/rTwNe509Qa4dRlWceBd8x/a/NsGW+I3VUmVPf0pqUUs6ai8sBT83A1PDvPgysXPw09S5tq0rZn7wfP61+F/kVOfw1IGn+i//ABLl58qkUE92n7O9MjuRVdbOddSW22aHY5SSD45k21wOhfXoUSm0tMz+uKVv0VYp5RYp5dCf5j5EKKtp4KNj2cyO8u13j5yJIR6kFla375J5+lPtZ8w1oLeHG98AF1+tn1AHUstTCXMPw07XpkCrsQ6yDsHoJX0PBg8Eprbcad+Sl5uNr6sBd8fejbm8ImYEp3MruFBy8cm1sdnImVzzMxGmjZjGHXF3sD1zO6cKT/X7Iwwkk0O9OJVdTqMVquC7oqymgYziGuJDzCuFQE8nYgPdWwsHFcMDS91H+4UQ02wqySXKS7tSqKxv4hdXjOn3WqYnspNtrYWkT7ReQ76jtfdOXjDjR1pTu/zEdudnVJiZtpZ9BJrqIGxOv+WzOdFXg2zGL2enxZlHbVkcoxXO7WqTx38uv5KGZmOX7SpujbkVb0dv/nnsn32TeZCYGOJJfZOm8GyFqW4mPrjrWNmsSB+OZZZ1nTWnGHJYqhQuB/YJIVKEECeFEKeEECdtKdilQHZZLa/vTefaSUFE96HnUUcmh3lhpxPsTWnpJ1SZD5n7tD5D7Q68DeycYH/74XhFtUUEOHeoKM74DhAQNqvf8tmckRPBI5TJFduJ9Ol9u+pwH2dCvZ35po1bJbE1yGz+/8fF3oW7x9/NgdwD7M/d3ze5B4FJodoDxPELpXye9jl/3P9H/nXiX7x/7n1yq6wzquTkhTItY7gbpTA7ypeGZmNrjYhi6GPp9LSrbCrFJcrfvjoLwC+u6GML6g64GuyYFOrJd+dbzPEznwKys1Jw9ob41XD8HVj8JLj4UttUS1VjFT5OPu2PTd+jtaRw8rKKjDZFCOrib2PW7v+Duo+A+F6eLlgwxo8PjmbR0GTEwU5HQk45rgY7wruJ99w49kbeSHqDdUfXMWPZjD7XmgwkQZ5O+LkZ2J6+j6PnnsbRzpHappb24C6BbL1ua3s3Yh84kVVOpK9Lt268aRHe6HWCvSlFzBnl26/rKQYGS9tcZACewNUtL8+WbYouSMqp4KNj2dwxO7zLEY99Yc4oX05ll1NW06C5jnzHaJPOOjLjPmiu18ZeAsW1mnXh49hGKTQ1wIWDEDbXavLZmtNRd7CleQYzz/+jtW6hN8wf40dNQzOHM7ReSqeyy4kJ7H4mgkFv4P74+zlVdIodmTv6LPtAIoQgLtiBY7X/IsQthF037uLo2qM8M+8Zcqpz+Drj636tL6XkRFZZl/EEE64GO+KDPS5at4ohj6VtLh4C3gb8W15vCSF+akvBhiN/+eIMC5/dxeT/28bVL+zB3dGe+y8bZdVrzB3li5RwOOm89pQfs9J8gNh/HEQtgoP/gaYGiutalEJbSyHnKDTVQnjv4gkHUot55P0Tg+InTi2q5ReNP6Y+YBJ8eG+3nVTNMSvKB3u94JtzhTQ1GzmdW8F4C9pfXx11NeHu4bx04qVh0wG0zPldmnXl/Hba/+Fs74y93p6lEUsJdQtlQ+KGfn2OvIo6CivrW5s1dsfsKF9OZpVTVd9zxb1i8LE0pnAXMENK+Xsp5e+BmcA9thNraFPT0NTpC7XzTAEv7UohwN2Rq+JG8KP5kWy4c3qvBupYQnyIJy4OeopPfqG1ph7TjWdv5v1QlQfbHqcoQ5s25uvUxoRP36P9DJ1t5mTznM2r5O4Nh9l0JIvPEwZ+jHZaUTVNOgP6H76rtdl+dy00WJ4H72qwY2qYN9+cLSSlsJq6RiNxFjQptNPZce+EezlXeo5dF3b14xMMDFtTt3K+5lsaChfRUBPcul0ndNwScwsJxQkcLzze5/WPZWrJDj1ZCqAp4maj5FCa6rg/HLBUKQig7WNhM9/TNhfFVfXMe2Yn97xxuDXdr7ahmd9vTmCUvysb7pzOU9eO55dLx7VWIVsTe72OmZE+uOXsAUdPCJzY9cFRCzVr4cC/Kf5Gy7X3OfT6xf0Z32mZSy4+5s/vQH5FHXe8dhAnBz1Bnk68dyirH5+kd2SV1vD4xwms/zZV607qHgDX/hsqc+Dg+l6ttWCsH2fyKtl+RqtZiLOw0vyqiKsIdg3m5ZMvD3lr4fljzxPrM56mkstbb+AmVkStwMPgwYbEDX1e/2BaCc4OeosKMqeEeeGg16nU1GFCbxriHRBCPCmEeBLYD/zXZlINYf7xdTKlNQ18fbqAR98/gdEoeWFnMhdKavnjNXE42Nm+lfKcKB8mNh6jJngu6PRdH6jTaTMNHkmmOP5GALz3vwwn34PmRsg8YHEqanV9E3dtOERZbSOv3j6Nm6eHsC+1mIzi6p5P7gd1jc08uTmRy/66i3cPZXLdpCBevmWKtjNsNoxaAnuegzrL24ovGKO1DX91TxpO9voeB/WYsNPZcff4u0ksTjQ753moUNlQSXZVNovDFjImwKNT/yFne2duHHMjOzJ3cKHiQp+usT+1mClhXha1Dne01zM5zFPFFYYJlgaa/w7cAZQApcAdUsp/2FKwoUhyfiXvHMxk7cwwHr1yLB8fz+HBd4+xfncq108OZmakZU/c/eVyv3ICRQmJTpMtO8HVn2JXHzwNntiHzoZPH4ITG6Gx2uJ4woZ96SRkV/DimsnEBXlw/ZRgdALeP2w7a+FcfiUrX/iO1/emc9O0EL559HL+fP0Egr3apKMu/B3UlcHeFyxed9wIN/zdDBRVNRAT6I6+F0WFK6JWMNJlJC+fGLrWgqnHVZRHFJNCPTmeWdpp8M7qcavR6/S8efrNXq9fVtPA2fxKpod7W3zO7ChfknIrtAQJxZDG0kBzKFAEfAx8BBS3bPte8dRnp3F20POzxWO4/7Io7pobwZaTuTg72PHbZWYygGxEeJmWL7+1Ktric4pqi7R4wg2vgYMrbH5Q22Fh5tHJC+VE+Lpw+Th/QGvbPH+MH5uOZNlkmMonx7O5+vk9FFfXt7rkAs1lcQVO1Kq5978E1Za5J4QQzG+xFiwJMrfFXm/PnXF3crzwOAfzDvbq3IEipSwFgFGeo5gU4kVFXVNraxAT/s7+rIhawftn3+dkYfuSo2ZjM0W1Xf9bHk4vRUqYHmG5UpgV5YOU2pQ2xdDGUl/HVmBLy2s7kAp8biuhhiLfnCtk19lCHlw4Gm8XB4QQPLYsml8uHcvzN0/Cx9UwYLKI1F0U2gfxaaa9xaMXi2uLtXRUtxGw6lUtY8l3LLiamcBmhtN5FUSPdGu37aapIeRV1LE72bo9duqbmnn84wRiAt35/KH5re6eLln4O2is0dxIFmJasy9NCq8dfS3+zv78cvcv2ZO9p9fn25rzZecx6A0EugYysbWIrXN33V9M/UXr56ho0Ir4ahpruO/r+1j8/mL+ceQf1DV1nrp7ML0EBzudRUFmE/HBWoLEjjNq8M5Qx1L30Xgp5YSW12hgOjD0vg02QkrJn7aeJszHmVtnXxysotMJ7r9sVOtT54DQ3Ajpe6gInEtxdQNn8ixrY1BUW3QxHTViHqx6Da78k0XnVtY1klFcQ0yHquxF0QF4uzjw/uG++aW7Yve5Iirqmnhw0Wj83CxQtr6jYeIaLeCcddiiayyJCeDhxWNYGjei1/IZ9AbWL1mPj5MPP/76x/zl0F9oaB46bpHUslQiPSLR6/SM8nPFzWDH0czOFcXuDu78ZcFfyK/O58m9T1JeX8492+7hUN4hZgfO5r8J/2XVp6s4lNe+ueKBtBImhnjiaN9NPKsDDnY6rowbwecJearlxRCnT1FRKeVR4HvTCymzpIaz+ZXcPTcCg53lXwSbkHUIGqrwmnAlwMXq5h4orituX6MQew2MXmzRuSbF07FVh4OdjmsnBbEtKZ9iK/bu/+R4Nt4uDsztTQXskv8Dt5Hwv7Va+48ecLTX89Di0bj1sqmeiSjPKN5Z9g43j7uZN5Pe5Fe7f9WndWzB+bLzRHlGAdqDy/QIb3adKTBrVcb7xfPTyT9lW8Y2Vn68ktPFp/nbgr/x0uKXeOWKV2g2NnPPV/dwpuQMAFX1TSRklzOjF64jE9dOCqKyromdZwZ2hrSid1gaU/h5m9cjQoh3gD77DIQQDwshEoUQCUKIjUIIRyFEhBDigBAiWQjxPyGEQ1/XtzZJLfMQemMu24yUnSB0eMcsIsLXhX2pPWd01DTWUNtU265G4XB6CUcyLPPvmkY7mpscd9O0EBqbJR8etU575Or6Jr4+nc+y8SMsymxpxdkbVr+jZSG9d6tWrW1jHO0c+e2M33LP+Hv4OvNr0soHdjayOaoaqsivyW9VCgArJgaSU17HwXTz/9+3x97O3KC51DTV8MKiF1gUtgiAmSNn8u7yd/EwePDU/qcwSiNHM0ppNspexRNMzI7yxc/NwEfHVCvtoYyl3zq3Ni8DWoxhZbdndIEQIgh4EG0mQxygB1YDzwDPtbinStEK5mxGb1IpE3Mq0OsEYwLcej7Y1qTsgKAp4OTJzEgfDqaV0NRDe+SOLS6MRskD7xzl5vUHLFIMSTkVeDnbM8LdsdO+MQFuTA71ZOOhTKtk42xLyqeu0cjKiUE9H9yREXGw8kW4sB++GLgn9zXRa7DX2fP26bcH7JpdkVKuBZmjPC4qhSUxATg76Pm4i5uxTuhYt3AdX17/JbMD2xcyehg8eHjKwxwvPM7mlM0cTCtBrxNMDu19ryy9TrAiPpCdZwtUFtIQxtKYwh/avJ6SUr4tpewcgbIcO8BJCGEHOAO5wEJgU8v+DcA1/Vi/Wz44ksVlz+7irIX++MScckb5ufbKh2p1Guvg/Ndaa4qohQDMjvLRzPmc7oe0F9VpLiaTpXDsQin5FfXodHDvG0fazRcwx+ncCqJHunfZCG719FBSC6s5lN7/TpifHM8myNOJKX246QAQdx3M+gkcfhWKkvstjyX4OvmyLGIZm1M2U15veb2ELWibeWTC2cGOpbEj2Hoqt50/v6HJ2Dqbw15nj5ej+X/zFVErmOg3keeOPMe+tAuMD/LAxdC3ZnrXTgqisVny2am8Pp2vsD3dKgUhxKdCiM1dvfpyQSllNvAskImmDMqBI0CZlNLUHCULMPuoKIS4VwhxWAhxuLCwbx6sRdH+ONvreWnXeYuOT8qt6NcozX5RVQgbb4a/RMBb14O9C8ReB9BaF7Gvh6Igk6VgUgpbT+bhYKfj/R/NpskoufP1Q1TUNZo9t6nZyJm8yk5B5rYsnzASN4Md7x7q3/jHkuoGvk0u4ur4wP4NJJr1gPYz6eN+ydMb1saspbaplo+SPxqwa5ojpSylNfOoLde0+PNNsySajZLbXztI/B++Yuk/dvPk5kQOdeFe0gkdj818jLL6Mk7Xv9+neIKJ2EB3Rvm7dmm1DBZSSp7+/DSbT+QMtiiDTk+WwrPA37p59RohhBea6ykCCARcMN+a26wvQkq5Xko5VUo51c+vb1k/ns4OrJ0Zxqcnckgv6t6NVFRVT35FvVl/+oCQ/BWc/QzG3wBr3oNHzmrN7tBGTI72d+0xrmDKOfdx8sFolHyekMv80X6MD/bgX2snk1ZUzSPvnTB7blpRNfVNxm7nQTg72LFiYiCfncptPxWul3x2Kpcmo2RFfGDPB3eHeyCEzIDET/q3Ti8Y5z2OqQFT2XhmI03GwWv8llKW0pp51JbZUT74uhr4+Jh20/vn9mT2phRz09QQ/NwM/O/QBW5ev5+EbPOWzjjvccwfsRKdxz58fft+QxdCcM3EQA6mlwyp2c3ZZbW8/E0qD248xiPvn6Cm4fvbvK8npZAmpfymq1cfr7m4Zd1CKWUj8CEwG/BscScBBAM2Vdl3zYvATq/jX7tSuj3OFGQeNKWQn6gNzFn+HIy5Ehza9/2fHeXD4fQSGpq6jisU1RahEzq8DF6cyCojt7yOZeNHtJzvy8NLxvBVUr7Z+EJSN0HmtqyeFkpdo5FPjvf9hrH5eA6j/V071UP0iZiVkH8Kirv//7Uma6PXklOdw84LOwfsmh1JKU8h0jOy03Y7vY4V8YHsOFPAZ6dyeX5HMqumBPPMqgm8edcM9v1mIb6uBh7+3/EuU0arcpcgmnzYmP40pXV9dxWa4kWfHB86T+UJLcOWro4P5IOjWSx/fg8phVWDLNXg0JNSaLW/hRAfWOmamcBMIYSz0JzUi4AkYCewquWY2wCbPub5uzmyeloIHx7LIqestsvjEluUQuzI3lW+Wo2CRM0y6KLH0awoH2oamtuP6OxAcV0xngZP9Do9nyfkYa8XLIq+OIHtjjnh+Lo68LevznU6Nym3Anu9IKqH/kDjgz2IDXRn48ELfQo4F1fVcyijhGXjR1pniE30Cu3nALqQLgu5jCDXIP594t9UN9q2J5Q5qhqqyKvOaxdPaMs1kwJpaDbyk3eOMsrPlf+3MrZ1n6ezA8+smkByQVXrcKi2nMoqZ+fpCq4N+jVl9aU8tucxjLJv859DvJ2ZGOLJV4lDJ66QkF2OXif466oJvH33DEqqG3hyc2LPJ16C9KQU2n47Oz9+9AEp5QG0gPJR4FSLDOuBXwE/F0KcB3wYgIZ7P1oQhZSwfndql8ck5pQT5Olk9RbYFpOfCP6xXe6eEeGDEN3HFUwtLqSUfHYql7mjfPFwuvh5nB3s+PFlo9ibUtxpndO5lYz2d7Oo0d/q6aGczq0wWz3bEzvPFiIlLI4O6PlgS/AMgaCp2iCiAUKv0/PbGb8lpSyFn2z/Seuks4Gibc8jc4wP8iDSzwUHOx0v/nAyzg7tg8ULxvixdmYo/9mTxv4OLsm/bzuLp7M9j16+kEenPcq32d/yeuLrfZZ1cbQ/J7LKKajsT76K9UjIKWe0v5ZMMjvKl7vmRPBtctH30lro6Zsuu/i9X0gpn5BSjpNSxkkpb5FS1kspU6WU06WUo6SUN0gprVcN1QVBnk5cOymIjQczya8w/8c5uEHmAqguhICYLg/xcnFg3Aj3buMKJbUl+Dj6kJBdQVZpLVeNH9npmB/OCCXA3cDft51t96SflFNh8XzpayYG4mqw4/W96RYd35btp/MJcDdYNNvAYmKvgdwTUDJw9QPzg+fzp7l/4kj+ER7e9fCAVjqbMo/a1ii0RQjBy2unsOm+2V2mV/92WTSh3s48/L/jrdbn0cxSdp4t5N75kbg52rN67GqWhC1h3dF1rdfsLSZLdSgUskkpScguJ7ZNC/XV00Nx0Ot4c9/3b8BkT0ohXghRIYSoBCa0/F4hhKgUQnSfBzlM+MlCzdR+dNPJThWf1fVNpBVVD248ASCga0sBtLjCkYxS6pvM+4JNlsLWU7nY6QRXxHR+Gne01/OTy0dxKL2Ub5O1wHRBZR1FVZYH2d0c7blhajBbT+Z2qWTN0dBkZPe5QhaOC7Du/ONWF9LAWQsAyyKX8cSsJ/gu+zse/+7xAbuuqedRkGvXNR6jA9yI66YJoLODHS+u0brvXvfSXv61K4Xntp3Dx8WB22aFA5py+d3M3yGEYNO5TV2u1R3jRrgR6OHI16cHXykUVNZTVNXA+DYPJH5uBn4wYSSbjmR97ybGdasUpJR6KaW7lNJNSmnX8rvp/SDdKa1LmI8Lv1sew+5zhWzYl95u35m8CqSk3RPEgFKQpP3sxn0EMCvSh/omY6dhKqA9BZlaXGxLymNWlA+ezuaLxW+cFkKQpxO/+fAUb+xLb+1o2V06akdunx1Os5S8tf/iE1ZdYzN/3JLEizvPsy+luFNmx4G0Yqobmlkc7W/xdSzCKwwCJw9oXMHE9WOu5774+/gs7TOOF/R9wllvSClPIcIjolPmUW+JC/Lg84fmcUVsAM98cYZvk4u4b0FUu9oEb0dvLg+5nC2pW/pkDQmhxbX2JBcNei8kU8ZVR2V566wwquqb+PDowA2TGgrYfiLMMGDtjFAWR/vz9OdnOJN30QAyZR4NmvsoPwlc/HrsZDo90huDnY4PjnT+461qrKK+uR5nvScphdXMjuq6n5DBTs/fbozHw8me33+SyIMbjwG9UwphPi4sGufP2wcyqWtsRkrJLzed5D970vjrl2e5+ZX9THjyKzYevFjTsP10AQY7Xbey9ZmYlZBzbECzkEzcEXsHXgYv/n3i3za/VpOxiXMl57p0HfUWT2cHXlwzmb9cP4EfTBjJ2plhnY65bvR1lNWX9Xk86aJof2obmy1q1WJLTmWXI0Tn3l6TQr2ID/Zgw950pJQUVtazbnsyG/amt05dvBRRSgHtqeWZ6yfg4WTPgxuPtZqLiTkVeDrbM9Kjc3uHASE/oUfXEYC7oz1rZ4bx4bFsUjsExkyFa2WV2meYEtZ9pfDMSB8+e2geW346l9tnh3PbrLBeB9nvmBNBSXUDm4/n8NKuFDafyOHRK8dy7PElvHb7NKaGe/Hk5kSS8yuRUvL16XzmjvLFycEGFePxq0FnDwdsf2PuiLO9M7fH3c53Od/Z3Fp4K+ktCmsLWRxqWZNDSxBCcOO0EF5cM9ns/82skbMIcA7gw/Mf9mn9mZE+ODvo2THILqSE7AoifV3MVmnfOiuclMJq7n3zCHOf2cHft53jic2JLF+3p8tiv+GOUgot+LgaePaGeJILqljy92/4/FQuiTlakNmqfm5LMTZD4ZkeXUcm7lsQhYNexz+3t2/tYCpcyyu1x14vmBBsmSssLsiDJ1fE8oeVcb2TGy3GMTbAjb9+dZa/fnmWlRMDuf+yKLxcHLh8nD/rbp6Ei8GOh987TmKOFvxeZK2so464jdAK/469DbX9b8PRW1aPXW1zayGjIoMXjr/AwpCFLApdZLPrdESv07Ny1Er2Zu8lr7r36aWO9nrmjvJl++n8QZ1il5hT3uWwpeXxI/F1NfDN2UKunRTEjl8sYP0tU6iqb+KGf+/juW2d07gHgk+OZ9us+E8phTYsGOPHpvtm4enswI/fPsqp7PJeuU6sSkkqNNVZZCmAFhi7dXYYm0/kcC7/Yk+n4jrNUkjNE8QFeQxI/yYhBHfMCaewsp74YA+euX5CO8Xq7+bIn64dT0J2BT9++wiguRJsxqz7tdGjR/o+qL6v2NpaMEojT+x9AgedA4/NfGzAH2CuGXUNEskn5/sWzF8U7U9OeR2ncy3rQ2ZtiqrqyS2v6zL4brDTs/knc9jza20UbKSfK1fEjmDbz+ezNHYE/9qVQmn1wDb3K6is4xfvneC179Jtsr5SCh2YEubNpz+ZwxNXxxDo4cjCcTZ6gu2J1syjrtNRO/Kj+VE42+v5x9cXn15MlsLZHPreZK4PXDs5iMeXx/DKbVPNKqKlcSNYNSWYCyW1jA/yIMBMB1arMWI8RMzXhvA0970NR1+xpbWw6dwmjuQf4dFpj+LvbEPF2gUhbiFMHzGdj89/3KdiNtN418GayNZanNpNMkmgpxP+bu3/Pp0d7Hho8Wgamo18OMB9nN49eIEmozQb57EGSimYwU6v4445Eez9zSJmRfn0fIItKEgCoQM/y2c/e7s4cOfcCD47lUdijpZRUVxbjE7oaah3ZGr4wCkFg52eu+ZG7zuYVgAAIABJREFUdPoyteWJq2OIDXTnxqnBthdo5gNQkT3g6amgWQtrotfwXc53fXKzdEV1YzXPHXmOmSNncs0omzUV7pFrR19LVlUWR/KP9PpcfzdH4oM92DFI9QqmzKPYPtTHRI90Jz7Ek/9ZqW28JTQ1G3nnQCbzRvsS4evS8wl9QCmFoUp+InhHgb2ZYfXdcPe8SNwMdry0U8u2Ka4rxknnDuiY3EOQeaBxc7Rn64PzuKUl/92mjL4CfEbBvhdhEPzXyyKWAfBV+ldWW3NbxjaqGqt4YOIDgxP3amFhyEIcdA7syNzRp/MXjPXn+IWyQZmxkJBdTriPM+59nMB387QQzuVXcdRMOrgt+Pp0PnkVddxqw++MUgpDlfzEXrmOTHg42fPDmWF8npBLRnG1ln3U7E6ot3O3T+2XPDodzPyxNo8i2Xo3ZksJdQ9lnPc4tmVs6/KYqoYqPj7/MZtTNrM9YzuJRd333tmSsoVQt1Di/eItF6Q0w+rpuc72zswMnMnOCzv79MR82Vg/jBJ2J1s2WtaaJOSUE9tNMV9PLI8PxNlBz//62TbeUt7Yl0GQpxMLx9nOVaiUwlCkvgpK0yzOPOrIHXPC0esE//k2jaLaImpqnZg6xKyEQWHSLeAfA58+NCiZSEvClnC88LhZF1JicSI3brmRx797nMf2PMbPdv2M1VtXsyV1i9m18qrzOJh3kOWRyy23Esqz4JWFsP5yqyuGy0MuJ7sqm3Olvc/GiQ/2xMvZvnXWw0BRUdfIhZJa4vpRnOpqsGNFfCCfnsilsouZJNbifEEle1OKWTMjFH1/5o30gFIKQ5FCbUi6pZlHHQlwd+TaSUG8f+QCuVUFNNQ7DznX0aBgZ4BrXtJ6Sn3x2wG//BVhVwCwPXN76zYpJW+ffpu1n62lobmB9UvWs/Xarby3/D2ivaN54dgLNJoJjm9J3YJEsjxqufmL5Z2CsgsX3zfWwf/WQlO9ZjW9+0Pt4cNKXBZyGQLRp7bhep1g3mg/dp8r7NRqxpZkl2oNC8N8nPu1zk3TQqhtbObTE7nWEKtL3tqfiYNex03TQmx6HaUUhiJpu7WfI8b3eYl750dS19hESV0xxib3AQ0yD2kCJ8G8X8CJd+Ds59q24hQ48S7U2zYtMtwjnNFeo9vFFV44/gJ/Pvhn5gbOZdPVm5gVOItQ91CifaJ5aPJDZFdl8/6599utI6Xk05RPmeQ/iRC3DjcIYzPsegZeng/PT4GdT0NjLWz9uVbZfd16WPUaFJ2FT+63WnzF18mX8X7j+zxL4rKxfhRVNbTO7xgIcss1pdDf4tSJIZ6MG+HGOwczbBZwrm1o5oMjWSwbPwJfV4NNrmFCKYWhRnMjHPoPhM/Tevf0kVH+biyIdgJhxIAno/2tMLjmUmH+oxAQBx/fD+smwfOT4aMfwcc/tnkQ+oqwKzhWcIyCmgK2Z2xn/cn1XDvqWtYtXIeno2e7Y2cHzmbaiGm8fPJlahovFir9//bOPK6qanvg38UoIA4oIioOOKDmiDibOablnA2OWVk2WGn9Kptfs/Z6r/GV9nLIyrLUV+ZQauac8zxPOAOCoghcBIH9+2NfcGLmXg7E/n4+fO495+yzz+LCPeustdewL3Yf4XHh9A2+wUpIiIbv7oKV70GTu6FRP1g1CT66BXbMgttehIZ3Qt2u0OMNHYm1+l8O+926BnVl3/l9BYqw6txAl3IpShdSZJwu2hhYPn/BHDciIgxvW5M9Zy6x7aRz3JJ/HogmPjmVe8OcayWAUQrFj/0LdOhku8cLPVXfUN0Yp07F6k71QZY43Dxg0BTwKKsjku74AG59Tn/2W6Y79dK317odheKrXV/x8tqXaVq5abZJZyLCuNBxxF6O5dt932buX3h0Ie4u7vSq3evq4LRUmN4LTm6A/p/ZLYJpMGohlK+h+3rfNuHq+A5P60zvFe/AwmcgtfCRP92CugEUqBZS5bKeNKtRnpUHC9Z3vSBEXryMq4vg71v4J++7QmvgW8bNaQllC3ZG4O/rSdtg54fI31zsw2AtG6dAxdrQoHehp/KvoJ+EBjTNe65DqaFqU3hm99Xt9HTtXlnyMtRsr/NDds2GlROhZge4YxJ4Fd4FF1whmHoV6jH74Gz8yvjxYZcP8XTN/qbU3L853YK6MWPvDJJSk7iUconfj/9Ol6AulPe8ZoH0yB86C/6embqPRAZ1boVHV988sQgM+lL3s173CUTvh3u/gbIFj2qpU74OtcrVYsWpFQxpOCTf59/WwJ/PVxwhznalSJpaRcZdJsDX0yEPTD6ebgxpHcT0dceJjEsqtPVxLfGXr/DnwWiGtXHuAnMGxlIoTpzZCqc2QptHs22/mR9ikvRTV5/GRinkiouLth48y8GcUTC1m3YnufvA7jnwRXs4/IdDLtU3uC9u4sa/b/s3VX2q5jr+6dCnSVfpzNw3k+Unl+Pv5c/IxiOvH7T9W11Rt2GfvAvi4go934LB0yBiB0y5FQ4szudvcxURoWtQVzZFbSI+Jf/rMxmhqWuOFI21EHUpiaoOLHZ5f/vaKKUc3phn6d6zpKSm0695NYfOmx1GKRQnNkwBD19oOcIh00XbohGESl4WZWWXNMpWgbu+hHOHIf4s3PUVPP4XPLIcypSHWYNh/ReFvsxDTR5i+b3LCasalqfxdSvUZf3Q9WwbsY1V961i/sD5tKzS8uqAhBg49LuuCOtagCfspnfDw8vApzLMHgpzH4LEguUMdA3qSmp6KqtOr8r3uS2CKlLey73IXEiRFy8TWMFxT/RBft70bBzAD5tOOrRHxIJdEVSv4EVozQq5D3YARikUF+KjYO/P0HI4lHFMEb4YWwyVvCrh7mJRf+mSSN1uMHYjPLUFmt2rLYhqLWHMKqjXE1a8q2/ChUBE8Cvjl69zXF1cs89H2PUjpKdCi0I8TFRtCo+sgK6vwL5f4YN6MKkmfNwMfhgGKYl5mqZFlRYE+QYx5+Cc3AffgKuL0LmBPysPRjs9NFUpRWTcZQIdXHPrwY51uGC7wvwdjqmHFJuYwtrD5+jXvFqRZa0bpeAsjq2Bhc/CrHvhiw76NTY8+/HrPtVf7DZjHCbCWdtZ/L1ybtBjyAL/EPC4oa6MexnobQ/vXP2BNXJlhVLadVSjNVQppJvQzQNuewEeWwtdXoRmQ6B6Kzi4CDZ+macpXMSF+0LuY1v0tgIlsvVoVIVzCSnsOO3cshFxSVdIupLmUEsBoG0dPxpW9eXLVeGcT8h7m/mT522kpN5cUPD3PVGkpiv6Nb+5r7qzMErBWSwcDzt/gPgIqBAEJ9fD5I76y5V+wx///FFdwbPlCKjkmM5ZoC2FAG+Lqrz+HalcH0JH6gil2GNWS6M5s00nOzrI5Qho5dLlRbjzn3DPDKjfC9Z9DEl5u1EPqDsAT1dPfjzwY74v3aVBFVxdhOX7nVs19Wo4qmMtBRHhlT6NOHMxiUFf/MWR6NwTBP/Yd5bOH6yg2ZtLGPbVBj5cdogFOyPYdfoiv2w/Q7C/T5GW8DdKwRnEhsP5IzoW/LG1MOxHeGID1OoAv70A3w26PlHqj3+Aqwd0e9WhYkTbovH3NpaCQ7ntRXBxgxXvWSfDqc0Qvko/TGydDm5eOuTUWXR7FS7HwV+f5Wl4hTIV6F27NwvCF5CQkr+s6fLe7rSuXZHlTu7G5qjEtay4tb4/s8e0w5aSyl1frOOnzaeYsuooT36/jdFfb76pR/m0tceoWq4MQ9vUJC7pCp/9eZinfthO//+sY9PxWPo2KzrXEZiQVOeQEaVS75rWiOWrw/C5sG2mdit9dzcMn6PLEexfAF1f1V3CHERKWgoXki9YUmP/b025QJ1DsvZD6PAUBDYr2uvv+F5HRV1L86EOW4fKksBm0GQwbJgMbR/NU9jqkIZDmH90Pr8e/ZVhjYbl63I9GgXwzqL9nIq1EeRXuBIU2eGoxLXsaFmzIj8/0ZGHvt7MC/N2AVC9ghdnLiYxY91xxnatB8DBqHjWh59nQu+GPN5FewlsKamcjLVx4ryN6EuX6d+iulNkzA5jKTiDw0t12esbXUEi0OoBuHs6nN4M3w2GJS9BuerQfqxDRcgIRzVKwQl0HKdzForaWojcqRPNat8KoxbAwCnQ402HW5hZ0vUV3Qlwzb/zNLxJ5SY0qdSEHw/+mO/SDxmtWZ3pQnJk4lp2BPl5M//Jjsx9rD3bX+vJuhe70aNRAFNWHc0sEz5z/XE83VwYck09I28PNxpWLUevW6oysn1tynsVbaCIUQqOJsUGx9fo+v3ZcctA7auN2Ka/6D3eAA/HPhHF2IxScBpeFaDtY3DoN4g5WDTXtMXCjyPBy0/XLqrTGVoMhU7jdcays6lUV69bbJ6mrZU83Ojva3gf4XHhbI7anK9L1ansQ7C/D8ud2HjHkYlrOeHt4UZYbT8q+ngA8HyvEBKSU5m86ihxtiv8vO0MA1pUyzxeHLBEKYhIBRGZKyIHRGS/iLQXET8RWSYih+2vJbOC2/G1+omqfo+cxzUeAENn66fOJnc7XIyzNv2UZaKPnETrh8GtTJ797Pkm7gxM7Qmz7oHFL+gKp5ci7FnHFv1Nu7+uo5x+eRy+vw8u5VwVtHft3lT0rMg3+77J96V6NgpgQ/h5p5WjdnTiWl4JqerLoBbV+XrdcT778zBJV9IY1aF2kcuRE1ZZCp8AvyulGgLNgf3Ai8BypVR9YLl9u+RxZJle+KvVKfex9XvqjFIXx/8ZMiwFE33kJHwqQ4vhOkcg3nEtNjPZ8T2c3qRvvDtm6ZpGd/4Tglo7/lp5xacyPLAIek/SlXw/bwM/P6bzay7H3TS8jFsZhjYayqrTqzh84XC+LtW9UQBX0hSrDzmn8Y6jE9fywzM9G5CuFFPXHqNNbb8c+0NbQZErBREpB3QGpgEopVKUUheBAcBM+7CZgHVNZwuKUno9Ifg2HdduIdG2aDxcPK6vj2NwLO3H6qq2eYzhz0QpSDyf85g983TNpcfXwkun4eUzEPZQwWV1FBkd7B5fByF36PLjcx6AD+pD+Mqbhg8NGYqXmxdf7/06X5cJrVmBCt7ufL/pBDtOXeRK2s0x/AXFWYlreSXIz5thbWoCFDsrAayxFIKBGGCGiGwXkaki4gMEKKUiAeyvWTrDRWSMiGwRkS0xMQXMLD13GOY/qRuOOJLzR+DC8eujjiwiOkmHo1rZu/dvT6W60KgvbJmWv4Y1S16GD4LhszBY/PzV/hkZnN0HMfuhiT3MVCTfvbqdTqW6uhLr80fhwd91Ls6vT92U+VyhTAXuqn8Xi8MXE5mQ9yY0bq4ujGhbi3VHzjPw83U0fWMJ42ZvJzm18OUjMhLXrHAfZfB/vUKYeFdTejdxXMSho7BCKbgBocBkpVRLIJF8uIqUUv9VSoUppcL8/QvoW714UmeBrv+8YOdnx2F7/936PR07bwGItkUb11FR0GGcdp1s/zb3sQAHFsGGL6DBHboa7rZvYWY/nXeQwd7/gbjodafijqsb1GoP/T7R36uVE68eUwpObeL+enehUPleW3iuVwgbX+7O58NCGRxag/k7Injmxx2kFbIERkY4ajWL3EcA5cq4M7SIqp7mFyuUwmngtFJqo317LlpJnBWRQAD7q/NCD+p1h4Z9dbmCOMfUKOHyJV1quXKI/rJbTIwtxiSuFQVBrbWbZ82HWfrVr+PiKd3YJ7AF3DsTRsyFF45C+Zqw9BXdNU0p7Tqq07lQZayLnNqdIPR+/aAVsUNnP88bDdN6Um3WMO4I7Mi8w/OIS87lM7qBgHJl6NMskHcHNeWVOxuxeHcUr83fU6gOZxmJa1ZaCsWZIlcKSqko4JSIhNh3dQf2Ab8Co+z7RgHznSpIr3dBpcNSB8R4nz8K03pC1B7o/Fzh5yskSinO2s6acNSiotc7kBgDq/6Z/Zi0VJj3sL7x3zND94sGXWOpxz90EuPOHyByh86Id2aGsrPo+RZ4V9Zd7KZ0gr2/QNvHIekCD2xfSFJqEj8c+KHA0z/SOZjHbqvL9xtP8tGy/NdVyiDTUnBS4prTuHxJW5RrP4afRsGOgn+WOWFV9NFTwCwR2QW0AN4DJgE9ReQw0NO+7Twq1oZOz2hT/diags2Rkqj/8b/qBglnYeTPurKmxSReSSQpNYkqXkYpFAnVW+kY/o1Tss9b+OsTOLUB+n0MfsHXH2syWId6Ln9bu5Nc3HQrzZKGV0UdIRVzQJfwHr1MNyd6dBUhFerQ2ZbEdzsmY1s1SUdTpeU/3HRC7xDuCwvi0z+PMGfLqQKJWRSJa5nYYuHUpsLPc3YffNwEvumvy+JEbIdk5/SztkQpKKV22NcFmimlBiqlLiilziuluiul6ttfY50uSMdxUKGmXuzL6z+oUrqi6dQeurTwnFHgG6jLDgffViAxlFL8dPAnHvj9AbZHby/QHNcSbdOeN2MpFCHd/6Eb8vw24ebErqSLsPYTCOmjexfciAj0mggJUXrRum538M5fae1iQ+OBeuH50TVQo5XeV74GPPgbj9TsTRzpzNn6H9069KMmOis8Hy5cEeGdQU3oWK8SL/+8m43huURxZUFk3GWqODtxTSnYNQf+E6a9CAufLXhgiy0WfhiiQ92Hz4UXjsH4XbrkiBMo3RnN7l7Q+30d6ZHH9H1W/ROWvaZdTx2ehuHzYMxK8KtTIBGupF3hrQ1v8faGt9l7bi+jfhvF+5vev65Re37JTFwzawpFR1l/6PoyhK+AAwuvP7ZhMiTHQdeXsj8/qLW2GOBq1FFJREQvPHuWvX6/uxct+vyHtlXb8nX1uiQPnqprKq36J3zcVOc75JIMlzmVqwtfDGtFUEVvHvtuKyfO563XQwZRl5KcUggvkwsn4Pt74X8Pa49Em0e1sp92u45OzA9pqTrkNz4S7vtOB7E4+YHBFMRreKeuG7/qfd1gJahN9mN3/ggr34Pmw2DgF/oLUAguXL7AuBXj2B69nUeaPsJDTR7i0+2f8t3+71h5aiUfd/2YEL+Q3Ce6gYy6Ryb6qIhp/fDVgocBt2g3UdIFHW3UqJ9uZJMTvd7TVmej/kUjrwWMaTaG0UtH87N7GkOGz9E3yU1f6dLx+36F256Hdk9cXXPJhvLe7kx7oDUDP1/H0P9uoGvDKjSuVo42tf2oH+Cb47mRFy/TyBmlqG2x+uFy03+1C7DXRP007+IKwV3gl8fg83b6fb3u2u148aTuj207Dx2f1p6LDJTSa57HVsGAL4oscVEKs4pvNWFhYWrLli2Fn+jyJb0wBrrU9Y0VJ9PT4cRa+PYuqNkORvxPNyQpJJ9u+5Tpe6Yz6dZJ9K7TO3P/lqgtTFgzgfiUeCZ2mkj3Wt3zNe/U3VP5ZNsnbBq+CS+3EraYVtKJOQjTe4OnLzy0RD8hrv4AHlsHVZtYLZ3lKKUY+dtIYmwxLLxr4dWugLHhsORV3dCnXA1o/4SOZvLM+Qa/9UQsHyw5yN6IS8RfTsVFYPaY9rSpk/XTtFKKxq8vYXjbmrzat3Hhf6G0VN1X/eBivR6UEg8thkGXl26uSXXhuC6LcngZXLy2j7PoNRivirrMfrWWOu9l0bM6Y77t43ptxoGIyFalVJb9YI1SyODkRpjRW9chqt9TZ2qGr9AKQ9kTZiqHwOiluiCaA3hh1QvsOb+HxXfd3Cw9xhbD+BXj2XVuF2NbjOXRZo/mORHtvY3vsTB8IX8N/cshchryyZmtMLM/lA+CuNNQr5uuWWQAYPXp1YxdPpa3OrzFoPqDrj949E9Y/S84sQ48y+sbbMM7oWb7HPtPK6U4GWtj2FcbKevpxsKnO+HuerN3/KIthRZvLePVPo14+NbgLGbKheQErQQid+jQ2+NrtDXo4g4NeulqsgG5KBultBKM2gUV60DlBlpJzLpHWww93oTNX+lk2C4vwa3/p60NB5KTUjDuowxqtoXOz2s30u6fwKcKhNypexy4uOuyFc2GOEwhAEQkRlDNp1qWx/y9/Zneezpv/PUGn+/4nNrla9O7du8sx95ItC3aRB5ZSfVWMGSW/pKnXdGNeQyZ3Fr9Vhr5NeLLXV/SN7gv7tfe7Ot20z+nt8Jfn2pLa+NkKFNeu217vZvlDVJEqFXJh3/0a8yYb7cyY90xxnS+uYthRvOeYH+fm47lysWT8HUf/Qr6hl6/ly73Ubdb3ntaiOiM8GtL61dpBA8v12sRvz0PZQPg/vk6X6WIMUrhWjq/oH16/g2hWqhTCtVdS2RiJO0D22d73NPVk3c6vsPBCwf5dNundK/Z/aq5nQMxthgTeWQ1wV1gxDx9A8ntybGUISI8Hfo0j//xOPMOz2NIwyE3D6rRSif4JSdoi33ffK0cUpOg78f6xqqUdsds+q+O7ElP5Xa3Mnxf+Rbm/nELkSFPEBhwdV0tLukKE3/bT/OgCnRpkM/vR9wZnXl+OQ6G/KC7KDrwAREA3wB4cLEuhth4gGXJi0YpXIurm2N73ebAlbQrxNhiqFY2a0shUyQXV8aHjmfs8rHMPTSXoQ2H5jr3WdtZ2ga2dZSohoJiwVNeSaFjtY60CmjFl7u+pH/d/ni7Z9NPxLOsXqRv1E+749Z+qBPkOj+nay3tnqM/Z79gvbhri6Xt0RV0cFlG6uT/QIOe0OweaHAHnyw9jGviWd4f3AKX7MJRk+Nh+3ew9WvwLKetgJrtdK00Wyzc/4u2BJ2Fhw+0ecR58+cBoxQs4qztLApFoE9grmNvrX4rrau2ZsrOKfSv2x8f9+xN3/iUeM4lnaOqT/ErtGUwZCAijAsdx/2/3c8PB35gdNPRuZ/U/XXtc1/zL9j1E8Sdgm6vaZ/7NettrmmpzFvwC+e3/I97j26kwqHfUOLG6yqV1z2BH9Hlv9td09ZUKZ0zsXGKTgqr0QbSUmD5m/q4R1mdnOpMhVBMMErBIiITdUx2YNnclYKI8EzoMwxbPIyZe2fyRIsnsh27OHwxaSqNbkHdHCarweAMWlZpSecanZm+Zzr3hNxDOY9cfPIi0Pcj7cI5shyGfK8XoW/E1Y2B/QfzhW9z2q04REfXA/Rw38OFVHce7NkaryOL4I839cJwRnb5+s9h9T91OHCn8Vdv/pci4MgfULUZVGvh2A+gmFK6k9csJFMp5MFSAGjq35Tba93O13u/Zu+5vVmOUUox9/BcGvo1pHEl48c2FH+ebvk0l1IuMX339Lyd4OIK93wNzx3KWiHYcXURnupen9/HdyE5qBMvxd+Nf5/X8OrwiM4xcnWHX5+2V3LdrEtHNOqno8SutQbKVdOhsaVEIYBRCpYRkRABkC83z/jQ8Xi5eTFk0RBeXPNi5hwZ7Du/jwOxBxhcf7Dpo2AoEYT4hdAvuB/f7PuGE5dO5H4CaIshjz3Na1f24dvRbVj1fBfuCQvSO8tVg9vf1uGk6z6BuQ/pff3/U+iE1L8DRilYRFRiFJXKVMLTNe9FuYLKBbFw0EJGNxnNHyf+oN/P/Vh+Ynnm8bmH51LGtQx9gvs4Q2SDwSk8G/Ysnq6eTNw4sVAlsbMjI1z1OkJH6QXqP/6hS0jc/bXjo4lKKEYpWEREQkSukUdZ4evhy/hW41kwcAEN/RoyYc0EdsXswnbFxuLwxfSq3Qtfj5yzQA2G4kRlr8o82fJJ1kWsY/nJ5bmf4AhEoN+nUK469J54tXifwSgFq4hMjMzzekJWBJYN5LPun+Hv5c9Tfz7F1N1TsaXauLtBFlU4DYZizn0h9xFSMYT3NxeuGGS+8KsDz+y1PAS0uGGUggUopQqtFAD8yvgxucdk0lQaX+3+irrl69Lcv7mDpDQYig43FzdeafcKUYlRfLT1I6e4kbLErCHchFEKFhB7OZbktOQ8haPmRu3ytfm066d4uXkxsvFIs8BsKLG0rNKS4Y2GM/vgbF7/63WupOe/CY+h8Jg8BQuISowC8h6OmhuhAaGsHbIWD9fCV241GKxkQusJ+Hr4MmXnFKISo/iwy4dmjayIMZaCBUQk6lDSgiw0Z4dRCIa/AyLC2Ba6guqWqC2MWDyC8IvhVotVqjBKwQIiE/KXuGYwlDYG1R/ElJ5TuJh8kSGLhrA4/Oby8gbnYJSCBUQmRuLt5p17Wr/BUIppG9iWn/r+lBl6PWnTpKJbgC7FGKVgARmRR2ZR2GDImQCfAKb1msbwRsOZtX8Wn23/zGqR/vaYhWYLiEiIcEjkkcFQGnB3cWdC6wlcTr3MV7u/oqpPVe4Nuddqsf62GKVgAVGJUTSpbPr1Ggx5RUR4td2rxCTF8O7Gd/F09aSZfzO83LxIV+kciztGeFw4ZxPP4uPuQ1mPspT3LE+gTyA1ytagincVXB3c0vLvilEKRYztio0LyRccGnlkMJQG3Fzc+KDzB4xeMppX172a5RhPV0+S05Jv2u/l5sX/tfo/7g25t1i6bZPTkolPiSfxSiJp6WnUKV/HMjmNUihiomyOzVEwGEoT3u7eTOs1jY2RG7Gl2khKTQKgVrlaBJcPxq+MH+kqnYQrCVxMvsiZhDOcSTjDsuPLeGfjO+w6t4vX2r3GxeSLzNgzgwVHF9C+WntebPMi/t7+TpFZKcWxuGNsiNxAREIE7au1p03VNri7urP33F5m7J3BshPLSFfpmed0rN6RN9u/SYBPQOYcRy4eoZxHOap4V3GqwhCrVvNFxBXYApxRSvUVkTrAbMAP2AaMVEql5DRHWFiY2rJli/OFdSDrzqzjsT8eY2bvmYQGhFotjsFQKkhX6UzZOYXJOydTvWx1ztrOgtI33/UR6/F09WR8q/Hc3eBuXCR/8TdKKZYcX8LmqM2kqTRS01NJSUsh/ko88SnxnEk4w7mkc4C2dlLTU/H18KWmb032nt+Lr7svA+sPpJZvLbzdvYm2RfPlri9xc3HjmVbPEJsUy4LwBZmlxf3K+NHQryED6g7gzuDse0rkhIhsVUqFZXXMSkthHLAfyIjLfB/4SCk1W0SmAKOByUUhiO2KLfsesQ5YaDKgAAAJZElEQVQmo7mOcR8ZDEWHi7jwRIsnaFK5Ce9vep/B9QfzUJOHqFa2GsfjjvP2hrd5e8Pb/Hr0V15r9xohfiEApKWnsS16G5W8KhFcPvimefee38ukjZPYEbMDXw9fvFy9cHVxxcPVg7LuZfH18KVdYDvCAsJoG9gWf29/NkRsYNmJZRy6cIjnwp5jcP3BlPUoe928PWr14NW1r/LW+rcACAsIY9Qto0hJS+FA7AEOxB4gJinGKZ+VJZaCiNQAZgLvAs8C/YAYoKpSKlVE2gNvKKV65TRPYSyF5LRklh5fyk8Hf2JHzA5aBbRiWMNhdKvZDTeXwunKPef28O2+b1FKMaLxCJr5NwMg8Uoib294m9+P/c6WEVsKfR2DweAYlFL8evRX/r3l31xKucTQhkNxd3Vn0dFFRCdFA9AlqAsPN32YAO8A1p5Zy+rTq1l5aiUVy1RkXOg4BtQd4NDF7LT0NDZEbqB2+dpUL1vdYfNCzpaCVUphLjAR8AWeAx4ANiil6tmPBwG/KaVuCtERkTHAGICaNWu2OnEij92armHlqZW8vu51LiRfoFa5WnSu0Zk/T/7JmYQz+Hv5U6tcLXw9fLXmd/PC280bL3cvutfsToOKDbKcMz4lnvUR65l9cDabozbj6+4LoveHBYRRvWx1lp5YSlJqEh2rdWRKzyn5lttgMDiXuOQ4Ptn2CXMPzcVVXOlUvRN96vYh/GI43x/4nrjkuMyxVX2qcmedO3m46cMlrj5TsVIKItIXuFMp9YSIdEErhQeB9TcohcVKqaY5zVVQS+FY3DE+3vox9zW8j3aB7XARF9LS01h9ejWLji3ifNJ54lO0PzBjMSs5LRlB6Fe3H2NbjKWCZwV2ndvFtrPb2BC5gV0xu0hTaVTxrsL9je/PbIk579A8vtn3DfEp8fSu05tB9QbR3L95sYyAMBgMmlPxp/B286aSV6XMfbYrNuYfnU9yajKdqneiboW6JfZ7XNyUwkRgJJAKlEGvKfwM9KII3Uf5JS45jmm7pzFr/yzSSUcpRZpKQxAaV2pMh2od6Fi9I838m+Hu4n7duWnpaaSTftN+g8FgsIJipRSuu7jdUrBHH80B5l2z0LxLKfVFTudbEX0UlRjFt/u+xdPVk9CAUJr7Ny9xpqPBYCjdFNfooxuZAMwWkXeA7cA0i+XJkqo+VXm+9fNWi2EwGAxOwVKloJRaCay0vw8H2lgpj8FgMJR2TJVUg8FgMGRilILBYDAYMjFKwWAwGAyZGKVgMBgMhkyMUjAYDAZDJkYpGAwGgyEToxQMBoPBkImlGc2FRURigPxXxNNUBs45UJyipKTKbuQuWozcRUtJkruWUirLrkIlWikUBhHZkl2ad3GnpMpu5C5ajNxFS0mV+0aM+8hgMBgMmRilYDAYDIZMSrNS+K/VAhSCkiq7kbtoMXIXLSVV7usotWsKBoPBYLiZ0mwpGAwGg+EGjFIwGAwGQyalUimISG8ROSgiR0TkRavlyQsiMl1EokVkj9Wy5AcRCRKRFSKyX0T2isg4q2XKCyJSRkQ2ichOu9xvWi1TfhARVxHZLiILrZYlP4jIcRHZLSI7RKRo2yoWAhGpICJzReSA/X+9vdUyFZRSt6YgIq7AIaAncBrYDAxVSu2zVLBcEJHOQALwjVKqidXy5BURCQQClVLbRMQX2AoMLAGftwA+SqkEEXEH1gLjlFIbLBYtT4jIs0AYUE4p1ddqefKKiBwHwpRSJSUJDAARmQmsUUpNFREPwFspddFquQpCabQU2gBHlFLhSqkUYDYwwGKZckUptRqItVqO/KKUilRKbbO/jwf2A9WtlSp3lCbBvulu/ykRT1AiUgPoA0y1WpbSgIiUAzpjbyGslEopqQoBSqdSqA6cumb7NCXgJvV3QERqAy2BjdZKkjfsLpgdQDSwTClVIuQGPgZeANKtFqQAKGCpiGwVkTFWC5NHgoEYYIbdZTdVRHysFqqglEalIFnsKxFPgCUZESkLzAPGK6UuWS1PXlBKpSmlWgA1gDYiUuzddiLSF4hWSm21WpYC0lEpFQrcAYy1u02LO25AKDBZKdUSSARKxFplVpRGpXAaCLpmuwYQYZEspQK7T34eMEsp9T+r5ckvdlfASqC3xaLkhY5Af7tvfjbQTUS+s1akvKOUirC/RgM/o929xZ3TwOlrLMm5aCVRIimNSmEzUF9E6tgXhIYAv1os098W+4LtNGC/UupDq+XJKyLiLyIV7O+9gB7AAWulyh2l1EtKqRpKqdro/+0/lVIjLBYrT4iIjz0YAbv75Xag2EfbKaWigFMiEmLf1R0o1oEUOeFmtQBFjVIqVUSeBJYArsB0pdRei8XKFRH5AegCVBaR08A/lFLTrJUqT3QERgK77f55gJeVUostlCkvBAIz7dFqLsBPSqkSFd5ZAgkAftbPEbgB3yulfrdWpDzzFDDL/qAZDjxosTwFptSFpBoMBoMhe0qj+8hgMBgM2WCUgsFgMBgyMUrBYDAYDJkYpWAwGAyGTIxSMBgMBkMmRikYDICIVLJX5twhIlEicuaa7b+cdM2WIpJtfSJ7rkRJCck0/E0odXkKBkNWKKXOAy0AROQNIEEp9S8nX/Zl4J0cZIoRkUgR6aiUWudkWQwGwFgKBkOuiEiC/bWLiKwSkZ9E5JCITBKR4fa+C7tFpK59nL+IzBORzfafjlnM6Qs0U0rttG/fdo1lsj0jsxf4BRheRL+qwWCUgsGQT5oD44Cm6EztBkqpNugy1U/Zx3wCfKSUag0MJusS1mFcX8LhOWCsvQDfrUCSff8W+7bBUCQY95HBkD82K6UiAUTkKLDUvn830NX+vgfQ2F6uAaCciPja+0lkEIgut5zBOuBDEZkF/E8pddq+Pxqo5vhfw2DIGqMUDIb8kXzN+/RrttO5+n1yAdorpZLIniSgTMaGUmqSiCwC7gQ2iEgPpdQB+5ic5jEYHIpxHxkMjmcp8GTGhoi0yGLMfqDeNWPqKqV2K6XeR7uMGtoPNaAEVAo1/H0wSsFgcDxPA2EisktE9gGP3TjAbgWUv2ZBebyI7BGRnWjL4Df7/q7AoqIQ2mAAUyXVYLAMEXkGiFdK5ZSrsBoYoJS6UHSSGUozxlIwGKxjMtevUVyHiPgDHxqFYChKjKVgMBgMhkyMpWAwGAyGTIxSMBgMBkMmRikYDAaDIROjFAwGg8GQiVEKBoPBYMjk/wFURZjo5gIJGgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "key = dict(mouse_id=0, session_number=1, scan_idx=1, seg_param_id=1)\n", + "\n", + "# ENTER YOUR CODE! - fetch 'time' from the Fluorescence table using fetch1()\n", + "time = (Fluorescence & key).fetch1('time')\n", + "\n", + "# ENTER YOUR CODE! - fetch 'trace' from the Fluorescence.Trace table using fetch()\n", + "traces = (Fluorescence.Trace & key).fetch('trace')\n", + "\n", + "plt.plot(time, np.vstack(traces).T)\n", + "plt.xlabel('Time (s)')\n", + "plt.ylabel('Fluorescence')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Summary" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Congratulations! You have successfully extended your pipeline with all types of tables. This pipeline is simple, but reflecting a typical preprocessin pipeline of calcium imaging. We have all the table tiers and major dependencies between DataJoint tables.\n", + "\n", + "**Table tiers**: \n", + "Manual table: green box \n", + "Lookup table: gray box \n", + "Imported table: blue oval \n", + "Computed table: red circle \n", + "Part table: plain text\n", + "\n", + "**Dependencies**: \n", + "One-to-one primary: thick solid line, share the exact same primary key \n", + "One-to-many primary: thin solid line, inherit the primary key from the parent table, but have additional field(s) as part of the primary key as well" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "data": { + "image/svg+xml": "\n\n%3\n\n\n\nFluorescence.Trace\n\n\nFluorescence.Trace\n\n\n\n\n\nFluorescence\n\n\nFluorescence\n\n\n\n\n\nFluorescence->Fluorescence.Trace\n\n\n\n\nMouse\n\n\nMouse\n\n\n\n\n\nSession\n\n\nSession\n\n\n\n\n\nMouse->Session\n\n\n\n\nScan\n\n\nScan\n\n\n\n\n\nSession->Scan\n\n\n\n\nSegmentation\n\n\nSegmentation\n\n\n\n\n\nSegmentation->Fluorescence\n\n\n\n\nSegmentation.Roi\n\n\nSegmentation.Roi\n\n\n\n\n\nSegmentation->Segmentation.Roi\n\n\n\n\nSegmentationParam\n\n\nSegmentationParam\n\n\n\n\n\nSegmentationParam->Segmentation\n\n\n\n\nSegmentation.Roi->Fluorescence.Trace\n\n\n\n\nAverageFrame\n\n\nAverageFrame\n\n\n\n\n\nAverageFrame->Segmentation\n\n\n\n\nScan->AverageFrame\n\n\n\n", + "text/plain": [ + "" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dj.Diagram(schema)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have covered most of the building elements of data pipeline design. Using these elements, we could design more sophiscated pipelines that facillitates your experimental recordings and data analyses." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.16" + }, + "vscode": { + "interpreter": { + "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} From a3441b4535c4594bb3732db652ed52bab54c806b Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:31:29 +0200 Subject: [PATCH 46/70] Revert "Rename including a space" This reverts commit e841910e598e87fb554456d3b6cd769b1f9010d7. --- .../03-Calcium Imaging Computed Tables.ipynb | 3089 ----------------- 1 file changed, 3089 deletions(-) delete mode 100644 completed_tutorials/03-Calcium Imaging Computed Tables.ipynb diff --git a/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb b/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb deleted file mode 100644 index 8d4c529..0000000 --- a/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb +++ /dev/null @@ -1,3089 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Working with automated computations: Computed tables" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example. \n", - "\n", - "In this session, we will learn to:\n", - "\n", - "* compute various statistics for each neuron by defining a `Computed` table\n", - "* define a `Lookup` table to store parameters for computation\n", - "* define another `Computed` table to perform spike detection and store the detected spikes\n", - "* automatically trigger computations for all missing entries with `populate`\n", - "* define a `Part` table to save the results computed with the master `Computed` table" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's import `datajoint` again." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import datajoint as dj" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As we are going to perform some computations, let's go ahead and import NumPy and Matplotlib." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "import numpy as np\n", - "import matplotlib.pyplot as plt\n", - "%matplotlib inline" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Similarly as before, to continue working with the tables we defined in the previous notebook, we can either redefine the classes for each table `Mouse`, `Session`, `Scan`, `AverageFrame` and populate them. Or, again for your convenience, we can import them from the `tutorial_pipeline.imaging` module. " - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Connecting shan@workshop-db.datajoint.io:3306\n" - ] - } - ], - "source": [ - "from tutorial_pipeline.imaging import schema, Mouse, Session, Scan, AverageFrame" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " Experiment session\n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

experiment_setup

\n", - " experiment setup ID\n", - "
\n", - "

experimenter

\n", - " experimenter name\n", - "
\n", - "

data_path

\n", - " \n", - "
02017-05-150Edgar Y. Walkerdata
02017-05-190Edgar Y. Walkerdata
52017-01-051Fabian Sinzdata
1002017-05-25100Jacob Reimerdata
\n", - " \n", - "

Total: 4

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date experiment_set experimenter data_path \n", - "+----------+ +------------+ +------------+ +------------+ +-----------+\n", - "0 2017-05-15 0 Edgar Y. Walke data \n", - "0 2017-05-19 0 Edgar Y. Walke data \n", - "5 2017-01-05 1 Fabian Sinz data \n", - "100 2017-05-25 100 Jacob Reimer data \n", - " (Total: 4)" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Session()" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

depth

\n", - " depth of this scan\n", - "
\n", - "

wavelength

\n", - " wavelength used\n", - "
\n", - "

laser_power

\n", - " power of the laser used\n", - "
\n", - "

fps

\n", - " frames per second\n", - "
\n", - "

file_name

\n", - " name of the tif file\n", - "
02017-05-151150.0920.026.015.0example_scan_01.tif
02017-05-152200.0920.024.015.0example_scan_02.tif
1002017-05-251150.0920.025.015.0example_scan_03.tif
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx depth wavelength laser_power fps file_name \n", - "+----------+ +------------+ +----------+ +-------+ +------------+ +------------+ +------+ +------------+\n", - "0 2017-05-15 1 150.0 920.0 26.0 15.0 example_scan_0\n", - "0 2017-05-15 2 200.0 920.0 24.0 15.0 example_scan_0\n", - "100 2017-05-25 1 150.0 920.0 25.0 15.0 example_scan_0\n", - " (Total: 3)" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Scan()" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

average_frame

\n", - " average fluorescence across frames\n", - "
02017-05-151=BLOB=
02017-05-152=BLOB=
1002017-05-251=BLOB=
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx average_fr\n", - "+----------+ +------------+ +----------+ +--------+\n", - "0 2017-05-15 1 =BLOB= \n", - "0 2017-05-15 2 =BLOB= \n", - "100 2017-05-25 1 =BLOB= \n", - " (Total: 3)" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "AverageFrame()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `imaging.py` also fill each table by inserting manually and loading data from the external tiff files." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Computations in data pipeline" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now it's time to perform more complicated analyses. \n", - "\n", - "When you perform computations in the DataJoint data pipeline, you focus and design tables in terms of **what** is it that you are computing rather than the **how**. You should think in terms of the \"things\" that you are computing!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, we would like to detect cells from the average image. The final product we would like is the binary **mask** for each individual cell, with the 1 in the region of interest (ROI) and 0 in other places." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So the new \"thing\" or entity here is `Roi`, where each entry corresponds the mask of one ROI. Let's start designing the table, paying special attention to the dependencies." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Detect cells from the average fluorescence image" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now let's perform the segmentation to isolate ROIs. Let's start by taking a look at one average fluoresence image." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "keys = AverageFrame.fetch('KEY')\n", - "\n", - "# pick one key\n", - "key = keys[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

average_frame

\n", - " average fluorescence across frames\n", - "
02017-05-151=BLOB=
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx average_fr\n", - "+----------+ +------------+ +----------+ +--------+\n", - "0 2017-05-15 1 =BLOB= \n", - " (Total: 1)" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# ENTER YOUR CODE! - preview an AverageFrame for a particular key\n", - "AverageFrame & key" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "avg_image = AverageFrame.fetch('average_frame')" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", - " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", - " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", - " ...,\n", - " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", - " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", - " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]]),\n", - " array([[ 29.53, 30.8 , 30.96, ..., 230.09, 230.78, 228.19],\n", - " [ 29.61, 29.69, 30.71, ..., 224.86, 227.21, 227.22],\n", - " [ 29.61, 28.28, 30.19, ..., 220.51, 224.43, 224.85],\n", - " ...,\n", - " [ 23.67, 23.05, 24.06, ..., 32.54, 32.07, 35.28],\n", - " [ 24.03, 22.45, 22.4 , ..., 31.67, 30.75, 32.72],\n", - " [ 25.78, 23.27, 21.92, ..., 31.12, 31.29, 32.94]]),\n", - " array([[41.55, 40.2 , 38.3 , ..., 55.67, 57.17, 58.65],\n", - " [43.66, 42.2 , 38.59, ..., 50.35, 54.83, 57.9 ],\n", - " [42.07, 41.46, 38.3 , ..., 49.46, 52.67, 55.61],\n", - " ...,\n", - " [30.04, 31.05, 30.92, ..., 45.04, 44.93, 44.81],\n", - " [28.88, 29.52, 29.93, ..., 46.5 , 45.65, 45.13],\n", - " [27.84, 27.34, 28.5 , ..., 42.75, 43.28, 45.17]])], dtype=object)" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "avg_image" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "It's a bit subtle, but `fetch` returns a NumPy array of the attribute, even if the attribute contains a NumPy array. So here, we actually got a NumPy array of NumPy array. We can of course just index into it," - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", - " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", - " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", - " ...,\n", - " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", - " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", - " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "avg_image[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "but if we knew that there was only one item, we can use `fetch1` instead to save some trouble" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [], - "source": [ - "avg_image = (AverageFrame & key).fetch1('average_frame')" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([[32.73, 30.77, 31.74, ..., 65.76, 62.77, 58.65],\n", - " [30.07, 27.96, 28.74, ..., 60.11, 58.1 , 54.59],\n", - " [28.91, 27.02, 27.12, ..., 58.01, 56.1 , 53.25],\n", - " ...,\n", - " [35.17, 35.56, 34.37, ..., 21.08, 21.45, 21.09],\n", - " [33.14, 35.73, 37.64, ..., 20.42, 21.24, 21.51],\n", - " [33.52, 37.65, 41.71, ..., 19.82, 20.29, 21.36]])" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "avg_image" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's plot to take a quick look:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO19fayt2V3Ws/Y+e5+vO7cz04/h2ja2TUa0kigNwQJGG+oHVEJjAqRIdISaRgMoqLGt/AEmkoAaBKMBJxQtpnRaPrQNoogVYkxkZBDEQimMgO3Qse3A3Hvmno+9zzl7+cfev/c8+3mftfbpvXPO3Z2znuTmPXfvd693vetd7/r9fs/vY6WcMxoaGq4uBve6Aw0NDfcWbRFoaLjiaItAQ8MVR1sEGhquONoi0NBwxdEWgYaGK44LWwRSSl+WUvpoSunJlNI7Luo6DQ0Nd4d0EXECKaUhgN8A8GcBPAXgFwB8bc751573izU0NNwVNi6o3S8E8GTO+bcAIKX0GIA3A7CLwHA4zBsbG+AFKaW0dMw5wy1Y8f1gMOiO8dl5wG3G33wtPerfpf6UrlE7Tz9z13aI71NK1TbOc83ZbHau+9PjZ4Lab2vjd6fjzvce8ySlhOFwCADY2Ji/Bpubm9je3u7+BoDRaNT9Ls6PNgDg5OQEAHB6enquvs5ms6Xv3XyNtk5OTrq/43ez2ax3rfjOXT/n3H3/7LPPPpNzfqn26aIWgZcD+Dj9/ykAf4JPSCm9DcDbAGA4HOLGjRs4OTnpBpgHH1gekDgCwHg8BgBcu3YNALCzs9P9Rh+Emyiz2ax7kHycTqcAgOPj46XvZrPZ0gOJ68Q1Y0INBoPu+jpRUkpdv7lPPEH52ny/fL5OkI2Nja7dwGQy6c6NScxtRX+jr5PJpLt3vi4wf07xXKKt0WjU/a2TkhclfhZxzTgOh8Ole+Bjzrm7hxhHJyyGw2Hv+cZ9nJycdP2Ol3xra6ubMy95yUsAAA8//DA+7/M+DwDwmte8BgDw0EMPAQCuX7+O++67D8B8jsX4PPvsswCAvb293jV1/s1ms969jMdjbG1tLY3b7du3AQDPPPNM1/7BwUF3jO8PDw+7a8az0jkzmUxwdHQEAHjf+973f2FwUYuAEw9LI5JzfhTAowAwHo/z8fExjo+Pey8wr7q9BmmV4xc4HrhOQF5FefLE+W6R0TbiOvyd+577rQ/ISZXT09PeQqJtu/4wTk5OelI2xuf09LTrW1w7pbQ0afX+9PosUePIi7PTyhQlKar3zM81rs8LrFuI475U2mp/gfmLFi9HPBfg7CX+5Cc/CQD4nM/5HADAAw880C0g/Nzj5bx582bXLoCubeBMqxgOh70Fdjwed/Mv+hu/vXXrFp577rmlPuace0Ll5OSk+z6O/Ny5Lw4XtQg8BeCV9P9XAPhE6eScM05PT5ekbMCp+fwCqXTjF50nexx10eCJHZ8dHx/3+uHMEu6HUwNLWg2rojwGKuni6BYlp/rHGAJ+IVGtxo3HbDbrLVDupeJr6v2GNhL3zW2U4DSAgKrtvAjES8UanS4QvPC4MY1xOTo6wq1btwCcvbj8YurLyr+Nlz9e2tu3b3d9i/HY2trqveiz2cxqvQCwv7/fLTLxXfSLr80LvAoyXqRLuCjvwC8AeDil9OqU0hjAWwB88IKu1dDQcBe4EE0g53ySUvomAD8NYAjgh3LOv1o6P6XUSXWVgryqq7q8sbFRVddVNXfqKa+UvLLq6lkjX5xd71T+EicR/VCJy+frd2wD18hLRxC59lXL0nuO35VUf3efg8GgahKopsbfsUbjTBwn3VQVdn0MbGxsdFKV+Zn4bbQVdvdgMOiZjY47ivP39/c7aR/z9fj4uBt7tud1Xkcfjo6OetrEeDxeMiGBM96HP4vfMV9QwkWZA8g5/xSAn7qo9hsaGp4fXNgi8JlCV8EASztd0UKDAJZJJmWrAyXJ5K7pbEdgLr2czazSh6+vNi1LcdZcVEKzVqF2LrfrGHLXfyd51W520pDvQ8ePORX3O7bjdfy4H8qH8HfKbzhNzUnqAHMw7vlEf/f29nrSNWzy7e3trh9xzuHh4RKpyPfBUjx+d/v2besxUu2H+Y44P+6Nr8fEprYb3zkvhaKFDTc0XHGshSbAnIBKGrZ/1AfKDDm7FmMFPk9wEUtPlti6Kjs7N9qfTqfVIBDHjKtd59xvztZnDiEkTJzP9rP2Z2Njw3IjTvKqdON7dx6XANvK0Qf1VnBMBWsf+lt+Fuq2dM91MBj0PAtubHnc1We/v7/feQdCA9jf3wcwZ+XVBbm/v9/91nljlNdi3ke9IdwP7re28eyzz/bmJrcbbcQ7cB6sxSIQpFUtJgDov8C8CDgVV3/HJgWruEG68PVLkXH8cq+K1HOTPaCBPnyePuTT09Pey8rkmDMb9D5dZBovuqwaqxmj/eN+cKDPecdF70+vw22xKl8juGazWe9Z8aKgrjN2p/KLHPcY6jT7/d3iHwsJRx3G/+NaLh7BBUrpYsdRtLxgOVey3p9b3Eto5kBDwxXHWmgCITE46EZXx1LAjFNxVaPg8zUyjkmmOI9ND12dmchhOGJT++bUcD5Xg12cOlsjefheVDo7d6BToTl0W+/DSXHXJ1blnfngXISqTteIRO6ji4hkKRvnKHHGpgr3V/sRUpzNRjbDoh8cDRpHdde5sZ1MJr3n7gKmnKuX29Ux0vuooWkCDQ1XHGujCYxGoyVJrUTRqoAHbivgEjhUUvKKzaSKS94JqMuvFNDibPtSH13oZ+2emDRyocrKQ7iAHO4bcwcu6EevqQFWfD5rRWqbjsdjyx2oW5dtXHXJOlcoj7cLNKvNJxcgxRpA9DHuKxJ+RqNR91lI5SAUOQgt2uLAINY6Vdt0AWc8b+NazBOECzH65p5xCWuxCABnE0H91szmqtrEGXO1F4xfQuf314k3nU6XYtL5GL8BluO4HSuvE5tj9h0J6BaouJ7LjHRxBbXoSCWJImmL731VKrFem5npmufA5V7wfZbMQE6K4qg5zZZkj47eOy8QfJ/uhdQxYhPDEaW8eALLKrrz2SvcYu7mHJ/vxkjP5byNVeRgMwcaGq441kYTAHw6La+OrphCbZVTYoZXetYwnBsrUCPCWOqqpB4Ohz0/fi1ewLXBWYjaljuPXVzafz4/JMhkMlmqN1Dqm4MbFxfpWMubqJkl3JaT4qp9ODejA2tXrq6Bi+6M3+lnTCDrs2VNoCTRow29dza1XFatmycuGxTwbtremFS/bWhoeMFjLTSBsHPYTtMjr9y1QAiOfef6AAEXh66uGUdUsQ3vuIbSPXE/WXq5CDO36gdU4vHqz+eUog1ZMoX0Pz4+7mlX3N8aD8HQiMuwR0suQgXbxQrmQ3gcNWCGn6PLftQ+uiCxjY2NjliL75wGFnCBOw48l2vagYPLBtXPWNq781dhLRYB4EzdL3kBeILzy+VIFRcVGCjFEPD5rPoxacR95fM5oi/AE0QnoIu8cyG5vJhpIomLi2A/t6r+0+m0I6iYRHVmV6m/zkziz1S9Zq+JMwt4fHScuX2NK+BFoEaiBbh9Zz5E+0w41rwrfM+uZFuco3NiOp1W1fXzRJGyCcJ90EQt7k9LIGpoaKhibTQBYFmlcySTfsaqOfuVnW864PziToVS6RNwPnn+3Em6WsQgH/X+uB31c/N5rLqWNB0XGcmEI6cB8/c6BkrIuRqALG2dRC3lN8T1+T5ZW+FUXmc2qtReRRa6HBPtW5DKjuQspUVz//n+WIrXSD33jGuuci72qtesmVrdNarfNjQ0vOCxNppASEJd+ZzdzSt+iRA572f8HUsmtbFYSjuJze68+L9KGleOnO+vxIeUtJX4O6QLS17nsgyNwUUHOptd3WX6W/1OJbAr/1YKjlESVzMk+Xx+7o5/cEVlzhNE5bgdPsdpGKWAsFVELM8NzQBkbbjGTbh8DOUXWMMsoWkCDQ1XHGulCTC7zRljwHLhDmdfulWa7dy4hgsMUp7AuesCTvtwATOj0ci68IAyN1A7/zyuHi5HpQUwnHeFr6+aDN8LHx33UdImWBtiW9vVH3Dehvi/jnMpmEuZdLat1X3pJCoHIamm48K6c849roalvu4F4PIsXIFZho4V3wv/X+d1KVfEYW0WgXDLRTy+LgLD4bBHGjn1jQekVGiB249rM3iyuwUiwJ/VEmpqLsISAalj4NJeXay5ywWIo77o/PLV1HunSvPv3IsV7UcfNReD++j8+E6FdQsJuzhLLwKTi0yO6pjzvbvIyxhbXjRqKbv6/NmU5HlU2m+C79ndC0PNJx7HVYtAMwcaGq441kITiNVtPB73zAFe2WJ14zrrjiysBd3wNaN91Saceu9WcCfFmSAquarcyjwYDKz0dufFsWbGqOt0Npv1XH58TUcyaeCMi0hkycRurIC6aQ8PD7vn53I6XICSjoeroMuagH6XUlrSKOOzz6QQh4tSdIFSaoYxXMYqa5Gq2XEAmZszjlTWOenqdiqaJtDQcMWxNprA1tbWksR0pbbUhcYrtyP/akQck1gu4MiRXHHUNhwx49xvbt+BQC3TzGW3MSnqXErcj2ijFszjSM44P6R4KTdB78+RhxGyPBqNbOCTI8C4Lb0n51ZzbrQ4vxb+G3Nnc3NzKe+B4Qjh09P+RqquKKoLIHPamysOex6OxHFY0Zaba4o7XgRSSq8E8MMAPgfADMCjOefvSyk9COB9AF4F4HcAfE3O+dlaW4PBoEvc4CoswLLf1ak8Svicxy8KLKtcOnD8AJ26rG24+AbOP3Apq64Nx8Zzm/yZU/1d4YuaWjibzXqTlglEXUzZc+AKZAT4PI1zd+PCz1YXNl5kXCwDLyjaXxYCq9TpOHKf+BzurzMHAmxauGShGtGsz5sjOvn3pYWRj3xvFxkncALg7+Sc/wiA1wP4xpTSawG8A8CHcs4PA/jQ4v8NDQ1rijvWBHLOTwN4evH3cymljwB4OYA3A3jD4rR3A/g5AG9f1V6oeLFS6j7uLouP1SC30rvUXEcClvIVFvcGYDmWPVAioqJdR1rqb/U6fH0XLencgXxN/S3fi44ll3Pj66vpEZJ9PB4XCS83HtzvOGdra6snUV3xDO4z9zfuSWMCuD0dKycJWXuLcTw4OFjKFeBrcnx+LcqT+6gkoXNLukhENq+c9qbXLEWxRl9X1ed8XojBlNKrAHw+gMcBPLRYIGKheFnhN29LKT2RUnqiplo2NDRcLO6aGEwpXQPw4wC+Jee8dx57HAByzo8CeBQAdnd3c2RHqURgW1FXSidteeUrrdLxW2A5EMe5ilQT4PPY5VbKK5d7Xro2f+Yi6fgcNy7uM0fK6T3z+GjxDM4xiPvb3t4GsLwtNmskOkZuGzgmNLVCr9NIXDCNBgjpefoZj7e2z5oAn6ckJxeTVfAzcG5VnX/sAufx0+xBF6HJ/XZuWtUA+BnXai0Ad7kIpJRGmC8A78k5/8Ti40+mlG7knJ9OKd0A8KlV7cxms24ThtK+c44gcvXZnI9f2yqd7yLjlBh0LDQ/GCavnCqnfePEGVcxWcGqo5sMblGM+3Bqp5aqHo/H9rM46qJxcnLS+f21qq5L9dbvo6+qOvOzcGp4oLboObOEx1HHyIUN8/kKF6vBz1jnsktpB/rkKfffLQLOc6Htcn8ujBhM85bfBeAjOefvoa8+COCRxd+PAPjAnV6joaHh4nE3msCXAPjLAP53SumXF5/9fQDfBeD9KaW3AvgYgK8+T2M5z3cTdumUQLlARYBXVi3BxZs16MrKkkb7w23UUlAdqcaSWldi5w50BCXfm5NktUiwEhkGLLssd3Z2AKA7ssrKm2tqv/kz3YKLJbdKSC6xpSQcg9Vrl4TkNAAn2fmo57s2XKQlf859Y+1U55BTwZ27jok+Z8a42Bin+tfMqQtLIMo5/zcAJT3jjXfabkNDw+ViLSIGB4MBdnd3kVLC4eEhgLOVOOwojpAr5QAA8xVZMxFrv3N2Vy0FdTQa9SLNWFK7bDx1uXEGYCC2YWOwJHN8gbPxlfxjjcNFMGoA02g06sYvjizlol3er0AzFx0x61xYrt+q7fHz4cAgJ6FLmgAHkPH5mvnJc8xxA+qq5DGt2d0ukzNwfHzcKwC7KsI1rsXaiiM+Aa85KFruQEPDFcdaaAIbGxt48YtfjO3tbdy6dQvAGdPMkkq1BOfmcTnbbF/qys1SlguDllw/wZhzH4+Pj21gTcBpB+dhyJ19V7P5OBhKj6tiyOO78Xjc4wLYLRhjxZ4A1bTC1o8jg8OG+Z7U9nWegFrWJvMEKg1LktAV9lQpG2AXdRxdiXL3XDgEWbkCtx8ka2WufVcno+bNWOUdWItFgHMHSu6SOA/whS+44IhOPvbr68NiVbE2aVj1UuKMyTROtnETCZi/HLGguUhE3TmXH7IjEvk+1Z3G967JMZG+DQC7u7vdMc7TlyRcucDZ7rtxr9xfXgTUt88mCN+v8/dHm26B0HiFnHMv3t8liZWuoe3qYsB9dO5LNcPOi5IgA+ZjxXkY0Q/+rfbDba5zKRGDDQ0Nn71YC00gVBxeiTWbq+QCjM9DE5hMJj1Vvpa95yKq3ArPGopG1HEaNGsRqg5GG9PptLeRZUrJVqIN1AKUnFajblJWLdl0CW0mogLjyP1m9VPr5TmpzL+rqaIu47KWb6G/0/Ep5QyUIu9cMFfJJZxS6saUzSU1Jd2eAS6aNZBzXtKS+HecFRr9drtMOdLXjUsJTRNoaLjiWBtNYDKZLNnySohMp9OevTMej7uVN4gqJ4nZFnfx4o5IUk2A7VwnsTWTjvuhRN9wOOwkLhNbjkjjc7R9vZYLaWao+8sFLTliKySfc1VGzgfgyc4An6NSizkVfZ5chISlufIPzl3nOAEmf1V74+xEV5PA5a5otivPJdXo2N3JKGlv3K5yTnwtJkW1jFqtZFpgbRaBIJH0IbiUUX5oGsE2Ho97LylPztKmItGPODrSLdrQl2k6nfYmDRNxOimd75ZfDvbBB7R9B35JS5FsCl3sJpOJjY2IPoYZEOPIOQ8uYUvVZd7mLMjgnZ2dLmJRiUQm5PjFZ88MsGwKBZzppLEjPAa8COiRX2o25dzmoPE7zQ9x8SdM5qqHiceTyWhXmcl9Bix7e0po5kBDwxXH2mgC6ussuU2AZVLP1SJwKbMBjbJjqc/nl9x73CcmyaK9WHU5AlC1Gad9cJ84SzL+79RBFxsf0ExARwxubW31MgVradSc5Vkjm1yUostmdPEPTpuIceF+uBh/F0WobQQcaelMIf6d+vP5N+dxM5e0D/0ta2K1LEnuq9NEtf/F3688o6Gh4QWNtdAEAi5GvWRvBs4TBcf/d662AEtWldq1OHTHZbjtogNsR3K7pUg31gTYlnQRZqUIR46WjPO3t7c7W/y+++7r2tD6AEF+HR8fd985DUzHmyMYmbtRd910Oi3az871p2MT96TPxcXPO9ewc7W5eaIao4vCZGnu5onOKw5yCihvxePBn3OBEv0snr/LQFWsxSKQc8bR0dHSw9JqwyklXLt2DcDyNlrxQsVNHxwc9MJXA65oiWPZOf7AvZhOxVVSx20TVgsfdT5+t8UWx0C4F10nQbTFpCG3ob7v8Xjci5bjCaubw3A/ddKzWcWTVCclk6KOKVezhxe0OI9jL1Rg8ELC9+TMhpJ3xcVlOG9MbU6klHoeFF6MlBR18yTuX8+L6ysJyBv1lNDMgYaGK4610AROT09x+/ZtbG5u9iReHHd2dvDAAw8AOItqOz4+7mLwWSqGGru/v9+dF0d2KQHLEVi1aDxWOx3JpOopk5bqQ+a/nYbhymm5xKG45xiX++67rxdrHjg6OurGiqWQbvLK5cUin4B33I3fcu6DxkhomTEeF4Yje10aszMNXQxI5DNoaq7Ls3AktEtMcqq0M09KZgG3W7oXnQur8mdc2rWai27OFdtceUZDQ8MLGmuhCQBngRcqMTiQQiWOszk5mMJdwxXdUK2Dd8RROJcO/81uNS1a4dKGXbSatunO39zcxIte9CIAZ5rA9evXu+sruXd8fNxJZrZVXVksDSByJKAL2AowuedILheoFXCurtB4HEnMXIPLXYjruLmjgWaOJ+Dn6VyQqjHws2buSu/XaZ21cQkw0ee2cdNndXp6utJN2DSBhoYrjrXRBNjWif8Dy3aahopOJpPuM0bJHmJpVAsDZnddLSCHJYjLSSgFLTlXVOkaeq2Qijs7O523JDiS4XDYjU2My3PPPQdgbsNH3zhzMDwLPN4ax88hqeox4EAVzo0o3Q9zKvy90xiiTReAo+eNRqPuXvS6Lry2VPqsFHDETL3TBEvFQhnOo8LBWW7PSsc/qJbKwVOqdfA+EiWszSIQKPnnT09POxWX49bdLq+qfqvvnM9n9YpdORqL7YpGsDurtgjovXG7tRwGvo5rK1ygt2/f7vqmCU9qFgDLbiRdbLmQRS0y0m0mouYPk17sW68tdgF+WV3uhfZja2uri3kIQjPGZX9/v5fOzQv9eZ6BS3d20ab8ErooSHVtspBTM8ClubObm00Al6QW1yslpnV9qn7b0NDwgsfaaAKh9pXIEV5NXe4AawTq9uLVWUkSDlRhNUwDjlxl4YBLVWVJo0E6JbeTahMub8Ih+ri1tdVJmJI2xP1hNZKJTdWc4siVcfk5hWYR7cb50+m06xvfu8vQ0xRiVpNVegJnkk5disCZ9sNZkOE+jO+cRsIBVQE3d1gCq4bh3J41QpjnvJsnbq6o9sFBS25+h6u8hKYJNDRccayNJhCrpLptXHYb255a2dYRck6Ks9R32X0ulDigJBZLj1Jed1wf8FK/Ziezbc2SJD4L6TmZTIpFJHZ2djrJGLbztWvXOgnqCmU4uPBYLpum965BPas0L/0d58MzZ6PPh0k0nUNcIZolpkpq3qFKtTanATpJzfepmhw/M8ctOU1AJbzLpeD5pP3gd6SE52NX4iGAJwD8bs75K1JKrwbwGIAHAfxPAH8551ztRcTBO3bWeQmYtdYXnfe6L1W84Xb5xeeJqA+LycNYmJg4O0+EGX9XilHnz/g+NKae2+OXz5XFBuaegPvvvx8AuviC3d1dW+Zcx5R3Gdb2T05OepF6ceT71UWPwVvMqbo8Go06ok9Tirk9Lm7ikpy03xx16Co+B5zpxC+kkoU1VZ7Hlp9/ydzj72rxJGwO6MLGAqSE58Mc+FsAPkL//24A/zTn/DCAZwG89Xm4RkNDwwXhbrcmfwWAvwDgOwH87TRfjr4UwF9anPJuAN8B4PtXtRX+ZpXALo7axWw79VQ1AF6J2UVXW7GVKHKuInbRsOuqFCfA7fM9qb+dfx+rf9wbkz2uDJWaLJw3wWShq42ouRSu1j1H8TkzIM51lXG1jY2NjaJmxK5hvheVkCcnJ73UZx5v1Q6cqr2xsdGZDi7yzhG22gb3y0Vj6jiyCazPgseK57lqyTyf7oUm8L0A/h6AuLsXA7iZc44RfArAy90PU0pvSyk9kVJ6wtnPDQ0Nl4M71gRSSl8B4FM5519MKb0hPjan2mUo5/wogEcBYHd3N5eIMbdyO1uTV0VXGz+gKyVHsPFKrGRhLVaetQmWCM59GddW3oLtYseLxDXZZemiw0pBIywpQ4vY2trq5by7MmQs6dUVxoFSPB5xVPLURd6xJqCaGu8oxePjAnciOIg3GA3os3BBOpubm708BZ4TzI1EP9wziP/X+CGehyrtnVZUi1wttRvHVcFZd2MOfAmAr0wpvQnAFoDrmGsG96eUNhbawCsAfOIurtHQ0HDBuONFIOf8TgDvBICFJvB3c85fl1L6UQBfhbmH4BEAH1jV1mw260pd10qCK0/AwRrOY1Crg88rtgtLrQXuuDhxDRbiABhXV8AF6ZxHIjhtQplhN37D4bBXQpwltXNjqXfg6Oio0ybcWKoNzOG9AdZg2I7lLeh5zBg8VvE910iIvzkgKI4qDZmvCLgxDY7APWNXcLQmdR3bz7+paUhOq+BnXHJtcnslXEScwNsBPJZS+ocAfgnAu87zo4gRdxFu+v+4wclk0puozv/KA6Qvh4vxduQYPyhdBFhNdrHjtcQTB/Wjr4JbjGox/uxmVOI1pdS9RHt7ewDOYvC5hBf3VSP6eBK7cmTqJnP3zsd4Edlk0BwJVyGar6NjyeZXYDY720fAEcJ6f/EboO9S5nFx7kA2G0uLgIuudOaD/q39vpRFIOf8cwB+bvH3bwH4wuej3YaGhovH2kQMhuro1FhgTto4CRlSykn7Wqklp2I7ItGRdWo+sAajaj6f51ALMuK+Oq3CSZpA3HNIysPDQ+s2jDTkiCLkTEuNz+eMPpbA0Z5zr+n+B3yvbh8BR8iqWXV0dNQbL84j0f67jT1dWS9uw7mmdV6xVqgarHPvlUho5xqMtnTu8Fzja5cC49xW8L37qn7b0NDwgsdaaAJs96kNxMEvumEj16sP1OxM/ttJYJVQfL4LpuG2HCFTc+eoFGJpr1KxFlbKRxfyGyG9ri9s93LGYDwL3isQmLsWOUcfmEtlJwUDmuXHQVRBMnKZeL23II35fFf6jQO2eN+++L9mKXJ/WdsrBYmx5sXPTrUDx8/w3DlPWTn+nUpxvk/WWlQTqLWvWItFIKV5OioTIVp0Iefcsb9cNScmOavyJZ8qkySrWHnnFYijW6h4swfAM8erUHtY7gVzC4+aL/y5JtRwnAD7ypkRZ5ycnPSY99FoZCs4x9GZGVqNuFbTkT1AzqzjF8FF0sVYaKLZ6elpb38H5x0IOE9HLbqSyUuX78HkqfaXyUXnSXFkpC7EjkgvoZkDDQ1XHGuhCQwGA2xvb+P09LSnygUODw87TSBcV3t7e91nvMLWNIGA8+uyaqdqHpsAujrzb50P2Wkf+p3eAx+du4lXf3YLar9ZzQ8SMGoTXrt2bUki8RE487drYRDtY6ma7dbWVnctHh+n8ehnLLlVzXdRcM6PH1oi1/tfFVOv4+G2eGNTRc1WF+fgyEInsWupxJwaXJvXtRiTEpom0NBwxbE2msDOzs7Sysp76AHehnMbTTpChqFS3PEQjsCpxXG7LC5uVyWls+FddlgtX8HdZykAJo6u+IfuWOSy5ejGF3QAACAASURBVNw+BXGdo6OjYh7/aDSydjdviBr91nJu0ZZz5TFqrkSO3nTkmz5HHovatVg7VOLOkcWqVcT3cSxJa37Gmg/D1xqNRsUK2+7eFWuzCGxtba1MdNDNMzc3N7vJ4iZNoKYanTchgyeAtsOJJOwHdhtexFHVwtoiwP2pReMxoen6qJGAg8GgKzAScBWFOaEoSMN4Fhy1Gep3eA5cLAP/HYvA5uZmLzmHx87V/dNnzC9dwL2QPB76cvCzdaSoxg7w+W7hduniLtVXTTG+tkYwusIxbkHTuVRDMwcaGq441kITAM6kmEpNTn8NScOJIloN2KmPKpEBT6Y4wk9/yyq3K3PlyEVdpUtx5SUXl8sJYFcek1GqwnMNwfiOzQGVZKxhaD+m02kvxZa3MndRa1psZTQa2W3CXD/iqH5xzhNgMjA+cySnSlQ2S1hDK917zrlX3ITNI+1/iSx18yRQIwsDTMTGeW5/B75OcxE2NDRUsTaaQEgNl3oKzO3Y2FKLo9WYCwB80QqnCayy8dXuYoJLpXgpOozb4+9Go5HNUlRCjqWWk/DhfgvEWABnAUEc96/lxVzmHW9NHueH5jWdTm3Mflw3zuex0/PZ9cvlwFRCsjTXKDh2j7L2UXrunHnH0DZ4/DRjsBSkpS485yIOOBeuywthjcBpNY5o1nGuaROKpgk0NFxxrIUmEDHSbI+GVAnpz+WxahKEWd87jcF3MeGucEYtmIeDlvR8LuHl4v5dJp0GsQDL4aVx1GuyxNEyZ9PptMsFYDdgaBGcgQj4jT2n02kxe489JIEIDItrRRulopyrymO5YC8NGnLhwKWgr4BqEyW7usTKu9+UcgGU73H36zZs5XuK+RTPjvkOlxG51Hb120vCYDDA9evXl9xBbjMKfYldUkfNrcYEVI10q7mK2Nzgl+o8lWJXvdS6Eairrxg4PT3tiFJHKGnF5cFg0Kni3H60we3GeZoHwS81u2bjujERV6miMVFjP4GTkxNb5Sf66iruuIQdfRE5bkHHiJ+7Lr5xr9yWuw+XcOTaYoGi5oMjsl26OM8N7QtXjXL1Ids2ZA0NDVWshSYwHA5x7do1nJyc4ObNmwDOioVwTTsl6ZwKDdTVfkf0aHAMk38uAs+RP+q24Wuoqsu1FDmaTLUPJ8m4H+fJkmRTyEVBalrvbDbrZURGW9euXetFGAJn0icyBgNM3MZ9himg46Obn8aRtQQOKIrfcnaiS7GNzx2Z6+C0H2A+ZjouXAnZBYQ5klO1PIbTNpzLkqMvA+5agVUl/Zsm0NBwxbEWmgBwFmYZJBQXtwTmq6RqAuwqci4/dbnp38ByLQBXhVfrGrAE5uAU57YpBYxsbGwsFbzQfqmE55Vcfxf9jWNpByK9vrahUlT/jvN1bGtZkEBfovJ58awdeeV4Bd1hiM8bj8fVkFnlK7iUGPe7lHvh7Hku5qF9zTn3yF+gnmVaIxldOTK+ZolLc+HRirVYBGazebnoyWTSDUhM9pgowPLLDyynVbpYABdtFeC4eN2Zl1lcfZk4ao4TVFRFY9+0Y5q14i77uXmSaVtu4XGxCa6CjVOFdVFkc0f7c/v27d7GHvwyherP3zmTRUlL3g1YX3guc87pxVogJaXUu76L8mS1WseUfxNw5CxHqbqNZeI6mvzDnhSdtzpGOn4cp+LI6pr5WhJG3fWq3zY0NLzgsTaawN7e3hKZEgQSR6PV4q5Zyjk1Vs+Pv7e2tnpkF6vVLtOM+w0sS0NWqzkugOEkJLub9J5c/Dy7uFgL0f6yWaNmzGw267klOcNRtQku9cXjo3s+sJbjog81h4ELh8Q1uTAImw1xjhKI/L3eO48Vj7uLt1fzxWlqHKWomgBDNR52B7JWqJocmwca4+KyKl2+B2txq8yBpgk0NFxxrIUmcHp6iv39/aU4aicVnaTRIBOWZK7qrErPra2tns3kiJxYWV38PBfscHnfjhNgPiE+cxl3wFzaqduQiVInhVyko3ICTIBqleK4Lo+Vq5vAefOaQ8ARjJy5qIVD2A2oROnR0dESERjXdhqPunr12TGcBObnyNeK79Qd7bQ31j7cBqYl253PY83EBYm5QDoXJanXLOGuFoGU0v0AfhDA5wHIAL4BwEcBvA/AqwD8DoCvyTk/W2snXgpWLTVtmEkYl4LqEkS0zp6LwBqPx/bB8KSN68f/NV2TXw5WGV1aZ/xfJyC/yMoWu3uZzWbWZNEFhE0RNaP4O/Z3q9qrBCGwvDDo4szkof52Op32wr+BM4JPffG86PKz1hLivMCrGeM8NUymscmiqem8KDkTLuCS0BzL78ZSF2cePw0lZgGiC7ieF21ctDnwfQD+Y875DwP4YwA+AuAdAD6Uc34YwIcW/29oaFhT3LEmkFK6DuBPAfirAJBzngKYppTeDOANi9PejfkehW9f1V6oz7qS8f+5oAZQTtekPi5951Z/F+HniD42MZwm4IpQqObC57iU0oDzFzvy0kmVaI9V8vhcNyTlKrzsmlU1VmsT8j0Nh8NOrdeNTjY2NnoaFZsDrNayxsfX5nP42WoBE5fizc/WlWJTEi3n3Kvlx1LZmV21CEQXp6Juyel02pPUrDnq+aXrqNta51wNd6MJvAbApwH8q5TSL6WUfjCltAvgoZzz04vOPQ3gZe7HKaW3pZSeSCk9oTZfQ0PD5eFuOIENAK8D8M0558dTSt+Hz0D1zzk/CuBRALh+/XqeTCZLEWlq77oAHl4BWSvQFd7tOsOSRstMsY3f3ayJ8a/tfsOSQ0lA/ru2YnOmnpPiCtYO9NpbW1s9baUUqRfnafVg5mXCFmcehz8DlglN5lScbV0KOGL+hIlT1RTZ3enyCtz4qrZXciVGmyrZmZNy3I6Cz9eAsLgvbqsWSMTX4rmvbkmXzt1rs/ptHU8BeCrn/Pji/z+G+aLwyZTSDQBYHD91F9doaGi4YNyxJpBz/n8ppY+nlD435/xRAG8E8GuLf48A+K7F8QPnaS9cXioheZV2edQqxQHvKQDKee4akOGyE52bKa7NocTODq0VnODrlHIeOItQ8xz4PC58qTbidDrtMe/uWrNZfwPQWnAMM/XOQ6KBQaw1cR9ceLGOmZNuLndBXYWsIbH0170OOMDLFfjQ58PPTMH9dqHbNS9BgN3AAZfD4PJZarkoiruNE/hmAO9JKY0B/BaAr8dcu3h/SumtAD4G4KtXNZLSfEPS4XDYqTOqirKKzi+OplA6kiTg1CKeDC7JpeT+YrCvnN03SkLW1E73IrgYglobLpqM23RxFnq//OLqS8KJKudZHFn1d9GBTuV3ppmOB88Fl/SlR2eaTafTXlyDu5Yz4QLOtHHuXf5OF13uty48fJ/OvezaDfAiuarG4F0tAjnnXwbwBearN95Nuw0NDZeHtYgYDE0g59yrHuwkGbuRdFciPs+pna7AgnNHlWLCS9F5Tt0sSSaWtk66OWKo5jLj71ST4nM06pCj7GpwLkjWlEo5D6yGK2Gl96f3ye0r+VZqoxR1WHL5llxzfA/cH1eUxZGFcU7tXvg653Hjcb9VS3Gmh4tOLaHlDjQ0XHGshSYAnJEbTkICy+4pjtNX+3IwGPRW/Rohdx5JyOc5+4tt9pqbkVdwxwmoXexWcw4GCaKP7egIunFuMicRdBej09OzfQRUM+Hz+DnpvbvyZU4Cs8TW587Xq4WGB7iP6iLma3HItJLQnOXnOBjVSFwOg+tjrQ3Hy7hgIT7WApTc+auwFovAbDbDwcEBUko9koYnmPvMqWPnIdiYyFMVilN3daLwtdzkcXCRY9oGP1w9h1+c0suin7l8CI055xeB/fQa0eeIRB2fUp+0P67AB7PbNULOxYxwXoiOqSPfuDiMi0moRea5eeWKeMTv3Rg56DMtlbXXtpz5EuDxXnX9Zg40NFxxrI0mcHR0tORzLqVoAt4HyjHeuqLWiks4KeT86DWp74gZFx3G11btwGkJrDa7fQ0cUeW2E9f2VbpwH7mwS8C50LivTkrFd3ot9xzdeLAkc2XUuNyWtuekud4Tk6hMNJci//jenWnjfPxOa3MmYm2Mas+Kz9F+8O+aJtDQ0FDFWmgCQN8l6NxOjvRwgSGl2GuX7eeCkLgftRhydx0mJZXAqbknWSPRVd3Zp4404kxB7Q9fO845PT3t7Uo0Ho97bsCIHDw+PrbSSvvL7lIXjenG2z2X6KtqAtyG3hO3UYvsY5vdcQfumTmXnwvm0fYdHNGnGgxre6XfAp6g5LYuup5AQ0PDZznWQhMIFxhLQw4zBZYlNts7zi4uSVInQRwLPZvNeu4uV2aK6xtoSSuWIMquc6l0x4artD0v0+x4CM7T135vb2/jvvvuA3DmKnRjw6XenUQtudX4eTI0fJlrAWj4NUuymjuXA5NcmKyWSuf7Ys3IeYP4GvqdahNOaytxRqU2AszsO02qttsQa2Cr3OBrsQjMZvN9Bziyy0WYBVgNci+6I2ni6FRAVTeZHFP/Mn/HLkVXj09JS+6H85WXCKKaK4jhVO3ow9HRUa/QiEtQ4d/oojsajXqf8eLi6vOpeVLqt778rj6gKwnnNlBxC49+5iI0ObVaI0adSu3mmlPzeQF3MSCl8XZuYwd2izt346pn0MyBhoYrjrXSBDijzxGDNTcTr9yqOtfqvrMLiqVQKe6/pJ6qys/qaS0YpBaNx206s0cxm51Vy9U0Zo7xZ1OHC3Vqv5XUc2nSHFilmoDbQty5WJlsrQVd8b24jVFVk2JtQjUjTot26cUaWFULXip9plrEYHC2nwXPTReJCMw1Hj3fRVc6rYZdxU0TaGhoqGItNAHn/tNyWm6F5Zj6AK/6jrByKzb/FljmDoKYZMmqEpulJ7dVIqrYpeOChLTQJwfH1MJ0WSKoFsL3FDg5OemVBCttdAoshxkHkehCcuM6bremWgAPg7Umd+9Om9DzXN6Je8bcVk1qOhd1jcdR6cxS2RGfOuedJsDzm+dVaV/K83ACa7MIxLZjOinj85RSz2NQqiakL0KAJywPqouyUrKLTQzH4q7yxXL7bNrwguaqEgPeV+7ILyaIAky46cuZcz91ezQadbED8aKzqaAbgbp4AVeJiPuoz2AwGPReGDZ/tK4i94NR8iLwC+mIPjd31ETUe4ij8+5of2I8x+Nx9TzdK4K9TkxM6watvACqB2NVQRGgmQMNDVcea6EJDAYD7OzsYDbrb6PMBJBKZZdKzESVEltcpZbJlfNkcfFR1Xx2tbkYBr7PuLYjwrQ2Hv/f7URTy2HQ/pckn0oK3qo9tDBXtdcV8+DCLsCyFOV+q+rsxpu1It19aXNzs0f6uh2fXPSm0wBrfvSa67kGNmNiXFLql/riupCa9zEej3sEaEqpR/oOBoOlZ8TfsclXQtMEGhquONZCEwD6RT4c2aQr5WAw6NlsbM+rDcznOynBbbisPcDHhDuCku1cDRBhVx7ftwskif+7oCgXoRfQ/pdIOiWjdnZ2sLOzA2BZgkUbSlRxsFAt85M/0/vk8Xf7K6o2w9KR6yDoODB3VHKpMtw5zD3oWLImqloNcyul3JJo0/0WWA6KYiLR7cmhmlFgVQ4DsCaLQMQJAH0yhZnTIKMi1JVVHVbRtYos4zzppm6yc191cWGTgh8yJ8Ho+brdlv6t13SRabVoSRd9xu3p+Tx+LvQUWCa2ov3JZNKNt94vE7E86UuMuvaDj6XxYLXXxQLwEVgWCC5E2PVJ++0EAoeQA/MXmYVV6T6dOcibs/I4x/mu/mZAvQ8tYrChoWEl1kYTiKIimtQR5NTm5mavpt5kMln6G5i7GHVLqwCTei52wLl+nLtJYxlcnIBL4WSJrb7hVf5/BUuygJNoLLVqWkGozrdv3+7FRrAZpgQUaw7Rbjwzp7az+cAakrp1eX8AlXyTyaSo/sa96lE1P1a13Vxw0lPnBD/fGJeQ4kz48YYxauq5xDgXjRlt8PbpvIenPisev1XmQNMEGhquONZCEwDOiD+XHgmUs+fUFufoMGcTansu4o2hEsRJWyb1+H5Ycum1XRahtss2q9sHQW1rRxYGXAqqC5KZTCbdParUB/pjyeOnrkV2GbJ25tJ6azwBFzXh/zNc3gmPgc6T0WhkA2xc4JiCn13cIwcEAcu2eIzH1tZWj1PhPjqOR3kixw85zoNJxlo+BnCXmkBK6VtTSr+aUvpwSum9KaWtlNKrU0qPp5R+M6X0vjTfoqyhoWFNcceaQErp5QD+JoDX5pwPU0rvB/AWAG8C8E9zzo+llH4AwFsBfP+KtjAcDpdipdVe4y3EeVXX+gOTyaS440rJZq6xvo71d8yxW7FLLO7GxkYvzp2lkHOTRvuuKGYtZLrk/eAjn89ait4nu8S4jxzEE/cXbZXctfwZS3G9NmuEXBhUg8TYTavam+METk5Oelon7yRV87w4TYAZ/WhLx5s5L5dXou5A9m654DIeK/Xo8Jy76NyBDQDbKaVjADsAngbwpQD+0uL7dwP4DqxYBELtce4p56cNHB4e4uDgAACW3II68fjh6eAzWcgEjbqmlLzR75xbSomhAKuK7mXVazg3j1sE3GLgVEH3nfutuul4oWJVn/3mwPIOxDqObPLxYq2LvksY437oM2bzy+UH6OI4nU57AseRrTy27L+PMdAFMBaD4XDYGz9+qdkEcYldcdT7ZKLUzWvNBTlPUZc7Ngdyzr8L4J9gvvPw0wBuAfhFADdzzvF0nwLwcvf7lNLbUkpPpJSeWMVeNjQ0XBzuxhx4AMCbAbwawE0APwrgy82plpXIOT8K4FEAGI/HOQgjjaPmbbJ0RT04OOhIIlYFdQXm1VwlAks3/k774VKaa+p4SqnrexBDAZZanA+h2YMaEMP9YbW9RiSxGuwIpZoW4VRo16caUeXcXqpW18iuo6OjHjHI0pOfu2oRNbcrmw+uUjEHSAFzSa+uT85wjGfNmZfRPmur+mxZW1LTs6QJqLnrittwWxdJDP4ZAL+dc/50zvkYwE8A+GIA96eUYlRfAeATd3GNhoaGC8bdcAIfA/D6lNIOgEMAbwTwBICfBfBVAB4D8AiAD5ynsSCHNFiIV7Swd8LmPDg46K2Kzl4MuJBLxwm4eGsnMVlaqPaRc7baQ1xbuQmWhmqzl6Sc4yFK0tC5y1ZxAnrPnM3IGonLJ9BrsrSNvx3RF4hnvL+/30nSGlHGQTd6ZDdmwIXwcl5DPLuQ/teuXevC1ZnPUc2hRuA5sphdsqqBuSzM4+PjnsvZZc7ynF61NfkdLwI558dTSj8G4H8COAHwS5ir9/8ewGMppX+4+Oxd52nPFZpgsGrkCCVXUVhfbn756D66v2uVd5m9dtGEqvbytUKddWy1u5aqb5xAxEc1j1hdd6SXM4W0/zW/OBOUTIRFwpEuArwo8cKgpKhTiWOiHxwcdHkl/LLqgum8PHxUr4OLmuTkHFXvt7e3ex4AfsaxaDFBrX3kRYbnle4C7Uw4F6fCHqOSyl+qKM24K+9AzvnbAXy7fPxbAL7wbtptaGi4PKxNxCDgV+faJh4bGxtV1fY80VksQQKsQmmUHcec88YoTt10Kn+pj3yeRtSNRiOrbjpJo1pKLYbAuadqLsW4Rmk8AlwUQyUlg6WikmLOHeie8Xk9S87UchpjLWJQx4aj8VTrc6q802BOT0975fJq7mJ2DZdMGv1/yyJsaGioYi00gZxzt9mlrmSBkq2qAUH8t2tLSa+4Nn/nIswcycQ1/nUHIgdXKMNJJnVdDQaDzgZ3EWZhqzJx5iSIcg5ce8Ht+ONsa2339PR0aRyAOYkGzDUB3deAOQEXnFUiuPReXGSkCxIqjYf7zhVvCSk9Ho+XcvS1TY3y4xwWDtxxmoPLLVHwOKj7kvkeV77ss6LaMENVHUdq8ITRPeadzzngGOTSAqELiVPfa8STY47dvbhFwIWnKmvNixEvGqqyKjGn7Wu0n4s14Pt0qdVcqRhY9q2ricPqKROI8SLokUkvF+9Rmvx6nwGX9MXVjJ1XKo7OvON4Br6n09PT3hzjZ8CflaJRHYHMEZc8PxzZG/emtR8VzRxoaLjiWAtNIOKyV7lyArxKO02gJr1L/n8GS1kXMeiqCDuJpO467oOT9tpvJ/U58k6llYuIXLUHgJbFYglWizRzppZKRZZkbudcNkF0/GpmiTMDXdw/j4WOLVcDZk1BVX4eR+eSLW2cW9JgXZxKaE4uTsBpk7UEMAUnq5XQNIGGhiuOtdEEXJVZoJzuCngXIa+eLujGRZXV4uedG0lXXY4E48xBrTbLbjJXWVbdPM49VXNZnRcsPXXc2T2qUo4lmZYUA/rSaDKZ9HbV4fZZ44nIPHdPSty6vAnmcZxW5qI21X7moCXVkJh/YnteXcnsunSFT1UL4vR51d54bjoy0kHHb1W0INA0gYaGK4+10ASAs5BN3VvAheYyK66S7OjoyO4fGOcoG+7a5bh8F8rr+ArHZZQyEE9OTnphqWw/q73NOeRafIP/dq4gl9HHfdVcAKAf+8+2u8sLCGj/S33Tz9g1V9O8WEI6745zDcb5zkOj88M9T/5O+8YegwDnt8R37HGI8Qut8OTkpMfe8725DUmdhqveAdbePmtchIPBYKmard786elpb/I437CLkV8VHahwqiWrmI7Uc8UldENP9uuG+hvf8eYZNdKtNHal+wuUJoIrUMHjoJ/VXj6Nu9/e3u69mC63gxcXjsLU6zh3Kj+/80QPukrBzjzSRDYmbmumIVf5DfDcVJKOC+nUNozhcVdzhwuYOFPVRWsymjnQ0HDFsRaawGAwwO7u7pJa5FZYJ7U1ss8RVbWgHnaJOY1B1UgXnz8ejzvpFxl1Ozs7S6WmuD+j0aj7jt1OJWJwd3e3t/q7bDJWTzVAhb9jLUfVR5Y+TvMKsJsqzovnF2PAxVS45JhmzR0fH3fpwpExyOqse7b6fJxUZujcKWUiapGQ0N547ji3oc4TJnq5r9rudDrt5aew6q95BW5zWp4LjmAvmUnd99VvGxoaXvBYG01ge3t7qSSTCyhxIcWO6HOhsIB3LbErh6Wt0wCiDUew8b0A85U+JKH2h4N/SiGjwPKq7rLm4m8uLqm7L9X2XihlS7qMxTi67dOZtwHOpD7fk6uSzPa/tsvPx9nxjqQraYUl4tFJyFLYNZN6XNla5wmHIDveQuECn1xNCn4+riK3zkXm1i60nsDziXgxSrsSM2kT5/CD54mnD8SxyjzgOun5e8f664NnYtCVnHYLj2PZSwsPg4kr9XO7qEA3Abkfrky3VgNm8lDJK0dUBTY3N7vzmDjlTTv1d1xRCJgXFaltLMux+iUTjoUF9zX+5nuKew6zJNT3wWCwZLoFdE6ySediE9grEP0peUZ4zgccec51G/k8YD7eqxaBZg40NFxxrI0mACyv5rEC84qpqyLXZ3NulYD6mfl8F3XIq7OL2df9BHZ2dnD9+nUAZ2m0m5ubVgOI+2QJBvgMM1Wz+ZqsKgZY4rlYBiUBWXNgKVRLo1bCkdtnDS3+rxWXr1271v2GzZjob3zGY8ckbnzmtkMLuLiPmkruTENXqVqJQZb2zvVcy2IMsMbooJoGpwazxuMyCwFf9EXRNIGGhiuOtdIEuDhjLffeRWzFare1tVWUVm6bKW63lFEILJfM4msBcxdeVKIN9xjneGvs+/HxcY9k4siumguIORCnYWjQCEtzJQ1dgUp2tSlZx4FYTHIyGcZjllLqNIDQlHZ2dnqRdM4N6DgKtqe1H+ySc/ekUpmDdlwAVCkbk8HPWH9X0jyU/HOSmjUpPZ81TEdQKt/CbZSwNotATGAXehrfu+SIuEHeBy8+i8Hkyjuqhjv/MrPxOmk44YP3otdrsmkT14qX7/DwsBdZNpvNuntQFdNFyPHEjs+m06klnrQN97K4RVG/K53j1O/4v5pOPOmdX59DpYF5GHiQdJGizJ/FmLIqXyPfOKFJFxLujy5o3AZDf1uL3uR5xaakEqTRf15gNeqUrzWbzbrPY05y8tqqRaCZAw0NVxxroQnMZrMudl4JmQCnefJqq+7Ara2tnqRhtUmrwToijIkWVYnZReO2nOIEH04YArAk0ZSQ45h6LcThfPHOh3x8fNyTggGXAMXqrDPD1HV13tRW3e2Xr8/RlXyeJoyxdhPPjDeduX37NoBlTSDG99atWwB85V9HFroiHnoPjtyrpZ47bZI/Y81E8wnYDewke4CLkWjEKpsFzUXY0NBQxdpoAgcHB0s2mboDXYEIltgu9djBxZAr2NWmUn97e7tnp3EWF0sEJZVYorlMOrUvmQxS8g1Aj4dwwT8BF5TELj/WglQas4aiLiiWhro70fb2dieZWKIFSbi7u9uNcSlQajKZ9DILJ5NJF0zE937z5s2lfoRGwKnYLuuwllvi5hNLeHUp8u8d4ajaAWuA+uzc3OT8BtYEXJFX4HnSBFJKP5RS+lRK6cP02YMppZ9JKf3m4vjA4vOUUvpnKaUnU0q/klJ63ar2Gxoa7i3Oown8awD/HMAP02fvAPChnPN3pZTesfj/2zHfmvzhxb8/AeD7F8cqcs44OjpaKqLgXC0aLsnsM9vMnFXHx9ls1rPJnPbBocQuR17tYJf3z54Ix1bX8hpUMrAtyYjVP75jl59yDi5kmnkLtmVLmoD+NsZHpU9I/+vXr+PBBx/s/o5xjPPDrbq5udnLlitpLjpWAZaooSWE5lXKLnRx+W4+Act8QU2yOk8K5/OX3MCuX9PptBcoxV4Nfv6qhcUYs/ZWwspFIOf8X1NKr5KP3wzgDYu/3w3g5zBfBN4M4IfzvJc/n1K6P6V0I+f89IprdDHoTk0GluOoXeosEy36wjDZyBF3wLJqzhO7pPY6Qo6vx754V6OPr1NqI8B9VMKU+1gjqpyL0Km/DkoCutwONskiJuD+++8HADzwwAPdy8/uTyZqA87lB/hYELdB62Aw6BYVrWG4t7fXW1y4PiCPmS58/aZ5cwAAFkNJREFUbq6xH780fkw88lx2BKKSkHzNWECYxNR9HjgfQ58PL44l3Ckx+FC82IvjyxafvxzAx+m8pxaf9ZBSeltK6YmU0hOrOtnQ0HBxeL6JQReVYEOncs6PYr6VOUajUS6twC6Vk1d1XYl5804Xz63fsXsvMJvNeuRfgDWHpRs3gSEBlULu96WgpfhOC3G4+2RT6DzBVu5aHOSi9+Ri1FmbCE3gRS96EYC5CaDur5xzJ92CyEspVTWB6DcTjloGbDwe48UvfjGAvptsd3e3R87u7e111+S+uSCrgCvo4oJ4Ampycq6G27ZOTbOUktXeVLPkWpsxfmp21HCnmsAnU0o3Fp26AeBTi8+fAvBKOu8VAD5xh9doaGi4BNypJvBBAI8A+K7F8QP0+TellB7DnBC8tYoPAM6kibO3Xew7r7Caucb5B2qTu8AgV3UY6BeJcCG857Gn47p8LSeRWavR6sRMDLKkcuNRCvll8L3XQn0D3C8tmZZS6pXM4v67bDyVypPJpCsvpryJGyt2e3HYeGgicX12LUb7TNK6Ap8aiu20NtYINLgofs/x/A5O2mslaSZu1QXI5x8fH3fBU6H9sIuwpgEC51gEUkrvxZwEfElK6SkA3475y//+lNJbAXwMwFcvTv8pAG8C8CSAAwBfv6r9riMbG0spti4xwxEtAfYhuxc8zuFad9Guqk4ussvFu7sFhKP3HEOv9+JKQ+siwCojM/eqnro6jC4SkCePRkS6mHfeiENLpfOGl7qByHQ6tQsK51AA8wjAeEkDXJVJ23Ami3uZ+NnpwsMvMCch6eLizAL+TnNAdCMRHj/+LUOFBC8CzlOjROLp6elSrgWAbjxXbUEGnM878LWFr95ozs0AvnHlVRsaGtYGaxExGCs7qz8qxXnVDbD6q4RS7VrAsmR32VuOeFK46DyOTdC+8IqvUshlqbFkqxUJcRWFA+xWK+VUxPfRD3Vbsqak7kCuEq1prCyduc8hrcKff3h4WKyN7zZqTSn13Iyz2awjBNXs4nnFz9URbCqV+eiKrajJySXNHClaSzXWWpTD4bAX6zKdTnukNWsMMY7PPfdcdx2nfTBa7kBDwxXHWmgCbN8GlBjilZM/01z9k5OTYqFOboPtKV1tJ5NJz/XDdrlKC9YmtIQXX5clt7PVXRFPbYvvSd1HzGW4ICqNUnPBK/yZ40o0QImzAlk7iN87Dsbl8auWEr87PDzsEb3T6dRuPxcuQg2c4T5y3QkdX977QfvNWaz8/EvlyJiQYwmvGoMjOR0nwNJf5yYXE9WoSa5rUULTBBoarjjWRhMIe6zEbrNtw9KulgtQc8kxXO0AlQgsgbWmv2PU3XWdy8+5gxSc7ee0Aqf5KEPO8eUhVdgl6/rhJLx+5oKWWALHGHBVIC235rwxLFl1XNwW4hxaHeC2lD2fTCa9cms8HqoJcHn7WlWqwGw2s8Ft6jHicdB+D4dDm++h3NhwOFzaKYmvyaHZJazFIgCc+Y9VharFufOEZWLLqevxnZoXriIuD7QjHJUM4lRfdv3pi87/r6ny+uD53l2tO5dPoOm33Aa7/Nwk0/TpBx54oDvyGPER8JuPaBLVc88911tYS2m3wLIL0rm7ODaBXZPAmUp88+ZN/P7v/373N4AuYQ1YJnPVVHEvfCm2pHRP7sXn3zuBEMdSHgy3x/kBaha4vSgUzRxoaLjiWAtNIKXUubBUpWONwElxJe7YbeOi7Fzgjgs+ciRQXMeRgLXoMOfec0E0LkAqrqmqtgteYZXfpWSrxHHBRfy9/n97e7vL1GOtRfMV4v+8exC74+IzR47VIjlVsgLLG4YqKRuawP7+fi+9mDU13nFJny3PFx2/mplZ0hxcungpSpGrY3O72haPKe/lEO03TaChoaGKtdAEgOUAHcC7S1QTYHuRSy1pnXpnT/OKGp9HqCXbYiqxOaCEJZNKTy45pjHhnA/vNB0XbnyecmuhUcXYMNj95UKIA1xYQ/vDmgPfi+YMhDRi+59dZ0pasttLY/DZrRZt8Z4LcR4XK1GSmF3Jzj7nOeZ4gmhDx5b7pmASkElOd3+OA4r+qI0/Ho97GgwHYGkWJJ9XwlosAkFacUEQfdGAfh3BUpx7idzhc/lvfVg8sZWscUUaOJrR1d6rFcXQWoZAvyIOxyHwYuPURzU9dILx+Qzn9w9/P1cFcoRgQBduvj824dw2WrrA8/joC8m+b14gYhHXDVUnk0lv7wK3sPLYuEVAow6dN8vlhygJrPeu1+b/u99yVGIc47y4vwAvGiU0c6Ch4YpjbTSBIIdUdWfJoK4iV/KJ/67V8WNtIdpgCVVaPVmKuyxGVv1VdXZ713MbWs+QpYAzT7TQiNNuasSfK1E2Ho87DeChhx4CALz0pS9d6heDiVjV3liz43NUQvLGsjpmjGiLs/045oEzBIGzLMWbN29ib28PALqUW3YNu2et48L9ZtJQx56fpzP5PlPoWJUIStV6nUZaQtMEGhquONZCEwi42HReaTVKzLn3XBw/S1RXODTAK6ySYmx/aYQe2258VBuSIxP1/KOjo57GEJLKFZfgvHzmI9TOdQFK/DslLV2OhAtGYu5BSTd+Thp5x1oNR+epa5i1JhcZqXUQ+PpxfyH19/f3O74gtAPmIVj7ZI2Ir+kyP2tBQ6V6Ai4SVbWeWgAZk5E8ptpvJklLGZqBtVgEmFhzKnwcHVlTY9n1RWP2PMAPgCe2ElV85E0ttf+sttUWgUBMWCZ0Qh3n7bq4smx8Fn+7qEBngsTYsv9ckfNZPcPf+73fW2rrxo0bXSVhfkk0nTteOK64zGpqPLO4ZxcKy2SnEmEuUtSRkQGXJMamJJ9femZuM1snQPiophj3wxGl2gb3jRdOTgEH5vNEi73UwqkVzRxoaLjiWBtNQKP8zkOmMCHCmkMpkovPdy5IVb3ceSU1X80XbsPdi8tvcKof4JN0mCjlNp07MtpQuCQWjpqLNjgJRclZdoU5t13Ane8KtaiJ49TlcCdzfzY2Nmy0XPxfnxn3g7UQtwFpnO/IXJXeTvV3KecBt4cCt+W0IOdejpwOLfHGqfUlNE2goeGKYy00gQDb7Co9eTVnIsdlrpUCWnglZpeVcxuWAj2c/eqi1ZyWwtqOSkvmN9TdyFKI7VK1//g3LkCKI+7iXLWVgX7VYOUouB/MfTjpqeMymUx6EW8uBZrJNxdcFO1eu3ataz84hiitFcfj4+Put6E5OMnuxpmfnX7Gc001Vv6OtbMaJ6USfjQa9biJ8Xjc46SYZ9F2XaFWRdMEGhquONZCEwgNgFczV2jU5Zxz/HR8VwrljGsB3k5n6aMaAIcRq7RghpeZXpdFFtD2ebVWXsFl+7EEYS+BxuW7IqgsiR1rrhI9vtvb2+u0iOjvzs5OV29A3ZM5515dA+YmWPNye0RGm+r2Go1Gvc/cPXMBERfq7TJKdZz5Gep3PHfOEyLsuCMX6s32v5YSc1uNT6fTHh9TC3NXrMUiAJz5j5XUqUW8cUVXXgxcFd64hhJlLrmE1WSXkqsvKVeAdZFuzg+sKi6bNm4y60LC1Y/YZeQqBcXnTDTGdfgeYkzjN9HurVu3AMxfJs0rGA6H3SIU24/xs4jfxgvJz4AJSBcHEf93m5soKcquMCX82FXJL6suAnzvARfjX8vHcPEB/CyU/CstDDou/Dt94U9OTnqxAC4vo4RmDjQ0XHGshSbgVLOAU2XcausyqtTVxqjtuMOSWgM5HBFWygRz32sbvOo71TyOLuU3oBKe+8tkZC0un/dXUMKRI9Q4ZTvaclFqcdRxnEwmvZwHJsziM1aDYz8Bvg4H8QBzEvDZZ5/t/gbOgpYODg5szUUn0RW1vBOG0wQU7prcbsC5nrUd4Ow58ia5mgezsbFx95pASumHUkqfSil9mD77xymlX08p/UpK6d+mlO6n796ZUnoypfTRlNKfX9V+Q0PDvcV5NIF/DeCfA/hh+uxnALwz53ySUvpuAO8E8PaU0msBvAXAHwXwBwD855TSH8o5V+MWYzV0sdhu9WVXmu4Bx3aXs4trdjTbZBqbzm1piCZLMkcI1VyQzH0o1+BsZnYfau44S169J+eG4/0SOfBEaymw/apSn/PVNXafSUa29VWrYQJM7ejNzc2lfQz0XuI53r59u9MEopho5A64IiQu6IbtZ+c+VO2N+SQX3qv2P5PKTvtQvmpzc7OqpTDhqKHEgdlstrLi8Hn2IvyvKaVXyWf/if778wC+avH3mwE8lnOeAPjtlNKTAL4QwH9fdZ2IbS/FOTv/v3txHPnCRJRLKtKJV1LXAf8icDUergRTqzJbi3TU42w265kUzkQo5UFEHzUikReNAJtTGstwfHzce5lu3rzZRalF8ZGASz3e3d21qr9LTIrvdGE7PDy0m5pqHIQbd15g3aKiHh0WDBydGOOjbHyAF3qX3xBwZDWr8o4E1IW7RG7H/axKIHo+iMFvAPAfFn+/HMDH6bunFp/1kFJ6W0rpiZTSE6tcGA0NDReHuyIGU0rfBuAEwHviI3OaZSVyzo8CeBQANjc3c/j3SwQhq9yOiKuRUbGC8yYUrMaxmy6OJZ8w+/852k+vxZqAk9jqJuN7cWNQStflz3jFV03GST6Wsq46raasAv0MxIODg84sednLXgbgbJ+C3d3dnoQfj8edK9H52VVbYddpSP+bN2925F/c8+3bt3v1DHm+uH0E9FouD8LF5ztNQOFKg7nSahzX4ua3PgN+Fqy5xr1r/gRrmCXc8SKQUnoEwFcAeGM+u8pTAF5Jp70CwCfu9BoNDQ0XjztaBFJKXwbg7QD+dM75gL76IIAfSSl9D+bE4MMA/sd52oxgllK0X2k1c7akkmG8/ZYWoSzZaSX7/PT0tLND47fb29tLMelxdCXHtH2+/xIppefFUQlHDghyhFKNQ2BtQrkM1nj0t6PRqIvfj3EJl97Ozk7Hm7CmobHvrDUFWPpHuyH9b9261ZF/XEBUK+5qjoK2rwQsawdKaG5tbXXP2El2bYs1gQDzSVwiTzNoWQtyZKvjjHSnJ66V4Ob4Ur+q38479l4AbwDwkpTSUwC+HXNvwCaAn1l0/Odzzn895/yrKaX3A/g1zM2Eb1zlGQgwobe4bu/oFgSt1suhsPqisX+Zz4kHp6GrfB6rZdEuh6zqC+OIQW7Xqf5KWjrCisdLSTTnpeAJ68gjJcI4zNT1W8eUQ3jjWvFi8u9iEvMiw+OjJlBM6oODg471j63E9vb2emYJt6sLIadiu4IgfG1dOHQcuY+rQnJVCLm0aBcnwOapK1ajwsUR3jUvhOI83oGvNR+/q3L+dwL4zpVXbmhoWAusRcQgk3m68jkpx4jfcVSZkigugYOlkap0vHo6dUyTYbg2foBXbAWrdKyKlggip/ozeFxK9RXdhp3cZ5YupchFllhx73t7e931Y7zDNcjVhl2BEXZ/6VjF+awJPPPMMwDmJKA+YyZK3XN0Kb9OC3JkYZyrGib325kFalpwvoczrVRqs3bI80SlvdMEuI+rtIGWO9DQcMWxFppAgINutH66I3cGg8FSzDswJ+nCleNSedVOc1II6NuC0SZH2fG5zlXkIhYBLwWcVGHJq3Yux4SzRHUu0BgfdQc6ycdEldMYVDIdHh52tnoQhKGVbW1t9QJgOGWcyT9tlzNBIwcgiMH9/X2bCxBw9vB5bGW2zzWC0WkCrKEF3HeOLOZnp3OeNTB1+fE9MCdQ0qD5WiU0TaCh4YpjLTSBnOclu1lacXAJsGxPBTjYhe1iV3YZmEsV1Sgmk4mVKi7YJtpytmFttXVSR8/n+6vVGnCbtnKbKklLrHa0G3D5BNoGe3A4P18lUzD2m5ubS1pBtKUeAOYh1MaPuaGf6Rg4tytrOep+TSZ7lEODNX+C3a8Mp1lGW+qR4Ou556kepqOjIzs39RmzFum4tFWcwFosAoGNjY1O7dYNODimPcDqZviIeeBc8o+60DgCy8XZ64Pkdvn/zsev/XU5DJ9pNWA+6sNNKfXGjdVxfflYjXRuQ4234DgLV/QjwIuLqsscy+4IPFcD0vWjBjcuznWqZokzL93Cw/1xQgjwQsuh5jbme68JErco1kwWRTMHGhquONIq0uBSOpHSpwHsA3jmXvcFwEvQ+sFo/VjGZ3M//mDO+aX64VosAgCQUnoi5/wFrR+tH60fl9uPZg40NFxxtEWgoeGKY50WgUfvdQcWaP1YRuvHMl5w/VgbTqChoeHeYJ00gYaGhnuAtgg0NFxxrMUikFL6sjTfp+DJlNI7Lumar0wp/WxK6SMppV9NKf2txecPppR+JqX0m4vjA5fUn2FK6ZdSSj+5+P+rU0qPL/rxvpTSeFUbz0Mf7k8p/Via7ynxkZTSF92L8UgpfevimXw4pfTelNLWZY1H8vts2DFIc/yzxbz9lZTS6y64Hxez30fEVd+rfwCGAP4PgNcAGAP4XwBeewnXvQHgdYu/7wPwGwBeC+AfAXjH4vN3APjuSxqHvw3gRwD85OL/7wfwlsXfPwDgb1xCH94N4K8t/h4DuP+yxwPz6tS/DWCbxuGvXtZ4APhTAF4H4MP0mR0DAG/CvNJ2AvB6AI9fcD/+HICNxd/fTf147eK92QTw6sX7NDz3tS56Yp3jZr8IwE/T/9+J+cYml92PDwD4swA+CuDG4rMbAD56Cdd+BYAPAfhSAD+5mFTP0ANfGqML6sP1xcuX5PNLHQ+cla1/EPPclp8E8OcvczwAvEpePjsGAP4lgK91511EP+S7vwjgPYu/l94ZAD8N4IvOe511MAfOvVfBRSHNN1f5fACPA3go5/w0ACyOL7uELnwvgL8HILJJXgzgZs45MnEuY0xeA+DTAP7Vwiz5wZTSLi55PHLOvwvgnwD4GICnAdwC8Iu4/PFglMbgXs7dO9rvw2EdFoFz71VwIRdP6RqAHwfwLTnnvcu6Ll3/KwB8Kuf8i/yxOfWix2QDc/Xz+3POn495Lsel8DOMhb39ZszV2j8AYBfAl5tT18G3fU/mbrqL/T4c1mERuGd7FaSURpgvAO/JOf/E4uNPppRuLL6/AeBTF9yNLwHwlSml3wHwGOYmwfcCuD+lFDnLlzEmTwF4Kuf8+OL/P4b5onDZ4/FnAPx2zvnTOedjAD8B4Itx+ePBKI3Bpc/ddLbfx9flhe5/t/1Yh0XgFwA8vGB/x5hvaPrBi75omidevwvAR3LO30NffRDAI4u/H8GcK7gw5JzfmXN+Rc75VZjf+3/JOX8dgJ/F2R6Pl9GP/wfg4ymlz1189EbMS8df6nhgbga8PqW0s3hG0Y9LHQ9BaQw+COCvLLwErwdwK8yGi0A62+/jK3N/v4+3pJQ2U0qvxmew3weAe08MLhazN2HOzv8fAN92Sdf8k5irTL8C4JcX/96EuT3+IQC/uTg+eInj8AaceQdes3iQTwL4UQCbl3D9Pw7gicWY/DsAD9yL8QDwDwD8OoAPA/g3mLPelzIeAN6LORdxjLmEfWtpDDBXw//FYt7+bwBfcMH9eBJz2z/m6w/Q+d+26MdHAXz5Z3KtFjbc0HDFsQ7mQENDwz1EWwQaGq442iLQ0HDF0RaBhoYrjrYINDRccbRFoKHhiqMtAg0NVxz/H/i1hqnqZEePAAAAAElFTkSuQmCC", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "plt.imshow(avg_image, cmap=plt.cm.gray)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "There are multiple ways to perform the segementation. To keep it simple, we just detect the cells by setting up the threshold on the average image." - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAASdUlEQVR4nO3dfYxc1XnH8e+vBuNAYhlDQH5TMZJJSxExyOIlVC3CSQ0U4VSCCooSN6GyWtGWJK2CXf6glRoptBGhkVrSFZA4FeE1tFiIdktdUFSpOKzBMQbHsIEUFjsYKiBVXLkmefrH3BXj9d3d2bkvc++c30da7cydO3PPnJn7nOece+ceRQRmlq5fGHQBzGywHATMEucgYJY4BwGzxDkImCXOQcAscZUFAUmXStoraVzSpqq2Y2bFqIrzBCTNA14EPgFMAE8D10bEC6VvzMwKOaai1z0PGI+IlwEk3QesB3KDwHwdFws4oaKimM3sjLMPDroItdix69BbEfHhqcurCgLLgNe67k8A53evIGkjsBFgAcdzvtZWVBSzmY2O7hx0EWoxb8n4f+Utr2pMQDnLjuh3RMRIRKyJiDXHclxFxTCb3bqlq1m3dPWgizEwVQWBCWBF1/3lwL6KtmVmBVQVBJ4GVklaKWk+cA2wtaJtmVkBlYwJRMR7kv4QGAXmAXdHxPNVbMvMiqlqYJCIeAx4rKrXN7NyVBYEzNpm3dLVjO5r95GCmQc4x3OX+rRhs8Q5EzDrMrUlLTMzyGulu19/UIcpHQTMZjC5Y+YFg5l22tF9O3vaqZtwfoK7A2aJcyZg1oMmtNhVcSZgljgHAbMKtClzcBAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS5yBgljgHAbPEOQiYJc5BwCxxDgJmiXMQMEtc30FA0gpJT0jaI+l5STdmyxdLelzSS9n/E8srrpmVrUgm8B7wJxHxy8AFwA2SzgQ2AdsiYhWwLbtvZg3VdxCIiP0R8Ux2+3+APcAyYD2wJVttC/DJooU0s+qUMiYg6TTgHGA7cGpE7IdOoABOmeY5GyWNSRo7zKEyimFmfSg874CkDwLfAT4XET+R1NPzImIEGAFYqMVRtBxWvemm5GrTlXXtaIWCgKRj6QSAeyLi4WzxG5KWRMR+SUuAA0ULaYM123x8k487GLRTkaMDAu4C9kTEbV0PbQU2ZLc3AI/0Xzwzq1qRTOAi4FPAc5Imm4o/A74MPCDpeuBV4OpiRbRBmeuMvM4I2qnvIBAR/wFMNwCwtt/XNbN6eUJSO8pcM4BJzgDayacNmyWuEUHgjLMP9t36WLn6+RzWLV3tLKDFGtUdGN2301+mlvDnNDwakQmY2eA0KhOwwem1G+AMYPg4EzBLXKOCgFuZ5hvdt9ODuEOmEUHgxV3HOwCYDUgjgoCZDY4HBhPn1N6cCZglrlFBwK1S8/nswOHTqCDgL1e9+gm6PjowfBoVBMysfq0aGJypBXIWMXvLPrWO1i1d7VbdnAmYpa5VmYDl66U1z8uU2pAFdJfR2V41nAmYJa7VmUDqLUO/GUBbdV/IdOp7H6b3WbdWBYHJD7oNaWwdUqiPXrsx7jb0z90Bs8S1KhOY5Eh/pO76qOva/01OzZtWnqZzJmCWuFZmAja9Xlq9MsYQ8rZTV/Zh5SpjVuJ5wBjwekRcIWklcB+wGHgG+FRE/F/R7VhxZe5E3d0B7/ztVkZ34EZgT9f9W4GvRsQq4G3g+hK2YWYVKRQEJC0HfhO4M7sv4BLgoWyVLcAni2zDmq2JrfRkduKfPfemaCZwO/BF4OfZ/ZOAdyLivez+BLAs74mSNkoakzR2mEMFi2Fm/ep7TEDSFcCBiNgh6eLJxTmrRt7zI2IEGAFYqMW561i52npy0UytuadDL67IwOBFwJWSLgcWAAvpZAaLJB2TZQPLgX3Fi2lmVek7CETEZmAzQJYJ/GlEXCfpQeAqOkcINgCPlFBOK0HbMoBeOAMororzBG4C7pP0l8CzwF0VbMMGZOpON4yBJTWlBIGIeBJ4Mrv9MnBeGa9rZtXzGYMJKWNgsOh5+Z5+vnn82wGzxCli8EfnFmpxnK+1s67nw0Hlqqo/n/erxl7Xt+r8Wzy0IyLWTF3equ6AvyzlqHowz4OF7eLugFniGpcJTNeKOAswq4YzAbPENSIInHH2wVnnuHM/c3j5sx2sRgQBq1cTf2LriU4Hx0HALHGtCQJNa7msGs4I6teaIGBm1WjcIcKpnAGYVcuZgFniHAQS1uQsy2MD9XEQMEucg4BZ4hwErNHcJaieg4BZ4hp7iLDJg1bDwq2sQYMzAY8OG7gxqENjg4CZ1aOx3QGr3rqlq3OvHlx3BubWfrCcCZglzplAIvKu9z/bpdzmmhF44tB2KhQEJC0C7gTOojP78GeBvcD9wGnAj4Dfjoi35/ra/rKUo3tHrmKnLuO1bLCKdgf+BviXiPgl4KPAHmATsC0iVgHbsvtm1lB9Tz4iaSHwfeD06HoRSXuBiyNiv6QlwJMR8ZGZXmvNRxfE90ZXHLHMLUf5em29XffDabrJR4pkAqcDbwLfkPSspDslnQCcGhH7AbL/p+Q9WdJGSWOSxt78758VKIaZFVEkCBwDnAvcERHnAD9lDql/RIxExJqIWPP26x9y62M2IEWCwAQwERHbs/sP0QkKb2TdALL/B4oV0cyq1HcQiIgfA69JmuzvrwVeALYCG7JlG4BHCpXQStPES43b4BU9T+CPgHskzQdeBj5DJ7A8IOl64FXg6tle5IyzDzI66t8JNIGDRHoKBYGI2AkcNdpIJyswsxbwGYMJ6m7t/UtN828HzBLXuEzAfdJ69fs7ARsejQgCL+463jv/gPiHPebugFniHATMEucgYJa4RowJ2OB4LMCcCZglzkHALHEOAmaJcxAwS5yDgFniHATMEucgYJY4BwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeIcBMwS16qfEuddB88/hbVhU/f33JmAWeJakwn4arjt0/2ZpZyxlfHdrfKCsIUyAUmfl/S8pN2S7pW0QNJKSdslvSTp/myKMjNrqL6DgKRlwB8DayLiLGAecA1wK/DViFgFvA1cX0ZB8yJgyq2LNd/ovp2tyGCLdgeOAT4g6TBwPLAfuAT4nezxLcCfA3cU3A7gnb5tUvy8ytzp66q/IlOTvw58hc7Mw/uBd4EdwDsR8V622gSwLO/5kjZKGpM0dphD/RbDzArqOxOQdCKwHlgJvAM8CFyWs2rkPT8iRoARgIVanLuOWVsUyQAGnTEVGRj8OPBKRLwZEYeBh4GPAYskTQaX5cC+gmU0swoVGRN4FbhA0vHA/wJrgTHgCeAq4D5gA/BI0UKaNUEVg3yDzgKg2JjAduAh4Bnguey1RoCbgC9IGgdOAu4qoZxmVpFCRwci4hbglimLXwbOK/K6Zlaf1pwxaNYm06X5TTxvwL8dMEucMwGrzOi+nY0Y+KrTbO+3ifXhIGCl6055q/zhS5O0+f25O2CWOGcCVrrJVnHYugPd76v7fts5EzBLnDMBq8ywtJRTDdv7ciZgljgHAbPEOQgMWFuuPmPDy0HALHEeGByAVFp+X224HZwJmCXOmUCFymrxp3udpreuTS+fdTgINNhsQWTYzlyz8uR9d+YtyV/X3QGzxDkTKNlcuwAzteJ5j7V9UHG28jur6U+R74UzAbPEORMoyVwicZHWrq0tZa/143GO+jkI1Mhf7N7lBQ3XXzXnXrg7YJY4ZwIlmRqVfbZc+YZ1UHGmLtBM77msQWJnAmaJcyZQkba2SmWr85Dm1G215TMYdDlnzQQk3S3pgKTdXcsWS3pc0kvZ/xOz5ZL0NUnjknZJOrfKwptZcb10B74JXDpl2SZgW0SsArZl96EzNfmq7G8jcEc5xbQ2avuJTU2wbunqyjOFWbsDEfFdSadNWbweuDi7vQV4ks5EpOuBb0VEAE9JWiRpSUTsL6vA1nxN2fmHaXC2u/xl12+/A4OnTu7Y2f9TsuXLgNe61pvIlh1F0kZJY5LGDnOoz2KYWVFlDwwqZ1nkrRgRI3SmMmehFueuY+3SlAwgj08+ml6/mcAbkpYAZP8PZMsngBVd6y0H9vVfPDOrWr9BYCuwIbu9AXika/mns6MEFwDvejzAmqhtWcDkBWmryGhm7Q5IupfOIODJkiaAW4AvAw9Iuh54Fbg6W/0x4HJgHDgIfKZQ6awVmtwNGBZTp0ArUy9HB66d5qG1OesGcEPRQplZfXzGoPXFrf9g5B0qLPpZ+LcDZolzELAkDUMmU9bZhA4CZonzmIDNyTC0oNC+Q4QzWbd09Yyfy/vvdTz3cQcBm9Ww7PgwXDt/L3p5v+4OmCXOQcBsCBQZJHQQMEucxwRsVlWeslqXVMYC+nmfDgI2qzbv/DY7dwfMEudMwKY1LBlAKl2BfjkTMEucg4BZ4twdaJkyU/TZ0uRhOCqQmn4mYHEmYJY4BwFrlDom2xhG011/sJcszkHALHEeE2iZMvrpTW5pp07TPazvs0mcCZglzkGgpYa9lZuuj2v5phtL6eV74u5Ai/XyAc80WDTsgSRF/XymzgTMEudMYMjltQzDnmY7w5mbWTMBSXdLOiBpd9eyv5b0A0m7JP2jpEVdj22WNC5pr6R1VRXczMrRS3fgm8ClU5Y9DpwVEWcDLwKbASSdCVwD/Er2nL+TNK+00lophvWEnGF9X1XrZS7C70o6bcqyf+26+xRwVXZ7PXBfRBwCXpE0DpwH/GcppbWByJv6yoZHGQODnwX+Obu9DHit67GJbNlRJG2UNCZp7DCHSiiGmfWj0MCgpJuB94B7JhflrBZ5z42IEWAEYKEW565jzTM13W5KZuBuQP/6DgKSNgBXAGuzKcmh0/Kv6FptObCv/+KZWdX6CgKSLgVuAn49Ig52PbQV+Lak24ClwCrge4VLaTaFW/7yzBoEJN0LXAycLGkCuIXO0YDjgMclATwVEb8fEc9LegB4gU434YaI+FlVhbc0OQCUq5ejA9fmLL5rhvW/BHypSKHMrD4+bdgKqbNV9nkA1XAQMEucfztghVV52NAtf/WcCZglbigzgelaonVLV/d1SWabm15/uei6b4ZWBwF/sdrDn0tzuTtglji9f8bvAAshvQn8FHhr0GUBTsbl6OZyHKnN5fjFiPjw1IWNCAIAksYiYo3L4XK4HPWWw90Bs8Q5CJglrklBYGTQBci4HEdyOY40dOVozJiAmQ1GkzIBMxsABwGzxDUiCEi6NJunYFzSppq2uULSE5L2SHpe0o3Z8sWSHpf0Uvb/xJrKM0/Ss5Ieze6vlLQ9K8f9kubXUIZFkh7K5pTYI+nCQdSHpM9nn8luSfdKWlBXfUwzz0ZuHajja9n3dpekcysuRzXzfUTEQP+AecAPgdOB+cD3gTNr2O4S4Nzs9ofozJ9wJvBXwKZs+Sbg1prq4QvAt4FHs/sPANdkt78O/EENZdgC/F52ez6wqO76oHN16leAD3TVw+/WVR/ArwHnAru7luXWAXA5nSttC7gA2F5xOX4DOCa7fWtXOc7M9pvjgJXZ/jSv521V/cXq4c1eCIx23d8MbB5AOR4BPgHsBZZky5YAe2vY9nJgG3AJ8Gj2pXqr6wM/oo4qKsPCbOfTlOW11gfvX7Z+MZ3ftjwKrKuzPoDTpux8uXUA/D1wbd56VZRjymO/BdyT3T5inwFGgQt73U4TugM9z1VQlWxylXOA7cCpEbEfIPt/Sg1FuB34IvDz7P5JwDsR8V52v446OR14E/hG1i25U9IJ1FwfEfE68BXgVWA/8C6wg/rro9t0dTDI725f833kaUIQ6Hmugko2Ln0Q+A7wuYj4SV3b7dr+FcCBiNjRvThn1arr5Bg66ecdEXEOnd9y1DI+0y3rb6+nk9YuBU4ALstZtQnHtgfy3S0y30eeJgSBgc1VIOlYOgHgnoh4OFv8hqQl2eNLgAMVF+Mi4EpJPwLuo9MluB1YJGnyp9511MkEMBER27P7D9EJCnXXx8eBVyLizYg4DDwMfIz666PbdHVQ+3e3a76P6yLL/YuWowlB4GlgVTb6O5/OhKZbq96oOtdKvwvYExG3dT20FdiQ3d5AZ6ygMhGxOSKWR8RpdN77v0fEdcATvD/HYx3l+DHwmqSPZIvW0rl0fK31QacbcIGk47PPaLIctdbHFNPVwVbg09lRgguAdye7DVXomu/jyjh6vo9rJB0naSVzne+jykGeOQyAXE5ndP6HwM01bfNX6aRMu4Cd2d/ldPrj24CXsv+La6yHi3n/6MDp2Qc5DjwIHFfD9lcDY1md/BNw4iDqA/gL4AfAbuAf6Ix611IfwL10xiIO02lhr5+uDuik4X+bfW+fA9ZUXI5xOn3/ye/r17vWvzkrx17gsrlsy6cNmyWuCd0BMxsgBwGzxDkImCXOQcAscQ4CZolzEDBLnIOAWeL+HzgSnDgghdkpAAAAAElFTkSuQmCC", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "threshold = 50\n", - "mask = avg_image > threshold\n", - "plt.imshow(mask)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The outcome is different across different threholds we set. Therefore, this threshold is a parameter we could potentially tweak." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQhUlEQVR4nO3df4wc5X3H8fen/hmTItsQkH+pGMlJS6vGoBMxoYoQTmpCEaYSSEYocVNXVivakqRSsMsfqFIjhTZKaNSK1AISp6IQ6tDaQrRX4xBFlYrDAa4xGPAFWjjsYFCBREFy7ObbP+a5eO/Y8+3N7Mzu7fN5Sdbuzs7ufD13+5nv8+zejiICM8vXL/W6ADPrLYeAWeYcAmaZcwiYZc4hYJY5h4BZ5moLAUlXSXpB0qikbXVtx8yqUR2fE5A0B3gR+AQwBjwB3BgRz3V9Y2ZWydyanvdSYDQiXgKQ9ACwEWgbAvO1IBZyVk2lmHXmg7/5bq9LqMWLBxcB8BPeejMiPjD5/rpCYAXwasvtMeAjrStI2gpsBVjIIj6i9TWVYtaZ4eEDvS6hFhuWrwXg0dj1P+3ur2tOQG2WTRh3RMSOiBiKiKF5LKipDLPOjb9YBkkn/6e6OoExYFXL7ZXA0Zq2ZWZJmSCrqxN4AlgjabWk+cAmYE9N2zKzCmrpBCLilKQ/BoaBOcC9EfFsHdsyG2TjR/bho2eer6gylKlrOEBEPAI8Utfzm1l31BYCZrPR5CPqdEfgTp+nqjonLf2xYbPMuRMwO4NBfNtwMncCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa50CEhaJekxSYclPSvplrR8qaS9ko6kyyXdK9fMuq1KJ3AK+LOI+DVgHXCzpIuAbcC+iFgD7Eu3zaxPlQ6BiDgWEU+l6z8BDgMrgI3AzrTaTuC6qkWaWX26Micg6QLgYmA/cH5EHIMiKIDzpnjMVkkjkkZOcqIbZZhZCZVDQNL7ge8An42IH3f6uIjYERFDETE0jwVVyzCzkiqFgKR5FAFwX0Q8lBa/LmlZun8ZcLxaiWZWpyrvDgi4BzgcEV9puWsPsDld3wzsLl+emdWtylmJLwc+BTwjafwk7n8OfAl4UNIW4BXghmolWj8aPlr8yHM4a++gKx0CEfEfgKa4e33Z5zWzZlXpBCxj7gAGhz82bJY5h4BZ5hwCZplzCJhlzhODNsH4W3/gyb9cuBMwy5w7AZvAR//8uBOwaQ0fPTBhmGCDxSFgljmHgFnmHAJmmfPEoAG0HfOPTxJ6snCwuROwKXlCMA8OAbPMeThg0/KnCAebOwGzzLkTGCB1H7HdBQwmdwJmmXMnkDnP/ptDYIC4XbcyPBwwy5xDwDrWz0OH8Q829XON/cohYJY5h4BZ5ipPDEqaA4wAr0XENZJWAw8AS4GngE9FxM+qbse6ZxBbZk+KlteNTuAW4HDL7TuAr0bEGuAtYEsXtmFmNal6avKVwO8Ad6fbAq4EdqVVdgLXVdmGmdWraidwJ/AF4Ofp9jnA2xFxKt0eA1a0e6CkrZJGJI2c5ETFMsysrNIhIOka4HhEPNm6uM2q0e7xEbEjIoYiYmgeC8qWYSVsWL7WY2j7hSoTg5cD10q6GlgInE3RGSyWNDd1AyuBo9XLNLO6lO4EImJ7RKyMiAuATcB3I+Im4DHg+rTaZmB35SqtL7h7GEx1/O3ArcADkv4SeBq4p4ZtWAUzfYvQL/7B1pUQiIjvAd9L118CLu3G85pZ/fxXhBlqPbJ30hWMr+OOYDD5Y8NmmXMnkLnxo/tMOoLWx9ns5xDI3CD+HYHNjIcDZplzJ5CxJrqAM53ezPqDOwGzzLkTsFI8STg4HAIZ27B8bVeGBJOfw6Ewu3g4YJY5h4B13XTf+utvBe4vDgGzzDkErDbTHe3dEfQHh4BZ5hTR9tu/GjX04YXxg+FVE5Z5hrlZvT4i++ddv0dj15MRMTR5ud8iNGBmf0jU7nFlHtvKf67cOx4OmGXOnYBNcKYj8VRH+m4OJdwRNM+dgFnm+i4E/J34/cs/l8Hk4YDNSLsgmEk4TDV0cMD0Tt91AmbWLHcC1igf8fuPOwGzzPVFJ/DiwUU+Qpj1SKVOQNJiSbskPS/psKTLJC2VtFfSkXS5pFvFmln3VR0O/A3wbxHxq8CHgcPANmBfRKwB9qXbZtanSoeApLOBj5FOOBoRP4uIt4GNwM602k7guqpFmll9qnQCFwJvAN+Q9LSkuyWdBZwfEccA0uV57R4saaukEUkjJzlRoQwzq6JKCMwFLgHuioiLgZ8yg9Y/InZExFBEDM1jQYUyzKyKKiEwBoxFxP50exdFKLwuaRlAujxerUQzq1PpEIiIHwGvSvpQWrQeeA7YA2xOyzYDuytVaGa1qvo5gT8B7pM0H3gJ+AxFsDwoaQvwCnBDxW2YWY0qhUBEHADe83VFFF2Bmc0C/tiwWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYqhYCkz0l6VtIhSfdLWihptaT9ko5I+nY6RZmZ9anSISBpBfCnwFBE/AYwB9gE3AF8NSLWAG8BW7pRqJnVo+pwYC7wPklzgUXAMeBKitOUA+wErqu4DTOrUZVTk78GfJnizMPHgHeAJ4G3I+JUWm0MWNHu8ZK2ShqRNHKSE2XLMLOKqgwHlgAbgdXAcuAs4JNtVo12j4+IHRExFBFD81hQtgwzq6jKcODjwMsR8UZEnAQeAj4KLE7DA4CVwNGKNZpZjaqEwCvAOkmLJAlYDzwHPAZcn9bZDOyuVqKZ1anKnMB+ignAp4Bn0nPtAG4FPi9pFDgHuKcLdZpZTeZOv8rUIuJ24PZJi18CLq3yvGbWHH9i0CxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMlfpOwbNZmr46IFfXN+wfG0PK7FxDgHrmdZAmMwBUU27fTtnWft1PRwwy5w7AetLZ+oSwJ3CVKbbb+24EzDLnDsBa0SZI1TZ58uxS6iyf6ftBCTdK+m4pEMty5ZK2ivpSLpckpZL0tckjUo6KOmS0pWZWSM6GQ58E7hq0rJtwL6IWAPsS7ehODX5mvRvK3BXd8q02Wr46IGudwE2UdX9O20IRMT3gf+dtHgjsDNd3wlc17L8W1F4nOI05VO8MWGDrlcv/tyCp+rwp+zE4PkRcQwgXZ6Xlq8AXm1Zbywtew9JWyWNSBo5yYmSZZhZVd2eGFSbZdFuxYjYQXEqc87W0rbr2OzUL0fhdnUM6qTh+P+rybcIXx9v89Pl8bR8DFjVst5K4GjJbZhZA8qGwB5gc7q+GdjdsvzT6V2CdcA748MGM+tP0w4HJN0PXAGcK2kMuB34EvCgpC3AK8ANafVHgKuBUeBd4DM11GwD5kwter8MLWaLMsOCaUMgIm6c4q71bdYN4OaOt25mPedPDFrXzPSo3ckkXes67go6N5P95r8dMMucOwGrrKkjtLuCck7vt9G297sTMMucOwGrpFdH5MnzCe4MynMI2IxUfbF1+xN7ndYzqJ8U7AYPB8wy5xCwjs3WlttdwJk5BMwy5xCwWW3D8rU+0lfkEDDLnEPALHN+i9Cm5QnBweZOwCxz7gRs4LgDmBl3Ajatbs7A13ESktk6XOkXDgGzzHk4YI0bP3JX6S589O8edwJmmXMnYNOq66hbtiNwF9Bd7gTMMucQsGnV/fn8To/sfiegHg4B61jdQTDVC9wv/no5BMwy54lBm5EqJ74sYybb8ScFy5m2E5B0r6Tjkg61LPtrSc9LOijpnyUtbrlvu6RRSS9I2lBX4WbWHZ10At8E/hb4VsuyvcD2iDgl6Q5gO3CrpIuATcCvA8uBRyV9MCL+r7tlW690owM40zcFl31+dwHldXIuwu9LumDSsn9vufk4cH26vhF4ICJOAC9LGgUuBf6zK9Vaz3VjOOBJvv7SjYnB3wf+NV1fAbzact9YWvYekrZKGpE0cpITXSjDzMqoNDEo6TbgFHDf+KI2q0W7x0bEDmAHwNla2nYd61+9PiWY2//uKR0CkjYD1wDr0ynJoTjyr2pZbSVwtHx5Zla3UsMBSVcBtwLXRsS7LXftATZJWiBpNbAG+EH1Ms1OcxfQXdN2ApLuB64AzpU0BtxO8W7AAmCvJIDHI+IPI+JZSQ8Cz1EME272OwPWLX7x16OTdwdubLP4njOs/0Xgi1WKMrPm+BODVlmdnyL00b9+/tsBs8y5E7Cu6fSo3Y2vF7PucSdgljl3AtY4dwD9xZ2AWeYcAmaZ0+lP/PawCOkN4KfAm72uBTgX19HKdUw0m+v4lYj4wOSFfRECAJJGImLIdbgO19FsHR4OmGXOIWCWuX4KgR29LiBxHRO5jokGro6+mRMws97op07AzHrAIWCWub4IAUlXpfMUjEra1tA2V0l6TNJhSc9KuiUtXyppr6Qj6XJJQ/XMkfS0pIfT7dWS9qc6vi1pfgM1LJa0K51T4rCky3qxPyR9Lv1MDkm6X9LCpvbHFOfZaLsPVPha+r09KOmSmuuo53wfEdHTf8Ac4IfAhcB84L+AixrY7jLgknT9l4EXgYuAvwK2peXbgDsa2g+fB/4ReDjdfhDYlK5/HfijBmrYCfxBuj4fWNz0/qD4duqXgfe17Iffa2p/AB8DLgEOtSxruw+Aqym+aVvAOmB/zXX8NjA3Xb+jpY6L0utmAbA6vZ7mdLytun+xOvjPXgYMt9zeTnFik6br2A18AngBWJaWLQNeaGDbK4F9wJXAw+mX6s2WH/iEfVRTDWenF58mLW90f3D6a+uXUvyB28PAhib3B3DBpBdf230A/D1wY7v16qhj0n2/C9yXrk94zQDDwGWdbqcfhgMdn6ugLunkKhcD+4HzI+IYQLo8r4ES7gS+APw83T4HeDsiTqXbTeyTC4E3gG+kYcndks6i4f0REa8BXwZeAY4B7wBP0vz+aDXVPujl726p83200w8h0PG5CmrZuPR+4DvAZyPix01tt2X71wDHI+LJ1sVtVq17n8ylaD/vioiLKf6Wo5H5mVZpvL2Roq1dDpwFfLLNqv3w3nZPfnernO+jnX4IgZ6dq0DSPIoAuC8iHkqLX5e0LN2/DDhecxmXA9dK+m/gAYohwZ3AYknj3/fQxD4ZA8YiYn+6vYsiFJreHx8HXo6INyLiJPAQ8FGa3x+tptoHjf/utpzv46ZIvX/VOvohBJ4A1qTZ3/kUJzTdU/dGVXxX+j3A4Yj4Sstde4DN6fpmirmC2kTE9ohYGREXUPzfvxsRNwGPcfocj03U8SPgVUkfSovWU3x1fKP7g2IYsE7SovQzGq+j0f0xyVT7YA/w6fQuwTrgnfFhQx1qO99HnZM8M5gAuZpidv6HwG0NbfO3KFqmg8CB9O9qivH4PuBIulza4H64gtPvDlyYfpCjwD8BCxrY/lpgJO2TfwGW9GJ/AH8BPA8cAv6BYta7kf0B3E8xF3GS4gi7Zap9QNGG/136vX0GGKq5jlGKsf/47+vXW9a/LdXxAvDJmWzLHxs2y1w/DAfMrIccAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhl7v8B+wWmT91sIhwAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "threshold = 60\n", - "mask = avg_image > threshold\n", - "plt.imshow(mask)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we could use scipy.ndimage to detect the blobs from this binary mask\n", - "\n", - "For the detailed tutorial, please refer to https://scipy-lectures.org/advanced/image_processing/index.html#segmentation" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [], - "source": [ - "from scipy import ndimage" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The method `label` marks each blob with different number, `label_im` is the image with different blobs marked with different number and `nb_labels` is the number of blobs that are detected" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "10\n" - ] - }, - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQzUlEQVR4nO3df4wc9X3G8fdTn30Ep8g2BGR8qLaDm9aKUqBXMFAhhJPwowhTCSRTRK6pK6sVbQmpFNvlD1SpUeI2CjRSRWoBiRM5EOrQ2kK0FAxRVDV2OcA1BkN8mBQOOxhUICmRjJ18+sd+D+8de769nZ3Zvf0+L+m0O7OzOx+Pd5/5fGd/jCICM8vXr3S6ADPrLIeAWeYcAmaZcwiYZc4hYJY5h4BZ5koLAUlXSHpR0oik9WWtx8yKURmfE5A0C/gR8ClgFHgSuCEinm/7ysyskL6SHvd8YCQiDgBIuh9YBTQMgTnqj5OYW1IpZs1R/5xOl1CKOPIeAD/jrTcj4iMTby8rBBYBr9ZNjwIX1C8gaS2wFuAkTuYCrSypFLPm9A0s7nQJpTh24McAPBZb/6fR7WUdE1CDeePGHRGxKSIGI2JwNv0llWHWvLEXSy9p5t9UVicwCpxVNz0AHCxpXWaWtBJkZXUCTwLLJC2RNAdYDWwvaV1mVkApnUBEHJP0Z8AjwCzg3oh4rox1mfWysT1739LFTS3XirKGA0TEw8DDZT2+mbVHaSFgNhNN3KNOtQdu9nGKKvOgpT82bJY5dwJmJ9CLbxtO5E7ALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzLUcApLOkvSEpH2SnpN0S5q/QNKjkvany/ntK9fM2q1IJ3AM+MuI+E1gBXCzpOXAemBHRCwDdqRpM+tSLYdARByKiKfT9Z8B+4BFwCpgc1psM3Bt0SLNrDxtOSYgaTFwLrALOCMiDkEtKIDTJ7nPWknDkoaPcqQdZZhZCwqHgKQPA98DPhcRP232fhGxKSIGI2JwNv1FyzCzFhUKAUmzqQXAloh4MM1+XdLCdPtC4HCxEs2sTEXeHRBwD7AvIr5ad9N2YChdHwK2tV6emZWtyFmJLwZuAp6VtDvN+yvgy8ADktYArwDXFyvRutGxy34bgL7Hn+pwJVZUyyEQEf8BaJKbV7b6uGZWrSKdgGXMHUDv8MeGzTLnEDDLnEPALHMOAbPM+cCgjfP2TRe+f33et3/YwUqsKu4EzDLnTsDG8d4/P+4EbEqvrbuI19Zd1OkyrCQOAbPMOQTMMucQMMucDwwaACN3rPjAvLNv3QnAoo3/WXU5ViF3AjapkTtWNAwH6y0OAbPMeThgU6rvBsaGCNY73AmYZc6dQA858J1z3r++9A92n2DJ1rgL6E3uBMwy504gcz76bw6BHlLGEMB6n4cDZplzCFjTunrosGPg+J9Ni0PALHMOAbPMFT4wKGkWMAy8FhFXS1oC3A8sAJ4GboqI94qux9qnq9v6Vq0c7XQFM1Y7OoFbgH110xuBOyJiGfAWsKYN6zCzkhQ9NfkA8HvA3WlawGXA1rTIZuDaIusws3IV7QTuBL4A/DJNnwq8HRHH0vQosKjRHSWtlTQsafgoRwqWYWatajkEJF0NHI6I+jNTNjpLcTS6f0RsiojBiBicTX+rZVgLzr51p78HYO8rcmDwYuAaSVcBJwGnUOsM5knqS93AAHCweJlmVpaWO4GI2BARAxGxGFgNPB4RNwJPANelxYaAbYWrtK7g7qE3lfHdgXXA/ZL+BngGuKeEdVgB032L0C/+3taWEIiI7wPfT9cPAOe343HNrHz+FmGG6vfszXQFY8u4I+hN/tiwWebcCWRubO8+nY6g/n428zkEMteT3yOwafFwwCxz7gQyVkUXcKLTm1l3cCdgljl3AtYSHyTsHQ6BjJ196862DAkmPoZDYWbxcMAscw4Ba7upTmnuU553F4eAWeYcAlaaqfb27gi6g0PALHNdEQJzlwe/s/sX4/6sGt3wU2PuBjrLbxEaML0vEjW6Xyv3reevK3dOV3QCZtY57gRsnBPtiSfb07eznXdHUD13AmaZ67oQePKcWTx5zqxOl2ENeO/cmzwcsGlpFATTCYfJhg4OmM7puk7AzKrlTsAq5T1+93EnYJa5rugE3n1ePhho1iGFOgFJ8yRtlfSCpH2SLpS0QNKjkvany/ntKtbM2q/ocODvgX+LiN8AfgvYB6wHdkTEMmBHmjazLtVyCEg6BbiEdMLRiHgvIt4GVgGb02KbgWuLFmlm5SnSCSwF3gC+IekZSXdLmgucERGHANLl6Y3uLGmtpGFJw0c5UqAMMyuiSAj0AecBd0XEucC7TKP1j4hNETEYEYOz6S9QhpkVUSQERoHRiNiVprdSC4XXJS0ESJeHi5VoZmVqOQQi4ifAq5I+lmatBJ4HtgNDad4QsK1QhWZWqqKfE/hzYIukOcAB4LPUguUBSWuAV4DrC67DzEpUKAQiYjcw2OCmlUUe18yq448Nm2XOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuUAhIulXSc5L2SrpP0kmSlkjaJWm/pO+mU5SZWZdqOQQkLQL+AhiMiI8Ds4DVwEbgjohYBrwFrGlHoWZWjqLDgT7gQ5L6gJOBQ8Bl1E5TDrAZuLbgOsysREVOTf4a8BVqZx4+BLwDPAW8HRHH0mKjwKJG95e0VtKwpOGjHGm1DDMrqMhwYD6wClgCnAnMBa5ssGg0un9EbIqIwYgYnE1/q2WYWUFFhgOfBF6OiDci4ijwIHARMC8NDwAGgIMFazSzEhUJgVeAFZJOliRgJfA88ARwXVpmCNhWrEQzK1ORYwK7qB0AfBp4Nj3WJmAd8HlJI8CpwD1tqNPMStI39SKTi4jbgdsnzD4AnF/kcc2sOv7EoFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZa7QbwyaTdeGl/a8f/1LH/1EByuxMQ4B65j6QJjIAVHMIwd3f2DerIWNl/VwwCxz7gSsK52oSwB3CpNp1AFMxZ2AWebcCVglptqzt/PxcuwSWukAxkzZCUi6V9JhSXvr5i2Q9Kik/elyfpovSV+TNCJpj6TzWq7MzCrRzHDgm8AVE+atB3ZExDJgR5qG2qnJl6W/tcBd7SnTZqoNL+1pexdg4xXpAqCJEIiIHwD/O2H2KmBzur4ZuLZu/reiZie105RP8saE9bpOvfhzC57Lzzyn0P1bPTB4RkQcAkiXp6f5i4BX65YbTfM+QNJaScOSho9ypMUyzKyodh8YVIN50WjBiNhE7VTmnKIFDZexmalb9sKN6ujVg4Zj3UCVbxG+Ptbmp8vDaf4ocFbdcgPAwRbXYWYVaDUEtgND6foQsK1u/mfSuwQrgHfGhg1m1p2mHA5Iug+4FDhN0ihwO/Bl4AFJa4BXgOvT4g8DVwEjwM+Bz5ZQs/WYE7Xo3TK0mClaGRZMGQIRccMkN61ssGwANze9djPrOH9i0NpmunvtZg7S1S/jrqB59W8bTtUV+LsDZplzJ2CFVbWHdlfQmuNdwUjD290JmGXOnYAV0qk98sTjCe4MWucQsGkp+mJr9yf2mq2nVz8p2A4eDphlziFgTZupLbe7gBNzCJhlziFgM9qXPvoJ7+kLcgiYZc4hYJY5v0VoU/IBwd7mTsAsc+4ErOe4A5gedwI2pXYegS/jJCQzdbjSLRwCZpnzcMAqN7bnLtJdeO/fPu4EzDLnTsCmVNZet9WOwF1Ae7kTMMucQ8CmVPbn85vds/udgHI4BKxpZQfBZC9wv/jL5RAwy5wPDNq0jHUDVe2Zp7Mef1KwNVN2ApLulXRY0t66eX8n6QVJeyT9s6R5dbdtkDQi6UVJl5dVuJm1h2pnDjvBAtIlwP8B34qIj6d5nwYej4hjkjYCRMQ6ScuB+4DzgTOBx4Bfj4hfnGgdp2hBXKAPnNXMulA7OoAyfinYXcDUHoutT0XE4MT5zZyL8AeSFk+Y9+91kzuB69L1VcD9EXEEeFnSCLVA+GGLdVuXacdwwAf5uks7Dgz+EfCv6foi4NW620bTvA+QtFbSsKThoxxpQxlm1opCBwYl3QYcA7aMzWqwWMPxRkRsAjZBbThQpA6rXqdPCeb2v31aDgFJQ8DVwMo4fmBhFDirbrEB4GDr5ZlZ2VoaDki6AlgHXBMRP6+7aTuwWlK/pCXAMuC/ipdpdpy7gPaashOQdB9wKXCapFHgdmAD0A88KglgZ0T8SUQ8J+kB4Hlqw4Sbp3pnwKxZfvGXo5l3B25oMPueEyz/ReCLRYoys+r4E4NWWJmfIvTev3z+7oBZ5twJWNs0u9dux8+LWfu4EzDLnDsBq5w7gO7iTsAscw4Bs8xN+VXiSoqQ3gDeBd7sdC3AabiOeq5jvJlcx69FxEcmzuyKEACQNNzou86uw3W4jnLr8HDALHMOAbPMdVMIbOp0AYnrGM91jNdzdXTNMQEz64xu6gTMrAMcAmaZ64oQkHRFOk/BiKT1Fa3zLElPSNon6TlJt6T5CyQ9Kml/upxfUT2zJD0j6aE0vUTSrlTHdyXNqaCGeZK2pnNK7JN0YSe2h6Rb0//JXkn3STqpqu0xyXk2Gm4D1XwtPW/3SDqv5DrKOd9HRHT0D5gFvAQsBeYA/w0sr2C9C4Hz0vVfBX4ELAf+Flif5q8HNla0HT4PfAd4KE0/AKxO178O/GkFNWwG/jhdnwPMq3p7UPt16peBD9Vthz+sansAlwDnAXvr5jXcBsBV1H5pW8AKYFfJdXwa6EvXN9bVsTy9bvqBJen1NKvpdZX9xGriH3sh8Ejd9AZgQwfq2AZ8CngRWJjmLQRerGDdA8AO4DLgofSkerPuP3zcNiqphlPSi08T5le6PTj+s/ULqH3B7SHg8iq3B7B4wouv4TYA/hG4odFyZdQx4bbfB7ak6+NeM8AjwIXNrqcbhgNNn6ugLOnkKucCu4AzIuIQQLo8vYIS7gS+APwyTZ8KvB0Rx9J0FdtkKfAG8I00LLlb0lwq3h4R8RrwFeAV4BDwDvAU1W+PepNtg04+d1s630cj3RACTZ+roJSVSx8Gvgd8LiJ+WtV669Z/NXA4Ip6qn91g0bK3SR+19vOuiDiX2nc5Kjk+Uy+Nt1dRa2vPBOYCVzZYtBve2+7Ic7fI+T4a6YYQ6Ni5CiTNphYAWyLiwTT7dUkL0+0LgcMll3ExcI2kHwP3UxsS3AnMkzT2ew9VbJNRYDQidqXprdRCoert8Ung5Yh4IyKOAg8CF1H99qg32Tao/Llbd76PGyP1/kXr6IYQeBJYlo7+zgFWUzt/QalU+630e4B9EfHVupu2A0Pp+hC1YwWliYgNETEQEYup/dsfj4gbgSc4fo7HKur4CfCqpI+lWSup/XR8pduD2jBghaST0//RWB2Vbo8JJtsG24HPpHcJVgDvjA0bylDa+T7KPMgzjQMgV1E7Ov8ScFtF6/xdai3THmB3+ruK2nh8B7A/XS6ocDtcyvF3B5am/8gR4J+A/grWfw4wnLbJvwDzO7E9gL8GXgD2At+mdtS7ku1B7azah4Cj1PawaybbBtTa8H9Iz9tngcGS6xihNvYfe75+vW7521IdLwJXTmdd/tiwWea6YThgZh3kEDDLnEPALHMOAbPMOQTMMucQMMucQ8Asc/8PbqC4fI92GrEAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "label_im, nb_labels = ndimage.label(mask)\n", - "print(nb_labels)\n", - "plt.imshow(label_im) " - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], dtype=int32)" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "np.unique(label_im)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "0 marks the background and 1 to 10 mark the detected blobs. Some of these are too small to be cells. We could set a cutoff on size to filter out the small blobs. Here the cutoff is another parameter that could be tweaked." - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQEElEQVR4nO3de4xU93nG8e9TLktMigA7trip4ArS0kq1rZWD4yqyTFJfahkq2RKWlWxTLNTKbZ2kUgz1H1alRgptlLhRK6cIOyGVC6HELchySwkhiirV1Gub2mBswHYLa4jBqu1EiUQgefvH/DYM61l2ds5lZvf3fCQ0c86cmfNyduY57+/M5SgiMLN8/VK3CzCz7nIImGXOIWCWOYeAWeYcAmaZcwiYZa6yEJB0q6RXJR2TtL6q9ZhZMaricwKSpgBHgE8AQ8CzwD0R8XLpKzOzQqZW9LjXA8ci4nUASduAVUDLEJiuvpjBzIpKMWuP+vq6XUIl4uxZAH7EO29HxIdG3l5VCCwATjRNDwEfaV5A0jpgHcAMLuMjWllRKWbtmbL4V7tdQiV+duQ1AL4TO/631e1VHRNQi3kXjTsiYlNE9EdE/zQmZwLbxDL8YplM2vk/VdUJDAGLmqYXAicrWpeZJZ0EWVWdwLPAUklLJE0H1gC7KlqXmRVQSScQEecl/TGwG5gCPB4Rh6pYl9lkNrxnn7Ls0scrigxlqhoOEBFPA09X9fhmVo7KQsBsIhq5Rx1rD9zu4xRV5UFLf2zYLHPuBMwuYTK+bTiSOwGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy13EISFokaZ+kw5IOSXogzZ8raY+ko+lyTnnlmlnZinQC54E/i4hfB1YA90taDqwH9kbEUmBvmjazHtVxCETEqYh4Pl3/EXAYWACsArakxbYAq4sWaWbVKeWYgKTFwLXAfuCqiDgFjaAArhzlPuskDUoaPMfZMsowsw4UDgFJHwS+DXwmIn7Y7v0iYlNE9EdE/zT6ipZhZh0qFAKSptEIgCci4sk0+y1J89Lt84DTxUo0syoVeXdAwGPA4Yj4ctNNu4CBdH0A2Nl5eWZWtSJnJb4R+CTwkqQDad6fA18EtktaCxwH7i5WoplVqeMQiIj/ADTKzSs7fVwzq5c/MWiWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGWuyLcIbRI5srn/ffOW3TfYhUqsbu4EbFRHNve3DAebXBwCZpnzcMDG1NwNeIgw+bgTMMucQ8Da5i5gcnIImGXOxwQy56P/5k7ALHMOAbPMOQSsbR46TE4OAbPMOQTMMlf43QFJU4BB4M2IuEPSEmAbMBd4HvhkRPy06HqsPG7rrVkZncADwOGm6Y3AVyJiKfAOsLaEdZhZRYqemnwh8LvA5jQt4GZgR1pkC7C6yDrMrFpFO4FHgM8DP0/TlwPvRsT5ND0ELGh1R0nrJA1KGjzH2YJlmFmnOg4BSXcApyPiuebZLRaNVvePiE0R0R8R/dPo67QM68Cy+wb9PQD7hSIHBm8E7pR0OzADmEWjM5gtaWrqBhYCJ4uXaWZV6bgTiIgNEbEwIhYDa4DvRsS9wD7grrTYALCzcJXWE9w9TE5VfIHoQWCbpL8EXgAeq2AdVsB43yL0i39yKyUEIuJ7wPfS9deB68t4XDOrnr9KnKHmPXs7XcHwMu4IJid/bNgsc+4EMje8dx9PR9B8P5v4HAKZ8/cIzMMBs8y5E8hYHV2AT2/W+9wJmGXOnYB1xAcJJw+HQMaW3TdYypBg5GM4FCYWDwfMMucQsNKNdUpzn/K8tzgEzDLnELDKjLW3d0fQGxwCZplzCGSuF35qzN1Ad/ktQgPG90WiVvfr5L7N/HXl7nEnYJY5dwJ2kUvtiUfb05fZzrsjqJ87AbPMOQSsbd47T04eDti4tAqC8YTDaEMHB0z3uBMwy5w7AauV9/i9x52AWeYcAmaZKxQCkmZL2iHpFUmHJd0gaa6kPZKOpss5ZRVrZuUr2gn8DfBvEfFrwG8Bh4H1wN6IWArsTdNm1qM6DgFJs4CPkU44GhE/jYh3gVXAlrTYFmB10SLNrDpFOoGrgTPA1yW9IGmzpJnAVRFxCiBdXtnqzpLWSRqUNHiOswXKMLMiioTAVOA64NGIuBb4MeNo/SNiU0T0R0T/NPoKlGFmRRQJgSFgKCL2p+kdNELhLUnzANLl6WIlmlmVOg6BiPgBcELSh9OslcDLwC5gIM0bAHYWqtDMKlX0E4N/AjwhaTrwOvBpGsGyXdJa4Dhwd8F1mFmFCoVARBwAWn0jZGWRxzWz+vgTg2aZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZcwiYZc4hYJY5h4BZ5hwCZplzCJhlziFgljmHgFnmHAJmmXMImGXOIWCWOYeAWeYcAmaZKxQCkj4r6ZCkg5K2SpohaYmk/ZKOSvpWOkWZmfWojkNA0gLgT4H+iPhNYAqwBtgIfCUilgLvAGvLKNTMqlF0ODAV+ICkqcBlwCngZhqnKQfYAqwuuA4zq1CRU5O/CXyJxpmHTwHvAc8B70bE+bTYELCg1f0lrZM0KGnwHGc7LcPMCioyHJgDrAKWAPOBmcBtLRaNVvePiE0R0R8R/dPo67QMMyuoyHDg48AbEXEmIs4BTwIfBWan4QHAQuBkwRrNrEJFQuA4sELSZZIErAReBvYBd6VlBoCdxUo0syoVOSawn8YBwOeBl9JjbQIeBD4n6RhwOfBYCXWaWUWmjr3I6CLiYeDhEbNfB64v8rhmVh9/YtAscw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMucQMMucQ8Ascw4Bs8w5BMwy5xAwy5xDwCxzDgGzzDkEzDJX6DcGzcZr98kDv7h+y/xruliJDXMIWNc0B8JIDoj6eDhgljl3AtaTLtUlgDuFMrkTMMucOwGrxVh79jIfz13C+IzZCUh6XNJpSQeb5s2VtEfS0XQ5J82XpK9KOibpRUnXVVm8mRXXznDgG8CtI+atB/ZGxFJgb5qGxqnJl6Z/64BHyynTJqrdJw+U3gVYucYMgYj4PvB/I2avArak61uA1U3zvxkNz9A4Tfm8soq1iaVbL34Hz/h0emDwqog4BZAur0zzFwAnmpYbSvPeR9I6SYOSBs9xtsMyzKyosg8MqsW8aLVgRGyicSpzZmluy2VsYuqVvXCrOnzQ8P067QTeGm7z0+XpNH8IWNS03ELgZOflmVnVOg2BXcBAuj4A7Gya/6n0LsEK4L3hYYOZ9aYxhwOStgI3AVdIGgIeBr4IbJe0FjgO3J0Wfxq4HTgG/AT4dAU12yRzqRa9V4YWk9mYIRAR94xy08oWywZwf9GizKw+/sSglWa8e+12DtI1L+OuoBr+7oBZ5twJWGF17aHdFVTDnYBZ5twJWCHd2iOPPJ7gzqBzDgEbl6IvtrI/sdduPf6k4Og8HDDLnEPA2jZRW253AZfmEDDLnEPAJrRb5l/jPX1BDgGzzDkEzDLntwhtTD4gOLm5EzDLnDsBm3TcAYyPOwEbU5lH4Ks4CclEHa70CoeAWeY8HLDaDe+5i3QX3vuXx52AWebcCdiYqtrrdtoRuAsolzsBs8w5BGxMVX8+v909u98JqIZDwNpWdRCM9gL3i79aDgGzzPnAoI3LcDdQ1555POvxJwU7M2YnIOlxSaclHWya99eSXpH0oqR/ljS76bYNko5JelXSLVUVbmblaKcT+Abwt8A3m+btATZExHlJG4ENwIOSlgNrgN8A5gPfkbQsIn5WbtnWLWV0AJf6peBOH99dQOfaORfh9yUtHjHv35smnwHuStdXAdsi4izwhqRjwPXAf5ZSrXVdGcMBH+TrLWUcGPwD4F/T9QXAiabbhtK895G0TtKgpMFznC2hDDPrRKEDg5IeAs4DTwzParFYtLpvRGwCNgHM0tyWy1jv6vYpwdz+l6fjEJA0ANwBrEynJIfGnn9R02ILgZOdl2dmVetoOCDpVuBB4M6I+EnTTbuANZL6JC0BlgL/VbxMswvcBZRrzE5A0lbgJuAKSUPAwzTeDegD9kgCeCYi/jAiDknaDrxMY5hwv98ZsLL4xV+Ndt4duKfF7McusfwXgC8UKcrM6uNPDFphVX6K0Hv/6vm7A2aZcydgpWl3r13Gz4tZedwJmGXOnYDVzh1Ab3EnYJY5h4BZ5nThE79dLEI6A/wYeLvbtQBX4DqauY6LTeQ6fiUiPjRyZk+EAICkwYjodx2uw3XUW4eHA2aZcwiYZa6XQmBTtwtIXMfFXMfFJl0dPXNMwMy6o5c6ATPrAoeAWeZ6IgQk3ZrOU3BM0vqa1rlI0j5JhyUdkvRAmj9X0h5JR9PlnJrqmSLpBUlPpeklkvanOr4laXoNNcyWtCOdU+KwpBu6sT0kfTb9TQ5K2ippRl3bY5TzbLTcBmr4anrevijpuorrqOZ8HxHR1X/AFOA14GpgOvDfwPIa1jsPuC5d/2XgCLAc+CtgfZq/HthY03b4HPCPwFNpejuwJl3/GvBHNdSwBbgvXZ8OzK57e9D4deo3gA80bYffr2t7AB8DrgMONs1ruQ2A22n80raAFcD+iuv4HWBqur6xqY7l6XXTByxJr6cpba+r6idWG//ZG4DdTdMbaJzYpO46dgKfAF4F5qV584BXa1j3QmAvcDPwVHpSvd30B79oG1VUw6z04tOI+bVuDy78bP1cGl9wewq4pc7tASwe8eJruQ2AvwfuabVcFXWMuO33gCfS9YteM8Bu4IZ219MLw4G2z1VQlXRylWuB/cBVEXEKIF1eWUMJjwCfB36epi8H3o2I82m6jm1yNXAG+HoalmyWNJOat0dEvAl8CTgOnALeA56j/u3RbLRt0M3nbkfn+2ilF0Kg7XMVVLJy6YPAt4HPRMQP61pv0/rvAE5HxHPNs1ssWvU2mUqj/Xw0Iq6l8V2OWo7PNEvj7VU02tr5wEzgthaL9sJ721157hY530crvRACXTtXgaRpNALgiYh4Ms1+S9K8dPs84HTFZdwI3Cnpf4BtNIYEjwCzJQ3/3kMd22QIGIqI/Wl6B41QqHt7fBx4IyLORMQ54Engo9S/PZqNtg1qf+42ne/j3ki9f9E6eiEEngWWpqO/02mc0HRX1StV47fSHwMOR8SXm27aBQyk6wM0jhVUJiI2RMTCiFhM4//+3Yi4F9jHhXM81lHHD4ATkj6cZq2k8dPxtW4PGsOAFZIuS3+j4Tpq3R4jjLYNdgGfSu8SrADeGx42VKGy831UeZBnHAdAbqdxdP414KGa1vnbNFqmF4ED6d/tNMbje4Gj6XJujdvhJi68O3B1+kMeA/4J6Kth/dcAg2mb/AswpxvbA/gL4BXgIPAPNI5617I9gK00jkWco7GHXTvaNqDRhv9det6+BPRXXMcxGmP/4efr15qWfyjV8Spw23jW5Y8Nm2WuF4YDZtZFDgGzzDkEzDLnEDDLnEPALHMOAbPMOQTMMvf/gBt0F9GgVmYAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "size_cutoff = 50\n", - "sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", - "\n", - "small_size_filter = sizes < size_cutoff\n", - "pixel_to_remove = small_size_filter[label_im]\n", - "\n", - "label_im[pixel_to_remove] = 0\n", - "plt.imshow(label_im)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now there are only two blobs left." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next let's separate out each mask." - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "metadata": {}, - "outputs": [], - "source": [ - "rois = []\n", - "for i in np.unique(label_im)[1:]: # 0 is the background\n", - " rois.append(label_im == i)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "There are two special notes about the analyses we performed above.\n", - "1. The segmentation results are dependent on the parameters `threshold` and `size_cutoff`. Rather than fixing the value of the threshold, we might want to try different values and see what works well.\n", - "2. For the segemtation analyses of each `AverageFrame`, the result is a number of ROIs instead of one. We need to perform the analyses on the level of each `AverageFrame`, but save the result on the granularity of each `Roi`.\n", - "\n", - "Next we would like to introduce two DataJoint table tiers to help handling these two needs." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Parameter `Lookup` table" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We would like to perform the segmentation for a **combination** of `AverageFrame`s and different set of paremeters of `threshold` and `size_cutoff` values. To do this while still taking advantage of the `make` and `populate` logic, you would want to define a table to house parameters for segmentation in a `Lookup` table!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's define `Param` table to hold different parameter configuration for our spike detection algorithm. We are going to define this table as a `Lookup` table, rather than a `Manual` table. By now, you know that `Lookup` must be yet another **table tier** in DataJoint. `Lookup` tables are depicted by gray boxes in the Diagram.\n", - "\n", - "This tier indicates that the table will contain information:\n", - "* that will be referenced by other tables\n", - "* that doesn't change much - usually contains a few pre-known entries" - ] - }, - { - "cell_type": "code", - "execution_count": 34, - "metadata": {}, - "outputs": [], - "source": [ - "@schema\n", - "class SegmentationParam(dj.Lookup):\n", - " definition = \"\"\"\n", - " seg_param_id : int # unique id for cell segmentation parameter set\n", - " ---\n", - " threshold : float\n", - " size_cutoff : float\n", - " \"\"\"" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session->Scan\n", - "\n", - "\n", - "\n", - "\n", - "SegmentationParam\n", - "\n", - "\n", - "SegmentationParam\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "AverageFrame\n", - "\n", - "\n", - "AverageFrame\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Scan->AverageFrame\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 25, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dj.Diagram(schema)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So far the `SegmentationParam` is an extra table that did not relate with any of the existing, but the `Segmentation` will depend on this table." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Master `Computed` table `Segmentation` and `Part` table for `Roi`" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As mentioned above, we performed the segmentation processing on each `AverageFrame`, but the product is the masks of each ROI. In this case, we could create a `Computed` table `Segmentation` containing the `make` method to drive the processing, while using a `Part` table to save results into `Roi`.\n", - "\n", - "`Computed` table and `Part` table are another two table tiers like `Manual`, `Lookup`, and `Imported` we introduced previously. \n", - "\n", - "`Computed` table is very similar to the `Imported` table, which also supports the definition of `make` function and `populate`. The only difference is that the computation in an `Imported` table is dependent on external data, while the computation in a `Computed` table only depends on data inside the database.\n", - "\n", - "Contents in a `Part` table is dependent on its **master** table and the master table could be any type of table. This current example is a very typical usage of Part table. The master `Computed` serves as the driver for computation and the major results with a smaller granularity are saved in the `Part` table." - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": {}, - "outputs": [], - "source": [ - "@schema\n", - "class Segmentation(dj.Computed):\n", - " definition = \"\"\"\n", - " -> AverageFrame\n", - " -> SegmentationParam\n", - " ---\n", - " segmented_masks : longblob # overview of segmented masks\n", - " \"\"\"\n", - " class Roi(dj.Part):\n", - " definition = \"\"\"\n", - " -> master\n", - " roi_idx : int # index of an roi\n", - " ---\n", - " mask : longblob # mask of this roi\n", - " \"\"\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Notice that in the definition of `Roi`, apart from inheriting the primary from its master table `Segmentation`, the Roi has another primary key attribute `roi_idx`. The relationship between `Segmentation` and `Roi` is **one-to-many**." - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session->Scan\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation\n", - "\n", - "\n", - "Segmentation\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation.Roi\n", - "\n", - "\n", - "Segmentation.Roi\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation->Segmentation.Roi\n", - "\n", - "\n", - "\n", - "\n", - "SegmentationParam\n", - "\n", - "\n", - "SegmentationParam\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "SegmentationParam->Segmentation\n", - "\n", - "\n", - "\n", - "\n", - "AverageFrame\n", - "\n", - "\n", - "AverageFrame\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "AverageFrame->Segmentation\n", - "\n", - "\n", - "\n", - "\n", - "Scan->AverageFrame\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 27, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dj.Diagram(schema)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The `Computed` table is labeled as a pink oval and the `Part` table is bare text. We see that `Segmentation` is a `Computed` table that depends on **both AverageFrame and SegmentationParam**. Finally, let's go ahead and implement the `make` method for the `Segmenation` table. " - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": {}, - "outputs": [], - "source": [ - "@schema\n", - "class Segmentation(dj.Computed):\n", - " definition = \"\"\"\n", - " -> AverageFrame\n", - " -> SegmentationParam\n", - " ---\n", - " segmented_masks : longblob # overview of segmented masks\n", - " \"\"\"\n", - " class Roi(dj.Part):\n", - " definition = \"\"\"\n", - " -> master\n", - " roi_idx : int # index of an roi\n", - " ---\n", - " mask : longblob # mask of this roi\n", - " \"\"\"\n", - " \n", - " def make(self, key): # key is one of the primary keys of the join product of AverageFrame and ParameterSet\n", - " \n", - " print('Populating for: ', key)\n", - " \n", - " # fetch average image from the previous table AverageFrame\n", - " avg_image = (AverageFrame & key).fetch1('average_frame')\n", - " \n", - " # fetch the parameters threshold and size_cutoff\n", - " threshold, size_cutoff = (SegmentationParam & key).fetch1(\n", - " 'threshold', 'size_cutoff')\n", - " \n", - " # perform the thresholding and blob detection\n", - " mask = avg_image > threshold\n", - " label_im, nb_labels = ndimage.label(mask)\n", - " sizes = np.array([np.sum(label_im==i) for i in np.unique(label_im)])\n", - "\n", - " small_size_filter = sizes < size_cutoff\n", - " pixel_to_remove = small_size_filter[label_im]\n", - "\n", - " label_im[pixel_to_remove] = 0\n", - " \n", - " rois = []\n", - " for i in np.unique(label_im)[1:]: # 0 is the background\n", - " rois.append(\n", - " dict(**key, # inherit primary key from master table\n", - " roi_idx=i, \n", - " mask=label_im==i))\n", - " \n", - " # insert into the master table first\n", - " self.insert1(\n", - " dict(**key, segmented_masks=label_im)\n", - " )\n", - " print('Detected {} ROIs!\\n'.format(len(rois)))\n", - " # then insert into the part table\n", - " self.Roi.insert(rois)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The implementation of the segmentation is pretty much what we had above, except that we now fetch the value of `threshold` and `size_cutoff` from the `SegmentationParam` table.\n", - "\n", - "**Important note: always insert into the master table first and then insert the corresponding entries in the part table.** If a master table entry does not exist, its corresponding entries would not be valid." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Looking at the `Segmentation` table, we see that it indeed inherits the primary key attributes from **both AverageFrame (`mouse_id`, `session_date`, `scan_idx`) and SegmentationParam (`seg_param_id`)**." - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 29, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And for the part table `Segmenation.Roi`, there was an additional primary key attribute `roi_idx`:`" - ] - }, - { - "cell_type": "code", - "execution_count": 30, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

roi_idx

\n", - " index of an roi\n", - "
\n", - "

mask

\n", - " mask of this roi\n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id *roi_idx mask \n", - "+----------+ +------------+ +----------+ +------------+ +---------+ +--------+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation.Roi()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Populating `Segmentation` table" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We are now ready to populate! When we call `populate` on `Segmentation`, DataJoint will automatically call `make` on **every valid combination of the parent tables - AverageFrame and SegmentationParam**." - ] - }, - { - "cell_type": "code", - "execution_count": 31, - "metadata": {}, - "outputs": [], - "source": [ - "# ENTER YOUR CODE! - populate the Segmentation table\n", - "Segmentation.populate()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Hm... `populate` doesn't seem to be doing anything... What could be the cause?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Looking at `SegmentationParam` reveals the issue:" - ] - }, - { - "cell_type": "code", - "execution_count": 32, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
\n", - " \n", - "

Total: 0

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "\n", - " (Total: 0)" - ] - }, - "execution_count": 32, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "That's right! We have not added a parameter set yet. Let's go ahead and add one." - ] - }, - { - "cell_type": "code", - "execution_count": 33, - "metadata": {}, - "outputs": [], - "source": [ - "SegmentationParam.insert1((0, 50, 50))" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
050.050.0
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "0 50.0 50.0 \n", - " (Total: 1)" - ] - }, - "execution_count": 35, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we should really be ready to perform the computation..." - ] - }, - { - "cell_type": "code", - "execution_count": 36, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 0}\n", - "Detected 6 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 0}\n", - "Detected 6 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 0}\n", - "Detected 9 ROIs!\n", - "\n" - ] - } - ], - "source": [ - "# ENTER YOUR CODE! - populate the Segmenation table for real!\n", - "Segmentation.populate()" - ] - }, - { - "cell_type": "code", - "execution_count": 37, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "0 2017-05-15 1 0 =BLOB= \n", - "0 2017-05-15 2 0 =BLOB= \n", - "100 2017-05-25 1 0 =BLOB= \n", - " (Total: 3)" - ] - }, - "execution_count": 37, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "...and we now have spike detection running!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Trying out other parameter values" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's see how different thresholds affect the results." - ] - }, - { - "cell_type": "code", - "execution_count": 38, - "metadata": {}, - "outputs": [], - "source": [ - "SegmentationParam.insert1((1, 60, 50)) # add another threshold and size cutoff" - ] - }, - { - "cell_type": "code", - "execution_count": 39, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
050.050.0
160.050.0
\n", - " \n", - "

Total: 2

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "0 50.0 50.0 \n", - "1 60.0 50.0 \n", - " (Total: 2)" - ] - }, - "execution_count": 39, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam()" - ] - }, - { - "cell_type": "code", - "execution_count": 40, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", - "Detected 3 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", - "Detected 2 ROIs!\n", - "\n", - "Populating for: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n", - "Detected 3 ROIs!\n", - "\n" - ] - } - ], - "source": [ - "# ENTER YOUR CODE! - populate the \"missing\" entry in Segmentation table\n", - "Segmentation.populate()" - ] - }, - { - "cell_type": "code", - "execution_count": 43, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
02017-05-1510=BLOB=
02017-05-1520=BLOB=
1002017-05-2510=BLOB=
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", - " \n", - "

Total: 6

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "0 2017-05-15 1 0 =BLOB= \n", - "0 2017-05-15 2 0 =BLOB= \n", - "100 2017-05-25 1 0 =BLOB= \n", - "0 2017-05-15 1 1 =BLOB= \n", - "0 2017-05-15 2 1 =BLOB= \n", - "100 2017-05-25 1 1 =BLOB= \n", - " (Total: 6)" - ] - }, - "execution_count": 43, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can see that the results of segmentation under different parameter settings can live happily next to each other, without any confusion as to what is what." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Deleting entries from \"upstream\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now let's say that we decided that we don't like the first threshold of `50`. While there is really nothing wrong keeping those results around, you might decide that you'd rather delete all computations performed with that threshold to keep your tables clean." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "While you can restrict `Segmentation` table to the specific parameter id (i.e. `seg_param_id = 0`) and delete the entries:" - ] - }, - { - "cell_type": "code", - "execution_count": 42, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "About to delete:\n", - "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", - "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", - "Proceed? [yes, No]: No\n", - "Cancelled deletes.\n" - ] - } - ], - "source": [ - "# Select 'No' when it pops up\n", - "(Segmentation & 'seg_param_id = 0').delete()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We can simply delete the unwanted paramter from the `SegmentationParam` table, and let DataJoint cascade the deletion:" - ] - }, - { - "cell_type": "code", - "execution_count": 44, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

threshold

\n", - " \n", - "
\n", - "

size_cutoff

\n", - " \n", - "
050.050.0
\n", - " \n", - "

Total: 1

\n", - " " - ], - "text/plain": [ - "*seg_param_id threshold size_cutoff \n", - "+------------+ +-----------+ +------------+\n", - "0 50.0 50.0 \n", - " (Total: 1)" - ] - }, - "execution_count": 44, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "SegmentationParam & 'seg_param_id = 0'" - ] - }, - { - "cell_type": "code", - "execution_count": 45, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "About to delete:\n", - "`shan_tutorial_pipeline`.`__segmentation__roi`: 21 items\n", - "`shan_tutorial_pipeline`.`__segmentation`: 3 items\n", - "`shan_tutorial_pipeline`.`#segmentation_param`: 1 items\n", - "Proceed? [yes, No]: yes\n", - "Committed.\n" - ] - } - ], - "source": [ - "(SegmentationParam() & 'seg_param_id = 0').delete()" - ] - }, - { - "cell_type": "code", - "execution_count": 46, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - "
\n", - " \n", - " \n", - " \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
\n", - "

mouse_id

\n", - " Unique animal ID\n", - "
\n", - "

session_date

\n", - " date\n", - "
\n", - "

scan_idx

\n", - " scan index\n", - "
\n", - "

seg_param_id

\n", - " unique id for cell segmentation parameter set\n", - "
\n", - "

segmented_masks

\n", - " overview of segmented masks\n", - "
02017-05-1511=BLOB=
02017-05-1521=BLOB=
1002017-05-2511=BLOB=
\n", - " \n", - "

Total: 3

\n", - " " - ], - "text/plain": [ - "*mouse_id *session_date *scan_idx *seg_param_id segmented_\n", - "+----------+ +------------+ +----------+ +------------+ +--------+\n", - "0 2017-05-15 1 1 =BLOB= \n", - "0 2017-05-15 2 1 =BLOB= \n", - "100 2017-05-25 1 1 =BLOB= \n", - " (Total: 3)" - ] - }, - "execution_count": 46, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Segmentation()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Visualize ROIs" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we have all ROI masks saved in the table `Segmentation.Roi`, let's quickly look at an example by fetching the `mask` from the table." - ] - }, - { - "cell_type": "code", - "execution_count": 51, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 51, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD7CAYAAABqkiE2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAPGklEQVR4nO3df+xddX3H8edrLS2CI1AVUloyStK5sSUT0iDoYozVgcwISzSBmNk5lmaL2/yxRMr8g+wPE9mMOrNF14haF0QZstEQN4YVY5bMzi/KEKjYCht8pVLMBI0mrJ3v/XFP5VK/pd/ec8/9fuHzfCTNvedzz73n3c/3e1/nc8493/tJVSGpXb+w1AVIWlqGgNQ4Q0BqnCEgNc4QkBpnCEiNGywEklyS5IEk+5JsG2o7kvrJENcJJFkBfBt4HTAPfA24sqrun/rGJPWycqDXvQDYV1UPAiT5LHAZsGAIrMrqOpGTBypFEsCP+MH3q+olR7YPFQLrgEfGlueBl4+vkGQrsBXgRE7i5dk8UCmSAL5YN//3Qu1DnRPIAm3POO6oqu1VtamqNp3A6oHKkHQsQ4XAPHDW2PJ64NGBtiWph6FC4GvAxiQbkqwCrgB2DrQtST0Mck6gqg4l+WPgdmAF8Imqum+IbUnqZ6gTg1TVF4AvDPX6kqbDKwalxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxk0cAknOSnJnkj1J7kvyjq59TZI7kuztbk+bXrmSpq3PSOAQ8GdV9avAhcDbk5wLbAN2VdVGYFe3LGmZmjgEqmp/VX29u/8jYA+wDrgM2NGttgO4vG+RkoYzlXMCSc4GzgN2A2dU1X4YBQVw+lGeszXJXJK5gzw1jTIkTaB3CCR5IfB54J1V9cPFPq+qtlfVpqradAKr+5YhaUK9QiDJCYwC4IaquqVrfizJ2u7xtcCBfiVKGlKfTwcCXA/sqaoPjj20E9jS3d8C3Dp5eZKGtrLHc18J/C7wzSR3d21/DrwfuCnJVcDDwJv7lShpSBOHQFX9G5CjPLx50teVNFteMSg1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1bhqzEq9I8o0kt3XLG5LsTrI3yeeSrOpfpqShTGMk8A5gz9jydcCHqmoj8APgqilsQ9JA+k5Nvh74beDj3XKA1wA3d6vsAC7vsw1Jw+o7Evgw8B7gp93yi4AnqupQtzwPrFvoiUm2JplLMneQp3qWIWlSE4dAkjcAB6rqrvHmBVathZ5fVduralNVbTqB1ZOWIamniacmB14JvDHJpcCJwCmMRganJlnZjQbWA4/2L1PSUCYeCVTVNVW1vqrOBq4AvlRVbwHuBN7UrbYFuLV3lZIGM8R1AlcD706yj9E5gusH2IakKelzOPAzVfVl4Mvd/QeBC6bxupKG5xWDUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuN6hUCSU5PcnORbSfYkuSjJmiR3JNnb3Z42rWIlTV/fkcBfA/9SVb8C/AawB9gG7KqqjcCublnSMjVxCCQ5BXgV3YSjVfW/VfUEcBmwo1ttB3B53yIlDafPSOAc4HHgk0m+keTjSU4Gzqiq/QDd7ekLPTnJ1iRzSeYO8lSPMiT10ScEVgLnAx+tqvOAH3McQ/+q2l5Vm6pq0wms7lGGpD76hMA8MF9Vu7vlmxmFwmNJ1gJ0twf6lShpSBOHQFV9D3gkyUu7ps3A/cBOYEvXtgW4tVeFkga1sufz/wS4Ickq4EHgbYyC5aYkVwEPA2/uuQ1JA+oVAlV1N7BpgYc293ldSbPjFYNS4wwBqXGGgNQ4Q0BqnCEgNc4QkBpnCEiNMwSkxhkCUuMMAalxhoDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS43qFQJJ3Jbkvyb1JbkxyYpINSXYn2Zvkc90UZZKWqYlDIMk64E+BTVX168AK4ArgOuBDVbUR+AFw1TQKlTSMvocDK4EXJFkJnATsB17DaJpygB3A5T23IWlAfaYm/y7wAUYzD+8HngTuAp6oqkPdavPAuoWen2Rrkrkkcwd5atIyJPXU53DgNOAyYANwJnAy8PoFVq2Fnl9V26tqU1VtOoHVk5Yhqac+hwOvBR6qqser6iBwC/AK4NTu8ABgPfBozxolDahPCDwMXJjkpCQBNgP3A3cCb+rW2QLc2q9ESUPqc05gN6MTgF8Hvtm91nbgauDdSfYBLwKun0Kdkgay8tirHF1VXQtce0Tzg8AFfV5X0ux4xaDUOENAapwhIDXOEJAaZwhIjTMEpMYZAlLjDAGpcYaA1DhDQGqcISA1zhCQGmcISI0zBKTGGQJS4wwBqXGGgNQ4Q0BqnCEgNa7XdwxKx+v2R+/+2f2Lz3zZElaiwwwBLZnxQDiSATE7Hg5IjXMkoGXp2UYJ4EhhmhwJSI1zJKCZONaefZqv5yjh+BxzJJDkE0kOJLl3rG1NkjuS7O1uT+vak+QjSfYluSfJ+UMWL6m/xRwOfAq45Ii2bcCuqtoI7OqWYTQ1+cbu31bgo9MpU89Vtz9699RHAZquY4ZAVX0F+J8jmi8DdnT3dwCXj7V/uka+ymia8rXTKlbPLUv15jd4js+kJwbPqKr9AN3t6V37OuCRsfXmu7afk2Rrkrkkcwd5asIyJPU17RODWaCtFlqxqrYzmsqcU7JmwXX03LRc9sIL1eFJw5836UjgscPD/O72QNc+D5w1tt564NHJy5M0tElDYCewpbu/Bbh1rP2t3acEFwJPHj5skLQ8HfNwIMmNwKuBFyeZB64F3g/clOQq4GHgzd3qXwAuBfYBPwHeNkDNep55tiH6cjm0eD47ZghU1ZVHeWjzAusW8Pa+RUmaHa8Y1NQc7157MSfpxtdxVDAM/3ZAapwjAfU2qz20o4JhOBKQGudIQL0s1R75yPMJjgwmZwjouPR9s037ir3F1uOVgkfn4YDUOENAi/ZcHXI7Cnh2hoDUOENAz2kXn/ky9/Q9GQJS4wwBqXF+RKhj8oTg85sjAalxjgT0vOMI4Pg4EtAxTfMM/BCTkDxXD1eWC0NAapyHA5q5w3vuPqML9/7T40hAapwjAR3TUHvdSUcEjgKmy5GA1DhDQMc09PX5i92z+0nAMAwBLdrQQXC0N7hv/mEZAlLjPDGo43J4NDCrPfPxbMcrBSdzzJFAkk8kOZDk3rG2v0ryrST3JPnHJKeOPXZNkn1JHkhy8VCFS5qOxYwEPgX8DfDpsbY7gGuq6lCS64BrgKuTnAtcAfwacCbwxSS/XFX/N92ytVSmMQJ4tm8KnvT1HQVMbjFzEX4lydlHtP3r2OJXgTd19y8DPltVTwEPJdkHXAD8+1Sq1ZKbxuGAJ/mWl2mcGPx94J+7++uAR8Yem+/afk6SrUnmkswd5KkplCFpEr1ODCZ5L3AIuOFw0wKr1ULPrartwHaAU7JmwXW0fC31lGAO/6dn4hBIsgV4A7C5m5IcRnv+s8ZWWw88Onl5koY20eFAkkuAq4E3VtVPxh7aCVyRZHWSDcBG4D/6lyk9zVHAdB1zJJDkRuDVwIuTzAPXMvo0YDVwRxKAr1bVH1bVfUluAu5ndJjwdj8Z0LT45h/GYj4duHKB5uufZf33Ae/rU5Sk2fGKQfU25FWE7v2H598OSI1zJKCpWexeexpfL6bpcSQgNc6RgGbOEcDy4khAapwhIDUuT1/xu4RFJI8DPwa+v9S1AC/GOsZZxzM9l+v4pap6yZGNyyIEAJLMVdUm67AO65htHR4OSI0zBKTGLacQ2L7UBXSs45ms45med3Usm3MCkpbGchoJSFoChoDUuGURAkku6eYp2Jdk24y2eVaSO5PsSXJfknd07WuS3JFkb3d72ozqWZHkG0lu65Y3JNnd1fG5JKtmUMOpSW7u5pTYk+SipeiPJO/qfib3JrkxyYmz6o+jzLOxYB9k5CPd7+09Sc4fuI5h5vuoqiX9B6wAvgOcA6wC/hM4dwbbXQuc393/ReDbwLnAXwLbuvZtwHUz6od3A58BbuuWbwKu6O5/DPijGdSwA/iD7v4q4NRZ9wejb6d+CHjBWD/83qz6A3gVcD5w71jbgn0AXMrom7YDXAjsHriO3wJWdvevG6vj3O59sxrY0L2fVix6W0P/Yi3iP3sRcPvY8jWMJjaZdR23Aq8DHgDWdm1rgQdmsO31wC7gNcBt3S/V98d+4M/oo4FqOKV78+WI9pn2B09/bf0aRn/gdhtw8Sz7Azj7iDffgn0A/B1w5ULrDVHHEY/9DnBDd/8Z7xngduCixW5nORwOLHqugqF0k6ucB+wGzqiq/QDd7ekzKOHDwHuAn3bLLwKeqKpD3fIs+uQc4HHgk91hyceTnMyM+6Oqvgt8AHgY2A88CdzF7Ptj3NH6YCl/dyea72MhyyEEFj1XwSAbT14IfB54Z1X9cFbbHdv+G4ADVXXXePMCqw7dJysZDT8/WlXnMfpbjpmcnxnXHW9fxmhYeyZwMvD6BVZdDp9tL8nvbp/5PhayHEJgyeYqSHICowC4oapu6ZofS7K2e3wtcGDgMl4JvDHJfwGfZXRI8GHg1CSHv+9hFn0yD8xX1e5u+WZGoTDr/ngt8FBVPV5VB4FbgFcw+/4Yd7Q+mPnv7th8H2+pbuzft47lEAJfAzZ2Z39XMZrQdOfQG83ou9KvB/ZU1QfHHtoJbOnub2F0rmAwVXVNVa2vqrMZ/d+/VFVvAe7k6TkeZ1HH94BHkry0a9rM6KvjZ9ofjA4DLkxyUvczOlzHTPvjCEfrg53AW7tPCS4Enjx82DCEweb7GPIkz3GcALmU0dn57wDvndE2f5PRkOke4O7u36WMjsd3AXu72zUz7IdX8/SnA+d0P8h9wD8Aq2ew/ZcBc12f/BNw2lL0B/AXwLeAe4G/Z3TWeyb9AdzI6FzEQUZ72KuO1geMhuF/2/3efhPYNHAd+xgd+x/+ff3Y2Prv7ep4AHj98WzLy4alxi2HwwFJS8gQkBpnCEiNMwSkxhkCUuMMAalxhoDUuP8Hh33yejtUZzUAAAAASUVORK5CYII=", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "# show one example ROI\n", - "masks = (Segmentation.Roi).fetch('mask')\n", - "plt.imshow(masks[2])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Fluorescence trace of each segmented ROI" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we got masks of ROIs in the table `Segmetation.Roi` obtained with different parameter combinations. We would like to extract the fluorescence trace of each segmentation. \n", - "\n", - "The table design is similar to the `Segmentation` and `Roi`. The master table `Fluorescence` is the driver for the computation, with a secondary attribute `time` shared across traces of all ROIs. The part table `Trace` saves the extracted trace for each ROI over time (frame). " - ] - }, - { - "cell_type": "code", - "execution_count": 57, - "metadata": {}, - "outputs": [], - "source": [ - "from skimage import io\n", - "import os\n", - "@schema\n", - "class Fluorescence(dj.Imported): # imported table because it also rely on the external tiff file.\n", - " definition = \"\"\"\n", - " -> Segmentation\n", - " ---\n", - " time : longblob # time for each frame\n", - " \"\"\"\n", - " \n", - " class Trace(dj.Part):\n", - " definition = \"\"\"\n", - " -> master\n", - " -> Segmentation.Roi\n", - " ---\n", - " trace : longblob # fluorescence trace of each ROI\n", - " \"\"\"\n", - " \n", - " # the master table is mainly to perform the computation, while the part table contains the result\n", - " def make(self, key):\n", - " \n", - " print('Populating: {}'.format(key))\n", - " # fetch data directory from table Session\n", - " data_path = '../01-Calcium_Imaging/' + (Session & key).fetch1('data_path')\n", - " \n", - " # fetch data file name from table Scan\n", - " file_name = (Scan & key).fetch1('file_name')\n", - " \n", - " # load the file\n", - " im = io.imread(os.path.join(data_path, file_name))\n", - " \n", - " # get dimensions of the image and reshape\n", - " n, w, h = np.shape(im)\n", - " im_reshaped = np.reshape(im, [n, w*h])\n", - " \n", - " # get frames per second to compute time\n", - " fps = (Scan & key).fetch1('fps')\n", - " \n", - " # insert into master table first\n", - " self.insert1(dict(**key, time=np.array(range(n))/fps))\n", - " \n", - " \n", - " # extract traces\n", - " roi_keys, masks = (Segmentation.Roi & key).fetch('KEY', 'mask')\n", - " \n", - " traces = []\n", - " for roi_key, mask in zip(roi_keys, masks):\n", - " \n", - " # reshape mask\n", - " mask_reshaped = np.reshape(mask, [w*h])\n", - " trace = np.mean(im_reshaped[:, mask_reshaped], axis=1)\n", - " \n", - " traces.append(dict(**roi_key, trace=trace))\n", - " \n", - " self.Trace.insert(traces)" - ] - }, - { - "cell_type": "code", - "execution_count": 58, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 1, 'seg_param_id': 1}\n", - "Populating: {'mouse_id': 0, 'session_date': datetime.date(2017, 5, 15), 'scan_idx': 2, 'seg_param_id': 1}\n", - "Populating: {'mouse_id': 100, 'session_date': datetime.date(2017, 5, 25), 'scan_idx': 1, 'seg_param_id': 1}\n" - ] - } - ], - "source": [ - "# ENTER YOUR CODE! - populate the Fluorescence table\n", - "Fluorescence.populate()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we could plot the traces of an example scan" - ] - }, - { - "cell_type": "code", - "execution_count": 61, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Text(0, 0.5, 'Fluorescence')" - ] - }, - "execution_count": 61, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEJCAYAAAB7UTvrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOydd3zU9f34n++7JJe9B2QPVgaEvYcyFCmCAxUp7lGrrdZWu6zV/r611tbWFkcrtiourOJCcCFDRPYmCSNkkr33zr1/f3xyIeOSXJK7DHw/H497JPcZ78/rIPd5fV5bSClRKBQKhQJAN9gCKBQKhWLooJSCQqFQKFpRSkGhUCgUrSiloFAoFIpWlFJQKBQKRStKKSgUCoWiFZspBSHEq0KIAiFEQoftPxVCnBVCJAoh/tJm+2+EEOdb9l1pK7kUCoVC0TV2Nlz7deAF4A3TBiHE5cBKYIKUsl4I4d+yPQZYDcQCgcDXQogxUspmG8qnUCgUig7YTClIKXcLIcI7bP4x8GcpZX3LMQUt21cC77ZsTxNCnAemA/u6u4avr68MD+94CYVCoVB0x5EjR4qklH7m9tnSUjDHGGCeEOIpoA54REp5CAgC9rc5LqtlWyeEEPcC9wKEhoZy+PBh20qsUCgUlxhCiIyu9g10oNkO8AJmAo8C7wkhBCDMHGu2/4aUcr2UcqqUcqqfn1lFp1AoFIo+MtBKIQv4UGocBIyAb8v2kDbHBQM5AyybQqFQfO8ZaKXwMbAQQAgxBnAAioDNwGohhEEIEQGMBg4OsGwKhULxvcdmMQUhxEbgMsBXCJEFPAG8CrzakqbaANwmtTatiUKI94AkoAl4QGUeKRQKxcAjhnPr7KlTp0oVaFYoFIreIYQ4IqWcam6fqmhWKBQKRStKKSgUCoWiFaUUFL2mpK6ED5M/ZDi7HhUKhXkGunhNcQnw7KFn+TT1UwKcA5gTNGewxVEoFFZEWQqKXpFcmsyW1C0AvH367UGWRqFQWBulFBS94oVjL+Bi78KacWv4NvtbMiq6rJZXKBTDEKUUFBZzsvAkOy7s4LbY27hnwj3Y6ezYeGbjYIulUCisiFIKCotZd3Qd3o7e3BJzC75OviwNX8rH5z+murF6sEVTKBRWQikFhUUcyD3AgbwD3D3+blzsXQBYM24N1Y3VfHL+k0GWTqFQWAuVfaSwiK2pW3FzcOPGsTe2bhvvN54JvhN4I+kNUstTSShKILc6lw1LNxDuET54wioUij6jLAVFj0gp2Ze7j5kjZ2LQG9rtuzX2VrKrstmSugWD3kBJXQnf5Xw3SJIqFIr+oiwFRY+kVaSRV53HvRPu7bTvyvArmew/GR8nHwSCRe8v4lTRqUGQUqFQWAOlFBQ9si9Hm4o6a+Qss/v9nC8OOxrvO55ThUopKBTDFeU+UvTIvpx9hLqFEuwW3OOxE/wmkFmZSVld2QBIplAorI1SCopuaWxu5GDeQWYFmrcSOjLBbwKAciEpFMMUpRQU3XKi8AS1TbUWK4VYn1h0QqeUgkIxTFFKQdEte3P2ohd6po+YbtHxzvbORHlGcbLopI0lUygUtkApBUW37M/dz3jf8bg5uFl8zgTfCSQUJajW2grFMEQpBUWXlNeXk1CUwOzA2b06b7zveMrry8mszLSRZAqFwlYopaDokgO5B5BIi+MJJsb7jQe0BnoKhWJ4oZSCokuO5B/Byc6JON+4Xp0X5RGFs52zUgoKxTBEKQVFl6SVpxHpEYmdrnc1jnqdnljfWJWBpFAMQ5RSUHRJekU6Ye5hfTp3vO94zpaepb653spSKRQKW6KUgsIstU215Fbn9rnb6QTfCTQZmzhdfNq6gikUCpuilILCLJkVWuZQhHtEn843xSESixOtJpNCobA9NlMKQohXhRAFQogEM/seEUJIIYRvy3shhFgnhDgvhDgphJhsK7kUlpFekQ7QZ0shwCUAPyc/Eoo6/fcrFIohjC0thdeBpR03CiFCgCVA2yT2q4DRLa97gX/ZUC6FBaSXpwMQ6hba5zVifWOVpaBQDDNsphSklLuBEjO7ngN+CbQtd10JvCE19gOeQoiRtpJN0TPpFemMcBmBs71zn9eI9YklvTydqoYqK0qmUChsyYDGFIQQK4BsKeWJDruCgAtt3me1bDO3xr1CiMNCiMOFhYU2klSRXp5OuHt4v9aI841DIkkqTrKOUAqFwuYMmFIQQjgDjwG/N7fbzDazjXOklOullFOllFP9/PzMHaLoJ1JK0iv6rxRifWIBFWxWKIYTAzl5LQqIAE4IIQCCgaNCiOlolkFIm2ODgZwBlE3RhuK6Yqoaq/ocZDbh5ehFkGuQCjYrFMOIAbMUpJSnpJT+UspwKWU4miKYLKXMAzYDt7ZkIc0EyqWUuQMlm6I9aeVpAP22FABifGKUpaBQDCNsmZK6EdgHjBVCZAkh7urm8M+AVOA88Apwv63kUvRMRkUG0Pd01LbE+caRXZVNaV1pv9dSKBS2x2buIynlzT3sD2/zuwQesJUsit6RXp6OQW9gpEv/E8DifLQitqTiJOYEzen3egqFwraoimZFJ9Ir0gl1D0Un+v/nEe0TDaDiCgrFMEEpBUUnrJF5ZMLNwY1w93ASipVSUCiGA0opKNrR2NxIVmWW1ZQCaJXNSUWqVkGhGA4opaBox4WqCzTLZiI8+tYIzxxxPnEU1BZQUFNgtTUVCoVtUEpB0Q5Tz6O+zlEwh6ljqoorKBRDH6UUFO3ob3dUc4zxGgNAcmmy1dZUKBS2QSkFRTsyKjLwdvTG3cHdams62zsz0mUkaRVpVltToVDYBqUUFO3IrMi0quvIRIRHBKllqVZfV6FQWBelFBTtyKzMJMQtpOcDe0mkRyTpFekYpdHqaysUCuuhlIKildqmWgpqCvo1WKcrIjwiqG2qJb863+prKxQK66GUgqKVrMosAELdbaMUAFLLlQtJoRjKKKWgaCWzUpuQagtLIdIjErjYgVWhUAxNlFJQtHKhQht+F+wWbPW1TRlNylJQKIY2SikoWsmszMTT4ImHwcPqawshiPSI7KQU1h1dx4HcA1a/nkKh6BtKKShayazMtInryESkZ2Q791FWZRavnHqF5448Z7NrKhSK3qGUgqKVCxUXCHG3fjqqiQj3CErqSiivLwdgd9ZuQJvhnFSsGuYpFEMBpRQUADQ0N5BbnWuTGgUTkZ7tg827s3Yz0mUkBr2BD859YLPrKhQKy1FKQQFAdlU2EmlT91HbtNSaxhoO5h1kSdgSrgy/kq1pW6lprLHZtRUKhWUopaAA4EKllnlkS0sh0CUQB50DqWWp7MvdR6OxkQXBC1g1ZhXVjdV8kf6Fza6tUCgsw2YzmhXDi8yKlhoFGxSumdDr9IR7hJNWkUZlYyWu9q5MCpiEnbBjlOcoNp3bxHWjr7PZ9RUKRc8oS0EBaJlHrvaueBm8bHqdSI9IUspS+DbrW+YEzcFeZ48QglVjVnGq6BRnSs7Y9PoKhaJ7lFJQABcb4QkhbHqdCI8IsquyKawtZEHwgtbtyyOXY9Ab+Cj5I5teX6FQdI9SCgpAS0e1pevIhKndhUAwN2hu63YPgwezAmfxTdY3SCltLodCoTCPUgoKGo2N5FTl2DTzyIQpAyneLx4vx/auqnlB88iuylbDeBSKQUQpBQV5VXk0ySabZh6ZCPcIx93BnSvDr+y0z2Q57MnaY3M5BpMDqcU88M5RGpvVbAnF0MNmSkEI8aoQokAIkdBm21+FEGeEECeFEB8JITzb7PuNEOK8EOKsEKLzHUNhM1q7ow6A+8igN/Dl9V+yJnpNp32BroFEeUTxbfa3NpdjMHnrQCZbT+by2ancwRZFoeiELS2F14GlHbZtA+KklBOAc8BvAIQQMcBqILblnJeEEHobyqZogy1bZpvD1cEVnTD/pzcveB5H8o9csoVsRqNkT3IhAK99lz64wigUZrCZUpBS7gZKOmz7SkrZ1PJ2P2Dq0bwSeFdKWS+lTAPOA9NtJZuiPZkVmTjZOeHr5DvYojA3aC6NxsZLtnNqQk45pTWNTA3z4viFMo5mlg62SApFOwYzpnAn8HnL70HAhTb7slq2dUIIca8Q4rAQ4nBhYaGNRfx+kFWVRZBrkM3TUS1hsv9knO2c2ZN9acYVdp/T/mb/fuNE3BztlLWgGHIMilIQQjwGNAFvmzaZOcxsXqKUcr2UcqqUcqqfn5+tRPxeUVRTRIBzwGCLAYC93p6ZI2eyJ3vPJZmauju5iNhAd0J9nLlpagifn8olt7x2sMVSKFoZcKUghLgNWA78UF781mcBbVNfgoGcgZbt+0pRXRE+Tj6DLUYrc4PnklOdc8lNaausa+RoRinzx2gPM7fNDscoJW/uyxhkyRSKiwyoUhBCLAV+BayQUraNJG4GVgshDEKICGA0cHAgZfu+IqWkqLZoSMQTTMwLmgdwybmQ9qUU02SUzB+tKYUQb2eWxATwzsFM6hqbB1k6hULDlimpG4F9wFghRJYQ4i7gBcAN2CaEOC6E+DeAlDIReA9IAr4AHpBSqm/JAFBeX06TsWlIKYURLiMY7TWarzO+HmxRrMru5EKcHfRMCbtYtLdmRhhlNY3sTSkaRMkUiovYMvvoZinlSCmlvZQyWEr5XynlKClliJRyYsvrvjbHPyWljJJSjpVSft7d2grrUVSr3YyGklIAWBm1kuOFxzlfen6wRbEa3yYXMSvSBwe7i1+7mZHeuDjo2ZZUMIiSKRQXURXN33OK6oamUrg66mrsdHZ8kHxpTGTLKK4mo7imNZ5gwmCnZ/4YP3acyb8kA+uK4YdSCt9zhqql4O3ozeLQxWxO2Ux9c/1gi9Nvdidr/84dlQLA4ugA8ivqSciuGGixFIpOKKXwPae4thgYekoB4Pox11PRUMG2jG2DLUq/ScqpwNvFgXAf5077Lh/nj07AttP5gyCZQtEepRS+5xTVFmHQG3C1dx1sUToxfcR0gl2D+eDc8HchZZXWEOLlZLZA0NvFgSlhXmxXSkExBLBIKQiNtUKI37e8DxVCqDYUlwCmdNShUM3cEZ3Qcf2Y6zmcf5i08uHdTju7tJZgr85WgolF0QEk5lSoQjbFoGOppfASMAu4ueV9JfCiTSRSDChFtUOrcK0j14y6Bjthx/vn3h9sUfqM0SjJKqsl2Mupy2MWR/sD8PVplYWkGFwsVQozpJQPAHUAUspSwMFmUikGjKLaInwdh148wYSvky9LI5byzul32J21e7DF6RNF1fU0NBm7VQpRfq6E+zgrF5Ji0LFUKTS2tLKWAEIIP0BNCLkEKK4tHpJB5rY8PvNxxnqP5ZFvHuFU4anBFqfXZJVqLqGgbpSCEIJF0QHsPV9MdX1Tl8cpFLbGUqWwDvgI8BdCPAXsAf5kM6kUA0KjsZHS+tIhrxSc7Z15cdGLeDt688D2B8isyBxskXqFSSl0F1MAWDTOn4ZmI/tTiwdCLIXCLBYpBSnl28AvgaeBXOAaKeXwdfIqACip1cZdDOWYgglfJ19eXvIyAD/f9fNhVeiVVaq1+Qry7NpSAJgS7oWjvY5vk1XLC8XgYWn20UwgW0r5opTyBSBLCDHDtqIpbM1QrWbuijD3MH425WecLT3L8cLjgy2OxWSV1uLt4oCLwa7b4wx2eqZH+PDdeaUUFIOHpe6jfwFVbd5Xt2xTDGOGcuFaVywNX4qLvQubzm0abFEsJru0tkcrwcTcUT4kF1SRV15nY6kUCvNYqhREm9kHSCmNQPePPYohz1BtcdEdzvbOLItYxlfpX1HRMDzaQmSV1nSbedSWOaO0/wtlLSgGC0uVQqoQ4kEhhH3L6yHg0pqA8j3EpBSGQ0yhLavGrKKuuY6tqVsHW5QekVKSVdp9jUJboke44+PiwB6lFBSDhKVK4T5gNpCNNiVtBnCvrYRSDAxFtUW4O7hj0BsGW5ReEeMTQ7R3NJvObbJ6wNlolDy1NYmE7HKrrFdU1UB9k7HHzCMTOp1g9ihf9pwvGlbBdMWlg6XZRwVSytVSSn8pZYCUco2UUpVeDnOG2sS13rBqzCrOlZ4joSjBquvmVtTxyrdp/OjNI5TXNvZ7PUszj9oyb5QvhZX1nMuv6vlghcLKWJp95CeE+K0QYr0Q4lXTy9bCKWzLcFYKyyKW4WTnZPV5CzllWk1Bdlktv/nwZL+f1rNb1gv2tlwpzBmt/Z8oF5JiMLDUffQJ4AF8DWxt81JYifyKOjYezORkVhnNxoFxGwz1vkfd4ergylURV7E1dWtrFpU1MCmFVVOC+exUHu8eutCv9VqrmXthKQR5OhHp68Ke5MJ+XVuh6AuWZhA5Syl/ZVNJvsfklNVy0/p9XCjRbiBujnYsGOPH326Mx2Cnt9l1h7OlAHB77O18lPwRbyS9wcNTHrbKmqab+JMrYsmvqOMPnyYyNcyL0QFufVyvBk9ne9wc7Xt13pxRvnxwNIuGJmO78Z0Kha2x9K9tixBimU0l+Z6SV17Hza/sp6y6kTfunM66mycxf7QfW07mcji91GbXrWmsobapdlgrhQiPCJZGLGXjmY3sSE7liU8SMPbTysopq8XL2R5Xgx1/uzEee52Ol3dbnmj31v4M/rvnYpvvrF7UKLRl7mhfahqaOZRe0utzFYr+YKlSeAhNMdQJISqEEJVCiOGRJD6EKajUFEJxVQMb7prO/DF+rIgP5M/Xj0cn4GCa7W4Iw7FGwRz3jr+XuqY6/rb/P2zYl8HX/ewymlNWS2DLTdzfzZGF0f7sOFNgkUvvxZ3n+d3HCfzps9OtAebsXqSjtmX+aD/cDHZ8cDSr1+cqFP3B0uwjNymlTkrpKKV0b3nvbmvhLnX++sVZcstr2XDnNCaHerVud3O0J3qku02fEluVwhBum20Jo7xGsSRsCZlNX4Guhpd2pfQrOJxTVteqFACuiBlBSXUDRzK6t9pe/iaFv355lsXRAQjgte/S29QoWJaO2hYnBz3L4wP5/FQeVaprqmIA6e3ktcdb3oeoyWv9o7ymkU9P5nDd5GCmhHl32j8t3JtjmWU0NtumQ/lwLVwzx1XBt4CunrCIwxy/UMa+fnQZzSlr7+5ZMNYPB72ObUl5XZ7z1v4Mnv78DMsnjOTfayfzgwkjefdgJunFNdQ2NvfJUgC4YWowtY3NbD2Z06fzFYq+0NvJa2ta3lehJq/1iw+PZVHXaGTN9FCz+6dHeFPb2Gy1IqqOXCruI4DsAk8aK2Kpc96Nr5vgpZ0pfVqnoq6RyvomAj0dW7e5GuyYPcqHr5Lyu7RAXt2TxpQwL567aSJ2eh33zIukuqGZZ788C/TcMrsrJoV4EuXnwvuHlQtJMXCoyWuDgJSStw9kEh/iSVyQh9ljpoVr1oOt4gpFtUXohR5Pg2fXBzU1QPXQz5Xfl1KMR+MCapqquHxSMXvOF3Eyq6zX65jSUQM7BIaXxASQUVxDckHnYrLq+ibSiquZP9oPe732dYoL8mBWpA9bT+UCvUtHbYsQghumhnA4o5TUQlXIphgYbDZ5raXArUAIkdBmm7cQYpsQIrnlp1fLdiGEWCeEOC+EOCmEmNzHzzMsOJReyvmCKn44w7yVAODnZiDC18VmcYXiumK8Hb3R67pJed35FPxjAuQcs4kM1sBolOxPK2Zu8Ex8HH2oMxzG3dGuT9ZCl0ohOgCAbUmdg9incyuQEuKC2ofY7pkf0fp7dxPXeuK6SUHodYJNR5S1oBgYbDl57XVgaYdtvwa2SylHA9tb3gNcBYxued3LJd6W++0DGbg52nH1hMBuj5sW7sWh9NJ+p1maw6IaheSvoLEa3rkJyjpMOzPnSsk5DpsfhJqBS6M8k1dJWU0js6P8uDL8Svbm7GH1DD++SMwjp6yW/bn7ufqjq8mv7jkrKbtMa1fd8cne392RiSGefJXYOa5gcu/FBra3+C4b488of1fcHe3wcOpdjULHay8Y48eHR7MHrKhR8f3GZpPXpJS7gY53h5XAhpbfNwDXtNn+htTYD3gKIUZa9hGGF8VV9Xx+Ko/rJwfj5NB9Ydr0CB/KaxvNui36S251Lv7O/l0fUFUABUkQvwYa6+DtG7Sb/ZnP4NWl8Ed/2PoIVOSC0Qh7n4f/LIajG+DERqvL2xWmoPKsKB+uiriKBmMD/iOSAfgiIZtnDj5DekU6bya92eNaOWW12OsFfq6dGwQuiQngRFZ5pzkHiTkV+Lg4EODe/hydTvDsDfH88drxff1ordwwJZi8ijq+OafajSlsz0BPXguQUuYCtPw03ZWCgLb9BLJatpmT5V4hxGEhxOHCwuHXBuDDo9k0NBtZ043ryMR0U1zByi6kZmMzGeUZRHhEdH1Q2u4WIe6G1W9BcQo8Fwvv3gzl2RB9NRx5DdZNhPUL4KvfwZgrwW8cJH3SvQBVBZoisQL7UooJ93Em0NOJeL94glyDOFy0k9H+rvzvzCecLztPsGsw7597n/L67oP2OWW1jPBwRKcTnfZdGdviQupQB5GQU0FMoDtCdD5nYognK+K7twYtYVF0AP5uBjbszej3WgpFTwyVyWudv1Et8YtOG6VcL6WcKqWc6ufnZ0URBoZ9qcWM9ndljAVtE0K8nQhwN3DIysHm7KpsGowNRHpEdn1Q6i4weMDIiRAxH657GYKmwPX/hQePwapX4SeHIe56KL8Ay5+Dm96CuFVw4QBUdJFGWVMC/5wIW3/e78/RbJQcSCtmVpSWViuEYGn4Uvbn7mdetB1ZfES0VyzPXf4cNU01vHf2vW7XyymrJdDDvP8/ys+VCF+XdnGF+qZmkvMru0wWsBYOdjrWzgzjm3OFnLeB1ahQtGWgJ6/lm9xCLT9N9nAWENLmuGDgkkzOTswpt/gmIoRgWrg3B9NKrNpbP7Vca9vQo6UQPhdMgei46+H2LTB+Fehb/uu9I+Cal+BX6TD1ThACYlZo+05vMb9u0sdanOLIa10fYyFJORVU1jUxM/JircVVEVfRLJs5WPMsOvtyprqvZZz3OOYEzuGt029R19T1mMucsrouM4WEEFwRE8C+lCIq67SW2sn5VTQZJbGBtq/jXDMjFAe9jg17021+LcX3m4GevLYZuK3l99vQuq+att/akoU0Eyg3uZkuJYqq6smvqO/VTWR6hDd5FXWtjdqsgUkpRHp2YSmUpkNZBkQu6P3ifmO7dyGd2gQ+o2BkPGz+qRaTACg4A29eB/tesvhS+1K1dNlZbZTCGK8xRHlEkVF1Hn1dNGfSNbfPnXF3UlJXwuaUzWbXamo2kldR122m0JKYABqbJbvOam7LroLMtsDX1cCKiYF8cDTLKnMeFIqusNnkNSHERmAfMFYIkSWEuAv4M7BECJEMLGl5D/AZmpI5D7wC3N/LzzEsSMzR2kXF9EIpmNpfHLvQ+7z7rkgtS8XXyRd3hy7kSP1G+xnRB6UAELMSMvdqsYO2lGdBxncw4SbNDdVYCx/fB7ufhZfnQcp22P7/LiqKHjiVXUGwlxP+7heLzYQQLI9ajk7oWBhwO3uSi6iqb2LaiGnE+cTxeuLrNBubO61VUFlPs1F2Skdty6RQL3xcHFpdSIk5Fbga7Ajz7ltxWm+5fXY4NQ3NvNfPdt4KRXfYbPKalPJmKeVIKaW9lDJYSvlfKWWxlHKRlHJ0y8+SlmOllPIBKWWUlHK8lPKwNT7cUCMxp+XJcqTlT5ZjR7hhsNNx0opKIa08rft4Qto34DpCe+rvCzErQRrhTAf3UELLQJzxq8B3NCx9Wotd7Pg/GLsM7t4Oxib45s+dljQrZlEVUX6unbbfFnsbm6/ZzOr4mTQ0G9l5pgAhBHfE3cGFygt8l/Ndp3O6qlFoi14nWBwdwM4zBTQ0GUnMKSdmpLvZwLQtiAvyYHq4Nxv2pav0VIXNsDT76C9CCPcW19F2IUSREGKtrYW71EjM0Z5sPZwtz1u31+uIDXTnRB8qdM0hpSS1PLXreIKUWjwhYr4WI+gL/jHgHdXZhXTqfQiaCt4tCmnK7bD4D3DT23DjBgieClPvgKNvQlFyj58jrbCaCF+XTvvsdfaEuYcxJcwLX1cHvmypL7g85HI8DZ5sSekcyzBNSAtq0+LCHEtiAqisb2JvShGncyt7ZfVZgzvmhJNVWssW1Q9JYSMsdR9dIaWsAJajuY/GAI/aTKpLlKScCmJG9v4mMiHYk4TsCpqs0ByvqLaIqsaqri2FgiSoLuxbPMGEEJq1kPYtVLc0pys4A3mnYPwN7Y+b+zOIXn5x2/xHwc4Rdvyx20vkV9RT3dBMlF9npWBCrxMsidGe7Osam7HX23Nl+JXsvLCT6sbq1uOajc1sz9oKuhpGdpF9ZGLuaF+c7PW8/E0qtY3NNs886siSmADigtz5w6dJFFR2HTQfaGobmq2aDKEYPCxVCqZH22XARpPbR2E5VfVNpBVV9ykoGR/iQW1jM+et0P+mxyBzazxhfv8uFHe95kL67xJI2QEJm0DoIPba7s9z9YfZP9GylLKPdHlYapH2bxFpxn3UlitiR1Dd0MzeFC0ovTxyOXXNdWzP3N56zCcpn7Cz+Hncgz/FxdB9Up2jvZ75Y3xbi+YGIvOoLXZ6Hc/dOJHq+iZ+/cGp1htxYWU9P3/vONv7OU+iL6QWVhH35Jcs+OsuntycyHfni5SCGMZYqhQ+FUKcAaYC21t6Hw2dx5RhwJlcLcjcl5tIfLDWtO6EFeIKrUqhK0sh6WPwGQ2ePRfXdcuIOLjlQ0DCm9fCd+u0wLVbQM/nzvoJOPvAa8u0SuqDr3RqnZFaqD3pm3MftWV2lA+uBju+StRulqYCN5MLqa6pjpeOv4SQdkiXYxzNP9qjeEtiRgBa/cAo/+6Vki0YHeDGr68ax44zBWw8eIF9KcUsW/ctHx7N5o9bT9ukLUp37E8todkoCfZyYuPBTH74nwO8czCz5xMVQxJLA82/RmudPVVK2QjUoLWmUFiIKfMoNqj3SiHcxwU3RztOZPW/jXZqWSqu9q74OZkp/Ms+ohWeTbur39cBIGoh/HgfXP4Y6OwsX9fRHW7bosUcipLhs0dg4+p2h6QVVeNkr2eEe/cxAIOdnsvH+bMtKZ9mo9SykyKXcyDvAAU1Bbx75l3ya/LxrroHB7x5+uDTZrOT2rJonIVVRC0AACAASURBVD86AeNGuLV2Rh1obpsVztxRvvzh00R++J/9uBnseGjRaNKKqtl5dmDbYZzMKsPT2Z63757B8d9fwcQQT9bvTlXB8GGKpYFmZ+ABLlYxB6JZDQoLScwpx9vFocebmDl0OkF8sKdVLAVT5pG5tgzs/zc4uMHEH/b7Oq3YO8KCX8JvW1pjWEpADFz1jFY9vfB3mrIqvdjmIbWwinBfF4syf66MDaC4zfS0H0T+AKM08t7Z93jl1CvMCZxDcVEUE51v4UzJGT5I/qDb9bxcHLhzTgQ3Tg3p9jhbotMJ/nrDBLycHbg6PpDNP53LTxeOItDDsd2M6IHg+IUyJgR7IoTAyUHPvfMjySiuMdtVVjH0sfQx5zWgAa1WAbRgc/eRQEU7EnMqiO2iR44lTAj24GxeJXWNXTzFlmVCc89jG1PLUwn3CO+8oyIXEj+CSWu1J3Vr09dMJiG01hnQLsU1taiayG6CzG1ZMEabnmbqchrhEUGcTxzrT66noqGCu+MeoLKuiWl+lzE1YCrPH3u+xz5Jv1sew9qZYX37TFZipIcT+36zkH+unoSrwQ47vY5bZ4ezN6WY07kDM0K9pqGJ5IIqJgZfjJVdERNAsJcT/93Tl/pWxWBjqVKIklL+BWgEkFLWYr5fkcIMDU1GzuX3L31xQrAnTUZJkrkve1kmrJsMm27vttFcZUMlhbWF5uMJh/+r1QjM6LYmcXDwjoAR4yFJq0ZuaDJyoaSGqB7iCSbcHO2ZM8qHL5PyWgOgP4j8ARLJVeFX4dCsPfGH+rjw6+m/prKhkgd3PEhlQ6VtPo8V6fiQcfO0UJzs9bw6QNZCYk4FzUbJhOCLw5rs9DrunBPBofRSjluxvkYxMFiqFBqEEE5cHLITBdTbTKpLjOSCShqbZb/aIUwM6SbYnLQZjI1w+lPY2bUB12WQubEODr8GY6+6WEMw1IheobmQKvPILKnGKCHCQksBtCykCyW1nM7VbvRXR13N8sjl/GzKz0jI1hRtXKAHY73H8uf5f+Zk4Unu+vIuSutKbfJxbIWHsz2rpgTzyfEciqps/xU1/T1OCGn/t33jtBDcHO34z7fKWhhuWKoUngC+AEKEEG+jDcj5pc2kusRoDTL3w1IY4eGIv5uBk+aCzUmfQMB4LTD77d/guPl5BqllXaSjJmyCmiKYcV+f5bM50SsACWe2tGYeRfpanvmzODoAIeCrJM2F5GHw4Ol5TxPoGsip7HLcHe0I8dZqFJaGL+WfC/9Jankqd3xxB4U1w6tF++1zwmloNvLOAdtnAJ3MKifQwxF/t/axMleDHWumh/J5Qh5ZpTU2l0NhPXpUCkKzT88A1wG3AxvRspB22VSyS4jE7HKcHfRE+Fj+ZGuOCcGenSuby7Mh6yDEroRlz2r1BZt/ChcOdTo/rTwNe509Qa4dRlWceBd8x/a/NsGW+I3VUmVPf0pqUUs6ai8sBT83A1PDvPgysXPw09S5tq0rZn7wfP61+F/kVOfw1IGn+i//ABLl58qkUE92n7O9MjuRVdbOddSW22aHY5SSD45k21wOhfXoUSm0tMz+uKVv0VYp5RYp5dCf5j5EKKtp4KNj2cyO8u13j5yJIR6kFla375J5+lPtZ8w1oLeHG98AF1+tn1AHUstTCXMPw07XpkCrsQ6yDsHoJX0PBg8Eprbcad+Sl5uNr6sBd8fejbm8ImYEp3MruFBy8cm1sdnImVzzMxGmjZjGHXF3sD1zO6cKT/X7Iwwkk0O9OJVdTqMVquC7oqymgYziGuJDzCuFQE8nYgPdWwsHFcMDS91H+4UQ02wqySXKS7tSqKxv4hdXjOn3WqYnspNtrYWkT7ReQ76jtfdOXjDjR1pTu/zEdudnVJiZtpZ9BJrqIGxOv+WzOdFXg2zGL2enxZlHbVkcoxXO7WqTx38uv5KGZmOX7SpujbkVb0dv/nnsn32TeZCYGOJJfZOm8GyFqW4mPrjrWNmsSB+OZZZ1nTWnGHJYqhQuB/YJIVKEECeFEKeEECdtKdilQHZZLa/vTefaSUFE96HnUUcmh3lhpxPsTWnpJ1SZD5n7tD5D7Q68DeycYH/74XhFtUUEOHeoKM74DhAQNqvf8tmckRPBI5TJFduJ9Ol9u+pwH2dCvZ35po1bJbE1yGz+/8fF3oW7x9/NgdwD7M/d3ze5B4FJodoDxPELpXye9jl/3P9H/nXiX7x/7n1yq6wzquTkhTItY7gbpTA7ypeGZmNrjYhi6GPp9LSrbCrFJcrfvjoLwC+u6GML6g64GuyYFOrJd+dbzPEznwKys1Jw9ob41XD8HVj8JLj4UttUS1VjFT5OPu2PTd+jtaRw8rKKjDZFCOrib2PW7v+Duo+A+F6eLlgwxo8PjmbR0GTEwU5HQk45rgY7wruJ99w49kbeSHqDdUfXMWPZjD7XmgwkQZ5O+LkZ2J6+j6PnnsbRzpHappb24C6BbL1ua3s3Yh84kVVOpK9Lt268aRHe6HWCvSlFzBnl26/rKQYGS9tcZACewNUtL8+WbYouSMqp4KNj2dwxO7zLEY99Yc4oX05ll1NW06C5jnzHaJPOOjLjPmiu18ZeAsW1mnXh49hGKTQ1wIWDEDbXavLZmtNRd7CleQYzz/+jtW6hN8wf40dNQzOHM7ReSqeyy4kJ7H4mgkFv4P74+zlVdIodmTv6LPtAIoQgLtiBY7X/IsQthF037uLo2qM8M+8Zcqpz+Drj636tL6XkRFZZl/EEE64GO+KDPS5at4ohj6VtLh4C3gb8W15vCSF+akvBhiN/+eIMC5/dxeT/28bVL+zB3dGe+y8bZdVrzB3li5RwOOm89pQfs9J8gNh/HEQtgoP/gaYGiutalEJbSyHnKDTVQnjv4gkHUot55P0Tg+InTi2q5ReNP6Y+YBJ8eG+3nVTNMSvKB3u94JtzhTQ1GzmdW8F4C9pfXx11NeHu4bx04qVh0wG0zPldmnXl/Hba/+Fs74y93p6lEUsJdQtlQ+KGfn2OvIo6CivrW5s1dsfsKF9OZpVTVd9zxb1i8LE0pnAXMENK+Xsp5e+BmcA9thNraFPT0NTpC7XzTAEv7UohwN2Rq+JG8KP5kWy4c3qvBupYQnyIJy4OeopPfqG1ph7TjWdv5v1QlQfbHqcoQ5s25uvUxoRP36P9DJ1t5mTznM2r5O4Nh9l0JIvPEwZ+jHZaUTVNOgP6H76rtdl+dy00WJ4H72qwY2qYN9+cLSSlsJq6RiNxFjQptNPZce+EezlXeo5dF3b14xMMDFtTt3K+5lsaChfRUBPcul0ndNwScwsJxQkcLzze5/WPZWrJDj1ZCqAp4maj5FCa6rg/HLBUKQig7WNhM9/TNhfFVfXMe2Yn97xxuDXdr7ahmd9vTmCUvysb7pzOU9eO55dLx7VWIVsTe72OmZE+uOXsAUdPCJzY9cFRCzVr4cC/Kf5Gy7X3OfT6xf0Z32mZSy4+5s/vQH5FHXe8dhAnBz1Bnk68dyirH5+kd2SV1vD4xwms/zZV607qHgDX/hsqc+Dg+l6ttWCsH2fyKtl+RqtZiLOw0vyqiKsIdg3m5ZMvD3lr4fljzxPrM56mkstbb+AmVkStwMPgwYbEDX1e/2BaCc4OeosKMqeEeeGg16nU1GFCbxriHRBCPCmEeBLYD/zXZlINYf7xdTKlNQ18fbqAR98/gdEoeWFnMhdKavnjNXE42Nm+lfKcKB8mNh6jJngu6PRdH6jTaTMNHkmmOP5GALz3vwwn34PmRsg8YHEqanV9E3dtOERZbSOv3j6Nm6eHsC+1mIzi6p5P7gd1jc08uTmRy/66i3cPZXLdpCBevmWKtjNsNoxaAnuegzrL24ovGKO1DX91TxpO9voeB/WYsNPZcff4u0ksTjQ753moUNlQSXZVNovDFjImwKNT/yFne2duHHMjOzJ3cKHiQp+usT+1mClhXha1Dne01zM5zFPFFYYJlgaa/w7cAZQApcAdUsp/2FKwoUhyfiXvHMxk7cwwHr1yLB8fz+HBd4+xfncq108OZmakZU/c/eVyv3ICRQmJTpMtO8HVn2JXHzwNntiHzoZPH4ITG6Gx2uJ4woZ96SRkV/DimsnEBXlw/ZRgdALeP2w7a+FcfiUrX/iO1/emc9O0EL559HL+fP0Egr3apKMu/B3UlcHeFyxed9wIN/zdDBRVNRAT6I6+F0WFK6JWMNJlJC+fGLrWgqnHVZRHFJNCPTmeWdpp8M7qcavR6/S8efrNXq9fVtPA2fxKpod7W3zO7ChfknIrtAQJxZDG0kBzKFAEfAx8BBS3bPte8dRnp3F20POzxWO4/7Io7pobwZaTuTg72PHbZWYygGxEeJmWL7+1Ktric4pqi7R4wg2vgYMrbH5Q22Fh5tHJC+VE+Lpw+Th/QGvbPH+MH5uOZNlkmMonx7O5+vk9FFfXt7rkAs1lcQVO1Kq5978E1Za5J4QQzG+xFiwJMrfFXm/PnXF3crzwOAfzDvbq3IEipSwFgFGeo5gU4kVFXVNraxAT/s7+rIhawftn3+dkYfuSo2ZjM0W1Xf9bHk4vRUqYHmG5UpgV5YOU2pQ2xdDGUl/HVmBLy2s7kAp8biuhhiLfnCtk19lCHlw4Gm8XB4QQPLYsml8uHcvzN0/Cx9UwYLKI1F0U2gfxaaa9xaMXi2uLtXRUtxGw6lUtY8l3LLiamcBmhtN5FUSPdGu37aapIeRV1LE72bo9duqbmnn84wRiAt35/KH5re6eLln4O2is0dxIFmJasy9NCq8dfS3+zv78cvcv2ZO9p9fn25rzZecx6A0EugYysbWIrXN33V9M/UXr56ho0Ir4ahpruO/r+1j8/mL+ceQf1DV1nrp7ML0EBzudRUFmE/HBWoLEjjNq8M5Qx1L30Xgp5YSW12hgOjD0vg02QkrJn7aeJszHmVtnXxysotMJ7r9sVOtT54DQ3Ajpe6gInEtxdQNn8ixrY1BUW3QxHTViHqx6Da78k0XnVtY1klFcQ0yHquxF0QF4uzjw/uG++aW7Yve5Iirqmnhw0Wj83CxQtr6jYeIaLeCcddiiayyJCeDhxWNYGjei1/IZ9AbWL1mPj5MPP/76x/zl0F9oaB46bpHUslQiPSLR6/SM8nPFzWDH0czOFcXuDu78ZcFfyK/O58m9T1JeX8492+7hUN4hZgfO5r8J/2XVp6s4lNe+ueKBtBImhnjiaN9NPKsDDnY6rowbwecJearlxRCnT1FRKeVR4HvTCymzpIaz+ZXcPTcCg53lXwSbkHUIGqrwmnAlwMXq5h4orituX6MQew2MXmzRuSbF07FVh4OdjmsnBbEtKZ9iK/bu/+R4Nt4uDsztTQXskv8Dt5Hwv7Va+48ecLTX89Di0bj1sqmeiSjPKN5Z9g43j7uZN5Pe5Fe7f9WndWzB+bLzRHlGAdqDy/QIb3adKTBrVcb7xfPTyT9lW8Y2Vn68ktPFp/nbgr/x0uKXeOWKV2g2NnPPV/dwpuQMAFX1TSRklzOjF64jE9dOCqKyromdZwZ2hrSid1gaU/h5m9cjQoh3gD77DIQQDwshEoUQCUKIjUIIRyFEhBDigBAiWQjxPyGEQ1/XtzZJLfMQemMu24yUnSB0eMcsIsLXhX2pPWd01DTWUNtU265G4XB6CUcyLPPvmkY7mpscd9O0EBqbJR8etU575Or6Jr4+nc+y8SMsymxpxdkbVr+jZSG9d6tWrW1jHO0c+e2M33LP+Hv4OvNr0soHdjayOaoaqsivyW9VCgArJgaSU17HwXTz/9+3x97O3KC51DTV8MKiF1gUtgiAmSNn8u7yd/EwePDU/qcwSiNHM0ppNspexRNMzI7yxc/NwEfHVCvtoYyl3zq3Ni8DWoxhZbdndIEQIgh4EG0mQxygB1YDzwDPtbinStEK5mxGb1IpE3Mq0OsEYwLcej7Y1qTsgKAp4OTJzEgfDqaV0NRDe+SOLS6MRskD7xzl5vUHLFIMSTkVeDnbM8LdsdO+MQFuTA71ZOOhTKtk42xLyqeu0cjKiUE9H9yREXGw8kW4sB++GLgn9zXRa7DX2fP26bcH7JpdkVKuBZmjPC4qhSUxATg76Pm4i5uxTuhYt3AdX17/JbMD2xcyehg8eHjKwxwvPM7mlM0cTCtBrxNMDu19ryy9TrAiPpCdZwtUFtIQxtKYwh/avJ6SUr4tpewcgbIcO8BJCGEHOAO5wEJgU8v+DcA1/Vi/Wz44ksVlz+7irIX++MScckb5ufbKh2p1Guvg/Ndaa4qohQDMjvLRzPmc7oe0F9VpLiaTpXDsQin5FfXodHDvG0fazRcwx+ncCqJHunfZCG719FBSC6s5lN7/TpifHM8myNOJKX246QAQdx3M+gkcfhWKkvstjyX4OvmyLGIZm1M2U15veb2ELWibeWTC2cGOpbEj2Hoqt50/v6HJ2Dqbw15nj5ej+X/zFVErmOg3keeOPMe+tAuMD/LAxdC3ZnrXTgqisVny2am8Pp2vsD3dKgUhxKdCiM1dvfpyQSllNvAskImmDMqBI0CZlNLUHCULMPuoKIS4VwhxWAhxuLCwbx6sRdH+ONvreWnXeYuOT8qt6NcozX5RVQgbb4a/RMBb14O9C8ReB9BaF7Gvh6Igk6VgUgpbT+bhYKfj/R/NpskoufP1Q1TUNZo9t6nZyJm8yk5B5rYsnzASN4Md7x7q3/jHkuoGvk0u4ur4wP4NJJr1gPYz6eN+ydMb1saspbaplo+SPxqwa5ojpSylNfOoLde0+PNNsySajZLbXztI/B++Yuk/dvPk5kQOdeFe0gkdj818jLL6Mk7Xv9+neIKJ2EB3Rvm7dmm1DBZSSp7+/DSbT+QMtiiDTk+WwrPA37p59RohhBea6ykCCARcMN+a26wvQkq5Xko5VUo51c+vb1k/ns4OrJ0Zxqcnckgv6t6NVFRVT35FvVl/+oCQ/BWc/QzG3wBr3oNHzmrN7tBGTI72d+0xrmDKOfdx8sFolHyekMv80X6MD/bgX2snk1ZUzSPvnTB7blpRNfVNxm7nQTg72LFiYiCfncptPxWul3x2Kpcmo2RFfGDPB3eHeyCEzIDET/q3Ti8Y5z2OqQFT2XhmI03GwWv8llKW0pp51JbZUT74uhr4+Jh20/vn9mT2phRz09QQ/NwM/O/QBW5ev5+EbPOWzjjvccwfsRKdxz58fft+QxdCcM3EQA6mlwyp2c3ZZbW8/E0qD248xiPvn6Cm4fvbvK8npZAmpfymq1cfr7m4Zd1CKWUj8CEwG/BscScBBAM2Vdl3zYvATq/jX7tSuj3OFGQeNKWQn6gNzFn+HIy5Ehza9/2fHeXD4fQSGpq6jisU1RahEzq8DF6cyCojt7yOZeNHtJzvy8NLxvBVUr7Z+EJSN0HmtqyeFkpdo5FPjvf9hrH5eA6j/V071UP0iZiVkH8Kirv//7Uma6PXklOdw84LOwfsmh1JKU8h0jOy03Y7vY4V8YHsOFPAZ6dyeX5HMqumBPPMqgm8edcM9v1mIb6uBh7+3/EuU0arcpcgmnzYmP40pXV9dxWa4kWfHB86T+UJLcOWro4P5IOjWSx/fg8phVWDLNXg0JNSaLW/hRAfWOmamcBMIYSz0JzUi4AkYCewquWY2wCbPub5uzmyeloIHx7LIqestsvjEluUQuzI3lW+Wo2CRM0y6KLH0awoH2oamtuP6OxAcV0xngZP9Do9nyfkYa8XLIq+OIHtjjnh+Lo68LevznU6Nym3Anu9IKqH/kDjgz2IDXRn48ELfQo4F1fVcyijhGXjR1pniE30Cu3nALqQLgu5jCDXIP594t9UN9q2J5Q5qhqqyKvOaxdPaMs1kwJpaDbyk3eOMsrPlf+3MrZ1n6ezA8+smkByQVXrcKi2nMoqZ+fpCq4N+jVl9aU8tucxjLJv859DvJ2ZGOLJV4lDJ66QkF2OXif466oJvH33DEqqG3hyc2LPJ16C9KQU2n47Oz9+9AEp5QG0gPJR4FSLDOuBXwE/F0KcB3wYgIZ7P1oQhZSwfndql8ck5pQT5Olk9RbYFpOfCP6xXe6eEeGDEN3HFUwtLqSUfHYql7mjfPFwuvh5nB3s+PFlo9ibUtxpndO5lYz2d7Oo0d/q6aGczq0wWz3bEzvPFiIlLI4O6PlgS/AMgaCp2iCiAUKv0/PbGb8lpSyFn2z/Seuks4Gibc8jc4wP8iDSzwUHOx0v/nAyzg7tg8ULxvixdmYo/9mTxv4OLsm/bzuLp7M9j16+kEenPcq32d/yeuLrfZZ1cbQ/J7LKKajsT76K9UjIKWe0v5ZMMjvKl7vmRPBtctH30lro6Zsuu/i9X0gpn5BSjpNSxkkpb5FS1kspU6WU06WUo6SUN0gprVcN1QVBnk5cOymIjQczya8w/8c5uEHmAqguhICYLg/xcnFg3Aj3buMKJbUl+Dj6kJBdQVZpLVeNH9npmB/OCCXA3cDft51t96SflFNh8XzpayYG4mqw4/W96RYd35btp/MJcDdYNNvAYmKvgdwTUDJw9QPzg+fzp7l/4kj+ER7e9fCAVjqbMo/a1ii0RQjBy2unsOm+2V2mV/92WTSh3s48/L/jrdbn0cxSdp4t5N75kbg52rN67GqWhC1h3dF1rdfsLSZLdSgUskkpScguJ7ZNC/XV00Nx0Ot4c9/3b8BkT0ohXghRIYSoBCa0/F4hhKgUQnSfBzlM+MlCzdR+dNPJThWf1fVNpBVVD248ASCga0sBtLjCkYxS6pvM+4JNlsLWU7nY6QRXxHR+Gne01/OTy0dxKL2Ub5O1wHRBZR1FVZYH2d0c7blhajBbT+Z2qWTN0dBkZPe5QhaOC7Du/ONWF9LAWQsAyyKX8cSsJ/gu+zse/+7xAbuuqedRkGvXNR6jA9yI66YJoLODHS+u0brvXvfSXv61K4Xntp3Dx8WB22aFA5py+d3M3yGEYNO5TV2u1R3jRrgR6OHI16cHXykUVNZTVNXA+DYPJH5uBn4wYSSbjmR97ybGdasUpJR6KaW7lNJNSmnX8rvp/SDdKa1LmI8Lv1sew+5zhWzYl95u35m8CqSk3RPEgFKQpP3sxn0EMCvSh/omY6dhKqA9BZlaXGxLymNWlA+ezuaLxW+cFkKQpxO/+fAUb+xLb+1o2V06akdunx1Os5S8tf/iE1ZdYzN/3JLEizvPsy+luFNmx4G0Yqobmlkc7W/xdSzCKwwCJw9oXMHE9WOu5774+/gs7TOOF/R9wllvSClPIcIjolPmUW+JC/Lg84fmcUVsAM98cYZvk4u4b0FUu9oEb0dvLg+5nC2pW/pkDQmhxbX2JBcNei8kU8ZVR2V566wwquqb+PDowA2TGgrYfiLMMGDtjFAWR/vz9OdnOJN30QAyZR4NmvsoPwlc/HrsZDo90huDnY4PjnT+461qrKK+uR5nvScphdXMjuq6n5DBTs/fbozHw8me33+SyIMbjwG9UwphPi4sGufP2wcyqWtsRkrJLzed5D970vjrl2e5+ZX9THjyKzYevFjTsP10AQY7Xbey9ZmYlZBzbECzkEzcEXsHXgYv/n3i3za/VpOxiXMl57p0HfUWT2cHXlwzmb9cP4EfTBjJ2plhnY65bvR1lNWX9Xk86aJof2obmy1q1WJLTmWXI0Tn3l6TQr2ID/Zgw950pJQUVtazbnsyG/amt05dvBRRSgHtqeWZ6yfg4WTPgxuPtZqLiTkVeDrbM9Kjc3uHASE/oUfXEYC7oz1rZ4bx4bFsUjsExkyFa2WV2meYEtZ9pfDMSB8+e2geW346l9tnh3PbrLBeB9nvmBNBSXUDm4/n8NKuFDafyOHRK8dy7PElvHb7NKaGe/Hk5kSS8yuRUvL16XzmjvLFycEGFePxq0FnDwdsf2PuiLO9M7fH3c53Od/Z3Fp4K+ktCmsLWRxqWZNDSxBCcOO0EF5cM9ns/82skbMIcA7gw/Mf9mn9mZE+ODvo2THILqSE7AoifV3MVmnfOiuclMJq7n3zCHOf2cHft53jic2JLF+3p8tiv+GOUgot+LgaePaGeJILqljy92/4/FQuiTlakNmqfm5LMTZD4ZkeXUcm7lsQhYNexz+3t2/tYCpcyyu1x14vmBBsmSssLsiDJ1fE8oeVcb2TGy3GMTbAjb9+dZa/fnmWlRMDuf+yKLxcHLh8nD/rbp6Ei8GOh987TmKOFvxeZK2so464jdAK/469DbX9b8PRW1aPXW1zayGjIoMXjr/AwpCFLApdZLPrdESv07Ny1Er2Zu8lr7r36aWO9nrmjvJl++n8QZ1il5hT3uWwpeXxI/F1NfDN2UKunRTEjl8sYP0tU6iqb+KGf+/juW2d07gHgk+OZ9us+E8phTYsGOPHpvtm4enswI/fPsqp7PJeuU6sSkkqNNVZZCmAFhi7dXYYm0/kcC7/Yk+n4jrNUkjNE8QFeQxI/yYhBHfMCaewsp74YA+euX5CO8Xq7+bIn64dT0J2BT9++wiguRJsxqz7tdGjR/o+qL6v2NpaMEojT+x9AgedA4/NfGzAH2CuGXUNEskn5/sWzF8U7U9OeR2ncy3rQ2ZtiqrqyS2v6zL4brDTs/knc9jza20UbKSfK1fEjmDbz+ezNHYE/9qVQmn1wDb3K6is4xfvneC179Jtsr5SCh2YEubNpz+ZwxNXxxDo4cjCcTZ6gu2J1syjrtNRO/Kj+VE42+v5x9cXn15MlsLZHPreZK4PXDs5iMeXx/DKbVPNKqKlcSNYNSWYCyW1jA/yIMBMB1arMWI8RMzXhvA0970NR1+xpbWw6dwmjuQf4dFpj+LvbEPF2gUhbiFMHzGdj89/3KdiNtN418GayNZanNpNMkmgpxP+bu3/Pp0d7Hho8Wgamo18OMB9nN49eIEmozQb57EGSimYwU6v4445Eez9zSJmRfn0fIItKEgCoQM/y2c/e7s4cOfcCD47lUdijpZRUVxbjE7oaah3ZGr4wCkFg52eu+ZG7zuYVgAAIABJREFUdPoyteWJq2OIDXTnxqnBthdo5gNQkT3g6amgWQtrotfwXc53fXKzdEV1YzXPHXmOmSNncs0omzUV7pFrR19LVlUWR/KP9PpcfzdH4oM92DFI9QqmzKPYPtTHRI90Jz7Ek/9ZqW28JTQ1G3nnQCbzRvsS4evS8wl9QCmFoUp+InhHgb2ZYfXdcPe8SNwMdry0U8u2Ka4rxknnDuiY3EOQeaBxc7Rn64PzuKUl/92mjL4CfEbBvhdhEPzXyyKWAfBV+ldWW3NbxjaqGqt4YOIDgxP3amFhyEIcdA7syNzRp/MXjPXn+IWyQZmxkJBdTriPM+59nMB387QQzuVXcdRMOrgt+Pp0PnkVddxqw++MUgpDlfzEXrmOTHg42fPDmWF8npBLRnG1ln3U7E6ot3O3T+2XPDodzPyxNo8i2Xo3ZksJdQ9lnPc4tmVs6/KYqoYqPj7/MZtTNrM9YzuJRd333tmSsoVQt1Di/eItF6Q0w+rpuc72zswMnMnOCzv79MR82Vg/jBJ2J1s2WtaaJOSUE9tNMV9PLI8PxNlBz//62TbeUt7Yl0GQpxMLx9nOVaiUwlCkvgpK0yzOPOrIHXPC0esE//k2jaLaImpqnZg6xKyEQWHSLeAfA58+NCiZSEvClnC88LhZF1JicSI3brmRx797nMf2PMbPdv2M1VtXsyV1i9m18qrzOJh3kOWRyy23Esqz4JWFsP5yqyuGy0MuJ7sqm3Olvc/GiQ/2xMvZvnXWw0BRUdfIhZJa4vpRnOpqsGNFfCCfnsilsouZJNbifEEle1OKWTMjFH1/5o30gFIKQ5FCbUi6pZlHHQlwd+TaSUG8f+QCuVUFNNQ7DznX0aBgZ4BrXtJ6Sn3x2wG//BVhVwCwPXN76zYpJW+ffpu1n62lobmB9UvWs/Xarby3/D2ivaN54dgLNJoJjm9J3YJEsjxqufmL5Z2CsgsX3zfWwf/WQlO9ZjW9+0Pt4cNKXBZyGQLRp7bhep1g3mg/dp8r7NRqxpZkl2oNC8N8nPu1zk3TQqhtbObTE7nWEKtL3tqfiYNex03TQmx6HaUUhiJpu7WfI8b3eYl750dS19hESV0xxib3AQ0yD2kCJ8G8X8CJd+Ds59q24hQ48S7U2zYtMtwjnNFeo9vFFV44/gJ/Pvhn5gbOZdPVm5gVOItQ91CifaJ5aPJDZFdl8/6599utI6Xk05RPmeQ/iRC3DjcIYzPsegZeng/PT4GdT0NjLWz9uVbZfd16WPUaFJ2FT+63WnzF18mX8X7j+zxL4rKxfhRVNbTO7xgIcss1pdDf4tSJIZ6MG+HGOwczbBZwrm1o5oMjWSwbPwJfV4NNrmFCKYWhRnMjHPoPhM/Tevf0kVH+biyIdgJhxIAno/2tMLjmUmH+oxAQBx/fD+smwfOT4aMfwcc/tnkQ+oqwKzhWcIyCmgK2Z2xn/cn1XDvqWtYtXIeno2e7Y2cHzmbaiGm8fPJlahovFir9//bOPK6qanvg38UoIA4oIioOOKDmiDibOablnA2OWVk2WGn9Kptfs/Z6r/GV9nLIyrLUV+ZQauac8zxPOAOCoghcBIH9+2NfcGLmXg7E/n4+fO495+yzz+LCPeustdewL3Yf4XHh9A2+wUpIiIbv7oKV70GTu6FRP1g1CT66BXbMgttehIZ3Qt2u0OMNHYm1+l8O+926BnVl3/l9BYqw6txAl3IpShdSZJwu2hhYPn/BHDciIgxvW5M9Zy6x7aRz3JJ/HogmPjmVe8OcayWAUQrFj/0LdOhku8cLPVXfUN0Yp07F6k71QZY43Dxg0BTwKKsjku74AG59Tn/2W6Y79dK317odheKrXV/x8tqXaVq5abZJZyLCuNBxxF6O5dt932buX3h0Ie4u7vSq3evq4LRUmN4LTm6A/p/ZLYJpMGohlK+h+3rfNuHq+A5P60zvFe/AwmcgtfCRP92CugEUqBZS5bKeNKtRnpUHC9Z3vSBEXryMq4vg71v4J++7QmvgW8bNaQllC3ZG4O/rSdtg54fI31zsw2AtG6dAxdrQoHehp/KvoJ+EBjTNe65DqaFqU3hm99Xt9HTtXlnyMtRsr/NDds2GlROhZge4YxJ4Fd4FF1whmHoV6jH74Gz8yvjxYZcP8XTN/qbU3L853YK6MWPvDJJSk7iUconfj/9Ol6AulPe8ZoH0yB86C/6embqPRAZ1boVHV988sQgM+lL3s173CUTvh3u/gbIFj2qpU74OtcrVYsWpFQxpOCTf59/WwJ/PVxwhznalSJpaRcZdJsDX0yEPTD6ebgxpHcT0dceJjEsqtPVxLfGXr/DnwWiGtXHuAnMGxlIoTpzZCqc2QptHs22/mR9ikvRTV5/GRinkiouLth48y8GcUTC1m3YnufvA7jnwRXs4/IdDLtU3uC9u4sa/b/s3VX2q5jr+6dCnSVfpzNw3k+Unl+Pv5c/IxiOvH7T9W11Rt2GfvAvi4go934LB0yBiB0y5FQ4szudvcxURoWtQVzZFbSI+Jf/rMxmhqWuOFI21EHUpiaoOLHZ5f/vaKKUc3phn6d6zpKSm0695NYfOmx1GKRQnNkwBD19oOcIh00XbohGESl4WZWWXNMpWgbu+hHOHIf4s3PUVPP4XPLIcypSHWYNh/ReFvsxDTR5i+b3LCasalqfxdSvUZf3Q9WwbsY1V961i/sD5tKzS8uqAhBg49LuuCOtagCfspnfDw8vApzLMHgpzH4LEguUMdA3qSmp6KqtOr8r3uS2CKlLey73IXEiRFy8TWMFxT/RBft70bBzAD5tOOrRHxIJdEVSv4EVozQq5D3YARikUF+KjYO/P0HI4lHFMEb4YWwyVvCrh7mJRf+mSSN1uMHYjPLUFmt2rLYhqLWHMKqjXE1a8q2/ChUBE8Cvjl69zXF1cs89H2PUjpKdCi0I8TFRtCo+sgK6vwL5f4YN6MKkmfNwMfhgGKYl5mqZFlRYE+QYx5+Cc3AffgKuL0LmBPysPRjs9NFUpRWTcZQIdXHPrwY51uGC7wvwdjqmHFJuYwtrD5+jXvFqRZa0bpeAsjq2Bhc/CrHvhiw76NTY8+/HrPtVf7DZjHCbCWdtZ/L1ybtBjyAL/EPC4oa6MexnobQ/vXP2BNXJlhVLadVSjNVQppJvQzQNuewEeWwtdXoRmQ6B6Kzi4CDZ+macpXMSF+0LuY1v0tgIlsvVoVIVzCSnsOO3cshFxSVdIupLmUEsBoG0dPxpW9eXLVeGcT8h7m/mT522kpN5cUPD3PVGkpiv6Nb+5r7qzMErBWSwcDzt/gPgIqBAEJ9fD5I76y5V+wx///FFdwbPlCKjkmM5ZoC2FAG+Lqrz+HalcH0JH6gil2GNWS6M5s00nOzrI5Qho5dLlRbjzn3DPDKjfC9Z9DEl5u1EPqDsAT1dPfjzwY74v3aVBFVxdhOX7nVs19Wo4qmMtBRHhlT6NOHMxiUFf/MWR6NwTBP/Yd5bOH6yg2ZtLGPbVBj5cdogFOyPYdfoiv2w/Q7C/T5GW8DdKwRnEhsP5IzoW/LG1MOxHeGID1OoAv70A3w26PlHqj3+Aqwd0e9WhYkTbovH3NpaCQ7ntRXBxgxXvWSfDqc0Qvko/TGydDm5eOuTUWXR7FS7HwV+f5Wl4hTIV6F27NwvCF5CQkr+s6fLe7rSuXZHlTu7G5qjEtay4tb4/s8e0w5aSyl1frOOnzaeYsuooT36/jdFfb76pR/m0tceoWq4MQ9vUJC7pCp/9eZinfthO//+sY9PxWPo2KzrXEZiQVOeQEaVS75rWiOWrw/C5sG2mdit9dzcMn6PLEexfAF1f1V3CHERKWgoXki9YUmP/b025QJ1DsvZD6PAUBDYr2uvv+F5HRV1L86EOW4fKksBm0GQwbJgMbR/NU9jqkIZDmH90Pr8e/ZVhjYbl63I9GgXwzqL9nIq1EeRXuBIU2eGoxLXsaFmzIj8/0ZGHvt7MC/N2AVC9ghdnLiYxY91xxnatB8DBqHjWh59nQu+GPN5FewlsKamcjLVx4ryN6EuX6d+iulNkzA5jKTiDw0t12esbXUEi0OoBuHs6nN4M3w2GJS9BuerQfqxDRcgIRzVKwQl0HKdzForaWojcqRPNat8KoxbAwCnQ402HW5hZ0vUV3Qlwzb/zNLxJ5SY0qdSEHw/+mO/SDxmtWZ3pQnJk4lp2BPl5M//Jjsx9rD3bX+vJuhe70aNRAFNWHc0sEz5z/XE83VwYck09I28PNxpWLUevW6oysn1tynsVbaCIUQqOJsUGx9fo+v3ZcctA7auN2Ka/6D3eAA/HPhHF2IxScBpeFaDtY3DoN4g5WDTXtMXCjyPBy0/XLqrTGVoMhU7jdcays6lUV69bbJ6mrZU83Ojva3gf4XHhbI7anK9L1ansQ7C/D8ud2HjHkYlrOeHt4UZYbT8q+ngA8HyvEBKSU5m86ihxtiv8vO0MA1pUyzxeHLBEKYhIBRGZKyIHRGS/iLQXET8RWSYih+2vJbOC2/G1+omqfo+cxzUeAENn66fOJnc7XIyzNv2UZaKPnETrh8GtTJ797Pkm7gxM7Qmz7oHFL+gKp5ci7FnHFv1Nu7+uo5x+eRy+vw8u5VwVtHft3lT0rMg3+77J96V6NgpgQ/h5p5WjdnTiWl4JqerLoBbV+XrdcT778zBJV9IY1aF2kcuRE1ZZCp8AvyulGgLNgf3Ai8BypVR9YLl9u+RxZJle+KvVKfex9XvqjFIXx/8ZMiwFE33kJHwqQ4vhOkcg3nEtNjPZ8T2c3qRvvDtm6ZpGd/4Tglo7/lp5xacyPLAIek/SlXw/bwM/P6bzay7H3TS8jFsZhjYayqrTqzh84XC+LtW9UQBX0hSrDzmn8Y6jE9fywzM9G5CuFFPXHqNNbb8c+0NbQZErBREpB3QGpgEopVKUUheBAcBM+7CZgHVNZwuKUno9Ifg2HdduIdG2aDxcPK6vj2NwLO3H6qq2eYzhz0QpSDyf85g983TNpcfXwkun4eUzEPZQwWV1FBkd7B5fByF36PLjcx6AD+pD+Mqbhg8NGYqXmxdf7/06X5cJrVmBCt7ufL/pBDtOXeRK2s0x/AXFWYlreSXIz5thbWoCFDsrAayxFIKBGGCGiGwXkaki4gMEKKUiAeyvWTrDRWSMiGwRkS0xMQXMLD13GOY/qRuOOJLzR+DC8eujjiwiOkmHo1rZu/dvT6W60KgvbJmWv4Y1S16GD4LhszBY/PzV/hkZnN0HMfuhiT3MVCTfvbqdTqW6uhLr80fhwd91Ls6vT92U+VyhTAXuqn8Xi8MXE5mQ9yY0bq4ujGhbi3VHzjPw83U0fWMJ42ZvJzm18OUjMhLXrHAfZfB/vUKYeFdTejdxXMSho7BCKbgBocBkpVRLIJF8uIqUUv9VSoUppcL8/QvoW714UmeBrv+8YOdnx2F7/936PR07bwGItkUb11FR0GGcdp1s/zb3sQAHFsGGL6DBHboa7rZvYWY/nXeQwd7/gbjodafijqsb1GoP/T7R36uVE68eUwpObeL+enehUPleW3iuVwgbX+7O58NCGRxag/k7Injmxx2kFbIERkY4ajWL3EcA5cq4M7SIqp7mFyuUwmngtFJqo317LlpJnBWRQAD7q/NCD+p1h4Z9dbmCOMfUKOHyJV1quXKI/rJbTIwtxiSuFQVBrbWbZ82HWfrVr+PiKd3YJ7AF3DsTRsyFF45C+Zqw9BXdNU0p7Tqq07lQZayLnNqdIPR+/aAVsUNnP88bDdN6Um3WMO4I7Mi8w/OIS87lM7qBgHJl6NMskHcHNeWVOxuxeHcUr83fU6gOZxmJa1ZaCsWZIlcKSqko4JSIhNh3dQf2Ab8Co+z7RgHznSpIr3dBpcNSB8R4nz8K03pC1B7o/Fzh5yskSinO2s6acNSiotc7kBgDq/6Z/Zi0VJj3sL7x3zND94sGXWOpxz90EuPOHyByh86Id2aGsrPo+RZ4V9Zd7KZ0gr2/QNvHIekCD2xfSFJqEj8c+KHA0z/SOZjHbqvL9xtP8tGy/NdVyiDTUnBS4prTuHxJW5RrP4afRsGOgn+WOWFV9NFTwCwR2QW0AN4DJgE9ReQw0NO+7Twq1oZOz2hT/diags2Rkqj/8b/qBglnYeTPurKmxSReSSQpNYkqXkYpFAnVW+kY/o1Tss9b+OsTOLUB+n0MfsHXH2syWId6Ln9bu5Nc3HQrzZKGV0UdIRVzQJfwHr1MNyd6dBUhFerQ2ZbEdzsmY1s1SUdTpeU/3HRC7xDuCwvi0z+PMGfLqQKJWRSJa5nYYuHUpsLPc3YffNwEvumvy+JEbIdk5/SztkQpKKV22NcFmimlBiqlLiilziuluiul6ttfY50uSMdxUKGmXuzL6z+oUrqi6dQeurTwnFHgG6jLDgffViAxlFL8dPAnHvj9AbZHby/QHNcSbdOeN2MpFCHd/6Eb8vw24ebErqSLsPYTCOmjexfciAj0mggJUXrRum538M5fae1iQ+OBeuH50TVQo5XeV74GPPgbj9TsTRzpzNn6H9069KMmOis8Hy5cEeGdQU3oWK8SL/+8m43huURxZUFk3GWqODtxTSnYNQf+E6a9CAufLXhgiy0WfhiiQ92Hz4UXjsH4XbrkiBMo3RnN7l7Q+30d6ZHH9H1W/ROWvaZdTx2ehuHzYMxK8KtTIBGupF3hrQ1v8faGt9l7bi+jfhvF+5vev65Re37JTFwzawpFR1l/6PoyhK+AAwuvP7ZhMiTHQdeXsj8/qLW2GOBq1FFJREQvPHuWvX6/uxct+vyHtlXb8nX1uiQPnqprKq36J3zcVOc75JIMlzmVqwtfDGtFUEVvHvtuKyfO563XQwZRl5KcUggvkwsn4Pt74X8Pa49Em0e1sp92u45OzA9pqTrkNz4S7vtOB7E4+YHBFMRreKeuG7/qfd1gJahN9mN3/ggr34Pmw2DgF/oLUAguXL7AuBXj2B69nUeaPsJDTR7i0+2f8t3+71h5aiUfd/2YEL+Q3Ce6gYy6Ryb6qIhp/fDVgocBt2g3UdIFHW3UqJ9uZJMTvd7TVmej/kUjrwWMaTaG0UtH87N7GkOGz9E3yU1f6dLx+36F256Hdk9cXXPJhvLe7kx7oDUDP1/H0P9uoGvDKjSuVo42tf2oH+Cb47mRFy/TyBmlqG2x+uFy03+1C7DXRP007+IKwV3gl8fg83b6fb3u2u148aTuj207Dx2f1p6LDJTSa57HVsGAL4oscVEKs4pvNWFhYWrLli2Fn+jyJb0wBrrU9Y0VJ9PT4cRa+PYuqNkORvxPNyQpJJ9u+5Tpe6Yz6dZJ9K7TO3P/lqgtTFgzgfiUeCZ2mkj3Wt3zNe/U3VP5ZNsnbBq+CS+3EraYVtKJOQjTe4OnLzy0RD8hrv4AHlsHVZtYLZ3lKKUY+dtIYmwxLLxr4dWugLHhsORV3dCnXA1o/4SOZvLM+Qa/9UQsHyw5yN6IS8RfTsVFYPaY9rSpk/XTtFKKxq8vYXjbmrzat3Hhf6G0VN1X/eBivR6UEg8thkGXl26uSXXhuC6LcngZXLy2j7PoNRivirrMfrWWOu9l0bM6Y77t43ptxoGIyFalVJb9YI1SyODkRpjRW9chqt9TZ2qGr9AKQ9kTZiqHwOiluiCaA3hh1QvsOb+HxXfd3Cw9xhbD+BXj2XVuF2NbjOXRZo/mORHtvY3vsTB8IX8N/cshchryyZmtMLM/lA+CuNNQr5uuWWQAYPXp1YxdPpa3OrzFoPqDrj949E9Y/S84sQ48y+sbbMM7oWb7HPtPK6U4GWtj2FcbKevpxsKnO+HuerN3/KIthRZvLePVPo14+NbgLGbKheQErQQid+jQ2+NrtDXo4g4NeulqsgG5KBultBKM2gUV60DlBlpJzLpHWww93oTNX+lk2C4vwa3/p60NB5KTUjDuowxqtoXOz2s30u6fwKcKhNypexy4uOuyFc2GOEwhAEQkRlDNp1qWx/y9/Zneezpv/PUGn+/4nNrla9O7du8sx95ItC3aRB5ZSfVWMGSW/pKnXdGNeQyZ3Fr9Vhr5NeLLXV/SN7gv7tfe7Ot20z+nt8Jfn2pLa+NkKFNeu217vZvlDVJEqFXJh3/0a8yYb7cyY90xxnS+uYthRvOeYH+fm47lysWT8HUf/Qr6hl6/ly73Ubdb3ntaiOiM8GtL61dpBA8v12sRvz0PZQPg/vk6X6WIMUrhWjq/oH16/g2hWqhTCtVdS2RiJO0D22d73NPVk3c6vsPBCwf5dNundK/Z/aq5nQMxthgTeWQ1wV1gxDx9A8ntybGUISI8Hfo0j//xOPMOz2NIwyE3D6rRSif4JSdoi33ffK0cUpOg78f6xqqUdsds+q+O7ElP5Xa3Mnxf+Rbm/nELkSFPEBhwdV0tLukKE3/bT/OgCnRpkM/vR9wZnXl+OQ6G/KC7KDrwAREA3wB4cLEuhth4gGXJi0YpXIurm2N73ebAlbQrxNhiqFY2a0shUyQXV8aHjmfs8rHMPTSXoQ2H5jr3WdtZ2ga2dZSohoJiwVNeSaFjtY60CmjFl7u+pH/d/ni7Z9NPxLOsXqRv1E+749Z+qBPkOj+nay3tnqM/Z79gvbhri6Xt0RV0cFlG6uT/QIOe0OweaHAHnyw9jGviWd4f3AKX7MJRk+Nh+3ew9WvwLKetgJrtdK00Wyzc/4u2BJ2Fhw+0ecR58+cBoxQs4qztLApFoE9grmNvrX4rrau2ZsrOKfSv2x8f9+xN3/iUeM4lnaOqT/ErtGUwZCAijAsdx/2/3c8PB35gdNPRuZ/U/XXtc1/zL9j1E8Sdgm6vaZ/7NettrmmpzFvwC+e3/I97j26kwqHfUOLG6yqV1z2BH9Hlv9td09ZUKZ0zsXGKTgqr0QbSUmD5m/q4R1mdnOpMhVBMMErBIiITdUx2YNnclYKI8EzoMwxbPIyZe2fyRIsnsh27OHwxaSqNbkHdHCarweAMWlZpSecanZm+Zzr3hNxDOY9cfPIi0Pcj7cI5shyGfK8XoW/E1Y2B/QfzhW9z2q04REfXA/Rw38OFVHce7NkaryOL4I839cJwRnb5+s9h9T91OHCn8Vdv/pci4MgfULUZVGvh2A+gmFK6k9csJFMp5MFSAGjq35Tba93O13u/Zu+5vVmOUUox9/BcGvo1pHEl48c2FH+ebvk0l1IuMX339Lyd4OIK93wNzx3KWiHYcXURnupen9/HdyE5qBMvxd+Nf5/X8OrwiM4xcnWHX5+2V3LdrEtHNOqno8SutQbKVdOhsaVEIYBRCpYRkRABkC83z/jQ8Xi5eTFk0RBeXPNi5hwZ7Du/jwOxBxhcf7Dpo2AoEYT4hdAvuB/f7PuGE5dO5H4CaIshjz3Na1f24dvRbVj1fBfuCQvSO8tVg9vf1uGk6z6BuQ/pff3/U+iE1L8DRilYRFRiFJXKVMLTNe9FuYLKBbFw0EJGNxnNHyf+oN/P/Vh+Ynnm8bmH51LGtQx9gvs4Q2SDwSk8G/Ysnq6eTNw4sVAlsbMjI1z1OkJH6QXqP/6hS0jc/bXjo4lKKEYpWEREQkSukUdZ4evhy/hW41kwcAEN/RoyYc0EdsXswnbFxuLwxfSq3Qtfj5yzQA2G4kRlr8o82fJJ1kWsY/nJ5bmf4AhEoN+nUK469J54tXifwSgFq4hMjMzzekJWBJYN5LPun+Hv5c9Tfz7F1N1TsaXauLtBFlU4DYZizn0h9xFSMYT3NxeuGGS+8KsDz+y1PAS0uGGUggUopQqtFAD8yvgxucdk0lQaX+3+irrl69Lcv7mDpDQYig43FzdeafcKUYlRfLT1I6e4kbLErCHchFEKFhB7OZbktOQ8haPmRu3ytfm066d4uXkxsvFIs8BsKLG0rNKS4Y2GM/vgbF7/63WupOe/CY+h8Jg8BQuISowC8h6OmhuhAaGsHbIWD9fCV241GKxkQusJ+Hr4MmXnFKISo/iwy4dmjayIMZaCBUQk6lDSgiw0Z4dRCIa/AyLC2Ba6guqWqC2MWDyC8IvhVotVqjBKwQIiE/KXuGYwlDYG1R/ElJ5TuJh8kSGLhrA4/Oby8gbnYJSCBUQmRuLt5p17Wr/BUIppG9iWn/r+lBl6PWnTpKJbgC7FGKVgARmRR2ZR2GDImQCfAKb1msbwRsOZtX8Wn23/zGqR/vaYhWYLiEiIcEjkkcFQGnB3cWdC6wlcTr3MV7u/oqpPVe4Nuddqsf62GKVgAVGJUTSpbPr1Ggx5RUR4td2rxCTF8O7Gd/F09aSZfzO83LxIV+kciztGeFw4ZxPP4uPuQ1mPspT3LE+gTyA1ytagincVXB3c0vLvilEKRYztio0LyRccGnlkMJQG3Fzc+KDzB4xeMppX172a5RhPV0+S05Jv2u/l5sX/tfo/7g25t1i6bZPTkolPiSfxSiJp6WnUKV/HMjmNUihiomyOzVEwGEoT3u7eTOs1jY2RG7Gl2khKTQKgVrlaBJcPxq+MH+kqnYQrCVxMvsiZhDOcSTjDsuPLeGfjO+w6t4vX2r3GxeSLzNgzgwVHF9C+WntebPMi/t7+TpFZKcWxuGNsiNxAREIE7au1p03VNri7urP33F5m7J3BshPLSFfpmed0rN6RN9u/SYBPQOYcRy4eoZxHOap4V3GqwhCrVvNFxBXYApxRSvUVkTrAbMAP2AaMVEql5DRHWFiY2rJli/OFdSDrzqzjsT8eY2bvmYQGhFotjsFQKkhX6UzZOYXJOydTvWx1ztrOgtI33/UR6/F09WR8q/Hc3eBuXCR/8TdKKZYcX8LmqM2kqTRS01NJSUsh/ko88SnxnEk4w7mkc4C2dlLTU/H18KWmb032nt+Lr7svA+sPpJZvLbzdvYm2RfPlri9xc3HjmVbPEJsUy4LwBZmlxf3K+NHQryED6g7gzuDse0rkhIhsVUqFZXXMSkthHLAfyIjLfB/4SCk1W0SmAKOByUUhiO2KLfsesQ5YaDKgAAAJZElEQVQmo7mOcR8ZDEWHi7jwRIsnaFK5Ce9vep/B9QfzUJOHqFa2GsfjjvP2hrd5e8Pb/Hr0V15r9xohfiEApKWnsS16G5W8KhFcPvimefee38ukjZPYEbMDXw9fvFy9cHVxxcPVg7LuZfH18KVdYDvCAsJoG9gWf29/NkRsYNmJZRy6cIjnwp5jcP3BlPUoe928PWr14NW1r/LW+rcACAsIY9Qto0hJS+FA7AEOxB4gJinGKZ+VJZaCiNQAZgLvAs8C/YAYoKpSKlVE2gNvKKV65TRPYSyF5LRklh5fyk8Hf2JHzA5aBbRiWMNhdKvZDTeXwunKPef28O2+b1FKMaLxCJr5NwMg8Uoib294m9+P/c6WEVsKfR2DweAYlFL8evRX/r3l31xKucTQhkNxd3Vn0dFFRCdFA9AlqAsPN32YAO8A1p5Zy+rTq1l5aiUVy1RkXOg4BtQd4NDF7LT0NDZEbqB2+dpUL1vdYfNCzpaCVUphLjAR8AWeAx4ANiil6tmPBwG/KaVuCtERkTHAGICaNWu2OnEij92armHlqZW8vu51LiRfoFa5WnSu0Zk/T/7JmYQz+Hv5U6tcLXw9fLXmd/PC280bL3cvutfsToOKDbKcMz4lnvUR65l9cDabozbj6+4LoveHBYRRvWx1lp5YSlJqEh2rdWRKzyn5lttgMDiXuOQ4Ptn2CXMPzcVVXOlUvRN96vYh/GI43x/4nrjkuMyxVX2qcmedO3m46cMlrj5TsVIKItIXuFMp9YSIdEErhQeB9TcohcVKqaY5zVVQS+FY3DE+3vox9zW8j3aB7XARF9LS01h9ejWLji3ifNJ54lO0PzBjMSs5LRlB6Fe3H2NbjKWCZwV2ndvFtrPb2BC5gV0xu0hTaVTxrsL9je/PbIk579A8vtn3DfEp8fSu05tB9QbR3L95sYyAMBgMmlPxp/B286aSV6XMfbYrNuYfnU9yajKdqneiboW6JfZ7XNyUwkRgJJAKlEGvKfwM9KII3Uf5JS45jmm7pzFr/yzSSUcpRZpKQxAaV2pMh2od6Fi9I838m+Hu4n7duWnpaaSTftN+g8FgsIJipRSuu7jdUrBHH80B5l2z0LxLKfVFTudbEX0UlRjFt/u+xdPVk9CAUJr7Ny9xpqPBYCjdFNfooxuZAMwWkXeA7cA0i+XJkqo+VXm+9fNWi2EwGAxOwVKloJRaCay0vw8H2lgpj8FgMJR2TJVUg8FgMGRilILBYDAYMjFKwWAwGAyZGKVgMBgMhkyMUjAYDAZDJkYpGAwGgyEToxQMBoPBkImlGc2FRURigPxXxNNUBs45UJyipKTKbuQuWozcRUtJkruWUirLrkIlWikUBhHZkl2ad3GnpMpu5C5ajNxFS0mV+0aM+8hgMBgMmRilYDAYDIZMSrNS+K/VAhSCkiq7kbtoMXIXLSVV7usotWsKBoPBYLiZ0mwpGAwGg+EGjFIwGAwGQyalUimISG8ROSgiR0TkRavlyQsiMl1EokVkj9Wy5AcRCRKRFSKyX0T2isg4q2XKCyJSRkQ2ichOu9xvWi1TfhARVxHZLiILrZYlP4jIcRHZLSI7RKRo2yoWAhGpICJzReSA/X+9vdUyFZRSt6YgIq7AIaAncBrYDAxVSu2zVLBcEJHOQALwjVKqidXy5BURCQQClVLbRMQX2AoMLAGftwA+SqkEEXEH1gLjlFIbLBYtT4jIs0AYUE4p1ddqefKKiBwHwpRSJSUJDAARmQmsUUpNFREPwFspddFquQpCabQU2gBHlFLhSqkUYDYwwGKZckUptRqItVqO/KKUilRKbbO/jwf2A9WtlSp3lCbBvulu/ykRT1AiUgPoA0y1WpbSgIiUAzpjbyGslEopqQoBSqdSqA6cumb7NCXgJvV3QERqAy2BjdZKkjfsLpgdQDSwTClVIuQGPgZeANKtFqQAKGCpiGwVkTFWC5NHgoEYYIbdZTdVRHysFqqglEalIFnsKxFPgCUZESkLzAPGK6UuWS1PXlBKpSmlWgA1gDYiUuzddiLSF4hWSm21WpYC0lEpFQrcAYy1u02LO25AKDBZKdUSSARKxFplVpRGpXAaCLpmuwYQYZEspQK7T34eMEsp9T+r5ckvdlfASqC3xaLkhY5Af7tvfjbQTUS+s1akvKOUirC/RgM/o929xZ3TwOlrLMm5aCVRIimNSmEzUF9E6tgXhIYAv1os098W+4LtNGC/UupDq+XJKyLiLyIV7O+9gB7AAWulyh2l1EtKqRpKqdro/+0/lVIjLBYrT4iIjz0YAbv75Xag2EfbKaWigFMiEmLf1R0o1oEUOeFmtQBFjVIqVUSeBJYArsB0pdRei8XKFRH5AegCVBaR08A/lFLTrJUqT3QERgK77f55gJeVUostlCkvBAIz7dFqLsBPSqkSFd5ZAgkAftbPEbgB3yulfrdWpDzzFDDL/qAZDjxosTwFptSFpBoMBoMhe0qj+8hgMBgM2WCUgsFgMBgyMUrBYDAYDJkYpWAwGAyGTIxSMBgMBkMmRikYDICIVLJX5twhIlEicuaa7b+cdM2WIpJtfSJ7rkRJCck0/E0odXkKBkNWKKXOAy0AROQNIEEp9S8nX/Zl4J0cZIoRkUgR6aiUWudkWQwGwFgKBkOuiEiC/bWLiKwSkZ9E5JCITBKR4fa+C7tFpK59nL+IzBORzfafjlnM6Qs0U0rttG/fdo1lsj0jsxf4BRheRL+qwWCUgsGQT5oD44Cm6EztBkqpNugy1U/Zx3wCfKSUag0MJusS1mFcX8LhOWCsvQDfrUCSff8W+7bBUCQY95HBkD82K6UiAUTkKLDUvn830NX+vgfQ2F6uAaCciPja+0lkEIgut5zBOuBDEZkF/E8pddq+Pxqo5vhfw2DIGqMUDIb8kXzN+/RrttO5+n1yAdorpZLIniSgTMaGUmqSiCwC7gQ2iEgPpdQB+5ic5jEYHIpxHxkMjmcp8GTGhoi0yGLMfqDeNWPqKqV2K6XeR7uMGtoPNaAEVAo1/H0wSsFgcDxPA2EisktE9gGP3TjAbgWUv2ZBebyI7BGRnWjL4Df7/q7AoqIQ2mAAUyXVYLAMEXkGiFdK5ZSrsBoYoJS6UHSSGUozxlIwGKxjMtevUVyHiPgDHxqFYChKjKVgMBgMhkyMpWAwGAyGTIxSMBgMBkMmRikYDAaDIROjFAwGg8GQiVEKBoPBYMjk/wFURZjo5gIJGgAAAABJRU5ErkJggg==", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "key = dict(mouse_id=0, session_number=1, scan_idx=1, seg_param_id=1)\n", - "\n", - "# ENTER YOUR CODE! - fetch 'time' from the Fluorescence table using fetch1()\n", - "time = (Fluorescence & key).fetch1('time')\n", - "\n", - "# ENTER YOUR CODE! - fetch 'trace' from the Fluorescence.Trace table using fetch()\n", - "traces = (Fluorescence.Trace & key).fetch('trace')\n", - "\n", - "plt.plot(time, np.vstack(traces).T)\n", - "plt.xlabel('Time (s)')\n", - "plt.ylabel('Fluorescence')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Summary" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Congratulations! You have successfully extended your pipeline with all types of tables. This pipeline is simple, but reflecting a typical preprocessin pipeline of calcium imaging. We have all the table tiers and major dependencies between DataJoint tables.\n", - "\n", - "**Table tiers**: \n", - "Manual table: green box \n", - "Lookup table: gray box \n", - "Imported table: blue oval \n", - "Computed table: red circle \n", - "Part table: plain text\n", - "\n", - "**Dependencies**: \n", - "One-to-one primary: thick solid line, share the exact same primary key \n", - "One-to-many primary: thin solid line, inherit the primary key from the parent table, but have additional field(s) as part of the primary key as well" - ] - }, - { - "cell_type": "code", - "execution_count": 62, - "metadata": {}, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "%3\n", - "\n", - "\n", - "\n", - "Fluorescence.Trace\n", - "\n", - "\n", - "Fluorescence.Trace\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Fluorescence\n", - "\n", - "\n", - "Fluorescence\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Fluorescence->Fluorescence.Trace\n", - "\n", - "\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "Mouse\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "Session\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Mouse->Session\n", - "\n", - "\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "Scan\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Session->Scan\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation\n", - "\n", - "\n", - "Segmentation\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation->Fluorescence\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation.Roi\n", - "\n", - "\n", - "Segmentation.Roi\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation->Segmentation.Roi\n", - "\n", - "\n", - "\n", - "\n", - "SegmentationParam\n", - "\n", - "\n", - "SegmentationParam\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "SegmentationParam->Segmentation\n", - "\n", - "\n", - "\n", - "\n", - "Segmentation.Roi->Fluorescence.Trace\n", - "\n", - "\n", - "\n", - "\n", - "AverageFrame\n", - "\n", - "\n", - "AverageFrame\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "AverageFrame->Segmentation\n", - "\n", - "\n", - "\n", - "\n", - "Scan->AverageFrame\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 62, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dj.Diagram(schema)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We have covered most of the building elements of data pipeline design. Using these elements, we could design more sophiscated pipelines that facillitates your experimental recordings and data analyses." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.16" - }, - "vscode": { - "interpreter": { - "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1" - } - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} From 211dd9370a31819e65c64dddd503bb884b9acbcb Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:34:43 +0200 Subject: [PATCH 47/70] Rename tutorial including a space --- ...uted Tables.ipynb => 03-Calcium Imaging Computed Tables.ipynb} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename completed_tutorials/{03-Calcium ImagingComputed Tables.ipynb => 03-Calcium Imaging Computed Tables.ipynb} (100%) diff --git a/completed_tutorials/03-Calcium ImagingComputed Tables.ipynb b/completed_tutorials/03-Calcium Imaging Computed Tables.ipynb similarity index 100% rename from completed_tutorials/03-Calcium ImagingComputed Tables.ipynb rename to completed_tutorials/03-Calcium Imaging Computed Tables.ipynb From 318eb449ac8b63a6c88801c320873680ca12b858 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:42:52 +0200 Subject: [PATCH 48/70] updated image path from`ca-imaging` to `electroph` --- tutorials/04-Electrophysiology Imported Tables.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/04-Electrophysiology Imported Tables.ipynb b/tutorials/04-Electrophysiology Imported Tables.ipynb index 83a37d3..a32834e 100644 --- a/tutorials/04-Electrophysiology Imported Tables.ipynb +++ b/tutorials/04-Electrophysiology Imported Tables.ipynb @@ -13,7 +13,7 @@ "source": [ "Welcome back! In this session, we are going to continue working with the pipeline for the mouse electrophysiology example.\n", "\n", - "![pipeline](../images/pipeline-calcium-imaging.svg)\n", + "![pipeline](../images/pipeline-electrophysiology.svg)\n", "\n", "In this session, we will learn to:\n", "\n", From 30e99aa1559be7429082b099d4b885042a6e06dc Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Thu, 14 Sep 2023 19:44:40 +0200 Subject: [PATCH 49/70] updated short tutorial `Datajoint in 30mins` --- ...ataJoint tutorial in 30min.ipynb => DataJoint in 30mins.ipynb} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename short_tutorials/{DataJoint tutorial in 30min.ipynb => DataJoint in 30mins.ipynb} (100%) diff --git a/short_tutorials/DataJoint tutorial in 30min.ipynb b/short_tutorials/DataJoint in 30mins.ipynb similarity index 100% rename from short_tutorials/DataJoint tutorial in 30min.ipynb rename to short_tutorials/DataJoint in 30mins.ipynb From b39409a31777995e79416fb87a1807d73b9c262d Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Sun, 17 Sep 2023 18:41:16 +0200 Subject: [PATCH 50/70] Revert changes to add in subsequent pull request --- tutorial_pipeline/mouse_session.py | 76 +++ tutorials/01-DataJoint Basics.ipynb | 528 ++++++------------ .../02-Calcium Imaging Imported Tables.ipynb | 244 +++----- 3 files changed, 338 insertions(+), 510 deletions(-) create mode 100644 tutorial_pipeline/mouse_session.py diff --git a/tutorial_pipeline/mouse_session.py b/tutorial_pipeline/mouse_session.py new file mode 100644 index 0000000..92f251d --- /dev/null +++ b/tutorial_pipeline/mouse_session.py @@ -0,0 +1,76 @@ +import datajoint as dj +from . import data_dir + + +# fail-safe user name retrieval +username = dj.conn().conn_info['user'] +schema = dj.schema('{}_tutorial_pipeline'.format(username)) + + +# Table definitions + +@schema +class Mouse(dj.Manual): + definition = """ + # Experimental animals + mouse_id : int # Unique animal ID + --- + dob=null : date # date of birth + sex="unknown" : enum('M','F','unknown') # sex + """ + + +@schema +class Session(dj.Manual): + definition = """ + # Experiment session + -> Mouse + session_date : date # date + --- + experiment_setup : int # experiment setup ID + experimenter : varchar(100) # experimenter name + data_path='' : varchar(255) # + """ + + +# Insert the following data into the table + +mouse_data = [ + {'dob': "2017-03-01", 'mouse_id': 0, 'sex': 'M'}, + {'dob': "2016-11-19", 'mouse_id': 1, 'sex': 'M'}, + {'dob': "2016-11-20", 'mouse_id': 2, 'sex': 'unknown'}, + {'dob': "2016-12-25", 'mouse_id': 5, 'sex': 'F'}, + {'dob': "2017-01-01", 'mouse_id': 10, 'sex': 'F'}, + {'dob': "2017-01-03", 'mouse_id': 11, 'sex': 'F'}, + {'dob': "2017-05-12", 'mouse_id': 100, 'sex': 'F'} +] + +session_data = [ + {'experiment_setup': 0, + 'experimenter': 'Edgar Y. Walker', + 'mouse_id': 0, + 'session_date': "2017-05-15", + 'data_path': data_dir.as_posix() + }, + {'experiment_setup': 0, + 'experimenter': 'Edgar Y. Walker', + 'mouse_id': 0, + 'session_date': "2017-05-19", + 'data_path': data_dir.as_posix() + }, + {'experiment_setup': 1, + 'experimenter': 'Fabian Sinz', + 'mouse_id': 5, + 'session_date': "2017-01-05", + 'data_path': data_dir.as_posix() + }, + {'experiment_setup': 100, + 'experimenter': 'Jacob Reimer', + 'mouse_id': 100, + 'session_date': "2017-05-25", + 'data_path': data_dir.as_posix() + } +] + +Mouse.insert(mouse_data, skip_duplicates=True) +Session.insert(session_data, skip_duplicates=True) \ No newline at end of file diff --git a/tutorials/01-DataJoint Basics.ipynb b/tutorials/01-DataJoint Basics.ipynb index 9d07fdf..f2f152a 100644 --- a/tutorials/01-DataJoint Basics.ipynb +++ b/tutorials/01-DataJoint Basics.ipynb @@ -11,44 +11,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! If you are reading this, you have successfully connected to the first DataJoint tutorial notebook: `01-DataJoint Basics`. \n", + "Now that you have successfully connected to DataJoint (if not, please visit [Connecting to DataBase](00-ConnectingToDatabase.ipynb) first), let's dive into using DataJoint! In this notebook, we will:\n", "\n", - "This tutorial will walk you through the major concepts and steps to use DataJoint: \n", - "\n", - "- Essential concepts and setup\n", - " - Data pipelines\n", - " - Concept\n", - " - Practical examples\n", - " - Schemas and tables\n", - " - Concept\n", - " - Practical examples\n", - " - Basic relational operators\n", - " - Create tables with dependencies\n", - " - Querying data\n", - "- Summary" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Essential concepts and setup\n" + "1. learn what a data pipeline is\n", + "2. create our first simple data pipeline in DataJoint\n", + "3. insert some data into the pipeline\n", + "4. basic queries to flexibly explore the data pipeline\n", + "5. fetch the data from the pipeline\n", + "6. delete entries from tables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Both interactive and local environments come with the latest DataJoint Python package pre-installed, along with many other popular [Python](https://www.python.org/) packages for scientific computations such as [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), and [Matplotlib](https://matplotlib.org/). \n", - "\n", - "Like any other package, to start using [DataJoint](https://datajoint.com/docs/), you must first import the package `datajoint`. The convention is to alias the package to `dj`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "*NOTE: Run code cells by clicking on the left-top corner bottom of the cell or using Ctrl+Enter shortcut.*" + "As always, let's start by importing the `datajoint` library." ] }, { @@ -64,55 +41,38 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Data pipelines\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Concept" + "# So... What is a data pipeline?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ + "If you visit the [documentation for DataJoint](https://docs.datajoint.io/introduction/Data-pipelines.html), we define a data pipeline as follows:\n", + "> A data pipeline is a sequence of steps (more generally a directed acyclic graph) with integrated storage at each step. These steps may be thought of as nodes in a graph.\n", "\n", - ">* A data pipeline is a collection of processes and steps for organizing the data, computations, and workflows. Data pipelines jointly perform complex data acquisition sequences, processing, and analysis with integrated storage at each step. These steps may be thought of as nodes in a graph. \n", - "\n", - ">* In other words, a data pipeline can be seen as a network where each node or entity set represents a **table**. The information involved in a research project or a whole research lab can be organized and stored in these entity sets or tables. Examples of these tables are: \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\". \n", - "\n", - ">* The data pipeline is formed by making these tables interdependent (as the nodes are connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms an entire cohesive data pipeline. \n", + "While this is an accurate description, it may not be the most intuitive definition. Put succinctly, a data pipeline is a listing or a \"map\" of various \"things\" that you work with in a project, with line connecting things to each other to indicate their dependecies. The \"things\" in a data pipeline tends to be the *nouns* you find when describing a project. The \"things\" may include anything from mouse, experimenter, equipment, to experiment session, trial, two-photon scans, electric activities, to receptive fields, neuronal spikes, to figures for a publication! A data pipeline gives you a framework to:\n", "\n", - "A [DataJoint pipeline](https://datajoint.com/docs/core/datajoint-python/0.14/concepts/terminology/) contains database table definitions, dependencies, and associated computations, together with the transformations underlying a DataJoint workflow. \n", + "1. define these \"things\" as tables in which you can store the information about them\n", + "2. define the relationships (in particular the dependencies) between the \"things\"\n", "\n", - "The following figure is an example pipeline using [DataJoint Element for Multi-photon Calcium Imaging](https://datajoint.com/docs/elements/element-optogenetics/0.1/):\n", + "A data pipeline can then serve as a map that describes everything that goes on in your experiment, capturing what is collected, what is processed, and what is analyzed/computed. A well designed data pipeline not only let's you organize your data well, but can bring out logical clarity to your experiment, and may even bring about new insights by making how everything in your experiment relates together obvious.\n", "\n", - "![pipeline](../images/pipeline-calcium-imaging.svg)\n", - "\n", - "A **well-designed data pipeline**: \n", - "- 1. Collects, organizes, and stores **every relevant piece of information during the scientific research**\n", - "- 2. Integrates, processes, and connects these pieces of information through **several steps**\n", - "- 3. Analyses and transforms the input data into **valuable insights for the research**, bringing together logical clarity to the experiments\n" + "Let's go ahead and build together a pipeline from scratch to better understand what a data pipeline is all about." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "##### Practical examples" + "# Building our first pipeline: " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The practical examples that will be used in the next tutorials will allow you to design and compute a data pipeline for a scientific project of two experiments on rodents: \n", - "- Single-electrode recording\n", - "- Calcium imaging recording\n", - "\n", - "Let's start with a brief description of this project's context:" + "Let's build a pipeline to collect, store and process data and analysis for our hypothetical single electrode recording or calcium imaging recording in mice. To help us understand the project better, here is a brief description:" ] }, { @@ -120,32 +80,31 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> * Your lab houses many mice, and a unique ID identifies each mouse. You also want to keep track of other information about each mouse, such as their date of birth and gender.\n", - "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse daily. However, a mouse undergoes at most one recording session on any given day.\n", - "> * For each experimental session, you want to record what mouse you worked with and when you experimented. You also want to keep track of other helpful information, such as the experimental setup you worked on. \n", + "> * Your lab houses many mice, and each mouse is identified by a unique ID. You also want to keep track of information about each mouse such as their date of birth, and gender.\n", + "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day! However, on any given day, a mouse undergoes at most one recording session.\n", + "> * For each experimental session, you would like to record what mouse you worked with and when you performed the experiment. You would also like to keep track of other helpful information such as the experimental setup you worked on. \n", "\n", - "> * In a session of electrophysiology:\n", - ">> * You record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you record.\n", - ">> * Neuron's activities are recorded as raw traces. Neuron's spikes need to be detected for further analysis to be perform.\n", - "\n", - "> * In a session of calcium imaging:\n", - ">> * You scan a brain region containing several neurons. You use recording equipment that produces separate data files for each scan you performed.\n", - ">> * You need to segment the frames and get the regions of interest (ROIs), and save a mask for each ROI\n", - ">> * In addition, you need to extract the trace from each segmented ROI" + "> * In a session of electrophysiology\n", + ">> * you record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you recorded.\n", + ">> * Neuron's activities are recorded as raw traces. Neuron's spikes needs to be detected for further analysis to be performed.\n", + "> * In a session of calcium imaging\n", + ">> * you scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", + ">> * you would like to segment the frames and get the regions of interest (ROIs), and save a mask for each ROI\n", + ">> * finally you would like to extract the trace from each segmented ROI" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The design of a data pipeline starts by identifying the **entities** or **tables** in your research project. Common entities include experimental subjects (e.g., mouse), recording sessions, and two-photon scans." + "Pipeline design starts by identifying **things** or **entities** in your project. Common entities includes experimental subjects (e.g. mouse), recording sessions, and two-photon scans." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's revisit the project description, this time paying special attention to **what** (e.g., nouns or entities) about your experiment. Here some particular entities are highlighted." + "Let's revisit the project description, this time paying special attention to **what** (e.g. nouns) about your experiment. Here I have highlighted some nouns in particular." ] }, { @@ -153,117 +112,94 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> * Your lab houses many **mice**, and a unique ID identifies each mouse. You also want to keep track of other information about each mouse, such as their date of birth and gender.\n", - "> * As a hard-working neuroscientist, you perform experiments every day, sometimes working with more than one mouse daily. However, a mouse undergoes at most one recording session on any given day.\n", - "> * For each **experimental session**, you want to record what mouse you worked with and when you experimented. You also want to keep track of other helpful information, such as the experimental setup you worked on. \n", - "\n", - "> * In a session of electrophysiology:\n", - ">> * You record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you record.\n", - ">> * Neuron's activities are recorded as raw traces. **Neuron's spikes** need to be detected for further analysis to be perform.\n", + "> * Your lab houses many **mice**, and each mouse is identified by a unique ID. You also want to keep track of information about each mouse such as their date of birth, and gender.\n", + "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with more than one mouse in a day! However, on an any given day, a mouse undergoes at most one recording session.\n", + "> * For each **experimental session**, you would like to record what mouse you worked with and when you performed the experiment. You would also like to keep track of other helpful information such as the experimental setup you worked on. \n", "\n", - "> * In a session of calcium imaging:\n", - ">> * You **scan** a brain region containing several neurons. You use recording equipment that produces separate data files for each scan you performed.\n", - ">> * You need to segment the frames and get the **regions of interest (ROIs)**, and save a mask for each ROI\n", - ">> * In addition, you need to extract the **trace** from each segmented ROI" + "> * In a session of electrophysiology\n", + ">> * you record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you recorded.\n", + ">> * Neuron's activities are recorded as raw traces. **Neuron's spikes** needs to be detected for further analysis to be performed.\n", + "> * In a session of calcium imaging\n", + ">> * you scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each **scan** you performed.\n", + ">> * you would like to segment the frames and get the **regions of interest (ROIs)**, and save a mask for each ROI\n", + ">> * finally you would like to extract the **trace** from each segmented ROI" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Just by going through the description, we can start to identify **entities** that needs to be stored and represented in our data pipeline:\n", + "Just by going though the description, we can start to identify **things** or **entities** that we might want to store and represent in our data pipeline:\n", "\n", - ">* Mouse\n", - ">* Experimental session\n", + "* mouse\n", + "* experimental session\n", "\n", - "For Ephys:\n", + "For ephys:\n", "\n", - ">* Neuron\n", - ">* Spikes\n", + ">* neuron\n", + ">* spikes\n", "\n", - "For Calcium Imaging:\n", + "For calcium imaging:\n", "\n", - ">* Scan\n", - ">* Regions of interest (ROI)\n", - ">* Trace" + ">* scan\n", + ">* regions of interest\n", + ">* trace" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In the next section, you will learn to design the tables and manipulate the data for `Mouse` and `Experimental sessions`. The rest of the pipeline (`Ephys` and `Calcium Imaging` will be addressed in the subsequent tutorials." + "In the current notebook, we will design the tables for mouse and experimental sessions, the rest of the pipeline will be designed in the subdirectory `electrophysiology` and `calcium_imaging`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Schemas and tables" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Concepts" + "In DataJoint data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each row of the table is a single example (instance) of the category of entity. \n", + "\n", + "For example, if we have a `Mouse` table, then each row in the mouse table represents a single mouse!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In a data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each table row is a single example (instance) of the entity's category. \n", - "\n", - "For example, if we have a `Mouse` table, each row in the mouse table represents a single mouse. \n", - "\n", - "It is essential to think about what information will **uniquely identify** each entry. \n", - "\n", - "In this case, the information that uniquely identifies the `Mouse` table is their **mouse IDs** - a unique ID number assigned to each animal in the lab. This attribute is named the **primary key** of the table.\n", - "\n", - "| Mouse_ID (*Primary key attribute*)|\n", - "|:--------: | \n", - "| 11234 |\n", - "| 11432 |" + "When constructing such table, we need to figure out what it would take to **uniquely identify** each entry. Let's take the example of the **mouse** and think about what it would take to uniquely identify a mouse." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "After some thought, we might conclude that each mouse can be uniquely identified by knowing its **mouse ID** - a unique ID number assigned to each mouse in the lab. \n", - "\n", - "The mouse ID is then a column in the table or an **attribute** that can be used to **uniquely identify** each mouse. \n", + "After some thought, we might conclude that each mouse can be uniquely identified by knowing its **mouse ID** - a unique ID number assigned to each mouse in the lab. The mouse ID is then a column in the table or an **attribute** that can be used to **uniquely identify** each mouse. Such attribute is called the **primary key** of the table.\n", "\n", - "Such an attribute is called the **primary key** of the table: the subset of table attributes uniquely identifying each entity in the table. The **secondary attribute** refers to any field in a table, not in the primary key.\n", - "\n", - "| Mouse_ID (*Primary key attribute*) \n", - "|:--------:| \n", - "| 11234 (*Secondary attribute*)\n", - "| 11432 (*Secondary attribute*)" + "| mouse_id* |\n", + "|:--------:|\n", + "| 11234 |\n", + "| 11432 |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Once we have successfully identified the table's primary key, we can now think about what other columns, or **non-primary key attributes** - additional information **about each entry in the table that need to be stored as well**." + "Once we have successfully identified the primary key of the table, we can now think about what other columns, or **non-primary key attributes** that we would want to include in the table. These are additional information **about each entry in the table that we want to store**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "For the case of `Mouse`, what other information about the mouse might you want to store? \n", - "\n", - "Based on the project description, we would probably want to store information such as the mouse's **date of birth** (DOB) and **sex**." + "For the case of mouse, what other information about the mouse you might want to store? Based on the project description, we would probably want to store information such as the mouse's **date of birth** and **gender**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "| Mouse_ID | DOB | sex |\n", + "| mouse_id* | dob | sex |\n", "|:--------:|------------|--------|\n", "| 11234 | 2017-11-17 | M |\n", "| 11432 | 2018-03-04 | F |" @@ -273,28 +209,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have an idea of how to represent information about the mouse, let's create the table using **DataJoint**!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##### Practical example" + "Now we have an idea on how to represent information about mouse, let's create the table using **DataJoint**!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "##### Schema" + "## Create a schema - house for your tables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Every table lives inside a schema - a logical collection of one or more tables in your pipeline. Your final pipeline will consist of many tables spread across one or more schemas. Let's go ahead and create the first schema to house our `Mouse` table using DataJoint." + "Every table lives inside a schema - a logical collection of one or more tables in your pipeline. Your final pipeline may consists of many tables spread across one or more schemas. Let's go ahead and create the first schema to house our table." ] }, { @@ -302,7 +231,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We create the schema using `dj.schema()` function, passing in the schema's name. For this tutorial, we create a schema called `tutorial`." + "We create the schema using `dj.schema()` function, passing in the name of the schema. For this workshop, you are given the database privilege to use any schema name. Let's create a schema called `tutorial`." ] }, { @@ -318,21 +247,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have a schema to place our table into let's go ahead and define our first table. " + "Now that we have a schema to place our table into, let's go ahead and define our first table!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "##### Table" + "## Creating your first table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In DataJoint, you define each table as a `class`, and provide the table definition (e.g., attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)." + "In DataJoint, you define each table as a class, and provide the table definition (e.g. attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)." ] }, { @@ -356,7 +285,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's take a look at our brand-new table" + "Let's take a look at our brand new table" ] }, { @@ -372,28 +301,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Basic relational operators" + "## Insert entries with `insert1` and `insert` methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "##### Insert operators" + "The table was successfully defined, but without any content, the table is not too interesting. Let's go ahead and insert some **mouse** into the table, one at a time using the `insert1` method." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The table was successfully defined, but with content, the table will be more interesting. Let's go ahead and **insert some mouse information** into the table, one at a time, using the `insert1` method." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's `insert1` a mouse with the following information:\n", + "Let's insert a mouse with the following information:\n", "* mouse_id: 0\n", "* date of birth: 2017-03-01\n", "* sex: male" @@ -421,7 +343,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can also `insert1` as a dictionary:" + "You could also insert1 as a dictionary" ] }, { @@ -459,7 +381,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can also insert multiple **mice** together using the `insert` method, passing in a list of data:" + "We can also insert multiple **mice** together using the `insert` method, passing in a list of data." ] }, { @@ -488,7 +410,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Of course, you can `insert` a list of dictionaries:" + "Of course, you can insert a list of dictionaries" ] }, { @@ -519,16 +441,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "##### Data integrity" + "## Data integrity" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "DataJoint checks for data integrity and ensures you don't insert a duplicate by mistake. Let's try inserting another mouse with `mouse_id: 0` and see what happens!\n", - "\n", - "*Note that the following code cell is intended to give an error code.* " + "DataJoint checks for data integrity, and ensures that you don't insert a duplicate by mistake. Let's try inserting another mouse with `mouse_id: 0` and see what happens!" ] }, { @@ -548,27 +468,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The error message suggests to include a new argument `skip_duplicates=True` in the `.insert1()` method to skip inserting duplicated entries. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "Mouse.insert1(\n", - "{'mouse_id': 0,\n", - " 'dob': '2018-01-01',\n", - " 'sex': 'M',\n", - "}, skip_duplicates=True)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's insert a few more mice into your table before moving on:" + "Go ahead and insert a few more mice into your table before moving on." ] }, { @@ -611,43 +511,45 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create tables with dependencies" + "## Create tables with dependencies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! We have successfully created your first table! We are ready to tackle and include other **entities** into the project's data pipeline. \n", + "Congratulations! We have successfully created your first table! We are now ready to tackle and include other **entities** in the project into our data pipeline. \n", "\n", - "Let's now have a look at representing an `experimental session`." + "Let's now take a look at representing an **experimental session**." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "As with `mouse`, we should consider **what information (i.e., attributes) is needed to identify an `experimental session`** uniquely. Here is the relevant section of the project description:\n", + "As with mouse, we should think about **what information (i.e. attributes) is needed to uniquely identify an experimental session**. Here is the relevant section of the project description:\n", "\n", - "> * As a hard-working neuroscientist, you perform experiments daily, sometimes working with **more than one mouse in a day**. However, on any given day, **a mouse undergoes at most one recording session**.\n", - "> * For each **experimental session**, you want to record **what mouse you worked with** and **when you performed the experiment**. You also want to keep track of other helpful information, such as the **experimental setup** you worked on. " + "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with **more than one mouse in a day**! However, on an any given day, **a mouse undergoes at most one recording session**.\n", + "> * For each experimental session, you would like to record **what mouse you worked with** and **when you performed the experiment**. You would also like to keep track of other helpful information such as the **experimental setup** you worked on." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Based on the above, it seems that you need to know these two data to uniquely identify a single experimental session:\n", + "Based on the above, it appears that you need to know:\n", "\n", "* the date of the session\n", - "* the mouse you recorded from in that session" + "* the mouse you recorded from in that session\n", + "\n", + "to uniquely identify a single experimental session." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note that to uniquely identify an experimental session (or simply a **session**), we need to know the mouse used in that the session. In other words, a session cannot be existing without a corresponding mouse! \n", + "Note that, to uniquely identify an experimental session (or simply a **session**), we need to know the mouse that the session was about. In other words, a session cannot existing without a corresponding mouse! \n", "\n", "With **mouse** already represented as a table in our pipeline, we say that the session **depends on** the mouse! We could graphically represent this in an **entity relationship diagram (ERD)** by drawing the line between two tables, with the one below (**session**) depending on the one above (**mouse**)." ] @@ -656,9 +558,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Thus, we will need both the **mouse** and the new attribute **session_date** to identify a single `session` uniquely. \n", + "Thus we will need both **mouse** and a new attribute **session_date** to uniquely identify a single session. \n", "\n", - "Remember that a **mouse** is uniquely identified by its primary key - **mouse_id**. In DataJoint, you can declare that **session** depends on the mouse, and DataJoint will automatically include the mouse's primary key (`mouse_id`) as part of the session's primary key, alongside any additional attribute(s) you specify." + "Remember that a **mouse** is already uniquely identified by its primary key - **mouse_id**. In DataJoint, you can declare that **session** depends on the mouse, and DataJoint will automatically include the mouse's primary key (`mouse_id`) as part of the session's primary key, along side any additional attribute(s) you specificy." ] }, { @@ -684,7 +586,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can generate something similar to an entity relationship diagram (ERD) on the fly by calling `dj.Diagram` with the schema object. Many of the symbols and features are the same as the ERD standard." + "You can actually generate something similar to an entity relationship diagram (ERD) on the fly by calling `dj.Diagram` with the schema object. Many of the symbols and features are the same as the ERD standard." ] }, { @@ -700,7 +602,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert a few sessions manually:" + "Let's try inserting a few sessions manually." ] }, { @@ -732,7 +634,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's insert another session for `mouse_id = 0` but on a different date:" + "Let's insert another session for `mouse_id = 0` but on a different date." ] }, { @@ -757,7 +659,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "And another session is done on the same date but on a different mouse:" + "And another session done on the same date but on a different mouse" ] }, { @@ -790,7 +692,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "What happens if we try to insert a session for a mouse that doesn't exist?:" + "What happens if we try to insert a session for a mouse that doesn't exist?" ] }, { @@ -807,13 +709,6 @@ "}" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "*Note: the following code line is intended to give an error code:*" - ] - }, { "cell_type": "code", "execution_count": null, @@ -829,16 +724,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Querying data" + "# Querying data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Oftentimes, you want only some of the data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your parser, narrowing your data to the subset before fetching is far more efficient.\n", + "Often times, you don't want all data but rather work with **a subset of entities** matching specific criteria. Rather than fetching the whole data and writing your own parser, it is far more efficient to narrow your data to the subset before fetching.\n", "\n", - "For this, DataJoint offers a very powerful yet intuitive **querying syntax** that lets you select the data you want before you fetch it.\n", + "For this, DataJoint offers very powerful yet intuitive **querying** syntax that let's you select exactly the data you want before you fetch it.\n", "\n", "It is also critical to note that the result of any DataJoint query represents a valid entity." ] @@ -847,41 +742,39 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We will introduce significant types of queries used in DataJoint:\n", - "* 1. Restriction (`&`) and negative restriction (`-`): filter the data with certain conditions\n", - "* 2. Join (`*`): bring fields from different tables together\n", - "* 3. Projection (`.proj()`): focus on a subset of attributes\n", - "* 4. Fetch (`.fetch()`): pull the data from the database\n", - "* 5. Deletion (`.delete()`): delete entries and their dependencies\n", - "* 6. Drop (`.drop()`): drop the table from the schema" + "We will introduce four major types of queries used in DataJoint:\n", + "* restriction (`&`) and negative restriction (`-`): filter data\n", + "* join (`*`): bring fields from different tables together\n", + "* projection (`.proj()`): focus on a subset of attributes\n", + "* aggregation (`.aggr()`): simple computation of one table against another table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### 1. Restrictions (`&` or `-`): filter the data with certain conditions" + "## Restrictions (`&`) - filter data with certain conditions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The **restriction** operation, `&`, allows you to specify the criteria to narrow down the table on the left." + "The **restriction** operation, `&`, let's you specify the criteria to narrow down the table on the left." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "##### Exact match" + "### Exact match" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse with `ID = 0`:" + "Mouse with id 0" ] }, { @@ -897,7 +790,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All the male (`M`) mice:" + "All male mice (`'sex = \"M\"'`)" ] }, { @@ -913,7 +806,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "All the female (`F`) mice:" + "All female mice (`'sex=\"F\"'`)" ] }, { @@ -929,7 +822,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can also use a dictionary as a restrictor, with one field or multiple fields:" + "We can also use as a dictionary as a restrictor, with one field or multiple fields" ] }, { @@ -959,7 +852,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse that is born `after 2017-01-01`:" + "Mouse that is born **after 2017-01-01**" ] }, { @@ -975,7 +868,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mouse that is born within a range of dates:" + "Mouse that is born within a range of dates" ] }, { @@ -991,7 +884,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mice that are `not male`:" + "Mouse that is **not** male" ] }, { @@ -1007,14 +900,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can easily combine multiple restrictions to narrow the entities based on various attributes." + "You can easily combine multiple restrictions to narrow down the entities based on multiple attributes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Let's find all mice that `are not male` and born `after 2017-01-01`:" + "Let's find all mice that **are not male** AND **born after 2017-01-01**." ] }, { @@ -1039,7 +932,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The result of one query can be used in another query! Let's first find `all the female mice` and `store the result`:" + "Result of one query can be used in another query! Let's first find **all female mice** and store the result." ] }, { @@ -1056,7 +949,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "It's your turn! Find and store the mice with a `mouse_id > 10`:" + "and among these mice, find ones with **mouse_id > 10**" ] }, { @@ -1072,44 +965,51 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In Computer Science and Math lingo, DataJoint operations are said to **satisfy closure property**. Practically speaking, this means that the result of a query can immediately be used in another query, allowing you to build more complex queries from simpler ones. " + "In computer science/math lingo, DataJoint operations are said to **satisfy closure property**. Practically speaking, this means that the result of a query can immediately be used in another query, allowing you to build more complex queries from simpler ones. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Restriction operator (`&`): all entities from one table for which there exist a matching entity in other table" + "### Restriction one table with another" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note that when restricting, for example, table A with table B (written A & B), the two tables must have common attributes (join-compatible). " + "All mice that have a session" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Mouse & Session " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "To select all the `mice` that have a `session`:" + "### Combining restrictions" ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ - "Mouse & Session " + "All the above queries could be combined " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All the above queries can be combined, for example, based on the `male mice` that are in a `session`:" + "Male mice that had a session" ] }, { @@ -1125,7 +1025,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Another example of how to select the `mice` that participated in an `experimental session` done `on or before 2017-05-19`:" + "Give me all mice that have had an experimental session done on or before 2017-05-19" ] }, { @@ -1141,14 +1041,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Negative restriction (`-`): subset of entities from one table for which there are no matching entities in other table" + "### Negative restriction - with the `-` operator" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All the `mice` that do `not have any session`:" + "All mice that do not have any session" ] }, { @@ -1164,7 +1064,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "It's your turn! Find and store the male mice that do not have any session:" + "Male mice that do not have any session" ] }, { @@ -1180,26 +1080,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 2. Join (`*`): bring fields from different tables together" + "## Joining (*) - bring fields from different tables together" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Sometimes you want to view and query information simultaneously from multiple tables combined. You can do this using the join `*` operator." + "Sometimes you want to see information from multiple tables combined together to be viewed (and queried!) simultaneously. You can do this using the join `*` operator." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The Join operator works as follows:\n", + "Behavior of join:\n", "\n", - "1. Match the common field(s) of the primary keys in the two tables\n", - "2. Do a combination of the non-matched part of the primary key\n", - "3. List out the secondary attributes for each combination\n", - "4. If two tables have secondary attributes that share a same name, it will throw an error. To join, we need to rename that attribute for at least one of the tables." + "1. match the common field(s) of the primary keys in the two tables\n", + "2. do a combination of the non-matched part of the primary key\n", + "3. listing out the secondary attributes for each combination\n", + "4. if two tables have secondary attributes that share a same name, it will throw an error. To join, we need to rename that attribute for at least one of the tables." ] }, { @@ -1216,7 +1116,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Each row represents a unique (and valid!) combination of a mouse and a session." + "Here each row represents a unique (and valid!) combination of a mouse and a session." ] }, { @@ -1249,15 +1149,15 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 3. Projection (`.proj()`): focus on attributes of interest\n", - "Besides restriction (`&`) and join (`*`) operations, DataJoint offers another type of operation: projection (`.proj()`). Projection is used to select attributes (columns) from a table, rename them, or create new calculated attributes. " + "## Projection .proj(): focus on attributes of interest\n", + "Beside restriction (`&`) and join (`*`) operations, DataJoint offers another type of operation: projection (`.proj()`). Projection is used to select attributes (columns) from a table, to rename them, or to create new calculated attributes. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "From the **Mouse** table, suppose we want to focus only on the `sex` attribute and ignore the others. This can be done as:" + "From the ***Mouse*** table, suppose we want to focus only on the `sex` attribute and ignore the others, this can be done as:" ] }, { @@ -1273,7 +1173,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that `.proj()` will always retain all attributes that are part of the primary key." + "Note that `.proj()` will always retain all attributes that are part of the primary key" ] }, { @@ -1282,7 +1182,7 @@ "metadata": {}, "source": [ "### Rename attribute with proj()\n", - "Say we want to rename the existing attribute `dob` of the `Mouse` table to `date_of_birth`. This can be done using `.proj()`:" + "Say we want to rename the existing attribute `dob` of the `Mouse` table to `date_of_birth`, this can be done using `.proj()`" ] }, { @@ -1299,7 +1199,7 @@ "metadata": {}, "source": [ "### Perform simple computations with proj()\n", - "Projection is perhaps most useful to perform simple computations on the attributes, especially on attributes from multiple tables, by using it in conjunction with the join (`*`) operation." + "Projection is perhaps most useful to perform simple computations on the attributes, especially on attributes from multiple tables by using in conjunction with the join (`*`) operation" ] }, { @@ -1316,7 +1216,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note: As you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation while removing all other non-primary attributes. To keep all the other attributes, you can use the `...` syntax." + "Note: as you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation, while removing all other non-primary attributes. To Keep all other attributes, you can use the `...` syntax" ] }, { @@ -1332,28 +1232,28 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 4. Fetch (`.fetch()`): pull the data from the database" + "# Fetch data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Once you have narrowed down to the entities you want, you can fetch the query results just by calling fetch on it!" + "Once you have successfully narrowed down to the entities you want, you can fetch the query results just by calling fetch on it!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Fetch one or multiple entries: `fetch()`" + "## Fetch one or multiple entries: `fetch()`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "All the `male mice`:" + "All male mouse" ] }, { @@ -1370,7 +1270,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch it!:" + "Fetch it!" ] }, { @@ -1386,7 +1286,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Or all in one step:" + "or all in one step" ] }, { @@ -1402,7 +1302,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch as a list of dictionaries:" + "Fetch as a list of dictionaries" ] }, { @@ -1418,7 +1318,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch as a [Pandas dataframe](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) if needed:" + "Fetch as a pandas dataframe" ] }, { @@ -1434,7 +1334,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch the primary key:" + "### Fetch the primary key" ] }, { @@ -1450,7 +1350,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Fetch specific fields:" + "### Fetch specific fields" ] }, { @@ -1484,7 +1384,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Or fetch them together as a list of dictionaries:" + "Or fetch them together as a list of dictionaries" ] }, { @@ -1501,14 +1401,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Fetch data from only one entry: `fetch1()`" + "## Fetch data from only one entry: `fetch1()`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "When there is only one result to be fetched back, we can use `.fetch1()`. `fetch1` will always return the fetched result in a dictionary format:" + "When knowing there's only 1 result to be fetched back, we can use `.fetch1()`. `fetch1` will always return the fetched result in a dictionary format" ] }, { @@ -1525,7 +1425,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "`fetch1()` can also fetch the primary key:" + "`fetch1()` could also fetch the primary key" ] }, { @@ -1541,7 +1441,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Or fetch specific fields:" + "or fetch specific fields:" ] }, { @@ -1575,22 +1475,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### 5. Deletion (`.delete()`): delete entries in the table " + "## Deletion (`.delete()`) - deleting entries and their dependencies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have a good idea of how to restrict table entries, this is an excellent time to introduce how to **delete** entries from a table." + "Now we have a good idea on how to restrict table entries, this is a good time to introduce how to **delete** entries from a table." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "To delete a specific entry, you restrict the table to the target entry, and call the `delete` method. Note that after running the following code line, you will have to confirm to commit the delete. \n", - "\n" + "To delete a specific entry, you restrict the table down to the target entry, and call `delete` method." ] }, { @@ -1606,7 +1505,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Calling `.delete()` method on an *unrestricted* table will attempt to delete the whole table!" + "Calling `delete` method on an *unrestricted* table will attempt to delete the whole table!" ] }, { @@ -1618,70 +1517,32 @@ "Mouse.delete()" ] }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "Mouse()" - ] - }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note that the `.delete()` method not only delete the entries in a table, but also all the corresponding entries in subsequent (downstream) tables!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### 6. Drop (`.drop()`): remove the table from the schema" + "# Summary" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Contrary to the `.delete()` method - where the table is preserved but its content is deleted - with `.drop()` we remove the whole table from the pipeline. \n", + "Congratulations! You have successfully created your first DatJoint pipeline, using dependencies to establish the link between the tables. You have also learned to query and fetch the data.\n", "\n", - "Again, `drop()` method not only drop the whole table, but also all the subsequent (downstream) tables." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.Diagram(schema)" + "In the next session, we are going to extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", + "\n", + "We will use both ephys and calcium imaging as example pipelines:\n", + "+ [02-electrophysiology](../02-Electrophysiology/02-Imported%20Tables%20-%20Interactive.ipynb)\n", + "+ [02-calcium imaging](../01-Calcium_Imaging/02-Imported%20Tables%20-%20Interactive.ipynb)" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ - "Remember, again, that after running the following method, you will be asked to confirm to commit the delete:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "Session.drop()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "dj.Diagram(schema)" + "# Clean up" ] }, { @@ -1690,26 +1551,7 @@ "metadata": {}, "outputs": [], "source": [ - "schema.drop()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Summary" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Congratulations! You have successfully created your first DataJoint pipeline, using dependencies to establish the link among the tables. You have also learned to query, fetch and delete the data.\n", - "\n", - "In the next session, we will extend our data pipeline with tables to represent **imported data** and define new tables to **compute and hold analysis results**.\n", - "\n", - "\n", - "Please, continue to the next notebook `02-Calcium Imaging.ipynb`." + "# schema.drop()" ] } ], @@ -1732,7 +1574,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.17" + "version": "3.7.16" }, "vscode": { "interpreter": { diff --git a/tutorials/02-Calcium Imaging Imported Tables.ipynb b/tutorials/02-Calcium Imaging Imported Tables.ipynb index 99faaee..6114d4b 100644 --- a/tutorials/02-Calcium Imaging Imported Tables.ipynb +++ b/tutorials/02-Calcium Imaging Imported Tables.ipynb @@ -4,36 +4,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Working with automated computations: Imported tables\n", - "# Application to Calcium Imaging" + "# Working with automated computations: Imported tables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Welcome back! The practical example of this session is Calcium Imaging! \n", + "Welcome back! In this session, we are going to continue working with the pipeline for the mouse calcium imaging example. \n", "\n", - "![pipeline](../images/pipeline-calcium-imaging.svg)\n", + "In this session, we will learn to:\n", "\n", - "During this session you will learn:\n", - "\n", - "* To import neuron imaging data from data files into an `Imported` table\n", - "* To automatically trigger data importing and computations for all the missing entries with `Populate`" + "* import neuron imaging data from data files into an `Imported` table\n", + "* automatically trigger data importing and computations for all missing entries with `populate`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Importing libraries" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "First thing first, let's import `DataJoint` again." + "First thing first, let's import `datajoint` again." ] }, { @@ -49,7 +39,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As we are going to perform some computations, let's go ahead and import `NumPy` and `Matplotlib`." + "As we are going to perform some computations, let's go ahead and import NumPy and Matplotlib" ] }, { @@ -67,36 +57,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Calcium imaging dataset" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In the `data` folder in this `DataJoint-Tutorials`, you can find a small dataset of three different cases of calcium imaging scans:\n", - "\n", - "- `/workspaces/datajoint-tutorials/data/example_scan_02.tif`\n", - "- `/workspaces/datajoint-tutorials/data/example_scan_03.tif`\n", - "- `/workspaces/datajoint-tutorials/data/example_scan_01.tif`\n", - "\n", - "As you might know, calcium imaging scans (raw data) are stored as *.tif* files. \n", - "\n", - "*NOTE: For this tutorial there is no need to deeper explore this small dataset. Nevertheless, if you are curious about visualizing these example scans, we recommend you to open the TIFF with [ImageJ](https://imagej.nih.gov/ij/download.html).*" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## First steps of the pipeline design: Schema, Mouse & Session" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The DataJoint pipeline commonly starts with a `schema` and the following classes for each table: `Mouse` and `Session`. Let's quickly create this pipeline's first steps as we learned it in the previous session:" + "Now we would like to continue working with the tables we defined in the previous notebook. To do so, we would need the classes for each table: `Mouse` and `Session`. We can either redefine it here, but for your convenience, we have included the schema and table class definitions in a package called `tutorial_pipeline.mouse_session`, from which you can import the classes as well as the schema object. We will use the schema object again to define more tables." ] }, { @@ -105,76 +66,14 @@ "metadata": {}, "outputs": [], "source": [ - "schema = dj.schema('tutorial')\n", - "\n", - "@schema\n", - "class Mouse(dj.Manual):\n", - " definition = \"\"\"\n", - " # Experimental animals\n", - " mouse_id : int # Unique animal ID\n", - " ---\n", - " dob=null : date # date of birth\n", - " sex=\"unknown\" : enum('M','F','unknown') # sex\n", - " \"\"\"\n", - "\n", - "@schema\n", - "class Session(dj.Manual):\n", - " definition = \"\"\"\n", - " # Experiment session\n", - " -> Mouse\n", - " session_date : date # date\n", - " ---\n", - " experiment_setup : int # experiment setup ID\n", - " experimenter : varchar(100) # experimenter name\n", - " data_path='' : varchar(255) # relative path\n", - " \"\"\"\n", - "\n", - "mouse_data = [\n", - " {'dob': \"2017-03-01\", 'mouse_id': 0, 'sex': 'M'},\n", - " {'dob': \"2016-11-19\", 'mouse_id': 1, 'sex': 'M'},\n", - " {'dob': \"2016-11-20\", 'mouse_id': 2, 'sex': 'unknown'},\n", - " {'dob': \"2016-12-25\", 'mouse_id': 5, 'sex': 'F'},\n", - " {'dob': \"2017-01-01\", 'mouse_id': 10, 'sex': 'F'},\n", - " {'dob': \"2017-01-03\", 'mouse_id': 11, 'sex': 'F'},\n", - " {'dob': \"2017-05-12\", 'mouse_id': 100, 'sex': 'F'}\n", - "]\n", - "\n", - "session_data = [\n", - " {'experiment_setup': 0,\n", - " 'experimenter': 'Edgar Y. Walker',\n", - " 'mouse_id': 0,\n", - " 'session_date': \"2017-05-15\",\n", - " 'data_path': '../data/'\n", - " },\n", - " {'experiment_setup': 0,\n", - " 'experimenter': 'Edgar Y. Walker',\n", - " 'mouse_id': 0,\n", - " 'session_date': \"2017-05-19\",\n", - " 'data_path': '../data/'\n", - " },\n", - " {'experiment_setup': 1,\n", - " 'experimenter': 'Fabian Sinz',\n", - " 'mouse_id': 5,\n", - " 'session_date': \"2017-01-05\",\n", - " 'data_path': '../data/'\n", - " },\n", - " {'experiment_setup': 100,\n", - " 'experimenter': 'Jacob Reimer',\n", - " 'mouse_id': 100,\n", - " 'session_date': \"2017-05-25\",\n", - " 'data_path': '../data/'\n", - " }\n", - "]\n", - "\n", - "Mouse.insert(mouse_data, skip_duplicates=True)\n", - "Session.insert(session_data, skip_duplicates=True)" + "from tutorial_pipeline.mouse_session import schema, Mouse, Session" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Take a quick look at the tables `Mouse` and `Session`:" + "Take a quick look at the tables Mouse and Session" ] }, { @@ -199,14 +98,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Define the Scan table and its primary key attributes" + "The `mouse_session.py` also fills each table with data to make sure we are all on the same page." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "First we define a table named `Scan` that describes a scanning in an experimental session of Calcium Imaging. This table will store the scans' metadata." + "## Define table `Scan` for meta information of each calcium imaging scan" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's define a table `Scan` that describes a scanning in an experimental session that stores the meta information of a particular scan." ] }, { @@ -233,11 +139,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The table `Scan` is dependent on the table `Session`, inheriting its primary key attributes. In addition, `Scan` has an additional primary key attribute `scan_idx`. \n", - "\n", - "One session might contain multiple scans - This is another example of **one-to-many** relationship. \n", - "\n", - "Take a look at the `Diagram` again:" + "This table is dependent on the table `Session`, inheriting its primary key attributes, with an additional primary key attribute `scan_idx`. One session could contain multiple scans, which is another example of **one-to-many** relationship. We could take a look at the Diagram again." ] }, { @@ -253,9 +155,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The thin solid line connecting [`Mouse`-`Session`] and [`Session`-`Scan`] indicates **one-to-many relationship**. \n", + "The thin solid line connecting `Mouse`-`Session`, and `Session`-`Scan` indicates **one-to-many relationship**. \n", "\n", - "The underline `____` indicates **additional primary key attribute(s)** apart from the ones inherited from its parents." + "The `____` indicates **additional primary key attribute(s)** apart from the ones inherited from its parents." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here we have prepared two tif files of scanning in the `data` folder `example_scan_01.tif` and `example_scan_02.tif` " ] }, { @@ -264,14 +173,25 @@ "metadata": {}, "outputs": [], "source": [ - "Scan()" + "from tutorial_pipeline import data_dir\n", + "data_dir" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for f in data_dir.glob('*.tif'):\n", + " print(f)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now we manually `insert` the metadata of the datasets and their file names in the table `Scan`:" + "Let's insert these meta information manually." ] }, { @@ -285,8 +205,6 @@ " 'depth': 150, 'wavelength': 920, 'laser_power': 26, 'fps': 15, 'file_name': 'example_scan_01.tif'},\n", " {'mouse_id': 0, 'session_date': '2017-05-15', 'scan_idx': 2, \n", " 'depth': 200, 'wavelength': 920, 'laser_power': 24, 'fps': 15, 'file_name': 'example_scan_02.tif'},\n", - " {'mouse_id': 0, 'session_date': '2017-05-15', 'scan_idx': 3, \n", - " 'depth': 200, 'wavelength': 920, 'laser_power': 24, 'fps': 15, 'file_name': 'example_scan_03.tif'} \n", "])" ] }, @@ -303,9 +221,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Calculation of the average frame\n", - "\n", - "Let's first load and look at the number of frames of the calcium imaging TIF files:" + "## Looking at the raw data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's first load one raw data and take a look at the data:" ] }, { @@ -316,18 +239,15 @@ "source": [ "import os\n", "from skimage import io\n", - "im = io.imread('../data/example_scan_01.tif')\n", - "print('Number of frames = ',im.shape[0])" + "im = io.imread(data_dir / 'example_scan_01.tif')\n", + "print(im.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Particularly, this example contains 100 frames. \n", - "\n", - "Let's calculate the average of the images over the frames and plot the result.\n", - "\n" + "This tiff file contains 100 frames. Let's take the average of the images over frames and look at it." ] }, { @@ -336,30 +256,23 @@ "metadata": {}, "outputs": [], "source": [ - "# ENTER YOUR CODE! \n", - "av_frame = np.mean(im,axis=0)\n", - "plt.imshow(av_frame)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "*TIP: compute the `average frame` of the `im` image using the mean function from NumPy (np.mean) with axis = 0. Then, plot the result with `imshow`*" + "# ENTER YOUR CODE! - compute the avg frame with np.mean of axis=0\n", + "avg_image = np.mean(im, axis=0)\n", + "plt.imshow(avg_image, cmap=plt.cm.gray)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Define the table for the average fluorescence " + "## Defining table for average fluorescence across frames" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's create a table `AverageFrame` to compute and save the average fluorescence across the frames. \n", + "Now let's create a table `AverageFrame` to compute and save the average fluorescence. \n", "\n", "For each scan, we have one average frame. Therefore, the table shares the exact same primary key as the table `Scan`" ] @@ -406,7 +319,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We defined `average_frame` as a `longblob`, which allow us to store a NumPy array. This NumPy array will be imported and computed from the file corresponding to each scan." + "We defined `average_frame` as a `longblob` so that it can store a NumPy array. This NumPy array will be imported and computed from the file corresponding to each scan." ] }, { @@ -429,7 +342,7 @@ "source": [ "In DataJoint, the tier of the table indicates **the nature of the data and the data source for the table**. So far we have encountered two table tiers: `Manual` and `Imported`, and we will encounter the two other major tiers in this session. \n", "\n", - "DataJoint tables in `Manual` tier, or simply **Manual tables**, indicate that its content is **manually** entered by either experimenters or a recording system, and its content **do not depend on external data files or other tables**. This is the most basic table type that you will encounter, especially as the tables at the beggining of the pipeline. In the Diagram, `Manual` tables are depicted by green rectangles.\n", + "DataJoint tables in `Manual` tier, or simply **Manual tables** indicate that its contents are **manually** entered by either experimenters or a recording system, and its content **do not depend on external data files or other tables**. This is the most basic table type you will encounter, especially as the tables at the beggining of the pipeline. In the Diagram, `Manual` tables are depicted by green rectangles.\n", "\n", "On the other hand, **Imported tables** are understood to pull data (or *import* data) from external data files, and come equipped with functionalities to perform this importing process automatically, as we will see shortly! In the Diagram, `Imported` tables are depicted by blue ellipses." ] @@ -454,7 +367,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Rather than filling out the content of the table manually using `insert1` or `insert` methods, we are going to make use of the `make` and `populate` logic that comes with `Imported` tables. These two methods automatically figure it out what needs to be imported, and perform the import." + "Rather than filling out the content of the table manually using `insert1` or `insert` methods, we are going to make use of the `make` and `populate` logic that comes with `Imported` tables to automatically figure out what needs to be imported and perform the import!" ] }, { @@ -468,17 +381,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "`Imported` table comes with a special method called `populate`. Let's call it for `AverageFrame`:\n", - "\n", - "*Note that the following code line is intended to generate a code error.*" + "`Imported` table comes with a special method called `populate`. Let's try calling it." ] }, { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "scrolled": true + }, "outputs": [], "source": [ + "# ENTER YOUR CODE! - call `populate` on the table\n", "AverageFrame.populate()" ] }, @@ -486,7 +400,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Notice that the `populate` call complained that a method called `make` is not implemented. Let me show you a simple `make` method that will help elucidate what this is all about." + "Notice that `populate` call complained that a method called `make` is not implemented. Let me show a simple `make` method that will help elucidate what this is all about." ] }, { @@ -519,7 +433,7 @@ "metadata": {}, "outputs": [], "source": [ - "# ENTER YOUR CODE! - call `populate` on the table AverageFrame\n", + "# ENTER YOUR CODE! - call `populate` on the table\n", "AverageFrame.populate()" ] }, @@ -608,14 +522,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Notice that we added the missing attribute information `average_frame` into the `key` dictionary, and finally **inserted the entry** into `self` (in this case, `self` corresponds to the `AverageFrame` table). The `make` method's job is to create and insert a new entry. This new entry corresponds to the `key` into this table." + "Notice that we added the missing attribute information `average_frame` into the `key` dictionary, and finally **inserted the entry** into `self` = `AverageFrame` table. The `make` method's job is to create and insert a new entry corresponding to the `key` into this table!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Finally, let's go ahead and call `populate` to actually populate the `AverageFrame` table with the new content, i.e., filling the table with the data loaded and computed from data files!" + "Finally, let's go ahead and call `populate` to actually populate the `AverageFrame` table, filling it with data loaded and computed from data files!" ] }, { @@ -695,7 +609,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can find all the `Scan` entries without their corresponding `AverageFrame` entries using the **negative restriction operator** `-`" + "We can find all `Scan` without corresponding `AverageFrame` entry with the **negative restriction operator** `-`" ] }, { @@ -704,7 +618,7 @@ "metadata": {}, "outputs": [], "source": [ - "# select all the `Scan` entries *without* a corresponding entry in `AverageFrame`\n", + "# select all Scan entries *without* a corresponding entry in AverageFrame\n", "Scan - AverageFrame" ] }, @@ -737,7 +651,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now instead of loading from the raw *.tif* file, we are able to fetch the average fluorescence image from this table." + "Now instead of loading from the raw tif file, we are able fetch the average fluorescence image from this table." ] }, { @@ -762,9 +676,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Congratulations! You have successfully:\n", - "- Extended your pipeline with a new table to represent the processed data (`AverageFrame` as `Imported` table)\n", - "- Learned and implemented the `make()` and `populate()` calls to load the external data to your tables" + "Congratulations! You have successfully extended your pipeline with a table to represent processed data (`AverageFrame` as `Imported` table), learned and implemented the `make()` and `populate()` call to load external data to your tables." ] }, { @@ -780,11 +692,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "At this point, our pipeline contains the core elements with the data populated, ready for further downstream analysis.\n", + "At this point, our pipeline contains the core elements with data populated, ready for further downstream analysis.\n", "\n", - "In the next session `03-Calcium Imaging Computed Tables`:\n", - "- We will introduce the concept of `Computed` table and `Lookup` table\n", - "- We will also set up an automated computation routine, essential to develop advanced data analyses in your experiments!" + "In the next [session](./03-Computed%20Table,%20Lookup%20Table,%20and%20Part%20Table%20-%20Interactive.ipynb), we are going to introduce the concept of `Computed` table, and `Lookup` table, as well as learning to set up a automated computation routine." ] } ], @@ -804,7 +714,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.17" + "version": "3.7.16" }, "vscode": { "interpreter": { From 159706dd4ff367a207325d786a53731e4a17a2a0 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 12:53:23 +0200 Subject: [PATCH 51/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 09a8d61..8ccc061 100644 --- a/README.md +++ b/README.md @@ -67,7 +67,7 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - Once you are done, see the options in the menu in the bottom-left corner. In Codespaces, you can `Stop Current Codespace`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. - **Local environment**: - - We highly recommend this option for users that who want to apply DataJoint to **their own neuroscience experiments** and lab research after exploring the tutorials. Additionally, this option is particularly advantageous for those who have a keen interest in **other modules of the DataJoint Elements Library** (e.g., Miniscope, DeepLabCut). For this option, ensure you have the following: + - Install the following: - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - [Docker](https://docs.docker.com/get-docker/) - On M1/M2 Mac, you have to: From 20331ec735ca32fd32b2f2103240b75fef422691 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 12:54:00 +0200 Subject: [PATCH 52/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 8ccc061..0a3630e 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,9 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - Once you are done, see the options in the menu in the bottom-left corner. When running DevContainer on your machine, you can `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. - For more detailed instructions, please check out the [User Guide in DataJoint Documentation](https://datajoint.com/docs/elements/user-guide/). +## Documentation + +- For more information on DataJoint Python, please refer to the [documentation](https://datajoint.com/docs/core/datajoint-python/). Before we start, remember that all the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that interest you in keeping the changes. From 805d7dae64ebaea16b0636648edf7527ca28cdfe Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 12:54:35 +0200 Subject: [PATCH 53/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 0a3630e..a934a7a 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,6 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - For more information on DataJoint Python, please refer to the [documentation](https://datajoint.com/docs/core/datajoint-python/). -Before we start, remember that all the edits you make in these tutorial notebooks are ***not persistent*** - they will be reset to the original content every time you restart the server. However, you can easily download the notebooks that interest you in keeping the changes. ## Support If you need help getting started or run into any errors, please open a GitHub Issue or contact our team by email at support@datajoint.com. From 218856cad4cbf84ea94fbc110b3fe80762a42b27 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 12:55:54 +0200 Subject: [PATCH 54/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index a934a7a..5f324f2 100644 --- a/README.md +++ b/README.md @@ -66,6 +66,7 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - Once you are done, see the options in the menu in the bottom-left corner. In Codespaces, you can `Stop Current Codespace`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. +## Developer Instructions - **Local environment**: - Install the following: - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) From 7a84d4957f9e035796ae9c7f24539c81cef28675 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:00:53 +0200 Subject: [PATCH 55/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5f324f2..aa07830 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ This document will guide you as a new DataJoint user through interactive tutoria *Please note that these hands-on DataJoint tutorials are friendly to non-expert users, and advanced programming skills are not required.* ## Table of contents -- In the `tutorials` folder are interactive Jupyter notebooks to learn DataJoint. The Calcium Imaging and Electrophysiology tutorials are relevant examples of data structure and analysis. In addition, some fill-in-the-blank sections are included for you to code yourself! +- In the [tutorials](./tutorials) folder are interactive Jupyter notebooks to learn DataJoint. The calcium imaging and electrophysiology tutorials provide examples of defining and interacting with data pipelines. In addition, some fill-in-the-blank sections are included for you to code yourself! - 01-DataJoint Basics - 02-Calcium Imaging Imported Tables - 03-Calcium Imaging Computed Tables From 0960b1b3d442909aea1277c0bcaaeaf16e81c99e Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:01:06 +0200 Subject: [PATCH 56/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index aa07830..a75c02d 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ This document will guide you as a new DataJoint user through interactive tutoria - In the `completed_tutorials` folder are Jupyter notebooks with the code sections completed and solved. -- You will find the following notebooks in the `short_tutorials` folder: +- You will find the following notebooks in the [short_tutorials](./short_tutorials) folder: - DataJoint in 30min - University From 97c102a5d72fdad7171da2cb115d70672c0b594d Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:01:19 +0200 Subject: [PATCH 57/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a75c02d..31f3ed0 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ This document will guide you as a new DataJoint user through interactive tutoria - 04-Electrophysiology Imported Tables - 05-Electrophysiology Computed Tables -- In the `completed_tutorials` folder are Jupyter notebooks with the code sections completed and solved. +- In the [completed_tutorials](./completed_tutorials) folder are Jupyter notebooks with the code sections completed and solved. - You will find the following notebooks in the [short_tutorials](./short_tutorials) folder: - DataJoint in 30min From 21b231d8e8d3721d0bb4fcd34394a5f60e669fee Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:01:30 +0200 Subject: [PATCH 58/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 31f3ed0..3b56f07 100644 --- a/README.md +++ b/README.md @@ -71,9 +71,9 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - Install the following: - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - [Docker](https://docs.docker.com/get-docker/) - - On M1/M2 Mac, you have to: - - enable Rosetta 2 on Docker advanced/experimental settings - - ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt + - On M1/M2 Mac: + - Enable Rosetta 2 on Docker advanced/experimental settings + - Ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). From 1417540ea449d3c1e51b2f8dfdae918956632de4 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:01:40 +0200 Subject: [PATCH 59/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3b56f07..74c6f19 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - Ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - - [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). + - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - `git clone` the codebase repository and open it in VSCode. - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - Once you are done, see the options in the menu in the bottom-left corner. When running DevContainer on your machine, you can `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. From d29fc852292045a3e033e4bfcbae1d38a9acfb30 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:01:51 +0200 Subject: [PATCH 60/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 74c6f19..0c9fbe3 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - - `git clone` the codebase repository and open it in VSCode. + - `git clone` your fork of the repository and open it in VSCode. - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - Once you are done, see the options in the menu in the bottom-left corner. When running DevContainer on your machine, you can `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. From e8e2113840057262a5c24a9d7ccad6ad189b8456 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:02:04 +0200 Subject: [PATCH 61/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0c9fbe3..0b3609a 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - `git clone` your fork of the repository and open it in VSCode. - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - - Once you are done, see the options in the menu in the bottom-left corner. When running DevContainer on your machine, you can `Reopen folder locally`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. + - Once you are done, you can stop the container by closing the `VS Code` window. ## Documentation From 8d3ecb26e802591e147c7634c2eb43358a6c7cd2 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:02:18 +0200 Subject: [PATCH 62/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 0b3609a..942b7ab 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,8 @@ DataJoint tutorials are easily accessible using an **interactive environment** t - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - `git clone` your fork of the repository and open it in VSCode. - - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. + - Use the `Dev Containers extension` to `Reopen in Container`. (More info in the `Getting started` included with the extension.) + - To begin, navigate to the [tutorials](./tutorials) directory located in the left panel and proceed through the sequentially organized Jupyter notebooks. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - Once you are done, you can stop the container by closing the `VS Code` window. ## Documentation From 5f1e9a7412f9dac1763cc64d7124c122c6c2f7c2 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 13:02:43 +0200 Subject: [PATCH 63/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 34 ++++++++++++++-------------------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 942b7ab..bae4536 100644 --- a/README.md +++ b/README.md @@ -45,26 +45,20 @@ Here is a summary of the content that you can expect to have learned: - `.populate()` for automated computation - `.populate(reserve_jobs=True)` for parallelization - -## Quick start - -### User installation -DataJoint tutorials are easily accessible using an **interactive environment** that contains all the software required to run the experiments. The setting is configured by [DevContainer] (https://containers.dev/). Here are two options to launch the interactive environment. - -*Please note that to use the DataJoint Python package with an interactive environment, you need a [GitHub](https://github.com/) account.* - -- **Cloud-based environment: GitHub Codespaces**: (*recommended*) - - This is the easiest option for **tutorial users**. You will immediately start coding using DataJoint and Python, without installing software or local environments. Cloud-based environments (IDEs), such as [GitHub Codespaces](https://github.com/features/codespaces), use built-in tools directly connected to the cloud and work on the browser. - - - Instructions: - - Fork the [datajoint-tutorials](https://github.com/datajoint/datajoint-tutorials) repository into your repository. - - From your `datajoint-tutorials` repository, click on `Code`, then click on `Codespaces` tab, and `+` option will `Create codespace on main` on your fork with default options. For more control, see the `...` where you may create `New with options...`. - - The building time for Codespaces is **~5m**. This is done infrequently and cached for convenience. - - The start time for Codespaces is **~30s**. This will pull the built codespace from the cache when you need it. - - *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of computing and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete Codespaces when not actively in use and recreate them when needed. We'll soon be creating prebuilds to avoid larger build times. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. - - *Tip*: GitHub auto names the Codespaces, but you can rename the Codespaces so that it is easier to identify later. - - To begin, navigate to the notebooks directory located in the left panel and proceed through the sequentially organized Jupyter notebooks, labeled by numbers. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - - Once you are done, see the options in the menu in the bottom-left corner. In Codespaces, you can `Stop Current Codespace`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. +## Interactive Environment + +- These interactive DataJoint tutorials can be accessed through a cloud-based environment on [GitHub Codespaces](https://github.com/features/codespaces). The following instructions will provide you with an environment that is configured with DataJoint for Python so that you can immediately begin to build and run a data pipeline. +- Instructions + - Sign up for a free a [GitHub](https://github.com/) account. + - Fork this repository. + - Launch the environment using GitHub Codespaces on your fork with the default options by selecting the green `Code` button, then the `Codespaces` tab, and then the green `Create codespace on main` button. For more control, under the `Codespaces` tab select the `...` button where you may create `New with options....`. + - The launch time for the Codespace is less that 2 minutes. + - You will know your environment has finished loading once the `pip install -e .` command has run and the terminal prompt is clear. + - To begin, navigate to the `tutorials` directory located in the left panel and proceed through the sequentially organized Jupyter notebooks. Execute the cells in the notebooks to begin your walkthrough of the tutorial. + - Once you are done, see the options in the menu in the bottom-left corner. In Codespaces, you can `Stop Current Codespace`. By default, GitHub will also automatically stop the Codespaces after 30 minutes of inactivity. +- *Tip*: Each month, GitHub renews a [free-tier](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts) quota of computing and storage. Typically we run into the storage limits before anything else since Codespaces consume storage while stopped. It is best to delete [Codespaces](https://github.com/codespaces) when not actively in use and recreate them when needed. Once any portion of your quota is reached, you will need to wait for it to be reset at the end of your cycle or add billing info to your GitHub account to handle overages. +- *Tip*: GitHub auto names the Codespaces, but you can rename the Codespace so that it is easier to identify later. +- *Tip*: All the edits you make in these tutorial notebooks are ***not persistent***. Edits will be reset to the original content every time you restart the server. However, you can easily commit the changes to your fork. ## Developer Instructions - **Local environment**: From cc529a766bc3c76b9be172b55ef9935a5d84ea43 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 18 Sep 2023 13:07:41 +0200 Subject: [PATCH 64/70] Move `Developer Instructions` to the end of README --- README.md | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index bae4536..0e12c07 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,7 @@ This document will guide you as a new DataJoint user through interactive tutoria *Please note that these hands-on DataJoint tutorials are friendly to non-expert users, and advanced programming skills are not required.* + ## Table of contents - In the [tutorials](./tutorials) folder are interactive Jupyter notebooks to learn DataJoint. The calcium imaging and electrophysiology tutorials provide examples of defining and interacting with data pipelines. In addition, some fill-in-the-blank sections are included for you to code yourself! - 01-DataJoint Basics @@ -20,6 +21,7 @@ This document will guide you as a new DataJoint user through interactive tutoria - DataJoint in 30min - University + ## Key learnings from the tutorials After completing this set of tutorials, you will gain real experience in the basics of the DataJoint framework. These skills will allow you to design, implement and manage data pipelines effectively applied to your scientific research. @@ -45,6 +47,7 @@ Here is a summary of the content that you can expect to have learned: - `.populate()` for automated computation - `.populate(reserve_jobs=True)` for parallelization + ## Interactive Environment - These interactive DataJoint tutorials can be accessed through a cloud-based environment on [GitHub Codespaces](https://github.com/features/codespaces). The following instructions will provide you with an environment that is configured with DataJoint for Python so that you can immediately begin to build and run a data pipeline. @@ -60,21 +63,6 @@ Here is a summary of the content that you can expect to have learned: - *Tip*: GitHub auto names the Codespaces, but you can rename the Codespace so that it is easier to identify later. - *Tip*: All the edits you make in these tutorial notebooks are ***not persistent***. Edits will be reset to the original content every time you restart the server. However, you can easily commit the changes to your fork. -## Developer Instructions -- **Local environment**: - - Install the following: - - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - - [Docker](https://docs.docker.com/get-docker/) - - On M1/M2 Mac: - - Enable Rosetta 2 on Docker advanced/experimental settings - - Ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt - - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt - - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - - `git clone` your fork of the repository and open it in VSCode. - - Use the `Dev Containers extension` to `Reopen in Container`. (More info in the `Getting started` included with the extension.) - - To begin, navigate to the [tutorials](./tutorials) directory located in the left panel and proceed through the sequentially organized Jupyter notebooks. Execute the cells in the notebooks to begin your walkthrough of the tutorial. - - Once you are done, you can stop the container by closing the `VS Code` window. ## Documentation @@ -84,6 +72,7 @@ Here is a summary of the content that you can expect to have learned: ## Support If you need help getting started or run into any errors, please open a GitHub Issue or contact our team by email at support@datajoint.com. + ## Additional DataJoint Tutorials - DataJoint Elements is a collection of curated modules for assembling data pipelines for several modalities of neurophysiology experiments. @@ -91,4 +80,21 @@ If you need help getting started or run into any errors, please open a GitHub Is - [Element Array Electrophysiology Tutorial](https://github.com/datajoint/workflow-array-ephys#interactive-tutorial) - [Machine Intelligence from Cortical Networks (MICrONS) program](https://www.microns-explorer.org/) - - [MICrONS Tutorial](https://github.com/datajoint/microns_phase3_nda#interactive-environment) \ No newline at end of file + - [MICrONS Tutorial](https://github.com/datajoint/microns_phase3_nda#interactive-environment) + + + ## Developer Instructions +- **Local environment**: + - Install the following: + - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) + - [Docker](https://docs.docker.com/get-docker/) + - On M1/M2 Mac: + - Enable Rosetta 2 on Docker advanced/experimental settings + - Ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt + - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt + - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) + - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). + - `git clone` your fork of the repository and open it in VSCode. + - Use the `Dev Containers extension` to `Reopen in Container`. (More info in the `Getting started` included with the extension.) + - To begin, navigate to the [tutorials](./tutorials) directory located in the left panel and proceed through the sequentially organized Jupyter notebooks. Execute the cells in the notebooks to begin your walkthrough of the tutorial. + - Once you are done, you can stop the container by closing the `VS Code` window. \ No newline at end of file From 482e4676177c0f4a76013c56a9242034875cb630 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 18 Sep 2023 14:18:31 +0200 Subject: [PATCH 65/70] example of `ca-imaging` pipeline in short tutorial --- short_tutorials/DataJoint in 30mins.ipynb | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/short_tutorials/DataJoint in 30mins.ipynb b/short_tutorials/DataJoint in 30mins.ipynb index f60a290..e52a610 100644 --- a/short_tutorials/DataJoint in 30mins.ipynb +++ b/short_tutorials/DataJoint in 30mins.ipynb @@ -1,5 +1,13 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "301be40e", + "metadata": {}, + "source": [ + "![pipeline-example-Calcium-Imaging](../images/pipeline-calcium-imaging.svg)" + ] + }, { "cell_type": "code", "execution_count": 12, @@ -2579,7 +2587,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.16" + "version": "3.9.18" } }, "nbformat": 4, From f4a048590050233877acfd05e1f83d3425b596d9 Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 16:51:17 +0200 Subject: [PATCH 66/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0e12c07..8323c49 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ If you need help getting started or run into any errors, please open a GitHub Is - [MICrONS Tutorial](https://github.com/datajoint/microns_phase3_nda#interactive-environment) - ## Developer Instructions +## Developer Instructions - **Local environment**: - Install the following: - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) From 47c5af903c943c5b65a27ce8b90ba7cccc6cb88c Mon Sep 17 00:00:00 2001 From: MilagrosMarin Date: Mon, 18 Sep 2023 16:51:34 +0200 Subject: [PATCH 67/70] Update README.md Co-authored-by: Kabilar Gunalan --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 8323c49..bc48157 100644 --- a/README.md +++ b/README.md @@ -84,16 +84,16 @@ If you need help getting started or run into any errors, please open a GitHub Is ## Developer Instructions -- **Local environment**: - - Install the following: - - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - - [Docker](https://docs.docker.com/get-docker/) - - On M1/M2 Mac: - - Enable Rosetta 2 on Docker advanced/experimental settings - - Ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt - - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt - - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) - - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). +- Local environment instructions + - Install the following: + - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) + - [Docker](https://docs.docker.com/get-docker/) + - On M1/M2 Mac: + - Enable Rosetta 2 on Docker advanced/experimental settings + - Ensure Rosetta is installed by typing `softwareupdate --install-rosetta` at a shell prompt + - `export DOCKER_DEFAULT_PLATFORM=linux/amd64` in .zshrc or at a shell prompt + - [Microsoft's Visual Studio Code (VS Code)](https://code.visualstudio.com/) + - VSCode [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). - `git clone` your fork of the repository and open it in VSCode. - Use the `Dev Containers extension` to `Reopen in Container`. (More info in the `Getting started` included with the extension.) - To begin, navigate to the [tutorials](./tutorials) directory located in the left panel and proceed through the sequentially organized Jupyter notebooks. Execute the cells in the notebooks to begin your walkthrough of the tutorial. From b1c403956baab3c08604291dacb9ba89cf35f636 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 18 Sep 2023 17:47:13 +0200 Subject: [PATCH 68/70] pipeline short tutorial image --- images/pipeline-short-tutorial.svg | 57 +++++++++++++++++++++++ short_tutorials/DataJoint in 30mins.ipynb | 2 +- 2 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 images/pipeline-short-tutorial.svg diff --git a/images/pipeline-short-tutorial.svg b/images/pipeline-short-tutorial.svg new file mode 100644 index 0000000..d6ebb49 --- /dev/null +++ b/images/pipeline-short-tutorial.svg @@ -0,0 +1,57 @@ + + +%3 + + + +Analysis + + +Analysis + + + + + +Param + + +Param + + + + + +Param->Analysis + + + + +Subject + + +Subject + + + + + +Session + + +Session + + + + + +Subject->Session + + + + +Session->Analysis + + + + \ No newline at end of file diff --git a/short_tutorials/DataJoint in 30mins.ipynb b/short_tutorials/DataJoint in 30mins.ipynb index e52a610..775e1f9 100644 --- a/short_tutorials/DataJoint in 30mins.ipynb +++ b/short_tutorials/DataJoint in 30mins.ipynb @@ -5,7 +5,7 @@ "id": "301be40e", "metadata": {}, "source": [ - "![pipeline-example-Calcium-Imaging](../images/pipeline-calcium-imaging.svg)" + "![pipeline-example-Calcium-Imaging](../images/pipeline-short-tutorial.svg)" ] }, { From 7fb8ca7baae697e61c430f546966490325560d74 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 18 Sep 2023 17:54:05 +0200 Subject: [PATCH 69/70] `pipeline` name of the image in short tutorial --- short_tutorials/DataJoint in 30mins.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/short_tutorials/DataJoint in 30mins.ipynb b/short_tutorials/DataJoint in 30mins.ipynb index 775e1f9..ec1c0a7 100644 --- a/short_tutorials/DataJoint in 30mins.ipynb +++ b/short_tutorials/DataJoint in 30mins.ipynb @@ -5,7 +5,7 @@ "id": "301be40e", "metadata": {}, "source": [ - "![pipeline-example-Calcium-Imaging](../images/pipeline-short-tutorial.svg)" + "![pipeline](../images/pipeline-short-tutorial.svg)" ] }, { From 4654d010ef1a0883ee9463bb9aab1eae0505e868 Mon Sep 17 00:00:00 2001 From: Milagros Marin Date: Mon, 18 Sep 2023 17:57:41 +0200 Subject: [PATCH 70/70] title and introduction in short tutorial --- short_tutorials/DataJoint in 30mins.ipynb | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/short_tutorials/DataJoint in 30mins.ipynb b/short_tutorials/DataJoint in 30mins.ipynb index ec1c0a7..b940f48 100644 --- a/short_tutorials/DataJoint in 30mins.ipynb +++ b/short_tutorials/DataJoint in 30mins.ipynb @@ -1,5 +1,21 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "ceada63c", + "metadata": {}, + "source": [ + "# DataJoint tutorial in 30 mins" + ] + }, + { + "cell_type": "markdown", + "id": "f6d12c96", + "metadata": {}, + "source": [ + "In this tutorial, you are going to learn how to build your first DataJoint pipeline, which is shown below:" + ] + }, { "cell_type": "markdown", "id": "301be40e",