demos/notebooks/demo_dendritic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analyzing dendritic data with CaImAn\n",
    "This notebook shows an example on how to analyze two-photon dendritic data with CaImAn. It closely follows the other notebooks, so if there are bits left unexplained, please see the main demos. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import glob\n",
    "from IPython import get_ipython\n",
    "import logging\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import psutil\n",
    "from pathlib import Path\n",
    "\n",
    "try:\n",
    "    cv2.setNumThreads(0)\n",
    "except():\n",
    "    pass\n",
    "\n",
    "try:\n",
    "    if __IPYTHON__:\n",
    "        get_ipython().run_line_magic('load_ext', 'autoreload')\n",
    "        get_ipython().run_line_magic('autoreload', '2')\n",
    "except NameError:\n",
    "    pass\n",
    "\n",
    "import caiman as cm\n",
    "from caiman.motion_correction import MotionCorrect\n",
    "from caiman.source_extraction.cnmf import cnmf as cnmf\n",
    "from caiman.source_extraction.cnmf import params as params\n",
    "from caiman.utils.utils import download_demo\n",
    "from caiman.utils.visualization import nb_view_patches\n",
    "from skimage.util import montage\n",
    "import bokeh.plotting as bpl\n",
    "import holoviews as hv\n",
    "bpl.output_notebook()\n",
    "hv.notebook_extension('bokeh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up a logger (optional)\n",
    "If you want information (printed to file or to console), run the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "use_logfile = False  # set to true to log to file\n",
    "if use_logfile:\n",
    "    log_file = Path(cm.paths.get_tempdir()) / 'cnmf_demo.log' # \n",
    "    print(f\"Will save logging data to {log_file}\")\n",
    "else:\n",
    "    log_file = None\n",
    "log_format = \"{asctime} - {levelname} - [{filename} {funcName}() {lineno}] - pid {process} - {message}\"\n",
    "logging.basicConfig(format=log_format,\n",
    "                    filename=log_file, \n",
    "                    level=logging.WARNING, style=\"{\") #DEBUG, INFO, WARNING, ERROR, CRITICAL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting the data\n",
    "\n",
    "We provide an example file from mouse barrel cortex, courtesy of Clay Lacefield and Randy Bruno, Columbia University. The `download_demo` command will automatically download the file and store it in your caiman_data folder the first time you run it. To use the demo in your own dataset you can set:\n",
    "\n",
    "```fnames = [/path/to/file(s)]```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fnames = [download_demo('data_dendritic.tif')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Viewing the data\n",
    "Prior to analysis you can view the data by setting `display_movie = False`. The example file provided here is small, with FOV size 128 x 128. To view it larger we set `magnification = 4` in the play command. This will result to a movie of resolution 512 x 512. For larger files, you will want to reduce this magnification factor (for memory and display purposes). The movie displays using OpenCV, and will open a separate window: to close it, press `q` on your keyboard when it is the active window."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display_movie = False\n",
    "if display_movie:\n",
    "    m_orig = cm.load_movie_chain(fnames)\n",
    "    ds_ratio = 0.25\n",
    "    m_orig.resize(1, 1, ds_ratio).play(\n",
    "        q_max=99.5, fr=30, magnification=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameter setting\n",
    "\n",
    "First we'll perform motion correction followed by CNMF.\n",
    "\n",
    "One of the main differences when analyzing dendritic data, is that dendritic segments are not localized and can traverse significant parts of the FOV. They can also be spatially non-contiguous. To capture this property, CaImAn has two methods for initializing CNMF: `sparse_nmf` which uses scikit-learn's built-in non-negative factorization ([documentation here](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.NMF.html)), and `graph-nmf`, where the Laplacian of the pixel affinity matrix is used as a regularizer to promote spatial components that capture pixels that have similar timecourses. These can be selected with the parameter `method_init`.\n",
    "\n",
    "In this demo we use (and recommend using) the `graph_nmf` initialization method although `sparse_nmf` can also be used, as indicated below.\n",
    "\n",
    "Since the dataset is small we can run CNMF without splitting the FOV in patches. This can be helpful because of the non-localized nature of the dendritic segments. To do that we set `rf = None`. Using patches is also supported as demonstrated below. Also note, that spatial and temporal downsampling can also be used to reduce the data dimensionality by appropriately modifying the parameters `ssub` and `tsub`. Here they are set to 1 since it is not necessary.\n",
    "\n",
    "The graph NMF method was originally presented in the paper:\n",
    "\n",
    "Cai, D., He, X., Han, J., & Huang, T. S. (2010). Graph regularized nonnegative matrix factorization for data representation. IEEE transactions on pattern analysis and machine intelligence, 33(8), 1548-1560."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dataset dependent parameters\n",
    "fr = 30                             # imaging rate in frames per second\n",
    "decay_time = 0.5                    # Length of a typical transient in seconds (see source/Getting_Started.rst)\n",
    "\n",
    "# motion correction parameters\n",
    "strides = (48, 48)          # start a new patch for pw-rigid motion correction every x pixels\n",
    "overlaps = (24, 24)         # overlap between patches (size of patch strides+overlaps)\n",
    "max_shifts = (6, 6)         # maximum allowed rigid shifts (in pixels)\n",
    "max_deviation_rigid = 3     # maximum shifts deviation allowed for patch with respect to rigid shifts\n",
    "pw_rigid = True             # flag for performing non-rigid motion correction\n",
    "\n",
    "# parameters for source extraction and deconvolution\n",
    "p = 0                       # order of the autoregressive system\n",
    "gnb = 2                     # number of global background components\n",
    "merge_thr = 0.75            # merging threshold, max correlation allowed\n",
    "rf =  None                  # half-size of the patches in pixels. e.g., if rf=25, patches are 50x50\n",
    "stride_cnmf = None          # amount of overlap between the patches in pixels\n",
    "K = 60                      # number of components per patch\n",
    "method_init = 'graph_nmf'   # initialization method (you could also use 'sparse_nmf' for dendritic data)\n",
    "alpha = 0.5                 # sparsity regularizer term (default is 0.5): used only for sparse_nmf\n",
    "ssub = 1                    # spatial subsampling during initialization\n",
    "tsub = 1                    # temporal subsampling during initialization\n",
    "\n",
    "opts_dict = {'fnames': fnames,\n",
    "            'fr': fr,\n",
    "            'decay_time': decay_time,\n",
    "            'strides': strides,\n",
    "            'overlaps': overlaps,\n",
    "            'max_shifts': max_shifts,\n",
    "            'max_deviation_rigid': max_deviation_rigid,\n",
    "            'pw_rigid': pw_rigid,\n",
    "            'p': p,\n",
    "            'nb': gnb,\n",
    "            'rf': rf,\n",
    "            'K': K, \n",
    "            'stride': stride_cnmf,\n",
    "            'method_init': method_init,\n",
    "            'alpha_snmf': alpha,  # used only for sparse_nmf\n",
    "            'rolling_sum': True,\n",
    "            'only_init': True,\n",
    "            'ssub': ssub,\n",
    "            'tsub': tsub,\n",
    "            'merge_thr': merge_thr}\n",
    "\n",
    "opts = params.CNMFParams(params_dict=opts_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# start a cluster for parallel processing \n",
    "# note if a cluster already exists it will be closed so a new session will be opened\n",
    "print(f\"You have {psutil.cpu_count()} CPUs available in your current environment\")\n",
    "if 'dview' in locals():  # locals contains list of current local variables\n",
    "    print('Closing previous cluster')\n",
    "    cm.stop_server(dview=dview)\n",
    "print(\"Setting up new cluster\")\n",
    "c, dview, n_processes = cm.cluster.setup_cluster(backend='multiprocessing', \n",
    "                                                 n_processes=None,  # set to less than cpu_count-1 if you have memory problems\n",
    "                                                 single_thread=False,\n",
    "                                                 ignore_preexisting=False)\n",
    "print(f\"Successfully set up cluster with {n_processes} processes active\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Motion correction\n",
    "First we create a motion correction object and then perform the registration. The provided dataset has already been registered with a different method so we do not expect to see a lot of motion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mc = MotionCorrect(fnames, dview=dview, **opts.get_group('motion'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "#%% Run piecewise-rigid motion correction using NoRMCorre\n",
    "mc.motion_correct(save_movie=True)\n",
    "m_els = cm.load(mc.fname_tot_els)\n",
    "border_to_0 = 0 if mc.border_nan == 'copy' else mc.border_to_0 \n",
    "    # maximum shift to be used for trimming against NaNs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Display registered movie next to the original (optional)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% compare with original movie\n",
    "if display_movie:\n",
    "    m_orig = cm.load_movie_chain(fnames)\n",
    "    ds_ratio = 0.2\n",
    "    cm.concatenate([m_orig.resize(1, 1, ds_ratio) - mc.min_mov*mc.nonneg_movie,\n",
    "                    m_els.resize(1, 1, ds_ratio)], \n",
    "                   axis=2).play(fr=60, q_max=99.5, magnification=4, offset=0)  # press q to exit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Memory mapping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% MEMORY MAPPING\n",
    "# memory map the file in order 'C'\n",
    "fname_new = cm.save_memmap(mc.mmap_file, base_name='memmap_', order='C',\n",
    "                           border_to_0=border_to_0) # exclude borders\n",
    "\n",
    "# now load the file\n",
    "Yr, dims, T = cm.load_memmap(fname_new)\n",
    "images = np.reshape(Yr.T, [T] + list(dims), order='F') \n",
    "    #load frames in python format (T x X x Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% restart cluster to clean up memory\n",
    "cm.stop_server(dview=dview)\n",
    "c, dview, n_processes = cm.cluster.setup_cluster(\n",
    "    backend='multiprocessing', n_processes=None, single_thread=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run CNMF with the chosen initialization method\n",
    "Note that in the parameter setting we have set `only_init = True` so only the initialization will be run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "cnm = cnmf.CNMF(n_processes, params=opts, dview=dview)\n",
    "cnm = cnm.fit(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Display components\n",
    "First compute the correlation image to use a reference image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CI = cm.local_correlations(images[::1].transpose(1,2,0))\n",
    "CI[np.isnan(CI)] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "cnm.estimates.hv_view_components(img=CI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot a montage array of all the components\n",
    "\n",
    "We can also plot a montage of all the identified spatial footprints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = cnm.estimates.A.toarray().reshape(opts.data['dims'] + (-1,), order='F').transpose([2, 0, 1])\n",
    "Nc = A.shape[0]\n",
    "grid_shape = (np.ceil(np.sqrt(Nc/2)).astype(int), np.ceil(np.sqrt(Nc*2)).astype(int))\n",
    "plt.figure(figsize=np.array(grid_shape[::-1])*1.5)\n",
    "plt.imshow(montage(A, rescale_intensity=True, grid_shape=grid_shape))\n",
    "plt.title(f'Montage of initial components {alpha=}');\n",
    "plt.axis('off');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run CNMF again to refine the components\n",
    "The components found by the initialization have captured some structure, but have likely not converged on a solution yet, and are not particularly sparse (especially if you used `graph_nmf`). Using the `refit` command will start with the initial solutions as seeds, and refine them, typically ending up with more sparse solutions. Note if the components already seemed good, you would not need to do this second refitting step, and could skip to component evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm2 = cnm.refit(images, dview=dview)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now plot the components again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm2.estimates.hv_view_components(img=CI)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A2 = cnm2.estimates.A.toarray().reshape(opts.data['dims'] + (-1,), order='F').transpose([2, 0, 1])\n",
    "Nc = A2.shape[0]\n",
    "grid_shape = (np.ceil(np.sqrt(Nc/2)).astype(int), np.ceil(np.sqrt(Nc*2)).astype(int))\n",
    "plt.figure(figsize=np.array(grid_shape[::-1])*1.5)\n",
    "plt.imshow(montage(A2, rescale_intensity=True, grid_shape=grid_shape))\n",
    "plt.title(f'Montage of refit components {alpha=}');\n",
    "plt.axis('off');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Component evaluation\n",
    "You will typically have many false positives after running CNMF. The next step of component evaluation helps remove them. Caiman includes three ways to evaluate components, two of which are relevant here:\n",
    "\n",
    "- *spatial*: the shape of each component must be correlated with the data\n",
    "- *temporal*: a minimum peak SNR is required over the length of a transient\n",
    "- *cnn*: each shape passes a CNN based classifier (trained on soma, so not relevant here)\n",
    "\n",
    "Once you run this you will see the evaluation metrics in the component viewer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "min_SNR = 4 # signal to noise ratio for accepting a component\n",
    "rval_thr = 0.85  # space correlation threshold for accepting a component\n",
    "\n",
    "cnm2.params.set('quality', {'decay_time': decay_time,\n",
    "                           'min_SNR': min_SNR,\n",
    "                           'rval_thr': rval_thr,\n",
    "                           'use_cnn': False}) # do not use cnn for dentritic data\n",
    "\n",
    "cnm2.estimates.evaluate_components(images, cnm2.params, dview=dview);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot selected and rejected components"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# accepted components\n",
    "cnm2.estimates.hv_view_components(img=CI, idx=cnm2.estimates.idx_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# rejected components\n",
    "cnm2.estimates.hv_view_components(img=CI, idx=cnm2.estimates.idx_components_bad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract DF/F values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm2.estimates.detrend_df_f(quantileMin=8, frames_window=250);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Play a movie with the results\n",
    "\n",
    "The movie will show the original, denoised and residual movies in the left, middle, and right panels respectively. \n",
    "The flag `include_bck = False` will remove the estimated background from the original and denoised panels. To include it set it to `True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm2.estimates.play_movie(images, q_max=99.9,\n",
    "                          magnification=4,\n",
    "                          include_bck=False);\n",
    "# press q to exit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the object\n",
    "You can save the results of the analysis for future use and/or to load them on the GUI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm2.estimates.Cn = CI # save the correlation image for displaying in the background\n",
    "cnm2.save('dendritic_analysis.hdf5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now try CNMF with patches\n",
    "As mentioned above, the file shown here is small enough to be processed at once. For larger files patches and/or downsampling can be used. We show how to use patches below. Note that unlike in somatic imaging, for dendritic imaging we want the patches to be large and have significant overlap so enough structure of the dendritic segments is captured in the overlapping region. This will help with merging the components. However, a too large patch size can result in memory issues due to large amount of data loaded concurrently onto memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% restart cluster to clean up memory\n",
    "cm.stop_server(dview=dview)\n",
    "c, dview, n_processes = cm.cluster.setup_cluster(backend='multiprocessing', \n",
    "                                                 n_processes=None, \n",
    "                                                 single_thread=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change some parameters to enable patches\n",
    "To enable patches the `rf` parameter should be changed from `None` to some integer value. Moreover the amount of overlap controlled by `stride_cnmf` should also change. Finally, since the parameter `K` specifies components per patch it should be reduced compared to its value when operating on the whole FOV at once. It is important to experiment with different values for these parameters since results can sensitive, due to the non stereotypical shapes and activity patterns of dendrites."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf = 48   # size of each patch is (2*rf, 2*rf) \n",
    "stride_cnmf = 16\n",
    "Kp = 25   # reduce the number of components as it is now per patch\n",
    "opts.change_params({'rf': rf,\n",
    "                    'stride': stride_cnmf,\n",
    "                    'K': Kp});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm_patches = cnmf.CNMF(n_processes, params=opts, dview=dview)\n",
    "cnm_patches = cnm_patches.fit(images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm_patches.estimates.hv_view_components(img=CI)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ap = cnm_patches.estimates.A.toarray().reshape(opts.data['dims'] + (-1,), order='F').transpose([2, 0, 1])\n",
    "Nc = Ap.shape[0]\n",
    "grid_shape = (np.ceil(np.sqrt(Nc/2)).astype(int), np.ceil(np.sqrt(Nc*2)).astype(int))\n",
    "plt.figure(figsize=np.array(grid_shape[::-1])*1.5)\n",
    "plt.imshow(montage(Ap, rescale_intensity=True, grid_shape=grid_shape))\n",
    "plt.title('Montage of found spatial components (patch run)');\n",
    "plt.axis('off');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Refit to sparsify\n",
    "As before we can pass the results through the CNMF `refit` function that can yield more sparse components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm_patches2 = cnm_patches.refit(images, dview=dview)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ap2 = cnm_patches2.estimates.A.toarray().reshape(opts.data['dims'] + (-1,), order='F').transpose([2, 0, 1])\n",
    "Nc = Ap2.shape[0]\n",
    "grid_shape = (np.ceil(np.sqrt(Nc/2)).astype(int), np.ceil(np.sqrt(Nc*2)).astype(int))\n",
    "plt.figure(figsize=np.array(grid_shape[::-1])*1.5)\n",
    "plt.imshow(montage(Ap2, rescale_intensity=True, grid_shape=grid_shape))\n",
    "plt.title('Montage of refit components with patches');\n",
    "plt.axis('off');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm_patches2.estimates.hv_view_components(img=CI)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnm_patches2.estimates.play_movie(images, q_max=99.9,\n",
    "                                  magnification=4,\n",
    "                                  include_bck=False);\n",
    "# press q to exit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compare the two approaches\n",
    "To identify how similar components and the two approaches yield, we can compute the cross-correlation matrix between the spatial footprints derived from the two approaches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "AA = np.corrcoef(cnm2.estimates.A.toarray(), cnm_patches2.estimates.A.toarray(), rowvar=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.imshow(AA[:cnm2.estimates.A.shape[1], cnm2.estimates.A.shape[1]:])\n",
    "plt.colorbar();\n",
    "plt.xlabel('with patches')\n",
    "plt.ylabel('without patches')\n",
    "plt.title('Correlation coefficients');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stop the cluster when you're done"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%% STOP CLUSTER and clean up log files\n",
    "cm.stop_server(dview=dview)\n",
    "log_files = glob.glob('*_LOG_*')\n",
    "for log_file in log_files:\n",
    "    os.remove(log_file)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}