New ILAB Kernel - 2024.02

[jordancaraballo]

We will update with the existing version of the environment.

[jordancaraballo]

@ssfinch is there awscli in the current ilab-pytorch environment?

[ssfinch]

@ssfinch is there awscli in the current ilab-pytorch environment?

Doesn't look like it..

[sstrong@ilab201 ilab]$ micromamba activate /panfs/ccds02/app/modules/jupyter/ilab/pytorch-kernel
(/panfs/ccds02/app/modules/jupyter/ilab/pytorch-kernel-bak) [sstrong@ilab201 ilab]$ micromamba list |grep awscli
(/panfs/ccds02/app/modules/jupyter/ilab/pytorch-kernel-bak) [sstrong@ilab201 ilab]$

[ssfinch]

DEV kernel build steps:

# Step 1, install pytorch, rapids, cuda

micromamba create -p /panfs/ccds02/app/modules/jupyter/ilab/dev/pytorch-dev-rapids-v23_10-v20 -c rapidsai -c conda-forge -c pytorch -c nvidia rapids python=3.11 cuda-version pytorch torchvision torchaudio pytorch-cuda

# Step 2, activate env

[sstrong@ilab201 envs]$ micromamba activate /panfs/ccds02/app/modules/jupyter/ilab/dev/pytorch-dev-rapids-v23_10-v##

# Step 3, install all the others

(pytorch-dev-rapids-v23_10-v17) [sstrong@ilab201 envs]$ micromamba install -c conda-forge -c legate pip xarray dask matplotlib numpy pandas scikit-learn netCDF4 ipympl nodejs gdal rasterio seaborn geoviews cartopy hvplot holoviews rioxarray mkl jupyter-dash ipysheet=0.6.0 pytest omegaconf black contextily rio-cogeo localtileserver jupyter-server-proxy dask-geopandas ipyleaflet=0.19.1 leafmap pooch pythreejs owslib cunumeric optuna ipykernel plotnine mizani tiler hydra-core ipywidgets datasets webdataset rasterstats optuna dask-geopandas h5py pyarrow earthaccess huggingface_hub psycopg2 -y

# Step 4 , install condo-forge::flask-restx=1.2.0

(pytorch-dev-rapids-v23_10-v17) [sstrong@ilab201 envs]$ micromamba install conda-forge::flask-restx=1.2.0

# Step 5, remove torchvision

[sstrong@ilab201 envs]$ micromamba activate pytorch-dev-rapids-v23_10-v##

# Step 6, Install torchvision (while still in the env)

(pytorch-dev-rapids-v23_10-v17) [sstrong@ilab201 envs]$ pip install torchvision

# Step 7, Install matplotlib-scalebar (while still in env)

micromamba install conda-forge::matplotlib-scalebar

# Step 8, Install yacs (while still in env)

pip install yacs

# Step 9 , Install timm torchgeo (while still in env). New request from Jordan

Pip install timm torchgeo opencv-python

# Step 10, New request on 10/24/24 from Jordan

Pip install geemap earthengine-api

# Step 11, had to re install hd4 and lingual-hdf4. Due to error on in modis validation NB 10/25/24

(/panfs/ccds02/app/modules/jupyter/ilab/pytorch-kernel-latest) [sstrong@ilab201 ilab]$ mamba install -c conda-forge hdf4 libgdal-hdf4

# Step 12, had to re install rioarray rasterio Due to error in modis water validation NB. 10/25/24

(/panfs/ccds02/app/modules/jupyter/ilab/pytorch-kernel-latest) [sstrong@ilab201 ilab]$ mamba install --force-reinstall -c conda-forge rioxarray rasterio