Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IA-4808] Upgrade DSVM version to 24.03.14 (again) #4492

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from
Draft

[IA-4808] Upgrade DSVM version to 24.03.14 (again) #4492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/unit_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
fail_ci_if_error: false
fail_ci_if_error: true
token: ${{ secrets.CODECOV_TOKEN }}
version: "v0.1.15"

Expand Down
21 changes: 17 additions & 4 deletions http/src/main/resources/init-resources/azure_vm_init_script.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,23 @@ set -e
# 'debconf: unable to initialize frontend: Dialog'
export DEBIAN_FRONTEND=noninteractive

# For welder to work correctly, the welder work directory must be owned by user 1001,
# the DSVM WORKING_DIR owned by jupyter-user 1001 gets mounted as the Welder container work directory owned by user 1001
# (which luckily is welder-user within the welder container)
# TODO: make this logic less brittle --> utilize a shared user group for welder & jupyter

# The azureuser and leoAdmin ids must be shifted so that the jupyter user can be 1001
# (context in https://broadworkbench.atlassian.net/browse/IA-4808)
sudo groupmod -g 2000 azureuser
sudo usermod -u 2000 -g 2000 azureuser

sudo groupmod -g 1000 leoAdmin
sudo usermod -u 1000 -g 1000 leoAdmin

#create user to run jupyter
VM_JUP_USER=jupyter

sudo useradd -m -c "Jupyter User" $VM_JUP_USER
sudo useradd -m -c "Jupyter User" -u 1001 $VM_JUP_USER
sudo usermod -a -G $VM_JUP_USER,adm,dialout,cdrom,floppy,audio,dip,video,plugdev,lxd,netdev $VM_JUP_USER

## Change ownership for the new user
Expand Down Expand Up @@ -189,9 +202,9 @@ echo "Y"| /anaconda/envs/py38_default/bin/python3 -m ipykernel install
sudo runuser -l $VM_JUP_USER -c "mkdir -p /home/$VM_JUP_USER/.jupyter"
sudo runuser -l $VM_JUP_USER -c "wget -qP /home/$VM_JUP_USER/.jupyter https://raw.githubusercontent.com/DataBiosphere/leonardo/ea519ef899de28e27e2a37ba368433da9fd03b7f/http/src/main/resources/init-resources/jupyter_server_config.py"
# We pull the jupyter_delocalize.py file from the base terra-docker python image, but it was designed for notebooks and we need to make a couple of changes to make it work with server instead
sudo runuser -l $VM_JUP_USER -c "wget -qP /anaconda/lib/python3.10/site-packages https://raw.githubusercontent.com/DataBiosphere/terra-docker/0ea6d2ebd7fcae7072e01e1c2f2d178390a276b0/terra-jupyter-base/custom/jupyter_delocalize.py"
sudo runuser -l $VM_JUP_USER -c "sed -i 's/notebook.services/jupyter_server.services/g' /anaconda/lib/python3.10/site-packages/jupyter_delocalize.py"
sudo runuser -l $VM_JUP_USER -c "sed -i 's/http:\/\/welder:8080/http:\/\/127.0.0.1:8081/g' /anaconda/lib/python3.10/site-packages/jupyter_delocalize.py"
sudo runuser -l $VM_JUP_USER -c "wget -qP /anaconda/lib/python3.11/site-packages https://raw.githubusercontent.com/DataBiosphere/terra-docker/0ea6d2ebd7fcae7072e01e1c2f2d178390a276b0/terra-jupyter-base/custom/jupyter_delocalize.py"
sudo runuser -l $VM_JUP_USER -c "sed -i 's/notebook.services/jupyter_server.services/g' /anaconda/lib/python3.11/site-packages/jupyter_delocalize.py"
sudo runuser -l $VM_JUP_USER -c "sed -i 's/http:\/\/welder:8080/http:\/\/127.0.0.1:8081/g' /anaconda/lib/python3.11/site-packages/jupyter_delocalize.py"

echo "------ Jupyter ------"
echo "Starting Jupyter with command..."
Expand Down
4 changes: 2 additions & 2 deletions http/src/main/resources/reference.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,15 +242,15 @@ azure {
publisher = "microsoft-dsvm"
offer = "ubuntu-2004"
sku = "2004-gen2"
version = "23.04.24"
version = "24.03.14"
}
custom-script-extension {
name = "vm-custom-script-extension",
publisher = "Microsoft.Azure.Extensions",
type = "CustomScript",
version = "2.1",
minor-version-auto-upgrade = true,
file-uris = ["https://raw.githubusercontent.com/DataBiosphere/leonardo/52aab3b7f252667f73b23682062ab3e0d9d533b9/http/src/main/resources/init-resources/azure_vm_init_script.sh"]
file-uris = ["https://raw.githubusercontent.com/DataBiosphere/leonardo/6aaa15eb5d58818525c96c1fef0eea3e982a2ab1/http/src/main/resources/init-resources/azure_vm_init_script.sh"]
}
listener-image = "terradevacrpublic.azurecr.io/terra-azure-relay-listeners:ad57bda"
}
Expand Down
4 changes: 2 additions & 2 deletions http/src/test/scala/org/broadinstitute/dsde/workbench/leonardo/dao/WsmCodecSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ class WsmCodecSpec extends AnyFlatSpec with Matchers {
| "publisher": "microsoft-dsvm",
| "offer": "ubuntu-2004",
| "sku": "2004-gen2",
| "version": "23.04.24"
| "version": "24.03.14"
| },
| "customScriptExtension": {
| "name": "vm-custom-script-extension",
Expand All @@ -108,7 +108,7 @@ class WsmCodecSpec extends AnyFlatSpec with Matchers {
| "minorVersionAutoUpgrade": true,
| "protectedSettings": [{
| "key": "fileUris",
| "value": ["https://raw.githubusercontent.com/DataBiosphere/leonardo/52aab3b7f252667f73b23682062ab3e0d9d533b9/http/src/main/resources/init-resources/azure_vm_init_script.sh"]
| "value": ["https://raw.githubusercontent.com/DataBiosphere/leonardo/6aaa15eb5d58818525c96c1fef0eea3e982a2ab1/http/src/main/resources/init-resources/azure_vm_init_script.sh"]
| },
| {
| "key": "commandToExecute",
Expand Down
4 changes: 2 additions & 2 deletions http/src/test/scala/org/broadinstitute/dsde/workbench/leonardo/http/ConfigReaderSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ class ConfigReaderSpec extends AnyFlatSpec with Matchers {
"microsoft-dsvm",
"ubuntu-2004",
"2004-gen2",
"23.04.24"
"24.03.14"
),
CustomScriptExtensionConfig(
"vm-custom-script-extension",
Expand All @@ -73,7 +73,7 @@ class ConfigReaderSpec extends AnyFlatSpec with Matchers {
"2.1",
true,
List(
"https://raw.githubusercontent.com/DataBiosphere/leonardo/52aab3b7f252667f73b23682062ab3e0d9d533b9/http/src/main/resources/init-resources/azure_vm_init_script.sh"
"https://raw.githubusercontent.com/DataBiosphere/leonardo/6aaa15eb5d58818525c96c1fef0eea3e982a2ab1/http/src/main/resources/init-resources/azure_vm_init_script.sh"
)
),
"terradevacrpublic.azurecr.io/terra-azure-relay-listeners:ad57bda",
Expand Down
2 changes: 1 addition & 1 deletion .../org/broadinstitute/dsde/workbench/leonardo/http/service/RuntimeV2ServiceInterpSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ class RuntimeV2ServiceInterpSpec extends AnyFlatSpec with LeonardoTestSuite with
val expectedRuntimeImage = Set(
RuntimeImage(
RuntimeImageType.Azure,
"microsoft-dsvm, ubuntu-2004, 2004-gen2, 23.04.24",
"microsoft-dsvm, ubuntu-2004, 2004-gen2, 24.03.14",
None,
context.now
),
Expand Down
Loading