Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PRISM-646 - Deploy new manager #9

Merged
merged 5 commits into from
Feb 5, 2025
Merged

PRISM-646 - Deploy new manager #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5969ae3
NOISSUE - Update SEV requirement (#330)
dependabot[bot] Jan 31, 2025
da88fe1
COCOS-346 - Explore cloud init for Cloud setup (#357)
smithjilks Jan 31, 2025
f45fd26
fix env
WashingtonKK Feb 3, 2025
199f831
update kernel and rootfs location
WashingtonKK Feb 3, 2025
68ee19e
update manager host
WashingtonKK Feb 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions .github/workflows/hal.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
name: Build and Release
name: Build and Release Hal

on:
push:
tags:
- '*'
- "*"

jobs:
build:
Expand Down Expand Up @@ -32,8 +32,8 @@ jobs:
with:
root-reserve-mb: 35000
swap-size-mb: 1024
remove-dotnet: 'true'
remove-android: 'true'
remove-dotnet: "true"
remove-android: "true"
- name: Check free space
run: |
echo "Free space:"
Expand All @@ -48,26 +48,33 @@ jobs:
- name: Checkout cocos
uses: actions/checkout@v4
with:
repository: 'ultravioletrs/cocos'
repository: "ultravioletrs/cocos"
path: cocos

- name: Checkout buildroot
uses: actions/checkout@v4
with:
repository: 'buildroot/buildroot'
repository: "buildroot/buildroot"
path: buildroot
ref: 2024.11-rc2

- name: Build
- name: Build hal
run: |
cd buildroot
make BR2_EXTERNAL=../cocos/hal/linux cocos_defconfig
make

- name: Build cocos
run: |
cd cocos
make

- name: Release
uses: softprops/action-gh-release@v2
with:
files: |
buildroot/output/images/bzImage
buildroot/output/images/rootfs.cpio.gz

cocos/build/cocos-agent
cocos/build/cocos-cli
cocos/build/cocos-manager
11 changes: 6 additions & 5 deletions cocos-manager.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ MANAGER_ATTESTATION_POLICY_BINARY=../../build
MANAGER_GRPC_CLIENT_CERT=
MANAGER_GRPC_CLIENT_KEY=
MANAGER_GRPC_SERVER_CA_CERTS=
MANAGER_GRPC_URL=localhost:7001
MANAGER_GRPC_PORT=6101
MANAGER_GRPC_HOST=0.0.0.0
MANAGER_GRPC_TIMEOUT=60s
MANAGER_EOS_VERSION=""

Expand All @@ -21,22 +22,22 @@ MANAGER_QEMU_MAX_MEMORY=30G
MANAGER_QEMU_OVMF_CODE_IF=pflash
MANAGER_QEMU_OVMF_CODE_FORMAT=raw
MANAGER_QEMU_OVMF_CODE_UNIT=0
MANAGER_QEMU_OVMF_CODE_FILE=/usr/share/OVMF/x64/OVMF_CODE.fd
MANAGER_QEMU_OVMF_CODE_FILE=/usr/share/edk2/x64/OVMF_CODE.fd
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

OVMF Path Verification Issue

  • The configuration references /usr/share/edk2/x64/OVMF_CODE.fd (and similarly /usr/share/edk2/x64/OVMF_VARS.fd).
  • The verification script output returned “OVMF files not found,” indicating these files are absent in the expected directory.
  • This discrepancy should be addressed to ensure compatibility on Dell machines—either by verifying that the edk2 package correctly installs these files on the target systems or by updating the path configuration accordingly.
🔗 Analysis chain

Verify OVMF paths for Dell machines.

The OVMF paths have been updated to use the edk2 package. Since this PR aims to enable deployment on Dell machines, please verify these paths:

  • /usr/share/edk2/x64/OVMF_CODE.fd
  • /usr/share/edk2/x64/OVMF_VARS.fd

Run this script to check path compatibility:

Also applies to: 31-31

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if the OVMF paths exist and are accessible
# Note: This should be run on a Dell machine to verify compatibility

# Check if edk2 package is available
if command -v dpkg >/dev/null 2>&1; then
    dpkg -l | grep edk2
elif command -v rpm >/dev/null 2>&1; then
    rpm -qa | grep edk2
fi

# Check if OVMF paths exist
ls -l /usr/share/edk2/x64/OVMF_*.fd 2>/dev/null || echo "OVMF files not found"

# Search for alternative OVMF locations
fd -g "OVMF_*.fd" /usr/share

Length of output: 156

MANAGER_QEMU_OVMF_VERSION=edk2-stable202408
MANAGER_QEMU_OVMF_CODE_READONLY=on
MANAGER_QEMU_OVMF_VARS_IF=pflash
MANAGER_QEMU_OVMF_VARS_FORMAT=raw
MANAGER_QEMU_OVMF_VARS_UNIT=1
MANAGER_QEMU_OVMF_VARS_FILE=/usr/share/OVMF/x64/OVMF_VARS.fd
MANAGER_QEMU_OVMF_VARS_FILE=/usr/share/edk2/x64/OVMF_VARS.fd
MANAGER_QEMU_NETDEV_ID=vmnic
MANAGER_QEMU_HOST_FWD_AGENT=7020
MANAGER_QEMU_GUEST_FWD_AGENT=7002
MANAGER_QEMU_VIRTIO_NET_PCI_DISABLE_LEGACY=on
MANAGER_QEMU_VIRTIO_NET_PCI_IOMMU_PLATFORM=true
MANAGER_QEMU_VIRTIO_NET_PCI_ADDR=0x2
MANAGER_QEMU_VIRTIO_NET_PCI_ROMFILE=
MANAGER_QEMU_DISK_IMG_KERNEL_FILE=/home/sammyk/Documents/cocos-ai/cmd/manager/img/bzImage
MANAGER_QEMU_DISK_IMG_ROOTFS_FILE=/home/sammyk/Documents/cocos-ai/cmd/manager/img/rootfs.cpio.gz
MANAGER_QEMU_DISK_IMG_KERNEL_FILE=/etc/cocos/bzImage
MANAGER_QEMU_DISK_IMG_ROOTFS_FILE=/etc/cocos/rootfs.cpio.gz
Comment on lines +39 to +40
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Action Required: Create and Set Permissions for /etc/cocos

The verification revealed that the /etc/cocos directory and its associated disk image files (bzImage and rootfs.cpio.gz) are missing. While sudo access is properly configured, you need to ensure that the /etc/cocos directory is created with correct ownership and permission settings to support system-wide deployment.

  • /etc/cocos Directory: Not found; it must be created.
  • Disk Image Files: Not present in /etc/cocos; ensure they are appropriately placed.
  • Sudo Configuration: Confirmed, but directory setup is required.
🔗 Analysis chain

Ensure /etc/cocos directory setup and permissions.

The disk image paths have been moved to /etc/cocos/, which is more appropriate for system-wide deployment. However, this requires proper directory setup and permissions.

Run this script to verify directory setup:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check directory setup and permissions for /etc/cocos

# Check if directory exists and its permissions
ls -ld /etc/cocos 2>/dev/null || echo "Directory not found"

# Check if files exist and their permissions
ls -l /etc/cocos/bzImage /etc/cocos/rootfs.cpio.gz 2>/dev/null || echo "Files not found"

# Check if MANAGER_QEMU_USE_SUDO is properly configured for file access
rg "MANAGER_QEMU_USE_SUDO.*true" && echo "Sudo access configured"

Length of output: 344

MANAGER_QEMU_SEV_ID=sev0
MANAGER_QEMU_SEV_CBITPOS=51
MANAGER_QEMU_SEV_REDUCED_PHYS_BITS=1
Expand Down
89 changes: 89 additions & 0 deletions hal/cloud/.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
#### memory config
MEMORY_SIZE=2048M
MEMORY_SLOTS=5
MAX_MEMORY=30G


#### ovmf code config
OVMF_CODE_IF=pflash
OVMF_CODE_FORMAT=raw
OVMF_CODE_UNIT=0
OVMF_CODE_FILE=/usr/share/OVMF/OVMF_CODE.fd
OVMF_CODE_READONLY=on
Comment on lines +11 to +12
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Standardize OVMF paths across configuration files.

The OVMF paths are inconsistent between this file and cocos-manager.env:

  • This file: /usr/share/OVMF/OVMF_CODE.fd and /usr/share/ovmf/x64/OVMF_CODE.4m.fd
  • cocos-manager.env: /usr/share/edk2/x64/OVMF_CODE.fd

Please standardize these paths across all configuration files.

Also applies to: 77-78

OVMF_VERSION=

#### ovmf vars config
OVMF_VARS_IF=pflash
OVMF_VARS_FORMAT=raw
OVMF_VARS_UNIT=1
OVMF_VARS_FILE=/usr/share/OVMF/OVMF_VARS.fd

#### net dev config
NET_DEV_ID=vmnic
NET_DEV_HOST_FWD_AGENT=7020
NET_DEV_GUEST_FWD_AGENT=7002

#### Virtio Net Pci Config
VIRTIO_NET_PCI_DISABLE_LEGACY=on
VIRTIO_NET_PCI_IOMMU_PLATFORM=true
VIRTIO_NET_PCI_ADDR=0x2
VIRTIO_NET_PCI_ROMFILE=

#### Disk image config
DISK_IMG_KERNEL_FILE=
DISK_IMG_ROOTFS_FILE=

KERNEL_COMMAND_LINE="quiet console=null"

#### Sev Config
SEV_ID=sev0
SEV_CBIT_POS=51
SEV_REDUCED_PHYS_BITS=1
SEV_HOST_DATA=

#### VSock Config
VSOCK_ID=vhost-vsock-pci0
VSOCK_GUEST_CID=3

BIN_PATH=qemu-system-x86_64
USE_SUDO=false
ENABLE_SEV=false
ENABLE_SEV_SNP=false

ENABLE_KVM=true

MACHINE=q35
CPU=EPYC
SMP_COUNT=8
SMP_MAXCPUS=64
MEM_ID=ram1

KERNEL_HASH=false

NO_GRAPHIC=true
MONITOR=pty
HOST_FWD_RANGE=6100-6200
CERTS_MOUNT=/etc/cocos/certs
ENV_MOUNT=/etc/cocos/environment

COCOS_AGENT_VERSION=v0.3.1

#### Base image URL and names
BASE_IMAGE_URL=https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
BASE_IMAGE=ubuntu-base.qcow2
CUSTOM_IMAGE=ubuntu-custom.qcow2

#### Paths for OVMF firmware
OVMF_CODE=/usr/share/ovmf/x64/OVMF_CODE.4m.fd
OVMF_VARS=/usr/share/ovmf/x64/OVMF_VARS.4m.fd

#### VM parameters
VM_NAME=cocos-vm
RAM=16G
DISK_SIZE=10G # Size for root filesystem
QEMU_BINARY=qemu-system-x86_64

AGENT_GRPC_SERVER_CERT=/etc/cocos/certs/server.pem
AGENT_GRPC_SERVER_KEY=/etc/cocos/certs/key.pem
AGENT_GRPC_SERVER_CA_CERTS=/etc/cocos/ca.pem
AGENT_GRPC_CLIENT_CA_CERTS=/etc/cocos/ca.pem
114 changes: 114 additions & 0 deletions hal/cloud/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Agent Cloud Init Setup

## Overview

The `hal/cloud` directory contains essential files required for setting up a virtual machine (VM) with cloud-init. This setup ensures the automated installation of dependencies, configuration of the environment, and deployment of the Cocos agent as a systemd service.

### Directory Contents

- **`config.yaml`**: This YAML file provides configuration instructions for the cloud image.
- **`meta-data`**: Contains VM metadata, such as instance-specific details and identifiers.
- **`qemu.sh`**: A Bash script for downloading and configuring a cloud image, running QEMU to simulate a VM with the cloud-init configuration.
- **`.env`**: Contains environment variables for starting the VM in different modes, configuring disk space, memory allocation, and other parameters.

## Configuration

### Preparing the Cloud-Config File

The `config.yaml` file defines system configurations, including user creation, package installations, file management, and command execution.
Ensure that the cloud-config file is set up with the following configurations:

- **User Credentials**: Specify the default username and password.
- **Certificates and Keys**: Certificate files for agent for secure communication.
- **Environment Variables**: Configuration parameters required by the system.

The `config.yaml` file is divided into multiple sections, each addressing a specific aspect of the setup process.

### 1. User Configuration

This section creates a default user with specific permissions and configurations:

- Creates a user named **`cocos_user`**.
- Adds `cocos_user` to the `sudo` and `docker` groups.
- Sets a default password (should be changed for production use).
- Configures the user’s shell as `/bin/bash`.

### 2. Package Installation

Installs essential system packages required for various operations:

- **`curl`**: For downloading files from the web.
- **`make`**: A utility for building software.
- **`git`**: Version control system for managing code repositories.
- **`python3` and `python3-dev`**: Required for running Python-based tools.
- **`net-tools`**: Provides networking utilities such as `ifconfig` and `route`.

### 3. File Management (write_files)

Creates and configures critical files required for the setup:

- **Certificates**: Cert files (`cert.pem`, `ca.pem`, `key.pem`) located at `/etc/cocos/certs/`.
- **Environment Variables**: An env file stored at `/etc/cocos/environment`.
- **Systemd Service File**: Cocos agent service configuration file at `/etc/systemd/system/cocos-agent.service` for managing the Cocos agent.
- **Agent Scripts**:
- `agent_setup.sh`: Configures network interfaces and resizes the root filesystem.
- `agent_start_script.sh`: Sets up Docker and starts the Cocos agent.

### 4. Execution of Commands (runcmd)

A sequence of commands is executed to finalize the setup:

- Creates necessary directories: `/cocos`, `/cocos_init`, `/var/log/cocos`, `/etc/cocos`.
- Downloads and installs the Cocos agent binary.
- Installs **Wasmtime** and configures its environment variables.
- Installs **Docker** and adds `cocos_user` to the Docker group.
- Reloads systemd and enables the Cocos agent service.

## Running the Agent

To test the cloud-init configuration, execute the `qemu.sh` script to bring up a VM using QEMU:

```bash
sudo ./qemu.sh
```

**Important:** The script must be executed as root.

Once the QEMU boots the VM, the Cocos agent will run as a systemd service. The service is configured to start automatically on boot and restart in case of failure.

## Debugging and Monitoring

For troubleshooting and monitoring the Cocos agent service, use the following commands within the VM:

### Manually Start the Service

To manually start the agent service, execute:

```bash
sudo systemctl start cocos-agent.service
```

### Verify Service Status

To check if the service is running properly, use:

```bash
sudo systemctl status cocos-agent.service
```

### View Service Logs

To inspect logs generated by the agent service, execute:

```bash
journalctl -u cocos-agent.service
```

### Check Standard Output and Error Logs

To check logs stored in the system, use:

```bash
cat /var/log/cocos/agent.stdout.log
cat /var/log/cocos/agent.stderr.log
```
Loading