From 716c145d40b068dc025bb917f062e8d312d38ad9 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Tue, 13 Aug 2024 11:47:44 +0300 Subject: [PATCH] init macos-arm docs --- docs/install.md | 9 ++++- docs/macos.md | 80 ++++++++++++++++++++++++++++++++++++++++ docs/manual/kinds/srl.md | 17 +++++++++ 3 files changed, 104 insertions(+), 2 deletions(-) create mode 100644 docs/macos.md diff --git a/docs/install.md b/docs/install.md index 931229e33..f2a6fb670 100644 --- a/docs/install.md +++ b/docs/install.md @@ -2,7 +2,10 @@ hide: - navigation --- -Containerlab is distributed as a Linux deb/rpm package and can be installed on any Debian- or RHEL-like distributive in a matter of a few seconds. + +# Installation + +Containerlab is distributed as a Linux deb/rpm/apk package for amd64 and arm64 architectures and can be installed on any Debian- or RHEL-like distributive in a matter of a few seconds. ## Pre-requisites @@ -259,7 +262,9 @@ In Windows 11 with WSL2 it is now possible to [enable KVM support](https://serve ## Apple macOS -Running containerlab on macOS is possible both on ARM (M1/M2) and Intel chipsets with certain limitations and caveats rooted in different architectures and underlying OS. +Running containerlab on macOS is possible both on ARM (M1/M2/M3/etc) and Intel chipsets. For a long time, we had many caveats around M-chipsets on Macs, but with the introduction of ARM64-native NOSes like Nokia SR Linux and Arista cEOS, powered by Rosetta emulation for x86_64-based NOSes, it is now possible to run containerlab on ARM-based Macs. + +Since we wanted to share our experience with running containerlab on macOS in details, we have created a separate - [Containerlab on macOS](macos.md) - guide. ### ARM diff --git a/docs/macos.md b/docs/macos.md new file mode 100644 index 000000000..9f8c91457 --- /dev/null +++ b/docs/macos.md @@ -0,0 +1,80 @@ +--- +comments: true +hide: +- navigation +--- + +# Containerlab on macOS + +For quite some time, we have been saying that containerlab and macOS is a challenging mix. This statement has been echoed through multiple workshops, presentations, and demos was based on the following reasons: + +1. **ARM64 Network OS images**: With the shift to ARM64 architecture made by Apple (and Microsoft[^1]), we found ourselves in a situation where 99% of existing network OSes were not compiled for ARM64 architecture. This meant that containerlab users would have to rely on x86_64 emulation via Rosetta or QEMU, which imposes a significant performance penalty, often making the emulation unusable for practical purposes. +2. **Docker on macOS**: Since containerlab is reliant on Docker for container orchestrations, it needs Docker to be natively installed on the host. + On macOS, Docker is always provided as a Linux/arm64 VM that runs the docker daemon, with docker-cli running on macOS natively. You can imagine, that dealing with a VM that runs a network topology poses some UX challenges, like getting access to the exposed ports or dealing with files on macOS that needs to be accessible to the Docker VM. +3. **Linux on macOS?** It is not only Docker that containerlab is based on. We leverage some Linux kernel APIs (like netlink) either directly or via Docker to be available to setup links, namespaces, bind-mounts, etc. + Naturally, Darwin (macOS kernel) is not Linux, and while it is POSIX compliant, it is not a drop-in replacement for Linux. This means that some of the Linux-specific features that containerlab relies on are simply not present on macOS. + +Looking at the above challenges one might think that containerlab on macOS is a lost cause. However, recently things have started to change quite rapidly, and we are happy to say that for certain labs Containerlab on macOS might be a better choice overall. + +As a long time macOS user, Roman recorded an in-depth video demonstrating how to run containerlab topologies on macOS using the tools of his choice. You can watch the video below or jump to the text version of the guide below. + +> VIDEO HERE + +## ARM64 Network OS and application images + +Finally :pray: some good news on this front, as vendors started to release or at least announce ARM64 versions of their network OSes. +**Nokia** first released the preview version of their freely distributed [SR Linux for ARM64](manual/kinds/srl.md#getting-sr-linux-image), and **Arista** announced the imminent cEOS availability sometime in 2024. + +You can also get [**FRR**](https://quay.io/repository/frrouting/frr?tab=tags) container for ARM64 architecture from their container registry. + +/// details | What about VM-based images? +Unfortunately, we don't know if we will see ARM64 versions of the VM-based network OSes from the likes of Cisco and Juniper. + +You may still try running some of them using Rosetta emulation, but if you manage to get the VM booted, it might be very slow and not usable for practical purposes. Still worth a try though. +/// + +Yes, SR Linux, cEOS, FRR do not cover all the network OSes out there, but it is a good start, and we hope that more vendors will follow the trend. + +The good news is that almost all of the popular applications that we see being used in containerlabs are **already** built for ARM. Your streaming telemetry stack (gnmic, prometheus/influx, grafana), regular client-emulating endpoints such as Alpine or a collection of network related tools in the network-multitool image had already been supporting ARM architecture. You can leverage the sheer ecosystem multi-arch applications that are available in the public registries. + +## Docker on macOS + +Ever since macOS switched to ARM architecture for their processors, people in "containers camp" have been busy making sure that Docker works well on macOS's new architecture. + +### How Docker runs on Macs + +But before we start talking about Docker on ARM Macs, let's remind ourselves how Docker works on macOS with Intel processors. + +
+
+
Docker on Intel Macs
+
+ +At the heart of any product or project that enables the Docker engine on Mac[^2] is a Linux VM that hosts the Docker daemon, aka "the engine". This VM is created and managed by the application that sets up Docker on your desktop OS. +The Linux VM is a mandatory piece because the whole container ecosystem is built around Linux kernel features and APIs. Therefore, running Docker on any host with an operating system other than Linux requires a Linux VM. + +As shown above, on Intel Macs, the macOS runs Darwin kernel on top of an AMD64 (aka a86_64) architecture, and consequently, the Docker VM runs the same architecture. The architecture of the Docker VM is the same as the host architecture allowing for a performant virtualization, since no processor emulation is needed. + +Now let's see how things change when we switch to ARM Macs: + +
+
+
Docker on ARM Macs
+
+ +The diagram looks 99% the same as for the Intel Macs, the only difference being the architecture that macOS runs on and consequently the architecture of the Docker VM. +Now we run ARM64 architecture on the host, and the Docker VM is also ARM64. + +### Native vs Emulation + +If Docker runs exactly the same on ARM Macs as it does on Intel Macs, then why is it suddenly a problem to run containerlab on ARM Macs?x` + +Well, it all comes down to the requirement of having ARM64 network OS images that we dicsussed earlier. Now when your Docker VM runs Linux/ARM64, you can run natively only ARM64-native software in it, and we, as a network community, are not having a lot of ARM64-native network OSes. It is getting better, but we are not there yet to claim 100% parity with the x86_64 world. + +You should strive to run the native images as much as possible, as it gives you the best performance and compatibility. But how do you tell if the image is ARM64-native or not? +A lot of applications that you might want to run in your containerlab topologies are already ARM64-native and often available as a multi-arch image. + +[^1]: With Microsoft Surface laptop released with ARM64 architecture +[^2]: The same principles apply to Docker Desktop on Windows + + diff --git a/docs/manual/kinds/srl.md b/docs/manual/kinds/srl.md index bce00f817..7f2c9e312 100644 --- a/docs/manual/kinds/srl.md +++ b/docs/manual/kinds/srl.md @@ -17,6 +17,23 @@ docker pull ghcr.io/nokia/srlinux To pull a specific version, use tags that match the released version and are listed in the [srlinux-container-image](https://github.com/nokia/srlinux-container-image) repo. +//// admonition | ARM64-native SR Linux container image + type: tip +SR Linux Network OS is also available as an ARM64-native container image in a preview mode. The preview mode means that not every release is available in ARM64, but the selected ones are. + +Users can freely pull the following ARM64-native images: +/// tab | 24.7.1 + +```bash +docker pull ghcr.io/nokia/srlinux:24.7.1-arm64-preview +``` + +/// + +ARM64 image unlocks running networking labs on [Apple macOS](../../macos.md) with M-chips, as well as cloud instances with ARM64 architecture and on new Microsoft Surface laptops. + +//// + ## Managing SR Linux nodes There are many ways to manage SR Linux nodes, ranging from classic CLI management all the way up to the gNMI programming.