Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Docker Compose depends_on long syntax #1718

Merged
merged 5 commits into from
Nov 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions it/docker-compose-dependon/Postgres.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM postgres:13-alpine
HEALTHCHECK --interval=5s --timeout=5s --retries=5 \
CMD "pg_isready" "-U" "postgres"
27 changes: 27 additions & 0 deletions it/docker-compose-dependon/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
version: "2.4"
services:
init:
image: alpine:latest
command:
- echo
- "Hello world!"
db:
image: localpg
build:
context: .
dockerfile: Postgres.Dockerfile
environment:
POSTGRES_PASSWORD: supersecret
tmpfs:
- /var/lib/postgresql/data
depends_on:
init:
condition: service_completed_successfully
web:
image: alpine:latest
command:
- echo
- "Hello foobar (after hello world)!"
depends_on:
db:
condition: service_healthy
77 changes: 77 additions & 0 deletions it/docker-compose-dependon/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<!--
Integration test demo with wait configurations mapped from Docker compose depends_on long syntax

Call it with: 'mvn verify'

The test does the following:

* Builds a custom postgres image with a healtcheck (because we do not support adding healthchecks from compose yet)
* Start an init container printing a message
* When successfully exited, database container starts
* Once Postgres enters state healthy, another simple container printing a message is started
* Stops and removes the containers.

-->

<parent>
<groupId>io.fabric8.dmp.itests</groupId>
<artifactId>dmp-it-parent</artifactId>
<version>0.44-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>dmp-it-docker-compose-dependon</artifactId>
<version>0.44-SNAPSHOT</version>

<build>
<plugins>
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<configuration>
<images>
<image>
<alias>web</alias>
<name>alpine:latest</name>
<external>
<type>compose</type>
<basedir>${project.basedir}</basedir>
</external>
</image>
</images>
</configuration>
<executions>
<execution>
<id>build</id>
<goals>
<goal>build</goal>
</goals>
<phase>verify</phase>
</execution>
<execution>
<id>start</id>
<goals>
<goal>start</goal>
</goals>
<phase>verify</phase>
<configuration>
<showLogs>true</showLogs>
</configuration>
</execution>
<execution>
<id>stop</id>
<goals>
<goal>stop</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
1 change: 1 addition & 0 deletions it/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
<module>buildx-dockerfile</module>
<module>buildx-push</module>
<module>docker-compose</module>
<module>docker-compose-dependon</module>
<module>dockerfile</module>
<module>dockerignore</module>
<module>healthcheck</module>
Expand Down
3 changes: 2 additions & 1 deletion src/main/asciidoc/inc/external/_docker_compose.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ In addition to the `docker-compose.yml` you can add all known options for <<buil

The following Docker Compose file keywords are not yet supported:

* `cgroup_parent`, `devices`, `env_file`, `expose`, `pid`, `security_opt`, `stop_signal`, `cpu_quota`, `ipc`, `mac_address`, `read_only` are not yet supported (but might be in a future version).
* `cgroup_parent`, `devices`, `env_file`, `expose`, `pid`, `security_opt`, `stop_signal`, `cpu_quota`, `ipc`, `mac_address`, `read_only`, `healthcheck` are not yet supported (but might be in a future version).
* `extend` for including other Docker Compose files is not yet implemented.
* Only **services** are currently evaluated, there is no supported yet for **volumes** and **networks**.
* When using https://docs.docker.com/compose/compose-file/compose-file-v2/#depends_on[`depends_on` with long syntax] in a Docker Compose file, be advised the plugin cannot apply all usage constellations expressible in it. The root cause is this plugin uses the concept of pausing execution based on <<start-wait,wait conditions>> attached to dependent containers, while Docker Compose applies checks when starting the depending container. Keep in mind that execution of a container is continued as soon as any wait condition is fulfilled.
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.io.*;
import java.util.*;
import java.util.stream.Collectors;

import io.fabric8.maven.docker.config.*;
import io.fabric8.maven.docker.config.handler.ExternalConfigHandler;
Expand Down Expand Up @@ -39,11 +40,11 @@ public String getType() {
@Override
@SuppressWarnings("unchecked")
public List<ImageConfiguration> resolve(ImageConfiguration unresolvedConfig, MavenProject project, MavenSession session) {
List<ImageConfiguration> resolved = new ArrayList<>();

DockerComposeConfiguration handlerConfig = new DockerComposeConfiguration(unresolvedConfig.getExternalConfig());
File composeFile = resolveComposeFileAbsolutely(handlerConfig.getBasedir(), handlerConfig.getComposeFile(), project);
Map<String, DockerComposeServiceWrapper> allServices = new LinkedHashMap<>();

// First retrieve all services from the compose file
for (Object composeO : getComposeConfigurations(composeFile, project, session)) {
Map<String, Object> compose = (Map<String, Object>) composeO;
validateVersion(compose, composeFile);
Expand All @@ -52,12 +53,35 @@ public List<ImageConfiguration> resolve(ImageConfiguration unresolvedConfig, Mav
String serviceName = entry.getKey();
Map<String, Object> serviceDefinition = (Map<String, Object>) entry.getValue();

DockerComposeServiceWrapper mapper = new DockerComposeServiceWrapper(serviceName, composeFile, serviceDefinition, unresolvedConfig, resolveAbsolutely(handlerConfig.getBasedir(), project));
resolved.add(buildImageConfiguration(mapper, composeFile.getParentFile(), unresolvedConfig, handlerConfig));
allServices.put(serviceName, new DockerComposeServiceWrapper(serviceName, composeFile, serviceDefinition,
unresolvedConfig, resolveAbsolutely(handlerConfig.getBasedir(), project)));
}
}

return resolved;

// Loop over all known services and add wait configurations where necessary
for (DockerComposeServiceWrapper service : allServices.values()) {
for (String dependentServiceName : service.getDependsOn()) {
DockerComposeServiceWrapper dependentService = allServices.get(dependentServiceName);

if (dependentService != null) {
if (service.equals(dependentService)) {
service.throwIllegalArgumentException("Invalid self-reference in dependent services");
}

// Note: for short syntax, we don't need to do anything else
if (service.usesLongSyntaxDependsOn()) {
dependentService.enableWaitCondition(service.getWaitCondition(dependentServiceName));
}
} else {
service.throwIllegalArgumentException("Undefined dependent service \"" + dependentServiceName + "\"");
}
}
}

// Now that we cross-correlated all dependencies from all services, let's build & return image configurations
return allServices.values().stream()
.map(svc -> buildImageConfiguration(svc, composeFile.getParentFile(), unresolvedConfig, handlerConfig))
.collect(Collectors.toList());
}

private void validateVersion(Map<String, Object> compose, File file) {
Expand Down Expand Up @@ -174,6 +198,7 @@ private RunImageConfiguration createRunConfiguration(DockerComposeServiceWrapper
// container_name is taken as an alias and ignored here for run config
// devices not supported
.dependsOn(wrapper.getDependsOn()) // depends_on relies that no container_name is set
.wait(wrapper.getWaitConfiguration())
.dns(wrapper.getDns())
.dnsSearch(wrapper.getDnsSearch())
.tmpfs(wrapper.getTmpfs())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;

import io.fabric8.maven.docker.config.Arguments;
import io.fabric8.maven.docker.config.ImageConfiguration;
Expand All @@ -15,6 +16,7 @@
import io.fabric8.maven.docker.config.RestartPolicy;
import io.fabric8.maven.docker.config.RunVolumeConfiguration;
import io.fabric8.maven.docker.config.UlimitConfig;
import io.fabric8.maven.docker.config.WaitConfiguration;
import io.fabric8.maven.docker.util.VolumeBindingUtil;


Expand Down Expand Up @@ -114,7 +116,135 @@ Arguments getCommand() {
}

List<String> getDependsOn() {
return asList("depends_on");
try {
return asList("depends_on");
// With the new long style of depends_on since compose 2.1+, this may be a map.
// Maps' keys still are the container names though.
} catch (ClassCastException e) {
return new ArrayList<>(asMap("depends_on").keySet());
}
}

boolean usesLongSyntaxDependsOn() {
return asObject("depends_on") instanceof Map;
}

/**
* <a href="https://docs.docker.com/compose/compose-file/compose-file-v2/#depends_on">Docker Compose Spec v2.1+ defined conditions</a>
*/
enum WaitCondition {
HEALTHY("service_healthy"),
COMPLETED("service_completed_successfully"),
STARTED("service_started");

private final String condition;
WaitCondition(String condition) {
this.condition = condition;
}

static WaitCondition fromString(String string) {
return Arrays.stream(WaitCondition.values()).filter(wc -> wc.condition.equals(string)).findFirst().orElseThrow(
() -> new IllegalArgumentException("invalid condition \"" + string + "\"")
);
}
}

/**
* Extract a required condition of another (dependent) service from this service.
* In a compose file following v2.1+ format this looks like this:
* <pre>{@code
* services:
* web:
* build: .
* depends_on:
* db:
* condition: service_healthy
* redis:
* condition: service_started
* }</pre>
* If this is the "web" service and the parameter to this method is "db", it will extract
* the validated condition via {@link WaitCondition}.
*
* @param dependentServiceName The dependent service's name (not alias!)
* @return The waiting condition if valid
* @throws IllegalArgumentException When condition cannot be extracted or is invalid
*/
WaitCondition getWaitCondition(String dependentServiceName) {
Objects.requireNonNull(dependentServiceName, "Dependent service's name may not be null");

Object dependsOnObj = asObject("depends_on");
if (dependsOnObj instanceof Map) {
Map<String, Object> dependsOn = (Map<String, Object>) dependsOnObj;
Object dependenSvcObj = dependsOn.get(dependentServiceName);

if (dependenSvcObj instanceof Map) {
Map<String, String> dependency = (Map<String,String>) dependenSvcObj;

if (dependency.containsKey("condition")) {
String condition = dependency.get("condition");
try {
return WaitCondition.fromString(condition);
} catch (IllegalArgumentException e) {
throwIllegalArgumentException("depends_on service \"" + dependentServiceName +
"\" has invalid syntax (" + e.getMessage() + ")");
}
}
throwIllegalArgumentException("depends_on service \"" + dependentServiceName + "\" has invalid syntax (missing condition)");
}
throwIllegalArgumentException("depends_on service \"" + dependentServiceName + "\" has invalid syntax (no map)");
}
throwIllegalArgumentException("depends_on does not use long syntax, cannot retrieve condition");
return null;
}

private boolean healthyWaitRequested;
private boolean successExitWaitRequested;

/**
* Switch on waiting conditions for this service.
* It will not yet check for conflicting conditions, this is done in {@link #getWaitConfiguration()}
* when building the image configurations.
* @param condition The condition to enable for this service
*/
void enableWaitCondition(WaitCondition condition) {
Objects.requireNonNull(condition, "Condition may not be null");

// We do not check for conflicting conditions here - this is done when the wrapper is asked for its WaitConfig
// Note: yes, we check here again, as we rely on Strings, not an enum
switch (condition) {
case HEALTHY:
this.healthyWaitRequested = true;
break;
case COMPLETED:
this.successExitWaitRequested = true;
break;
case STARTED:
// No need to do anything here.
// This is equivalent to the short syntax and simple startup ordering doesn't need a wait configuration.
default:
// Do nothing when unknown condition
}
}

/**
* Build the actual wait configuration for this service.
* <p>Please note: while Docker Compose allows you to create a dependency graph which will allow to wait
* for a dependent service to exit from one service and at the same time wait for it to be healthy from another,
* this is not possible with this Maven Plugin.</p>
* <p>The reasoning behind this limitation is rooted in the data model. Docker Compose attaches the
* requested conditions at the depending service level, while this plugin only supports wait conditions
* on the dependent service. Once these are fulfilled, all depending services will be started.</p>
* @return The wait configuration
*/
WaitConfiguration getWaitConfiguration() {
if (successExitWaitRequested && healthyWaitRequested) {
throwIllegalArgumentException("Conflicting requested conditions \"service_healthy\" and \"service_completed_successfully\" for this service");
} else if (healthyWaitRequested) {
return new WaitConfiguration.Builder().healthy(healthyWaitRequested).build();
} else if (successExitWaitRequested) {
return new WaitConfiguration.Builder().exit(0).build();
}
return null;
}

List<String> getDns() {
Expand Down Expand Up @@ -448,9 +578,21 @@ private Map<String, String> convertToMap(List<String> list) {
return map;
}

private void throwIllegalArgumentException(String msg) {
void throwIllegalArgumentException(String msg) {
throw new IllegalArgumentException(String.format("%s: %s - ", composeFile, name) + msg);
}


@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof DockerComposeServiceWrapper)) return false;
DockerComposeServiceWrapper that = (DockerComposeServiceWrapper) o;
return Objects.equals(name, that.name);
}

@Override
public int hashCode() {
return Objects.hash(name);
}
}

Loading
Loading