The bad situation with bind volumes

[steveiliop56]

Drupal is really cool but the docker image has an issue where it makes it really annoying to host using docker images, bind volumes. For example a lot of us just create a docker compose with bind volumes to our data and start the app but due to the (bad) design choice of Drupal's dockerfile the volume location has already data which are getting replaced by docker. The solution is as simple as storing the data on a template location and copy them just before the web server starts. I would really like to see this implemented as it would make self hosting Drupal with docker extremely easy.

[LaurentGoderre]

The volume has data to get started quickly. If you bind your own folder, it would use that folder instead of the image one

[steveiliop56]

Yeah but I want to store my Drupal data on a folder in my filesystem and if I use a bind volume to do so Drupal's data will be replaced with nothing. The container should simply copy the data from another folder which isn't binded to the folder that is binded and probablem solved.

[mstenta]

For what it's worth, we do something like this in the farmOS Docker image (which extends from drupal). We build our codebase in a temporary directory (/var/farmOS) and then copy it into /opt/drupal towards the end of the Dockerfile build process. This serves to have a copy of the code available elsewhere in the image.

https://github.com/farmOS/farmOS/blob/3a73ccf6360ba309ee657e2d645d29c23994cbc6/docker/Dockerfile#L143

Then, to handle bind-mounted volumes (which are empty when the container first starts), we use a custom ENTRYPOINT script that checks to see if either /opt/drupal (for development environments, which bind-mount the entire codebase) or /opt/drupal/web/sites (for production environments, which only bind-mount the sites directory) are empty, and if they are then it copies the files from the clean codebase that was built in /var/farmOS.

https://github.com/farmOS/farmOS/blob/3.x/docker/docker-entrypoint.sh

This serves us well. However it is somewhat custom to our use-case, and an entrypoint script seems to be the only way to achieve it. It's not possible to do this in the Dockerfile alone because the logic for copying files needs to happen when the container starts (and therefore when volumes are bind-mounted).

That means that in order for this image (drupal) to provide the same, it would need to do it in a docker-entrypoint.sh script, too. And it might need to make similar assumptions about dev/prod environment needs. The drupal image does not currently use ENTRYPOINT (I don't think?) so that would be a departure from the current approach, and would add complexity.

@steveiliop56 It feels to me like this doesn't need to be implemented in the drupal image, since it can be easily added to a downstream derivative image that inherits from drupal. But I suppose I could see arguments either way.

Just my 2 cents, and how we're achieving it. Hope that helps!

[steveiliop56]

I mean drupal is essentially a webserver serving php, from what I see it's using apache or fpm so the entry point should be fairly easy to change.

@mstenta I didn't understand this:

It feels to me like this doesn't need to be implemented in the drupal image, since it can be easily added to a downstream derivative image that inherits from drupal

What do you mean downstream derivative image?

[mstenta]

What do you mean downstream derivative image?

I mean creating your own Dockerfile that inherits FROM drupal as a base image, and adding your own ENTRYPOINT script on top.

For farmOS, we create a farmos/farmos:3.x image that inherits FROM drupal:10.2. Our users can then run the "farmOS" image instead of the "Drupal" image, and it will include this entrypoint logic for them (among other customizations we need).

[mstenta]

I mean drupal is essentially a webserver serving php, from what I see it's using apache or fpm so the entry point should be fairly easy to change.

That's correct. And it's exactly what we do in our downstream image.

So I think the question is whether or not this "official upstream" image (drupal) should do it for us.

That's probably more a question for the maintainers of this project. But I can see the argument for keeping this simple and letting downstream users implement it however they want in their own images (like we do).

[LaurentGoderre]

If the entrypoint is all you need, you don't even need to create another image, you can just mount the entrypoint to the base image.

ex:

docker run --rm -it -v ./entrypoint.sh:/entrypoint.sh --entrypoint /entrypoint.sh alpine

[mstenta]

Good point @LaurentGoderre!

Apart from the entrypoint, I think the only other piece that's missing from the current drupal image is what I described above:

This serves to have a copy of the code available elsewhere in the image.

The entrypoint needs to copy files from the "official" Drupal build to the bind-mounted volume. But if /opt/drupal is the only place that Drupal exists, then it gets replaced by the bind-mount and files can't be copied from it.

[steveiliop56]

If the entrypoint is all you need, you don't even need to create another image, you can just mount the entrypoint to the base image.

ex:

docker run --rm -it -v ./entrypoint.sh:/entrypoint.sh --entrypoint /entrypoint.sh alpine

Wouldn't work, from the moment the container starts the data directory is gone. The only solution I can think of is to mount only a /data directory and make the entry point copy the data from the var directory if /data is empty then simply create a symbolink link but that's again harder than it should be.

[LaurentGoderre]

How are you mounting your data? Are you doing it the way shown in the documentation in the Volumes section?

https://hub.docker.com/_/drupal

[mstenta]

The documentation itself points out the issue that @steveiliop56 is trying to solve:

but handling of /var/www/html/sites is somewhat more complex, since the contents of that directory do need to be initialized with the contents from the image.

If using bind-mounts, one way to accomplish pre-seeding your local sites directory would be something like the following:

$ docker run --rm drupal tar -cC /var/www/html/sites . | tar -xC /path/on/host/sites

This workaround works, but I think @steveiliop56 is suggesting the possibility of building a solution that would not require the workaround.

[steveiliop56]

Exactly, I don't see the reason why this design choice was selected.

[mstenta]

To be very specific, the reason that an empty /var/www/html/sites directory is annoying is because it means you need to manually create a bunch of files and directories that ship with Drupal core in that directory. Whereas when those files exist, Drupal installation can all be done through the Drupal UI without needing to create those files.

For example, Drupal expects there to be a default.settings.php file that it can copy and modify automatically during installation. But when a bind-mount volume is used, that file does not exist, so it must be created manually.

[steveiliop56]

To add to this I am a contributor/maintainer at a project called runtipi, it's basically a self hosted app manager like casa os, umbrel etc. you probably have seen it. The problem here is that when adding apps to these kind of appstores having to use hacks to make the app works makes it a lot harder to ship something that just works.

[mstenta]

Exactly, I don't see the reason why this design choice was selected.

I think this is more an issue with the design choices of Drupal itself... which were made before Docker (and its expectation of an empty data volume) were created. So it is not the fault of this drupal image maintainers. The current approach is just a simple and standard way to do things with Docker. It's just a bit annoying in the context of Drupal specifically.

So there is an opportunity to improve the "default" experience of drupal image users. But, I think there's also an argument for keeping it simple and leaving it to downstream users to extend from with their own images.

Personally, I don't find the Drupal codebase that is included in this image useful at all, because most Drupal deployments are built and managed with Composer these days (or should be as best practice). So I think in most cases you will want your own derivative image that runs composer install at the very least regardless.

[steveiliop56]

So there is an opportunity to improve the "default" experience of drupal image users. But, I think there's also an argument for keeping it simple and leaving it to downstream users to extend from with their own images.

I believe for most of the users self hosting Drupal, the bind volume is the simple thing they expect.

[mstenta]

@steveiliop56 Are you willing to make a pull request to demonstrate the solution you would like to see? As an open source maintainer myself I can say that it will be a lot more likely to happen if you help with the effort. And if it doesn't get in, at least you have the script you need for your downstream derivative image. :-)

[steveiliop56]

I definitely could open a pull request but there are a billion docker images and a lot of tags to change. It would change how the Drupal image works but I think it's worth a try. I will open a pull request as soon as I can (in a week or so), I will ping you to ask for your feedback @mstenta and @LaurentGoderre

[mstenta]

Indeed, and the documentation would also need to be updated, which is in a separate repo: https://github.com/docker-library/docs/tree/master/drupal

[kjostling]

I dont see why you would ever need two copies of the code base to get you application to work. That doesnt sound like a "bad design choice of Drupal's dockerfile", but rather a bad design of the app, imho.

[steveiliop56]

I don't think you understood the issue @kjostling. We don't need 2 copies of the data, we just need a bind volume. Drupal stores the data in the www directory which if you bind a volume into it, it will get replaced with basically an empty directory. The correct way to implement this would be to check if the directory is empty and if it is fill it with Drupal data else serve whatever the contents are.

[mstenta]

That doesnt sound like a "bad design choice of Drupal's dockerfile", but rather a bad design of the app, imho.

If by "bad design of the app" you mean "the app (Drupal) was not designed with Docker data volumes in mind", that's true. Drupal is more than 10 years older than Docker, and many of the "good design" patterns that Docker advocates were not standard practice when Drupal was designed. And making major changes to Drupal's core design pattern now is very difficult (due to community momentum, rules around breaking changes, etc). And so, "it's not our fault but it is our problem" that we need to work around in order to use Drupal in Docker.

[kjostling]

Ah, you'd like a "Wordpress version" of this image?

I believe that will break alot of current usages of this image since recreation of a container won't reinstall the app-files. That's what I expect should happen when I recreate a container from an image. Volumes are indentended for data that should be persisted after recreation, ie, sites/default/files (for drupal). Im not sure about mounts since I havent used that before. Looks alot similar though.

I do believe the Drupal 8 branch is 2 years younger than docker and it use composer as you pointed out earlier. Deployment should make the necessary "composer require/install" statements to set things up as the site in question wants it to be.

Hovever in a usecase where users should be able to "on-click-install" drupal and change the code in the GUI I guess this is a suboptimal image.