Figure out how to specify redis version

[rfay]

This is a great question. I'm a n00b on redis though, so need to understand more about

• Why people use other than latest stable
• How they would configure it.

[hussainweb]

Honestly, I haven't had to change the version of Redis as far as Drupal was concerned. The latest version (6) works just fine. The only reason I have is that maybe the hosting people use might have an older version and they might want to match that in DDEV as well.

This is somewhat similar to why you might want to have a different version of Solr. Features change significantly across versions and you might want to match your host's version.

For configuration, I don't have good ideas, but I do have a mediocre idea: if people could specify a random parameter name in .ddev/config.yml, we could have addon-specific definitions. So, someone might be able to put in redis_version: 6 in their configuration and this should be made available to the addon somehow. What do you think?

[rfay]

On the solr example, basically each version requires a separate recipe.

If that's not the case with redis, we might be able to figure out giving it an argument. OTOH maybe people should just change it themselves if they want something else.

But this remains a valid consideration, worthy of some more thought.

[thijskaspers]

Would be nice if the image is configurable. We prefer using redis:6-alpine over redis:6.

[rfay]

Could you say why alpine is important to you @thijskaspers ? Different images have different behaviors typically. In this case they are likely similar since both versions of the official image.

[thijskaspers]

Could you say why alpine is important to you @thijskaspers ? Different images have different behaviors typically. In this case they are likely similar since both versions of the official image.

Alpine images are generally much smaller, saves disk space. For example, the Alpine Linux image is about 15mb, versus almost 200MB for the regular image (uncompressed sizes).

But yes, the behaviour is slightly different for the images, because of the different OS. It isn't a 100% drop-in replacement, but would be nice to have the option to pick what you prefer?

[rfay]

I definitely understand about preferences and sizing, but DDEV is built for local development purposes, where size is not as important. And the key to things is not to make them complicated where they're harder to maintain.

Please take a look at https://hub.docker.com/_/redis/tags

The compressed size of the alpine one is about 10MB, the compressed size of the standard one (6.0.18) is about 40MB.

The difference on your computer is trivial.

You can easily fork this project though and experiment with using the alpine image. Maybe it will be fine. You can also just edit the config.redis.yaml on your computer and remove the #ddev-generated line and change the image specified, and see if you can get it working.

[markdorison]

On the solr example, basically each version requires a separate recipe.

If that's not the case with redis, we might be able to figure out giving it an argument.

This is not the case with Redis. I have been through many Redis major versions and have yet to experience breaking changes with standard Drupal configurations.

[rfay]

See:

• ddev add-on get should allow specifying optional settings ddev#4430

Suggestions are welcome on how to do that.

[seebeen]

Hello friends,

From a devops/SRE perspective, there is almost never a reason to pin a redis version to a specific lower major version. Redis team is not known for working fast and breaking things. They have an almost rock solid release system.

That being said, the issues you mentioned here are solvable via a first_run hook.
If DDEV had a hook that would be run only on first run after getting an addon, it would be possible to code out a shell (or maybe even a golang) script which could take user input and modify the addon.

I'm more than willing to code out a customizable script that could do that. I just need a first_run hook.

Is that something you could implement @rfay

P.S. When you have the time can you check this out - https://github.com/oblakstudio/ddev-redis-7

[rfay]

That being said, the issues you mentioned here are solvable via a first_run hook.

Thanks for this idea. I'd love for you to continue with still more ideas in

• ddev add-on get should allow specifying optional settings ddev#4430

I'm not sure a first-run hook is a feature that needs to be done in DDEV, it can easily be done already; create a file on first run; if it doesn't exist ask the question. However, interactive response does break a number of things, including testing. But we often use and respect the DDEV_NONINTERACTIVE environment variable in DDEV for this purpose...

Thanks so much for brainstorming on this because I think it's a super important problem to solve.

P.S. When you have the time can you check this out - https://github.com/oblakstudio/ddev-redis-7

Can you say a bit about what's different between it and ddev/ddev-redis, and why you built it from ddev-addon-template instead of just forking ddev-redis? I would prefer to have fewer confusing versions of things around, but we have to solve the problem above.

I wonder if the redis version here can just be bumped.

[seebeen]

I've covered everything in the readme file. It's all there. I'll explain in more detail sometime during the day :)

[seebeen]

Thanks so much for brainstorming on this because I think it's a super important problem to solve.

P.S. When you have the time can you check this out - https://github.com/oblakstudio/ddev-redis-7

Can you say a bit about what's different between it and ddev/ddev-redis, and why you built it from ddev-addon-template instead of just forking ddev-redis? I would prefer to have fewer confusing versions of things around, but we have to solve the problem above.

I wonder if the redis version here can just be bumped.

From the top:

1. It's from ddev/ddev-addon-template because back then ddev didn't have an official redis plugin
2. It took me a while to configure it and make it performant, fast, and fully ddev compatible

You can see the full stack of differences in the readme file on the repo I linked, but the keypoints are:

1. persistence - Mine keeps data across restarts
2. major version bump - official is on v6, mine is on v7
3. No anon volumes in mine, everything is named and self-contained
4. Mine runs as a non-root user, keeping everything in check.
5. Mine uses 4x smaller image, which uses an additional multi-stage build step, to reduce the footprint even further.

Other things are, better testing, fully setup workflows and PRs, auto updates via renovate...

If you're so inclined, you can grant me maintaner access on the official repo, and I can backport the changes, and archive my repo.

[rfay]

It doesn't take maintainership to propose features, just PRs.

@stasadev is the maintainer of this one. You might consider a PR doing the basics. Maybe he'll want to take a look at https://github.com/oblakstudio/ddev-redis-7

[stasadev]

@seebeen

Hi, this is great that you want to make things better with this add-on. Feel free to create PR.

persistence - Mine keeps data across restarts

Agree.

major version bump - official is on v6, mine is on v7

There is reason #17 why we use bullseye (to cover all use cases), we can migrate it to redis:7-bullseye.

No anon volumes in mine, everything is named and self-contained

Agree.

Mine runs as a non-root user, keeping everything in check.

Can you please explain this further, as I can see the root user inside?

Mine uses 4x smaller image, which uses an additional multi-stage build step, to reduce the footprint even further.

This is a win if you have only one project, but not when you have many DDEV projects with Redis. In this case, they will each have their own image instead of using the same base image (the size is not reused).

Side note: when you delete a project, the image ddev-${DDEV_SITENAME}-redis is not deleted (this is because DDEV looks for ddev/ddev-* when deleting).

@rfay this is another reason to use image tags (labels) when searching for something created with DDEV ddev/ddev#5073 (comment)

Other things are, better testing, fully setup workflows and PRs, auto updates via renovate...

Agree.

optimized config

It's hard to tell which options are set by default when you're provided with an almost complete config. And given that DDEV isn't intended for production use, I'm not sure if it's necessary for everyone using this add-on.

ddev redis-flush Clears all the Redis Databases

Would be a great addition to commands.

[seebeen]

There is reason #17 why we use bullseye (to cover all use cases), we can migrate it to redis:7-bullseye.

#17 is related to the default redis setup and lunacy from the Redis/RedisLabs team which handles docker. They don't understand it. I saw that ticket way back. That's why a custom dockerfile exists, and that's why the docker-compose is more specific than the ddev-redis one.

Feel free to test the image in any possible use cases - I guarantee that it will work.

Can you please explain this further, as I can see the root user inside?

Sorry, I misspoke - I mixed it up with the redisinsight docker image I build

Mine uses 4x smaller image, which uses an additional multi-stage build step, to reduce the footprint even further.

This is a win if you have only one project, but not when you have many DDEV projects with Redis. In this case, they will each have their own image instead of using the same base image (the size is not reused).

This is always a win, because it reduces the download footprint for users on shitty internet, and keeps the disk usage lower.

Side note: when you delete a project, the image ddev-${DDEV_SITENAME}-redis is not deleted (this is because DDEV looks for ddev/ddev-* when deleting).

@rfay this is another reason to use image tags (labels) when searching for something created with DDEV ddev/ddev#5073 (comment)

Yes, I couldn't figure out how to delete it except manually, but haven't gotten around to that.
Still, a named volume is waaaaay better than anon volume which keeps getting recreated on every restart

One Restart -> One anon volume. Which sucks :(

It's hard to tell which options are set by default when you're provided with an almost complete config. And given that DDEV isn't intended for production use, I'm not sure if it's necessary for everyone using this add-on.

There is nothing default in the config. I optimized the config in line with my experience with redis and complex redis clusters.

Having a dev environment doesn't mean not having a fast dev environment.

If we're all inline - I can make a full PR to inline it 1:1 with my repo :)

[stasadev]

Feel free to test the image in any possible use cases - I guarantee that it will work.

I'll check it out (with an older Docker in VM) and get back here.

This is always a win, because it reduces the download footprint for users on shitty internet, and keeps the disk usage lower.

Ok, I could be wrong, I'm not an expert on how Docker layers use disk space with multi-stage builds.

[seebeen]

Sure, check it out, let me know - and I'll submit a PR, and you can grant me axx to the repo :)

[seebeen]

Also, can you please add renovate to the org / repo? :)

[stasadev]

I tested ddev get oblakstudio/ddev-redis-7 with:

$ ddev version
 ITEM             VALUE                                                
 DDEV version     v1.22.1                                              
 architecture     amd64                                                
 db               ddev/ddev-dbserver-mariadb-10.4:v1.22.0              
 ddev-ssh-agent   ddev/ddev-ssh-agent:v1.22.0                          
 docker           20.10.7                                              
 docker-compose   v2.20.0                                              
 docker-platform  Ubuntu 20.04.6 LTS                                   
 mutagen          0.17.2                                               
 os               linux                                                
 router           traefik:v2.10                                        
 web              ddev/ddev-webserver:20230803_php_serialize_precision

$ ddev redis --version
redis-cli 7.2.0

Can confirm that it works.

There is nothing default in the config. I optimized the config in line with my experience with redis and complex redis clusters.

On the other hand, it's good to provide best practices, so people know how to set up Redis later in production.

Sure, check it out, let me know - and I'll submit a PR, and you can grant me axx to the repo :)
Also, can you please add renovate to the org / repo? :)

You can submit a PR right now from a GitHub fork.

Regarding access and renovate (that's an alternative to dependabot, right?) - it's up to @rfay.

[seebeen]

Renovate is Dependabot - on steroids. I have a custom config set on the repo, which will automatically update the redis version with automerge.

I'm also bringing in semantic-release, which means that all releases will happen automatically based on the commit type, etc...

[rfay]

Mine runs as a non-root user, keeping everything in check

I'm a big fan of running as non-root user. DDEV core containers run as the same UID as the user on the host. This is important for security reasons and also usability reasons... where the container will successfully run that way; not all will. Glad to see you running it as non-root. I'd love to see that be standard on add-ons.

renovate vs dependabot

I haven't found dependabot all that inspiring for small projects like this, but I don't have a problem with either of these (and we use dependabot on ddev). I don't know of add-ons currently using either one except ddev-redis-7. The bottom line is maintainers paying attention, not automated updates. As I follow ddev-redis-7, I have seen way more activity than I would expect from renovate :)

This is a win if you have only one project, but not when you have many DDEV projects with Redis. In this case, they will each have their own image instead of using the same base image

The image is shared among all docker projects. For example, ddev-webserver of a particular tag is just one image, you don't have many images for many projects. You have different containers if you're running multiple instances.

this is another reason to use image tags (labels) when searching for something created with DDEV ddev/ddev#5073 (comment)

Absolutely!

@seebeen If you're going to do PRs here, please do them one feature at a time. I know that's work for you given that you've already sorted things out, but that's how PRs work, so they can be successfully evaluated on the focused thing they are about.

[seebeen]

I'll do at as you wish, but - again, I'd like to clear some things out.

My endgame with the ddev-redis is to migrate all of the features of the plugin I wrote here. Make a breaking change - and set the version to 2.0 - Add a notification on upgrade for all users (about auth being required, etc...)

I'm an active maintainer - I maintain OSS projects that I wrote and that have a userbase.
That being said - you OK with me taking over this one? :)

[rfay]

My endgame with the ddev-redis is to migrate all of the features of the plugin I wrote here. Make a breaking change - and set the version to 2.0 - Add a notification on upgrade for all users (about auth being required, etc...)

I wonder if versioning should be based on the redis version in use, using 6.x then 7.x, or if that's valuable. Since DDEV now supports explicit add-on versions, if we do a major version with breaking change it is a solvable problem.

I'm an active maintainer - I maintain OSS projects that I wrote and that have a userbase.
That being said - you OK with me taking over this one? :)

As above, @stasadev is the maintainer here and has just taken this over a few weeks ago after it being little maintained after a long time.

It's not usually a good thing for a new contributor to come into a project and talk about "taking it over". The normal path is to increase contributions, show goodwill and long-term staying power, and then be invited to maintain with privileges.

[stasadev]

Addition to my previous reply:

I thought there was something special about the Dockerfile for https://github.com/oblakstudio/ddev-redis-7 that made it work on older Docker.

I tried redis:7-alpine and it works the same as redis:6-bullseye.

This is always a win, because it reduces the download footprint for users on shitty internet, and keeps the disk usage lower.

So I'm not sure if we need to pull two more files (Dockerfile and docker-entrypoint.sh) just to save a few MB of space.

That way, we don't have to think about the leftovers problem on ddev delete.

[rfay]

My general bias is toward Debian/Ubuntu because of overall better support (and familiarity) but I do understand Alpine's size advantage in some cases.

[seebeen]

@stasadev I don't mind using the base image.
@rfay If you want to choose the redis version I can add a command ddev redis-version which can get from docker hub some of the most popular images - that way the user can choose 🤷‍♀️

Can't think of a better way

[rfay]

@rfay If you want to choose the redis version I can add a command ddev redis-version which can get from docker hub some of the most popular images - that way the user can choose 🤷‍♀️

A docker-compose.redis-version.yaml could override the image; that seems like a good way. I'm not done thinking about it :)

[seebeen]

I can place it in a file which wouldn't be #ddev-generated and then post_install / pre_start get it from there?

[rfay]

Your add-on can put image: <something> in a docker-compose.redis-version.yaml and see how it goes. Give it a try. There is no processing required. Nothing needs to be done in post_install or pre_start.

[seebeen]

I've made the tweaks inline with your suggestions.

https://github.com/oblakstudio/ddev-redis-7

If we're all on the same page about the setup, I'll modify the release workflows and make a pull request.

Did you have the time to think about maintainer status?

[rfay]

@seebeen @stasadev has said he's willing to add you as a maintainer. Then we need to talk about the full scope of what you want to do and how it can be done with the most benefit and the least disturbance for existing users.

Please do these things:

1. Open an issue here in ddev/ddev-redis requesting to be added as a maintainer
2. Open another issue here proposing the changes you envision.
3. Join us in Discord for a conversation about this, https://discord.com/channels/664580571770388500/1152623153265250374/1152623153265250374. - If you haven't already joined Discord, here's the invitation, https://discord.gg/hCZFfAMc5k