Provide identity to docker build to allow fetching deps from git repositories

[tlvenn]

As discussed in #17 , here is a generic implementation that is fully covering the use case of providing your ssh identity to docker build so that it can fetch your mix dependencies from any git repositories, either public or private ones.

As explained, the gist of it is to serve your identify over an http server and then fetch it, use it and delete it within one atomic RUN docker command.

The fetch part is done in a failsafe manner so that if the host cant be resolved or if any other error happens, it will silently fail and move on so that if you actually don't need to fetch any dependencies from a git repository, the build will succeed.

This PR introduces 2 command line arguments:

• --host which is your host IP address from where the container can fetch your identity. If you dont provide this variable, no http server is started, otherwise, just before starting the docker build an http server is started on this host on port 4000. The identity is served at the following url: http://$host:4000/identity. The http server is automatically stopped once the build is finished.

• --identity-file to specify which identity file should be served. It defaults to ~/.ssh/id_rsa

Let me know if you have any question @teamon and I hope you will merge this PR which I believe add a lot of values.

This PR should not be merged before #16

[teamon]

I must say this is a tough one:

• This is definitely an important feature, possibly a deal breaker for some users
• But it brings plug & cowboy as dependencies,
• and requires changes in the default dockerfile

Apart from changing the default dockerfile I do not see anything that would require putting the server inside mix_docker. After all, the server could be run with simple python -m SimpleHTTPServer 8888 right?

[teamon]

Maybe mix_docker should allow for build hooks and there could be a mix_docker_identity_server package?

[tlvenn]

Providing build hooks could be an interesting idea but at the same time, the core of mix docker should be able to build any elixir project and so imho, it should take care of providing your ssh identity for you transparently which is what this PR is doing.

Otherwise you limit severely the scope of this project as to when it can be used or you are forcing people to solve this on their own and many will make bad unsafe choices as not everyone is fluent with the subtlety that comes with Docker layered file system.

Updating the build Dockerfile from time to time is expected and I dont see this as a problem. Pulling plug & cowboy dependencies which are very well known should not really be an issue at all for building your image.

[tlvenn]

I have rebased on master and updated the code to cope with your latest changes.

[tlvenn]

@teamon any more though on the subject ? I am also genuinely curious on how you actually solve this at Recruitee.

[teamon]

the core of mix docker should be able to build any elixir project
Building any elixir project was never the intention of this package. From the very beginning it was supposed to be a starting point.

Let me give you some background on how we are using mix_docker in @Recruitee.

mix_docker started as a bash script inside one of our elixir projects. All it did was run docker build twice with correct arguments (that's exactly what mix_docker is doing). Back then we were quite attached to the version numbers for every app. It was both important for image tagging and getting the distillery release package from correct directory. This was the main reason to use elixir to get the version number from mix project metadata. In the end it's still just executing shell commands.

As it turned out, we are not really bumping version numbers, or even care about them. We have moved into continuos integration and delivery. The version numbers became both frozen (no need to get it from mix metadata directly) and obsolete (no need to tag images with version number).

Every elixir project of ours ended up with custom Dockerfiles anyway - a additional package required here, a custom command needed there. So fo us the benefits of default dockerfiles in zero.

Yet another piece of story is that we have unified our deployment for every single app - everything runs in docker now. But there were not only elixir apps, but also some ruby, java or python. For them we obviously couldn't use mix_docker. Since not every app required two-phases build I ended up making two simple bash scripts that were put directly in apps repositories. Despite all these apps being actively developed I haven't touched the build scripts for more than 6 months.

And since we have an CI/CD automation in place, nobody is really using mix tasks or build scripts - it is all handled by CI.

As one of my colleagues put "Docker is not Heroku" - you really need to know how things work. It's not a silver bullet, you can't hide the complexities. I'm all for reducing complexity on general, but it must be done with responsibility from the solutions' author.

I'm sorry to say this, but the future of mix_docker is somehow unknown. We are using quite old versions of it in our projects since we don't need any new features. And given the heterogeneity of our stack we will probably move away from mix_docker and stick to simple shell scripts. (The other reason is that right now we need to build the app twice - once to compile mix_docker tasks, then inside docker to make a release which is quite dumb in the CI environment)

To answer you question about private dependencies accessed over ssh - we simply don't have these. Which is yet another reason I don't feel like putting ssh keys handling code inside mix_docker - I have no reliable way (or need) to test that, answer questions or solve any issues that can come up.