The use case:
We have several "Release jobs" in Jenkins that build and push a Docker image of the application to a docker registry, update the project version in various files and finally push the release tag to the corresponding Git repository.This whole process, runs inside an isolated Docker-in-Docker container, which means that the Docker cache is completely blank each time these Jenkins jobs are being executed.
In short:Jenkins instance --> Starts a DinD container --> The Git repo is being cloned inside the DinD container --> Dockerfiles with several layers including the actual build process of the applications are being built --> Push docker images to registry --> Push release to Git.
The issue:
While on one hand this isolation helps to avoid some problems, on the other hand it makes the whole docker build process particularly slow.
Docker pull and docker push processes surely contribute to this delay up to a degree, but this is a network speed issue that we cannot deal with atm.
However another reason for this tardiness is that, since the actual application (maven or angular) is being built inside a "clean" docker container where the .m2 or node_modules directories are empty every time, all dependencies must be downloaded/installed upon each run. We can obviously mount an .m2 repository from Jenkins inside the DinD container, but the images which are being build inside this DinD container will have no access to it.
We tried to tar .m2 and node_modules directories, COPY them inside the image through the Dockerfile, untar them and move them to the right path, but this workaround saved as 1-2 minutes tops.We also tried to cache Maven dependencies using buildkit, e.g. https://www.baeldung.com/ops/docker-cache-maven-dependencies#caching-using-buildkit but it's obviously not exactly what we need.
AFAIK it's not possible to mount volumes upon docker build, which would be the ideal solution in our "blank cache" situation.
Has anyone met a similar problem and found a workaround to it?
In general, we would appreciate any suggestions on how to minimize the execution time of our release jobs, and optimize the whole process.
Thank you in advance.