We use GitLab CI extensively to build and deploy BeyondTracks. However we have some heavy dependencies, which we used to build and install every time our CI process ran. This added an extra 5 minutes at the start of every build, which is just wasteful.
To speed things up we wanted to follow best practice by installing our dependencies into a Docker image and reused that in our build.
It was important that this Docker image had a well document build process, which didn’t depend on any local configuration, meaning it needed to be built by GitLab CI. So we created a new repository,
beyondtracks-docker with just two files,
GitLab makes this dead easy. Under Repository > Files select New > New file. Then choose the Template
.gitlab-ci.yml, applying a GitLab CI Yaml template for
Docker. This provides us a base GitLab CI configuration to build and deploy our Docker image.
image: docker:latest services: - docker:dind before_script: - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY build-master: stage: build script: - docker build --pull -t "$CI_REGISTRY_IMAGE" . - docker push "$CI_REGISTRY_IMAGE" only: - master build: stage: build script: - docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" . - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" except: - master
We just need one more file, a
Dockerfile which documents how to build our dependencies. This is what ours looks like.
RUN apt-get -y update && apt-get -y install curl gnupg openssh-client rsync RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list RUN curl -sL https://deb.nodesource.com/setup_11.x | bash - RUN apt-get -y update && apt-get -y install nodejs yarn RUN apt-get -y install imagemagick webp postgresql postgis postgresql-contrib osm2pgsql wget osmium-tool gdal-bin python-pip pgtap s3cmd parallel git unzip build-essential libsqlite3-dev zlib1g-dev RUN pip install mapboxcli RUN npm install -g @mapbox/geojsonhint RUN git clone --depth=1 https://github.com/mapbox/tippecanoe.git && cd tippecanoe && make -j && make install && cd ..
This builds a Docker image straight to our Container Registry which we can then reference in all our other GitLab CI with
Finally, to ensure our dependencies stay fresh with new software versions we can add a CI Schedule to rebuild our docker image at a set frequency.