For the runner, we use an amazon ec2 instance and as registry, we use docker hub<\/a>. If you like to have the same setup you need to have the following accounts: <\/p>\n\n\n\n The first step to create a pipeline is to enable the CI\/CD features in Gitlab.<\/p>\n\n\n\n You can enable the feature under settings > general > Visibility, project features, permissions > pipelines.<\/p>\n\n\n\n A Pipeline in Gitlab is configured with a YAML file. The file called .gitlab-ci.yml<\/em>. The YAML file defines the structure and determines what the Gitlab runner should execute.<\/p>\n\n\n\n As mentioned before I use an AWS ec2 Instance for the runner. But you can also use any other VM or computer you like because we will use the runner inside docker.<\/p>\n\n\n\n Launch an ec2 instance and install docker. Amazon<\/a> provides an easy setup for that. <\/p>\n\n\n\n Make sure docker is installed correctly with the hello-world <\/em>image.<\/p>\n\n\n\n In order to configure the runner, a folder and other resources must be mounted.<\/p>\n\n\n\n Now we can connect the runner to our repository. Therefore we must run the register<\/em> command of the runner.<\/p>\n\n\n\n Enter the Hdm Gitlab Instance URL.<\/p>\n\n\n\n Add your Runner Token. (You can find the token under settings > CI\/CD > runners)<\/p>\n\n\n\n Add Tags and Description. <\/p>\n\n\n\n Enter the runner executor. (Use docker as executor)<\/p>\n\n\n\n Choose the default docker image for the docker executor. This image will be used if no one is defined in the YAML.<\/p>\n\n\n\n Now you should see the runner online under the settings page. <\/p>\n\n\n\n To create the pipline you could either create a file called .gitlab-ci.yml or you can press the “Set up CI\/CD” Button under the Project dashboard. <\/p>\n\n\n\n Now let’s define the Pipeline. You can edit the YAML file directly in Gitlab.<\/p>\n\n\n\n First, we define our stages. For the tests, we will use 2 stages called test-api <\/em>and test-web<\/em>. The two stages run sequentially. The former one cover tests for a backend written in Go and the latter one for a vue.js frontend. <\/p>\n\n\n\n First, we define a base image inside the job with the image<\/em> keyword. Since I use Go the image in the first job is golang:alpine. In the second stage, I use a node:alpine image.<\/p>\n\n\n\n In the script part, we need to prepare the environment for the tests. You could either make a docker image with all dependencies installed or install it during the job. After that, we only need to call the test command.<\/p>\n\n\n\n The tests in the first job are designed to be tested against a functional application with database access. With the services<\/em> keyword, we can run a docker image during a job. Access to the running container is as usual with hostname container name and port.<\/p>\n\n\n\n Now commit the changes and watch if the tests run correctly. Under CI\/CD Pipelines you can see the process. <\/p>\n\n\n\n If a stage has an error the whole pipeline will fail and all stages after will not run. So the next stage will only run if the test stages succeed. <\/p>\n\n\n\n Now we define the build<\/em> stage.<\/p>\n\n\n\n This stage is very simple and good to reuse. Let’s break it down. <\/p>\n\n\n\n The first step in the script part is to call docker info<\/em> just, to reconstruct the build if anything fails. After that we login to docker hub with our credentials. For that, we use the environment variables in Gitlab. You can enter environment variables under settings > CI\/CD > variables. <\/p>\n\n\n\n NOTE: Make sure to protect the variables and the branch you use. Otherwise, other members can see your password. <\/strong><\/p>\n\n\n\n To build a docker image in the pipeline we need the docker in docker (dind<\/em>) service.<\/p>\n\n\n\n We need to modify the runner configuration a bit to do that. <\/p>\n\n\n\n Open the mounted config file from the runner and set privileged to true.<\/p>\n\n\n\n NOTE: Docker in privileged mode disables all of the security mechanisms of containers. <\/strong>https:\/\/docs.docker.com\/engine\/reference\/run\/#runtime-privilege-and-linux-capabilities<\/a><\/p>\n\n\n\n Now we can build the images. You can pull your latest image first to use it as cache to speed up your build. Tag it with the predefined environment variable CI_COMMIT_SHA<\/em> to identify the image for later processing. Afterward, you can push the image to the registry.<\/p>\n\n\n\n For the web build, you can copy the whole job and change only the environment variables and point to the correct dockerfile<\/em>. <\/p>\n\n\n\n The last stage is the push<\/em> stage. Here we only pull the image from the build<\/em> stage and tag it again with meaningful tags. So the first part is the same as the build. After that, we tag it again with latest<\/em> you can add other tags such as version numbers. This stage should be done only in the master branch. We can set the Git strategy to none because we don’t need to copy the code from the repository.<\/p>\n\n\n\n To complete your CI\/CD process you can use a webhook to auto-deploy your application to your production server. You can find a very simple node.js webhook under my Gitlab account<\/a>. The complete project is also available on Gitlab<\/a>.<\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":" Introduction For all my university software projects, I use the HdM Gitlab instance for version control. But Gitlab offers much more such as easy and good ways to operate a pipeline. In this article, I will show how we can use the CI\/CD functionality in a university project to perform automated testing and an automated […]<\/p>\n","protected":false},"author":927,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[120,659,650],"tags":[144,150,98],"ppma_author":[800],"class_list":["post-7154","post","type-post","status-publish","format-standard","hentry","category-cloud-technologies","category-devops","category-scalable-systems","tag-ci-pipeline","tag-ci-cd","tag-gitlab"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":3348,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/continuous-integration-pipeline-for-unity-development-using-gitlab-ci-and-aws\/","url_meta":{"origin":7154,"position":0},"title":"Continuous Integration Pipeline for Unity Development using GitLab CI and AWS","author":"Jonas Graf, Christian Gutwein","date":"30. March 2018","format":false,"excerpt":"This blog entry describes the implementation of a Continous Integration (CI) pipeline especially adapted for Unity projects. It makes it possible to automatically execute Unity builds on a configured build server and provide it for a further deployment process if required.","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":3513,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/ci-cd-with-gitlab-ci-for-a-web-application-part-3\/","url_meta":{"origin":7154,"position":1},"title":"CI\/CD with GitLab CI for a web application – Part 3","author":"Nina Schaaf","date":"30. March 2018","format":false,"excerpt":"Hosting your own GitLab server Some users might have concerns regarding security using GitLab for a variety of purposes, including commercial and business applications. That is, because GitLab is commonly used as a cloud-based service - on someone else's computer, so to speak. So setting it up for running it\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3503,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/ci-cd-with-gitlab-ci-for-a-web-application-part-2\/","url_meta":{"origin":7154,"position":2},"title":"CI\/CD with GitLab CI for a web application – Part 2","author":"Nina Schaaf","date":"30. March 2018","format":false,"excerpt":"GitLab Our first approach was to use the existing GitLab instance of HdM for our project. For them, a shared runner was already defined on which we could run our jobs, so we were able to focus on the CI process itself. This plan worked out at first. We simply\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"Shaky Pipeline GitLab","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/pipeline-gitlab-1024x156.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/pipeline-gitlab-1024x156.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/pipeline-gitlab-1024x156.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3496,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/ci-cd-with-gitlab-ci-for-a-web-application-part-1\/","url_meta":{"origin":7154,"position":3},"title":"CI\/CD with GitLab CI for a web application – Part 1","author":"Nina Schaaf","date":"30. March 2018","format":false,"excerpt":"Introduction When it comes to software development, chances are high that you're not doing this on your own. The main reason for this is often that implementing components like UI, frontend, backend, servers and more is just too much to handle for a single person leading to a slow development\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"Shaky architecture","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/01_shaky-architecture-300x106.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":21064,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2021\/09\/11\/how-do-you-get-a-web-application-into-the-cloud\/","url_meta":{"origin":7154,"position":4},"title":"How do you get a web application into the cloud?","author":"af094","date":"11. September 2021","format":false,"excerpt":"by Dominik Ratzel (dr079) and Alischa Fritzsche (af094) For the lecture \"Software Development for Cloud Computing\", we set ourselves the goal of exploring new things and gaining experience. We focused on one topic: \"How do you get a web application into the cloud?\". In doing so, we took a closer\u2026","rel":"","context":"In "Cloud Technologies"","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2021\/09\/availableRunners-150x118.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":4405,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2019\/01\/04\/radcup-part-3-automation\/","url_meta":{"origin":7154,"position":5},"title":"Radcup Part 3 – Automation with Gitlab CI\/CD","author":"Immanuel Haag","date":"4. January 2019","format":false,"excerpt":"Written by: Immanuel Haag, Christian M\u00fcller, Marc R\u00fcttler The goal of this blog entry is to automate the previously performed steps. At the end all manual steps should be automated when new code changes are added to the repository. The new version of the backend will be made available in\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/01\/Gesamtstruktur-der-Architektur-Kurzform-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/01\/Gesamtstruktur-der-Architektur-Kurzform-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/01\/Gesamtstruktur-der-Architektur-Kurzform-1.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/01\/Gesamtstruktur-der-Architektur-Kurzform-1.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/01\/Gesamtstruktur-der-Architektur-Kurzform-1.png?resize=1050%2C600&ssl=1 3x"},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":800,"user_id":927,"is_guest":0,"slug":"nr037","display_name":"nr037","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/5ef625a5295dc48a3348857e6fb5f67c448faf816fc6eef08891dabcddd17fbd?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/7154","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/927"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=7154"}],"version-history":[{"count":24,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/7154\/revisions"}],"predecessor-version":[{"id":11631,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/7154\/revisions\/11631"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=7154"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=7154"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=7154"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=7154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Prepare Gitlab<\/h2>\n\n\n\n
Setup the runner<\/h3>\n\n\n\n
docker run hello-world<\/code><\/pre>\n\n\n\ndocker run -d --name gitlab-runner --restart always \\\n -v \/srv\/gitlab-runner\/config:\/etc\/gitlab-runner \\\n -v \/var\/run\/docker.sock:\/var\/run\/docker.sock \\\n gitlab\/gitlab-runner:latest<\/code><\/pre>\n\n\n\ndocker run --rm -t -i -v \/srv\/gitlab-runner\/config:\/etc\/gitlab-runner gitlab\/gitlab-runner register<\/code><\/pre>\n\n\n\nPlease enter the gitlab-ci coordinator URL (e.g. https:\/\/gitlab.com )\nhttps:\/\/gitlab.mi.hdm-stuttgart.de <\/code><\/pre>\n\n\n\nPlease enter the gitlab-ci token for this runner\nxxx<\/code><\/pre>\n\n\n\nPlease enter the gitlab-ci description for this runner\n[hostname] gitlab-runner<\/code><\/pre>\n\n\n\nPlease enter the gitlab-ci tags for this runner (comma separated):\na-tag,another-tag<\/code><\/pre>\n\n\n\nPlease enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell:\ndocker<\/code><\/pre>\n\n\n\nPlease enter the Docker image (eg. ruby:2.1): \ndocker:stable<\/code><\/pre>\n\n\n\nCreate a Pipeline<\/h2>\n\n\n\n
![\"\"](\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13-1024x124.png\")
<\/figure>\n\n\n\nstages:\n - test-api\n - test-web\n - build\n - push<\/code><\/pre>\n\n\n\ntest-api:\n image: golang:alpine\n stage: test-api\n services: \n - redis:latest\n script:\n - apk update && apk add --no-cache git\n - cd api && go mod download\n - CGO_ENABLED=0 go test\n\ntest-web:\n image: node:alpine\n stage: test-web\n script: \n - cd web && npm install \n - npm test<\/code><\/pre>\n\n\n\nbuild-api:\n stage: build\n script:\n - docker info\n - echo "$REGISTRY_PWD" | docker login -u "$REGISTRY_USER" --password-stdin\n - docker pull "$REGISTRY_API":latest || true \n - docker build --cache-from "$REGISTRY_API":latest -t "$REGISTRY_API":"$CI_COMMIT_SHA" .\/api\n - docker push "$REGISTRY_API":"$CI_COMMIT_SHA"\n<\/code><\/pre>\n\n\n\nservices:
- docker:dind
<\/pre>\n\n\n\npush-api:\n stage: push\n variables:\n GIT_STRATEGY: none\n script:\n - echo "$REGISTRY_PWD" | docker login -u "$REGISTRY_USER" --password-stdin\n - docker pull "$REGISTRY_API":"$CI_COMMIT_SHA"\n - docker tag "$REGISTRY_API":"$CI_COMMIT_SHA" "$REGISTRY_API":latest\n - docker push "$REGISTRY_API":latest\n only:\n - master\n<\/code><\/pre>\n\n\n\n