{"id":3491,"date":"2018-03-31T14:01:13","date_gmt":"2018-03-31T12:01:13","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=3491"},"modified":"2023-06-09T14:22:37","modified_gmt":"2023-06-09T12:22:37","slug":"continuous-integration-deployment-for-a-cross-platform-application-part-2","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/31\/continuous-integration-deployment-for-a-cross-platform-application-part-2\/","title":{"rendered":"Continuous Integration & Deployment for a Cross-Platform Application – Part 2"},"content":{"rendered":"

In the first part<\/a> we pointed out how we set up the infrastructure for our CI system. Now we would like to explain how we build a pipeline for our cross-platform application and what features of GitLab CI we made use of.<\/p>\n

<\/p>\n

Building a Pipeline in GitLab CI<\/h2>\n

Our first goal when we faced our fresh installed GitLab instance obviously was to get a build done – initiated by a commit to the repository.<\/p>\n

The CI functionality in GitLab CI is simply enabled by just putting a gitlab-ci.yml file into the projects root. The entire pipeline is defined in this file.<\/p>\n

So we created a job for Android which executed a build, signed the application and uploaded it to Google play. For the Web and iOS build we defined similar jobs. So far so good. Pretty fast we realized the need of a proper pipeline since our three jobs were running fully independently from each other. So no matter if the iOS job fails after a minute, the Android job would deploy a new version which is not desirable regarding consistency. This is where stages help out.<\/p>\n

 <\/p>\n

Stages<\/h3>\n

Instead of putting all steps in a single job we created multiple jobs containing only the steps that are necessary for the respective stage.<\/p>\n

All of the jobs in a stage are executed in parallel (there need to be enough runners available). If they all succeed, the pipeline moves on to the next stage. If one jobs fails, the next stage is not executed.<\/p>\n

 <\/p>\n

\"\"
Pipeline including Versioning, Build, Test and Deploy stage<\/figcaption><\/figure>\n

 <\/p>\n

By using multiple jobs in different stages we needed to have a mechanism that passes artifacts from previous jobs to the following ones. How could the Android deployment job get at the APK file which was built by the build job? We use another building blocks of common CI systems: Artifacts.<\/p>\n

 <\/p>\n

Artifacts<\/h3>\n

Artifacts is a list of files and directories which are attached to a job after it completes successfully. The artifacts will be sent from the runner to GitLab and will be available for download in the GitLab UI. We also liked that there is an API to access them from another client. To pass artifacts between different jobs, the job which depends on the results of a previous job declares the dependency as follows:<\/p>\n

 <\/p>\n

build_android:\n    stage: build\n    ...\n    artifacts:\n        paths:\n            - platforms\/android\n        expire_in: 2 weeks\n        \ndeploy_android:\n    stage: deploy\n    ...\n    dependencies:\n        - build_android<\/pre>\n
 <\/p>\n
The APK file which needs to be deployed is located in the platforms\/android<\/code>  folder which is defined as artifact of the build_android job. By setting expire_in we tell GitLab to delete the artifact after two weeks. Otherwise it would be kept by GitLab and use considerable amounts of disk space.<\/p>\n
By defining the dependency on build_android we make sure that our deploy_android job gets the required files to be able to do the deployment. When the job starts to run, it will download the artifacts.<\/p>\n
Downloading artifacts for build_android...\nDownloading artifacts from coordinator... ok<\/pre>\n
 <\/p>\n
Configuring build jobs<\/h2>\n
 <\/p>\n
Defining a reusable docker container<\/h3>\n
As a basis for the pipeline, a self-configured Docker image<\/a> is used as a base for the GitLab runners. It contains all necessary prerequisites and dependencies for the build and deployment process and is declared as base image for docker runners as simple as that:<\/p>\n
image: marcomaisel\/ionic:latest<\/pre>\n
It contains Ionic 3, Cordova, node & npm, Java, Android SDK, Gradle, fastlane, Chrome and all necessary prerequisites.<\/p>\n
 <\/p>\n
Increment version number<\/h3>\n
The automated upload to the Google Play Store only works if the version code is incremented for each new upload. Therefore, each time a commit is made, a custom script needs to be executed at the beginning of the runner\u2019s building process. This script increases the version code in the associated config.xml and package.json files and commits the changes to the repository.<\/p>\n
The automatic upload to the Google Play Store is made possible by the open source platform ‘fastlane’. In the associated Fastfile, which is located in the respective repository (in our case this is the frontend repository), the upload process is specified and necessary information such as package names, folder structure or keys are stored.<\/p>\n
To make fastlane compatible with an Ionic\/Cordova project, we need to use a workaround because the Android build job only works if fastlane is located inside the Android subfolder. However, we don\u2019t want to put the fastlane files in that subfolder because of the danger to overwrite something and because we also need fastlane for our iOS build. Therefore we need to wrap all the android actions in a chdir do:<\/p>\n
lane :playstore do\n    Dir.chdir(\"..\/platforms\/android\/src\") do\n        \/\/ Gradle Tasks\n    end\n  end\n<\/pre>\n
Because the folder structure of Cordova projects changed with Cordova versions 7 and higher, we need to run the commands in a different folder:<\/p>\n
lane :playstore do\n    Dir.chdir(\"..\/platforms\/android\/app\") do\n        \/\/ Gradle Tasks\n    end\n  end\n<\/pre>\n
 <\/p>\n
Speeding up the cycle time<\/h2>\n
After reducing our cycle time – the time consumed by a successful run of a pipeline – by using more performant hardware for the runners (as mentioned previously in part 1) we took a closer look at our build processes.<\/p>\n
 <\/p>\n
Optimizing Build processes<\/h3>\n
We went through our build jobs line by line. We found some steps like generating app icons that we removed from the CI as it doesn’t need to be done for every build.<\/p>\n
Even though we installed Gradle as a dependency in our base docker image there was always some activity loading and installing Gradle dependencies during the Android build. To avoid this time-consuming activity we had to make use of GitLab’s Caching feature.<\/p>\n
\"\"
Gradle dependencies were installed during every build<\/figcaption><\/figure>\n
 <\/p>\n
Caching<\/h3>\n
Caching is great if you need to dynamically install certain dependencies during your job and can\u2019t pre-build them into a CI image for some reason. This example demonstrates how we cached Gradle dependencies folders between builds:<\/p>\n
build_android:\n    stage: build\n    before_script:\n        - ...\n    script:\n        - ...\n    tags:\n        - ...\n    cache:\n        untracked: false\n        key: \"buildandroid\"\n        paths:\n        - node_modules\/\n        - plugins\/\n        - .gradle\/wrapper\n        - .gradle\/caches\n    artifacts:\n        - ...\n<\/pre>\n
 <\/p>\n
As you can see we used this feature for caching NPM’s node_modules folder as well which reduced the execution time for npm install<\/code> from about two minutes to about 20 seconds. As installing npm dependencies is done in all of our four stages it saved us more than 5 minutes for the entire pipeline.<\/p>\n
All in all caching reduced the cycle time of your pipeline almost by half – from 25-30 minutes to 12-14 minutes. In our opinion this duration is acceptable and provides notable faster feedback for the involved developers.<\/p>\n
 <\/p>\n
Deployment for different environments<\/h2>\n
The last stage in our pipeline contains the deployment jobs. We defined a staging<\/strong> and production<\/strong> environment. Both environments are hosted on Amazon S3 instances, they only differ in the URL. Every successful web build will be deployed to our staging environment.<\/p>\n
Deployment to the production is a manual step that you can trigger from the GitLab UI. This way of deployment can be easily changed to a branching model. So e.g. if you commit to a “release” branch it will deploy to production automatically instead of doing it manually.<\/p>\n
In the end it depends on the deployment strategy and the requirements the project has regarding releases. Technically it’s just about changing a few lines in our CI config.<\/p>\n
Hint:<\/strong> Working with the Amazon AWS CLI we had to deal with API keys which have to be kept secret. GitLab CI has a place for that which is called secret variables<\/a>. These variables can be added in the UI and will be turned into environment variables. In this way you can access them e.g. in your GitLab config file without adding them to the repository.<\/p>\n
 <\/p>\n
Conclusion<\/h2>\n
The main advantage of hybrid app development is its ability to develop on many platforms simultaneously with a single code base. Ionic is also compatible with Electron, Github\u2019s framework for developing cross-platform desktop applications. So it would be possible to develop for even more platforms with the same code base.<\/p>\n
We managed to set up the infrastructure and a pipeline at very low cost, but the resulting problems (especially regarding performance) took a lot of time. Debugging a build that takes 50 minutes is really annoying. From the experience gained, we would suggest to invest a little more money in future projects to avoid our initial problems.<\/p>\n
For longer-term projects that are to be developed simultaneously on several platforms (Android, iOS, Web), we would continue to rely on Gitlab CI. We liked the simple syntax of the config file and the GitLab UI integrating repository and CI all-in-one.<\/p>\n
If the pipeline is successfully set up, future deployment will be greatly facilitated by the resulting automation.<\/p>\n","protected":false},"excerpt":{"rendered":"
In the first part we pointed out how we set up the infrastructure for our CI system. Now we would like to explain how we build a pipeline for our cross-platform application and what features of GitLab CI we made use of.<\/p>\n","protected":false},"author":188,"featured_media":3462,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[659,650,651,2,657],"tags":[],"ppma_author":[746],"class_list":["post-3491","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","category-scalable-systems","category-system-designs","category-system-engineering","category-teaching-and-learning"],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg","jetpack-related-posts":[{"id":3348,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/continuous-integration-pipeline-for-unity-development-using-gitlab-ci-and-aws\/","url_meta":{"origin":3491,"position":0},"title":"Continuous Integration Pipeline for Unity Development using GitLab CI and AWS","author":"Jonas Graf, Christian Gutwein","date":"30. March 2018","format":false,"excerpt":"This blog entry describes the implementation of a Continous Integration (CI) pipeline especially adapted for Unity projects. It makes it possible to automatically execute Unity builds on a configured build server and provide it for a further deployment process if required.","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CI_process.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":3503,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/ci-cd-with-gitlab-ci-for-a-web-application-part-2\/","url_meta":{"origin":3491,"position":1},"title":"CI\/CD with GitLab CI for a web application – Part 2","author":"Nina Schaaf","date":"30. March 2018","format":false,"excerpt":"GitLab Our first approach was to use the existing GitLab instance of HdM for our project. For them, a shared runner was already defined on which we could run our jobs, so we were able to focus on the CI process itself. This plan worked out at first. We simply\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"Shaky Pipeline GitLab","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/pipeline-gitlab-1024x156.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/pipeline-gitlab-1024x156.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/pipeline-gitlab-1024x156.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":7154,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2019\/08\/31\/setting-up-a-ci-cd-pipeline-in-gitlab\/","url_meta":{"origin":3491,"position":2},"title":"Setting up a CI\/CD pipeline in Gitlab","author":"nr037","date":"31. August 2019","format":false,"excerpt":"Introduction For all my university software projects, I use the HdM Gitlab instance for version control. But Gitlab offers much more such as easy and good ways to operate a pipeline. In this article, I will show how we can use the CI\/CD functionality in a university project to perform\u2026","rel":"","context":"In "Cloud Technologies"","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":3496,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/30\/ci-cd-with-gitlab-ci-for-a-web-application-part-1\/","url_meta":{"origin":3491,"position":3},"title":"CI\/CD with GitLab CI for a web application – Part 1","author":"Nina Schaaf","date":"30. March 2018","format":false,"excerpt":"Introduction When it comes to software development, chances are high that you're not doing this on your own. The main reason for this is often that implementing components like UI, frontend, backend, servers and more is just too much to handle for a single person leading to a slow development\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"Shaky architecture","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/01_shaky-architecture-300x106.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":10392,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2020\/02\/29\/attempts-at-automating-the-build-process-of-a-net-wpf-application-with-gitlabs-ci-cd-pipeline\/","url_meta":{"origin":3491,"position":4},"title":"Attempts at automating the build process of a .NET WPF application with GitLab’s CI\/CD pipeline","author":"Felix Messner","date":"29. February 2020","format":false,"excerpt":"(Originally written for System Engineering and Management in 02\/2020) Introduction In the System Engineering course of WS1920, I took the opportunity to look into automating the build process of a Windows desktop application. Specifically, the application in question is built in C#, targeting .NET Framework 4.0 and using Windows Presentation\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/08\/windows_runner_Tree.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/08\/windows_runner_Tree.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/08\/windows_runner_Tree.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/08\/windows_runner_Tree.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":3314,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/28\/continuous-integration-deployment-for-a-cross-platform-application-part-1\/","url_meta":{"origin":3491,"position":5},"title":"Continuous Integration & Deployment for a Cross-Platform Application – Part 1","author":"Tobias Eberle, Marco Maisel, Tobias Staib, Mario Walz","date":"28. March 2018","format":false,"excerpt":"When we started the project \"Flora CI\" for the lecture \"System Engineering\", we planned to deal with Continuous Integration. As an important aspect of software engineering all of us have previously been involved in projects where code of developers had to be merged and builds had to be automated somehow.\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/flora-app.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":746,"user_id":188,"is_guest":0,"slug":"mm248","display_name":"Tobias Eberle, Marco Maisel, Tobias Staib, Mario Walz","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/a12752d216e492995fc5c8d7b5ae0bea6937c816b1c31526029e032a119b3041?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/3491","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/188"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=3491"}],"version-history":[{"count":23,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/3491\/revisions"}],"predecessor-version":[{"id":24761,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/3491\/revisions\/24761"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media\/3462"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=3491"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=3491"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=3491"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=3491"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}