Continuous Integration & Deployment for a Cross-Platform Application – Part 2

In the first part we pointed out how we set up the infrastructure for our CI system. Now we would like to explain how we build a pipeline for our cross-platform application and what features of GitLab CI we made use of.

Building a Pipeline in GitLab CI

Our first goal when we faced our fresh installed GitLab instance obviously was to get a build done – initiated by a commit to the repository.

The CI functionality in GitLab CI is simply enabled by just putting a gitlab-ci.yml file into the projects root. The entire pipeline is defined in this file.

So we created a job for Android which executed a build, signed the application and uploaded it to Google play. For the Web and iOS build we defined similar jobs. So far so good. Pretty fast we realized the need of a proper pipeline since our three jobs were running fully independently from each other. So no matter if the iOS job fails after a minute, the Android job would deploy a new version which is not desirable regarding consistency. This is where stages help out.

Stages

Instead of putting all steps in a single job we created multiple jobs containing only the steps that are necessary for the respective stage.

All of the jobs in a stage are executed in parallel (there need to be enough runners available). If they all succeed, the pipeline moves on to the next stage. If one jobs fails, the next stage is not executed.

By using multiple jobs in different stages we needed to have a mechanism that passes artifacts from previous jobs to the following ones. How could the Android deployment job get at the APK file which was built by the build job? We use another building blocks of common CI systems: Artifacts.

Artifacts

Artifacts is a list of files and directories which are attached to a job after it completes successfully. The artifacts will be sent from the runner to GitLab and will be available for download in the GitLab UI. We also liked that there is an API to access them from another client. To pass artifacts between different jobs, the job which depends on the results of a previous job declares the dependency as follows:

build_android:
    stage: build
    ...
    artifacts:
        paths:
            - platforms/android
        expire_in: 2 weeks
        
deploy_android:
    stage: deploy
    ...
    dependencies:
        - build_android

The APK file which needs to be deployed is located in the platforms/android  folder which is defined as artifact of the build_android job. By setting expire_in we tell GitLab to delete the artifact after two weeks. Otherwise it would be kept by GitLab and use considerable amounts of disk space.

By defining the dependency on build_android we make sure that our deploy_android job gets the required files to be able to do the deployment. When the job starts to run, it will download the artifacts.

Downloading artifacts for build_android...
Downloading artifacts from coordinator... ok

Configuring build jobs

Defining a reusable docker container

As a basis for the pipeline, a self-configured Docker image is used as a base for the GitLab runners. It contains all necessary prerequisites and dependencies for the build and deployment process and is declared as base image for docker runners as simple as that:

image: marcomaisel/ionic:latest

It contains Ionic 3, Cordova, node & npm, Java, Android SDK, Gradle, fastlane, Chrome and all necessary prerequisites.

Increment version number

The automated upload to the Google Play Store only works if the version code is incremented for each new upload. Therefore, each time a commit is made, a custom script needs to be executed at the beginning of the runner’s building process. This script increases the version code in the associated config.xml and package.json files and commits the changes to the repository.

The automatic upload to the Google Play Store is made possible by the open source platform ‘fastlane’. In the associated Fastfile, which is located in the respective repository (in our case this is the frontend repository), the upload process is specified and necessary information such as package names, folder structure or keys are stored.

To make fastlane compatible with an Ionic/Cordova project, we need to use a workaround because the Android build job only works if fastlane is located inside the Android subfolder. However, we don’t want to put the fastlane files in that subfolder because of the danger to overwrite something and because we also need fastlane for our iOS build. Therefore we need to wrap all the android actions in a chdir do:

lane :playstore do
    Dir.chdir("../platforms/android/src") do
        // Gradle Tasks
    end
  end

Because the folder structure of Cordova projects changed with Cordova versions 7 and higher, we need to run the commands in a different folder:

lane :playstore do
    Dir.chdir("../platforms/android/app") do
        // Gradle Tasks
    end
  end

Speeding up the cycle time

After reducing our cycle time – the time consumed by a successful run of a pipeline – by using more performant hardware for the runners (as mentioned previously in part 1) we took a closer look at our build processes.

Optimizing Build processes

We went through our build jobs line by line. We found some steps like generating app icons that we removed from the CI as it doesn’t need to be done for every build.

Even though we installed Gradle as a dependency in our base docker image there was always some activity loading and installing Gradle dependencies during the Android build. To avoid this time-consuming activity we had to make use of GitLab’s Caching feature.

Caching

Caching is great if you need to dynamically install certain dependencies during your job and can’t pre-build them into a CI image for some reason. This example demonstrates how we cached Gradle dependencies folders between builds:

build_android:
    stage: build
    before_script:
        - ...
    script:
        - ...
    tags:
        - ...
    cache:
        untracked: false
        key: "buildandroid"
        paths:
        - node_modules/
        - plugins/
        - .gradle/wrapper
        - .gradle/caches
    artifacts:
        - ...

As you can see we used this feature for caching NPM’s node_modules folder as well which reduced the execution time for npm install from about two minutes to about 20 seconds. As installing npm dependencies is done in all of our four stages it saved us more than 5 minutes for the entire pipeline.

All in all caching reduced the cycle time of your pipeline almost by half – from 25-30 minutes to 12-14 minutes. In our opinion this duration is acceptable and provides notable faster feedback for the involved developers.

Deployment for different environments

The last stage in our pipeline contains the deployment jobs. We defined a staging and production environment. Both environments are hosted on Amazon S3 instances, they only differ in the URL. Every successful web build will be deployed to our staging environment.

Deployment to the production is a manual step that you can trigger from the GitLab UI. This way of deployment can be easily changed to a branching model. So e.g. if you commit to a “release” branch it will deploy to production automatically instead of doing it manually.

In the end it depends on the deployment strategy and the requirements the project has regarding releases. Technically it’s just about changing a few lines in our CI config.

Hint: Working with the Amazon AWS CLI we had to deal with API keys which have to be kept secret. GitLab CI has a place for that which is called secret variables. These variables can be added in the UI and will be turned into environment variables. In this way you can access them e.g. in your GitLab config file without adding them to the repository.

Conclusion

The main advantage of hybrid app development is its ability to develop on many platforms simultaneously with a single code base. Ionic is also compatible with Electron, Github’s framework for developing cross-platform desktop applications. So it would be possible to develop for even more platforms with the same code base.

We managed to set up the infrastructure and a pipeline at very low cost, but the resulting problems (especially regarding performance) took a lot of time. Debugging a build that takes 50 minutes is really annoying. From the experience gained, we would suggest to invest a little more money in future projects to avoid our initial problems.

For longer-term projects that are to be developed simultaneously on several platforms (Android, iOS, Web), we would continue to rely on Gitlab CI. We liked the simple syntax of the config file and the GitLab UI integrating repository and CI all-in-one.

If the pipeline is successfully set up, future deployment will be greatly facilitated by the resulting automation.