Hosting your own GitLab server

Some users might have concerns regarding security using GitLab for a variety of purposes, including commercial and business applications. That is, because GitLab is commonly used as a cloud-based service – on someone else’s computer, so to speak. So setting it up for running it on your own server is the conclusion, whether it be a NAS, real dedicated server or even a Raspberry Pi. So, as a side quest, we decided to set things up on a Raspberry Pi Model 3 for comparison. The following part will cover the installation procedure (mostly according to the official GitLab page) as well as hints to some potential pitfalls.

Basically, running GitLab on a Raspberry Pi became possible back in September 2016, when the Pi model 2 was silently upgraded to the BCM2837, which in fact is a 64bit ARM SoC, and GitLab requires an 64bit architecture to run. This is the same chip that powers the Pi 3, why we chose the latter and installed a 32GB microSD card. While the up-to-date Raspian distribution Stretch still is an 32bit operating system, we initially were hoping we could push things further and overcome the expected performance (or lack thereof) by using the 64bit OS Pi64 (https://github.com/bamarni/pi64). Following the installation routine described further on, we were forced to a sudden stop as the gitlab-ce apt package is only available in a 32bit format. Bummer.

So we started over with a fresh install of the Raspian Stretch distro. You most definitely want to resize your partition via raspi-config to use all of the available memory of the microSD card and have enough space for your files. Another point is that GitLab will want to use 4GB of RAM, as described earlier. As this is just physically impossible on a Pi, GitLab will fall back to its bare minimum requirement of 1GB by creating an additional 3GB swap file, a setting that is strongly discouraged by GitLab itself and which results in the already not-so-strong CPU being heavily loaded with moving data from swap to RAM and vice versa, thus narrowing down the Pi’s performance even further.

Accepting all those limitations, the actual setup process mainly works as described by GitLab (https://about.gitlab.com/installation/#raspberry-pi-2), even if that guide is meant to be for the Pi 2. After the initial sudo apt-get update && sudo apt-get upgrade, things were getting serious:

sudo apt-get install curl openssh-server ca-certificates apt-transport-https
curl https://packages.gitlab.com/gpg.key | sudo apt-key add -
sudo apt-get install -y postfix

During Postfix setup, you can just apply all of the default settings.

sudo curl -sS https://packages.gitlab.com/install/repositories/gitlab/raspberry-pi2/script.deb.sh | sudo bash 

Again, this works for the Pi 3 as well as for the older Pi 2, so don’t mind that path name.

Next, you’ll need to meet some requirements that are not really documented there: First, apt-get install gitlab-ce will fail because of that package being only available for the older Raspian distribution Jessie. So you have to trick your system to identify as such by modifying the file /etc/apt/sources.list.d/gitlab_gitlab-ce.list (e.g. with sudo nano) and just replace stretch with jessie in both lines. Retrieving the package will still fail unless you have run apt-get update one more time.

Besides, depending on how you want to access your GitLab instance it might be necessary to have a DynDNS account ready at hand, because the next step will automatically set up the server to connect to its address which you fill in by replacing the gitlab.example.com default. Otherwise, make sure you have assigned a static IP address to the Pi in your network so you can permanently access it locally. You should also pay attention to opening ports 22 and 80 for SSH and HTTP, respectively.

 sudo EXTERNAL_URL="http://gitlab.example.com" apt-get install gitlab-ce

This will probably take some time, so go grab a coffee meanwhile.

sudo gitlab-ctl reconfigure

After that, you will be able to access your GitLab instance via the hostname and you can start with configuring it, the same way we described earlier.

Hosting your own Runner server

Now, compared to the Gitlab instance setup, installing a GitLab Runner on Raspberry Pi is quite a walk in the park.

Since we want to use the Docker executor, we have to install Docker in the first place:

curl -sSL https://get.docker.com/ | sh

While a simple apt package install like

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner

would probably do the trick (eventually after altering /etc/apt/sources.list.d/gitlab_gitlab-runner.list similar to the GitLab repo setup), we decided to take the manual route:

sudo wget -O /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-arm
sudo chmod +x /usr/local/bin/gitlab-runner
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner

Control the Runner via:

sudo gitlab-runner start
sudo gitlab-runner stop

Finally, you just need to register your new (running, that is: started) Runner according to the previously described steps, using the token provided by your GitLab instance:

sudo gitlab-runner register

More details on this can be found at https://gitlab.com/gitlab-org/gitlab-runner/blob/master/docs/install/linux-manually.md.

Conclusion and outlook

It was very instructive to set up a CD pipeline from scratch, considering that these processes are becoming more and more important and present in software development. We had one or two challenges to overcome, from setting up the web server to combining all the small steps to a functioning and well interacting pipeline. But in the end we learned a lot, not only for an exam but for future working routine.

CI promotes more structured working, since every developer has insight into the process at all times, including the possible errors. Testing and subdividing the pipeline into stages and jobs makes error identification much easier and problems can be resolved more quickly.

One hurdle that we came across during our project was the firewall that blocked the SSH connection to our server. Luckily, this problem occurred relatively early in our project, so we were able to react without problems by setting up our own GitLab instance. Apart from that, we had to struggle again and again with smaller challenges, for which we always found a solution relatively quickly.

As expected, using an instance of GitLab on a Raspberry Pi isn’t really fun because every step takes ages. No wonder regarding the aforementioned specs and limitations. It still is a great step to more freedom and serves as a good educational example. On one point we couldn’t even manage to connect via SSH, but that turned out to be another huge problem, that is often mentioned related to cloud-connected Raspberry Pi projects: always change the default password when setting up your Pi. Apparently our Pi 3 got hacked and ran some mysterious code which tried to gain access to some Portuguese company networks (probably as a part of a larger botnet), as we figured out when examining  the terminal output on an attached HDMI screen.

Speaking of low-end performance, running tests on such an instance is virtually impossible, so we didn’t even try to install a GitLab Runner there. But as SBCs are getting dirt-cheap nowadays, we just got hands on another Pi 3 for that very purpose, only difference being a memory card half the size of the Pi running the GitLab instance.

Surprisingly, the Runner on a Pi 3 is a lot more practicable than the GitLab instance on the same device. This might even be enough for smaller projects with 5-7 team members working on not too CPU consuming tests. However, since the HdM Runners are shared with other projects, it’s hard to compare the actual oomph, let alone the fact we don’t even know the hardware specs of the machines running those.

Working with GitLab CI was intuitive and relatively easy most of the time. Because the tool is part of GitLab, all tools are bundled in one place, which makes the work a little easier. GitLab CI also has a very good documentation and a growing community, so problems can usually be solved quickly.
What we were missing was that the tool was not very customizable, unlike Jenkins, for example. We would have liked to have a dashboard, which gives us a brief overview of the most important activities. The publication of test results in the CI Tool directly would also be advantageous for the purpose of transparency. Using GitLab Pages, however, we found a good way to publish the test results close to the CI tool.