When we started the project “Flora CI” for the lecture “System Engineering”, we planned to deal with Continuous Integration. As an important aspect of software engineering all of us have previously been involved in projects where code of developers had to be merged and builds had to be automated somehow. Anyway we felt we could dive deeper into CI pipelines and learn what it needs to get them running in a performant, useful and reliable way.
On our journey building such a system we also planned the containerization of all components in order to achieve the strongest possible encapsulation of the individual processes and to gain experience in this topic as well. Another planned goal was the automatic deployment into the corresponding Android and iOS App Stores as well as the provision on a web server.
Related articles: ►CI/CD infrastructure: Choosing and setting up a server with Jenkins as Docker image ►Dockerizing Android SDK and Emulator for testing ►Automated Unit- and GUI-Testing for Android in Jenkins ►Testing a MongoDB with NodeJS, Mocha and Mongoose
During the winter term 2017/2018, we created an app called Take Me Home. The purpose of the app is to guide you home in case you are not able to find the way back home by yourself. There are several situations in (a student’s) life when this functionality can be very useful. For example, under the influence of a specific amount of alcohol even an easy and short path back home can get rather challenging. In this case one can use Take Me Home to find a fast and save way back home.
Related articles: ►Take Me Home – Project Overview ►CI/CD infrastructure: Choosing and setting up a server with Jenkins as Docker image ►Automated Unit- and GUI-Testing for Android in Jenkins ►Testing a MongoDB with NodeJS, Mocha and Mongoose
During our Android development project, we had to cope with several technological and organizational challenges with regard to construct a stable CI pipeline. Due to our chosen stack of Docker containers for building and deploying we had to confront with the topic how to integrate and deploy a building and testing environment for Android in Docker.
For a better understanding of this challenge following illustration of our stack can be consulted:
Related articles: ►Take Me Home – Project Overview ►CI/CD infrastructure: Choosing and setting up a server with Jenkins as Docker image ►Android SDK and emulator in Docker for testing ►Testing a MongoDB with NodeJS, Mocha and Mongoose
In this article we would like to describe, how to write unit- and gui-tests for an Android-Application in Android Studio and how-to automat these tests on Jenkins.
Tools and frameworks
We used the following tools and frameworks to write and automate our tests:
- Android Studio
To test the basic functionality of your app, the best way is to run a unit-test. Continue reading
Related articles: ►Take Me Home – Project Overview ►CI/CD infrastructure: Choosing and setting up a server with Jenkins as Docker image ►Android SDK and emulator in Docker for testing ►Automated Unit- and GUI-Testing for Android in Jenkins
Setting up the testing environment and workflow
- Jenkins CI Docker Container
- MongoDB Docker Container
- Production-MongoDB on mlab.com
- NodeJS Web-Application, hosted on heroku.com
Regarding our database tests we wanted to achieve two things: first of all, we wanted to test any functions of our web application using mongoose which change persistent data. Secondly, we needed database tests to test if eventual model-changes are compatible with our data in the production database.
In relational database management systems one defines constraints in the database, which are tested by the database tests. Since we are using MongoDB though, we don’t really have any constraint in the database. Instead we can define any needed constraints with mongoose right in the application. Therefore, cloning the production database for testing is only required when migrating data (or when having constraints defined) but non-essential for the consistency of the as-is backend.
So alternatively when testing a MongoDB we could also use a shell-script or a “before”-function, that gets called once before starting the tests, to define our database testing environment, like creating the same collections we have in the production database and put in test data for our test cases.
You might have already heard of IBM’s artificial intelligence “Watson”, which beat two former champions of the american television game show “Jeopardy!” back in 2011. What you probably don’t know is that today lots of predefined Watson services are publicy available on IBM’s cloud platform “Bluemix”. These services cover different aspects of AI-backed applications like Visual Recognition, Language Translation or Text to Speech. This post glances on Natural Language Understanding, which is about analyzing text by extracting different kinds of information, and shows how this can be achieved by using Bluemix services.
At the beginning of this article series we introduced the core concepts of Hadoop and Spark in a nutshell. Both, Apache Spark and Apache Hadoop are frameworks for efficient processing of large data on computer clusters. The question arises how they differ or relate to each other.
Hereof it seems the opinions are divided. In discussions some people opine that Apache Spark is edge Hadoop away, because Spark is among others for some use cases more efficient and distinctly faster. To get a clearer understanding of this we compare the both frameworks in the following section.
The main reason for choosing Spark was a second project which we developed for the course “Programming Intelligent Applications”. For this project we wanted to implement a framework which is able to monitor important events (e.g. terror, natural disasters) on the world through Twitter.
To separate important tweets from others we use Latent Dirichlet Allocation (LDA) which is an algorithm for topic modelling. LDA is able to extract distinct topics through relations between words. An example for such a relation would be that the word “terror” is likely to be used with the word “attack”. This information is enough to generate separate topics for a bunch of documents, in our case tweets. Therefore, the need to annotate training data to learn word distributions for topics is not necessary and LDA is an algorithm which can learn completely unsupervised. For most machine learning applications the performance is getting better with more training data, and this also applies for our application.
Our objective in this project was to build an environment that could be practical. So we set up a virtual Hadoop test cluster with virtual machines. Our production environment was a Hadoop Cluster in the IBM Bluemix cloud which we could use for free with our student accounts. We developed and tested the logic of our Spark applications with a small amount of data local or in the virtual test cluster. After that we run the applications in the cloud with perhaps higher amount of data. The figure below shows our clusters and their capabilities.