Category: System Designs
Spying on everyone – Cheap and Simple
Introduction Espionage, or less formally “syping” has become a huge public topic in Summer 2013. Edward Snowden revealed that The American NSA (National Security Agency) and the allied Great Britain GCHQ (Government Communications Headquarters) are surveilling, storing and examing all network traffic in- and outbound the US and Great Britain. The debate covered the Newspapers…
Social Engineering: Firewall-Rules for your brain – Part 1
Humans can be regarded as one of the biggest weaknesses for secure systems. Their interaction with technology and awareness for information security makes them usually the “weak link” for gaining access to enterprise networks and private information. From an attackers point of view the investment of using a human to hack a system is much…
Blockchain – Revolution or hype?
Welcome to our journey through the blockchain, since the emergence of Bitcoin, one of the most trending topics of the global digital village. After reading this blog post, you’ll have a basic understanding of the technology, a wide overview of future use cases and are able to differentiate between realistic potential and hype. In the…
Analyzing text with IBM Watson services on Bluemix
You might have already heard of IBM’s artificial intelligence “Watson”, which beat two former champions of the american television game show “Jeopardy!” back in 2011. What you probably don’t know is that today lots of predefined Watson services are publicy available on IBM’s cloud platform “Bluemix”. These services cover different aspects of AI-backed applications like…
Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services – Part 6 – Apache Spark and/vs Apache Hadoop?
At the beginning of this article series we introduced the core concepts of Hadoop and Spark in a nutshell. Both, Apache Spark and Apache Hadoop are frameworks for efficient processing of large data on computer clusters. The question arises how they differ or relate to each other. Hereof it seems the opinions are divided. In…
Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services – Part 5 – Spark applications in PIA project
The main reason for choosing Spark was a second project which we developed for the course “Programming Intelligent Applications”. For this project we wanted to implement a framework which is able to monitor important events (e.g. terror, natural disasters) on the world through Twitter. To separate important tweets from others we use Latent Dirichlet Allocation…
Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services – Part 4 – Big Data Engineering
Our objective in this project was to build an environment that could be practical. So we set up a virtual Hadoop test cluster with virtual machines. Our production environment was a Hadoop Cluster in the IBM Bluemix cloud which we could use for free with our student accounts. We developed and tested the logic of…
Building an HdM Alexa Skill – Part 4
We present our own HdM Alexa Skill and share the experience we gained throughout this project. This time: Automating tests and deployment with Continuous Integration via Jenkins.
Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services – Part 3 – What is Apache Spark?
Apache Spark is a framework for fast processing of large data on computer clusters. Spark applications can be written in Scala, Java, Python or R and can be executed in the cloud or on Hadoop (YARN) or Mesos cluster managers. It is also possible to run Spark applications standalone, that means locally on a computer.…
Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services – Part 2 – Apache Hadoop Ecosystem
In our project we primarily implemented Spark applications, but we used components of Apache Hadoop like the Hadoop distributed file system or the cluster manager Hadoop YARN. For our discussion in the last part of this blog article it is moreover necessary to understand Hadoop MapReduce for comparison to Apache Spark. Because of this we…