{"id":2820,"date":"2017-08-31T00:17:41","date_gmt":"2017-08-30T22:17:41","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=2820"},"modified":"2023-08-06T21:50:25","modified_gmt":"2023-08-06T19:50:25","slug":"iot-with-the-raspberry-pi-part-1","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/iot-with-the-raspberry-pi-part-1\/","title":{"rendered":"IoT with the Raspberry Pi &#8211; Part 1"},"content":{"rendered":"<h3><span style=\"font-weight: 400;\">Introduction to the project<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">As part of the lecture \u201cSoftware Development for Cloud Computing\u201d in summer term 2017 we primarily wanted to work on a project that has something to do with the Internet of Things. In more detail we decided to measure air quality using a Raspberry Pi with the MQ135 Gas sensor and send this data to our self-built cloud application to analyze it.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We started out by getting IBM Bluemix student accounts. Playing around with different Bluemix Services we found the <\/span><a href=\"https:\/\/internetofthings.ibmcloud.com\/#\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">IBM Watson IoT Platform<\/span><\/a><span style=\"font-weight: 400;\"> which can display data you can send to it from other devices out of the box. Despite displaying the data in charts it is also possible to create rules and send alerts based on them and to authorize other Bluemixusers to manage your devices as well.<\/span><br \/>\n<!--more--><br \/>\n<span style=\"font-weight: 400;\">But we didn\u2019t like that it is all integrated into Bluemix and therefore requires you to set up an user account which turned out to be way more complicated as we expected and you need to navigate through Bluemix\u2019 sometimes quite confusing user interface.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For that reason and way more important because we wanted to learn how these things work in detail and get some experience with them, we decided to implement our own application with partly related functionality.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In our blogposts we want to share our gained insights and therefore these posts are going to cover the following topics:<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400;\">The setup of Raspberry Pi and MQ135 gas sensor including the physical connection between both as well as calibration of the sensor and reading and sending the data using a python script<\/span><\/li>\n<li><span style=\"font-weight: 400;\">An overview of the Dataflow programming tool Node-Red which we tried out for building our cloud application<\/span><\/li>\n<li><span style=\"font-weight: 400;\">An introduction to the MQTT Protocol which we used to transfer our sensor data to our cloud app including setting up our own MQTT Broker<\/span><\/li>\n<li><span style=\"font-weight: 400;\">How we built our actual web application upon the IBM Bluemix Cloud-Platform as a Service using Node.js, MongoDB, Bootstrap 4, Webpack, Gitlab CI and a couple of Frameworks like Express, Passport, Chart.js and others<\/span><\/li>\n<\/ul>\n<h3><span style=\"font-weight: 400;\">Setting up Raspberry Pi and MQ-135 Gas Sensor<\/span><\/h3>\n<h4><span style=\"font-weight: 400;\">Connection between Raspberry Pi and MQ-135 Gas Sensor<\/span><\/h4>\n<p><strong>Accessories:<\/strong><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">The MQ-135 (as well as many other MQ-X gas sensors) needs 5V to work correctly. Since it is not recommended to put 5V on Raspberry Pi\u2019s input GPIO, we need to put a 5V to 3.3V Logic Level Converter between them.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Because our sensor\u2019s output is an analog signal, we also need an Analog-Digital Converter to convert the signal for our Raspberry Pi.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Furthermore a Breadboard and of course jumper wires are required for the connection.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">For setting up we mainly followed <\/span><a href=\"https:\/\/tutorials-raspberrypi.com\/configure-and-read-out-the-raspberry-pi-gas-sensor-mq-x\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">this tutorial<\/span><\/a><span style=\"font-weight: 400;\"> except that we used a different Logic Level Converter. Being our first project using a Raspberry Pi\u2019s GPIO parts of this tutorial seemed to be a bit complicated. That is why we are going to try to explain things in different maybe easier way using numbers in the picture here again.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" data-attachment-id=\"2838\" data-permalink=\"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/iot-with-the-raspberry-pi-part-1\/raspberry_pi_gas_sensor-steckplatine\/\" data-orig-file=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine.png\" data-orig-size=\"1600,812\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"raspberry_pi_gas_sensor-steckplatine\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine-1024x520.png\" class=\"alignnone wp-image-2838 size-large\" src=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine-1024x520.png\" alt=\"\" width=\"656\" height=\"333\" srcset=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine-1024x520.png 1024w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine-300x152.png 300w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine-768x390.png 768w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_pi_gas_sensor-steckplatine.png 1600w\" sizes=\"auto, (max-width: 656px) 100vw, 656px\" \/><\/p>\n<ol>\n<li><span style=\"font-weight: 400;\">To supply power to our sensor we used the 5V pin from our Raspberry Pi even though this not recommended when more other external devices (keyboard etc.) are plugged in.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Instead of what looks like a split wire here we just used another wire from the other 5V pin of our Raspberry Pi to connect it to the Logic Level Converter.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Connect the Ground pin to the breadboard.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">And from there connect it to the sensor.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">As well as to the logic level converter once on each side (8)<\/span><\/li>\n<li><span style=\"font-weight: 400;\">And twice to the analog digital converter (10)<\/span><\/li>\n<li><span style=\"font-weight: 400;\">The green wire is the analog response signal from the sensor which first needs to be converted from 5V to 3.3V by the logic level converter.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">As logic level converter we used a 8 channel one called SKU:AC-LLC8-V2 that was included in a Raspberry Pi starter package we bought but unfortunately not labeled the same way as the one in the tutorial nor documented in any way. Luckily every logic level converter works mainly the same way. So we figured out that we can put our response signal wire (7) onto almost any pin (we chose A0) as long as we plug a 5V wire (2) onto the same side (pin VCCA). Onto the counterpart of VCCA (VCCB) we need to plug the lower voltage we want to convert our signal to. So this is 3.3V coming from the Raspberry Pi (12). Then we need to plug ground wires to the \u201cGND\u201d labeled pins of each side (5).<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Finally we can receive our response signal converted to 3.3V from A0\u2019s counterpart B0 and wire it up to our.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Analog digital converter (MCP3008). There are more detailed tutorials on how to use an analog digital converter to convert and read analog signals using the I2C bus of a Raspberry Pi. (<\/span><a href=\"https:\/\/tutorials-raspberrypi.de\/raspberry-pi-mcp3008-analoge-signale-auslesen\/\"><span style=\"font-weight: 400;\">here<\/span><\/a><span style=\"font-weight: 400;\"> you can find a German one for example). But for here it is enough to just wire it up to the data from our sensor (9), ground (6) and 3.3V (12) from our Raspberry Pi.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Then it is possible to read the signal using the Raspberry Pi by connecting it with the analog digital converter as shown.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Raspberry Pi\u2019s 3.3V are connected to the breadboard and used by the analog digital converter (10) and the logic level converter (8) mainly as reference value.<\/span><\/li>\n<\/ol>\n<p><span style=\"font-weight: 400;\">Our real-life result looked like this and worked as expected.<\/span><img loading=\"lazy\" decoding=\"async\" data-attachment-id=\"2836\" data-permalink=\"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/iot-with-the-raspberry-pi-part-1\/raspberry_gesteckt\/\" data-orig-file=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt.jpg\" data-orig-size=\"1164,653\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"raspberry_gesteckt\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt-1024x574.jpg\" class=\"alignleft wp-image-2836 size-medium_large\" src=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt-768x431.jpg\" alt=\"\" width=\"656\" height=\"368\" srcset=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt-768x431.jpg 768w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt-300x168.jpg 300w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt-1024x574.jpg 1024w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/raspberry_gesteckt.jpg 1164w\" sizes=\"auto, (max-width: 656px) 100vw, 656px\" \/><\/p>\n<h3><span style=\"font-weight: 400;\">Calibration, reading and sending the data<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">For calibration of a MQ-X gas sensor first of all the according datasheet to the sensor is needed. By googling \u201cmq135 datasheet\u201d a couple of them can be found for example <\/span><a href=\"https:\/\/www.olimex.com\/Products\/Components\/Sensors\/SNS-MQ135\/resources\/SNS-MQ135.pdf\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">this one<\/span><\/a><span style=\"font-weight: 400;\">. But unfortunately none of these provided the chart that is needed for calibration with a completely readable legend. This problem has been solved by prolonged searching for a datasheet that doesn\u2019t contain a broken font. We found it <\/span><a href=\"https:\/\/upverter.com\/datasheet\/05a4d494d8c28d681c71285ffeaa8c509a605037.pdf\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">here<\/span><\/a><span style=\"font-weight: 400;\">, it was listed on page 5 of the results on the international Google search. Now we were able to translated the chinese characters easily with with Google Translate and were able to fully detect all the gases the sensor responds to.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">We also tried configuration and calibration of our sensor as explained in the already mentioned tutorial. But this didn\u2019t really work out for our sensor because we ended up with very unrealistic values. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">To get realistic values we&#8217;ve looked at another guide which can be found<\/span><a href=\"http:\/\/davidegironi.blogspot.de\/2014\/01\/cheap-co2-meter-using-mq135-sensor-with.html#.WZ3I8HdJYy4\" target=\"_blank\" rel=\"noopener\"> <span style=\"font-weight: 400;\">here<\/span><\/a><span style=\"font-weight: 400;\">. <\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" data-attachment-id=\"2840\" data-permalink=\"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/iot-with-the-raspberry-pi-part-1\/mq135_sensitivity_legend\/\" data-orig-file=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend.jpg\" data-orig-size=\"1450,1144\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"mq135_sensitivity_legend\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend-1024x808.jpg\" class=\"alignnone size-large wp-image-2840\" src=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend-1024x808.jpg\" alt=\"\" width=\"656\" height=\"518\" srcset=\"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend-1024x808.jpg 1024w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend-300x237.jpg 300w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend-768x606.jpg 768w, https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mq135_sensitivity_legend.jpg 1450w\" sizes=\"auto, (max-width: 656px) 100vw, 656px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">In the logarithmic graph for sensitivity characteristics it can be seen that the gas curves are power functions.<\/span><\/p>\n<p><i><span style=\"font-weight: 400;\">y = a*x^b<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">In order to determine the sensitivity for the desired gas (CO<sub>2<\/sub><\/span><span style=\"font-weight: 400;\">) we have read the coordinates of two points, once at x = 10 and once at x = 200, on the CO<sub>2<\/sub><\/span><span style=\"font-weight: 400;\">&nbsp;curve. Through power regression we can obtain a scaling factor (a) and an exponent (b) for the gas which is necessary to compute the value of the output.<\/span><\/p>\n<p><i><span style=\"font-weight: 400;\">ppm = a*(Rs\/Ro)^b<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">To calibrate the sensor also a value of the gas in clean air is required as a reference. We have taken it from <\/span><a href=\"http:\/\/co2now.org\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">http:\/\/co2now.org\/<\/span><\/a><span style=\"font-weight: 400;\">. With all these values, the measured CO<sub>2<\/sub><\/span><span style=\"font-weight: 400;\">&nbsp;content can now be calculated in ppm. This guide provides a script written in C that takes this task. We have translated this script for the Raspberry Pi into Python.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In order to send the determined data to Bluemix, first the Raspberry Pi must be registered as a device in the IoT Platform. At the beginning we sent the data to the cloud using the Websocket protocol from the Python application. <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction to the project As part of the lecture \u201cSoftware Development for Cloud Computing\u201d in summer term 2017 we primarily wanted to work on a project that has something to do with the Internet of Things. In more detail we decided to measure air quality using a Raspberry Pi with the MQ135 Gas sensor and [&hellip;]<\/p>\n","protected":false},"author":481,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1,654,650,22],"tags":[],"ppma_author":[732],"class_list":["post-2820","post","type-post","status-publish","format-standard","hentry","category-allgemein","category-internet-of-things","category-scalable-systems","category-student-projects"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":2859,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/iot-with-the-raspberry-pi-final-application-part-3\/","url_meta":{"origin":2820,"position":0},"title":"IoT with the Raspberry Pi \u2013 Final application \u2013 Part 3","author":"mr143@hdm-stuttgart.de","date":"31. August 2017","format":false,"excerpt":"In our final application, we have put together a solution consisting of four different modules. First, we have again the Raspberry Pi which raises and sends the sensor data using the already presented Python script. We changed the transfer protocol in the final application to MQTT, which gives us more\u2026","rel":"","context":"In &quot;Allgemein&quot;","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mqtt-1024x465.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mqtt-1024x465.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/mqtt-1024x465.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":11460,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2020\/09\/29\/get-car-location-using-raspberrypi-and-google-cloud-iot-core\/","url_meta":{"origin":2820,"position":1},"title":"Get car location using Raspberry Pi and Google Cloud IoT Core","author":"Simon L\u00f6bert","date":"29. September 2020","format":false,"excerpt":"Project idea Have you ever been in the situation, that you parked your car somewhere in the city and some hours later, you couldn't remember where you parked it? You may wish to have an application on your smartphone, which is able to locate your car. From this consideration, the\u2026","rel":"","context":"In &quot;Allgemein&quot;","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/09\/grafik.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":2843,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/iot-with-the-raspberry-pi-node-red-part-2\/","url_meta":{"origin":2820,"position":2},"title":"IoT with the Raspberry Pi \u2013 Node RED &#8211; Part 2","author":"mr143@hdm-stuttgart.de","date":"31. August 2017","format":false,"excerpt":"As already stated in the introduction to our project, we decided to create a Cloud Foundry-Application in IBM Bluemix. We used the boilerplate called \u201cInternet of Things Platform Starter\u201d. Using this boilerplate Node Red is deployed initially. Node Red is a software tool for graphical dataflow programming. It was developed\u2026","rel":"","context":"In &quot;Allgemein&quot;","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/websocketnode_config-300x143.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":3114,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/09\/01\/sport-data-stream-processing-on-ibm-bluemix-real-time-stream-processing-basics\/","url_meta":{"origin":2820,"position":3},"title":"Sport data stream processing on IBM Bluemix:  Real Time Stream Processing Basics","author":"nk065@hdm-stuttgart.de","date":"1. September 2017","format":false,"excerpt":"New data is created every second. Just on Google the humans preform 40,000 search queries every second. By 2020 Forbes estimate 1.7 megabytes of new information will be created every second for every human on our planet. However, it is about collecting and exchanging data, which then can be used\u2026","rel":"","context":"In &quot;Allgemein&quot;","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/09\/Real-Time-Stream-Processing-Basics_6.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/09\/Real-Time-Stream-Processing-Basics_6.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/09\/Real-Time-Stream-Processing-Basics_6.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":2157,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/03\/09\/of-apache-spark-hadoop-vagrant-virtualbox-and-ibm-bluemix-services-part-4-big-data-engineering\/","url_meta":{"origin":2820,"position":4},"title":"Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services &#8211; Part 4 &#8211; Big Data Engineering","author":"bh051, cz022, ds168","date":"9. March 2017","format":false,"excerpt":"Our objective in this project was to build an environment that could be practical. So we set up a virtual Hadoop test cluster with virtual machines. Our production environment was a Hadoop Cluster in the IBM Bluemix cloud which we could use for free with our student accounts. We developed\u2026","rel":"","context":"In &quot;Student Projects&quot;","block_context":{"text":"Student Projects","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/student-projects\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/03\/dev-env-spark-768x512.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/03\/dev-env-spark-768x512.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/03\/dev-env-spark-768x512.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":2143,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/03\/08\/of-apache-spark-hadoop-vagrant-virtualbox-and-ibm-bluemix-services\/","url_meta":{"origin":2820,"position":5},"title":"Of Apache Spark, Hadoop, Vagrant, VirtualBox and IBM Bluemix Services &#8211; Part 1 &#8211; Introduction","author":"bh051, cz022, ds168","date":"8. March 2017","format":false,"excerpt":"As part of the lecture \u201cSystem Engineering and Management\u201d in the winter semester 2016\/17, we run a project with Apache Spark and the Apache Hadoop Ecosystem. In this article series firstly we want to introduce Apache Spark and the Apache Hadoop Ecosystem. Furthermore we want to give an overview of\u2026","rel":"","context":"In &quot;Student Projects&quot;","block_context":{"text":"Student Projects","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/student-projects\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":732,"user_id":481,"is_guest":0,"slug":"mr143","display_name":"mr143@hdm-stuttgart.de","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/2a169003f6b79b3c75d54001ae4c144bd4e03ff2bbcafda15f2a705800c27636?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/2820","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/481"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=2820"}],"version-history":[{"count":12,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/2820\/revisions"}],"predecessor-version":[{"id":24728,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/2820\/revisions\/24728"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=2820"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=2820"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=2820"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=2820"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}