{"id":4147,"date":"2018-08-30T14:45:05","date_gmt":"2018-08-30T12:45:05","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=4147"},"modified":"2023-06-09T12:00:31","modified_gmt":"2023-06-09T10:00:31","slug":"using-the-power-of-google-cloud-api-a-dockerized-node-app-counting-words-in-prasentations","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/08\/30\/using-the-power-of-google-cloud-api-a-dockerized-node-app-counting-words-in-prasentations\/","title":{"rendered":"Using the power of google cloud API: A dockerized node app counting words in prasentations."},"content":{"rendered":"

For the Dev4Cloud <\/a>lecture at HdM Stuttgart<\/a>, we created a simple Go\/NodeJS\/React App, which helps people to keep track of often used words during presentations. In a presentation setting, most people tend to use too many fill words and to train against this, we want to introduce our presentation counter to you.
\n
\n\"\"<\/a><\/p>\n

The presentation counter consists of 3 parts, the React frontend, the GO backend and the NodeJS speech server for the communication with Google Cloud platform. To make it short, the frontend captures the microphone audio, sends it to the speech server, and the speech server gets the audio transcript from Google. Then the transcript is send back to the Go Backend which saves the relevant words in an Alpine db and updates the frontend.<\/p>\n

Frontend<\/h2>\n

Static compiled react frontend, contain code to communicate via Websocket with the Go and NodeJS server. As well as capturing the microphone audio. Capturing audio needs a bit of boilerplate code:<\/p>\n

AudioContext = window.AudioContext || window.webkitAudioContext;\n    context = new AudioContext();\n    processor = context.createScriptProcessor(bufferSize, 1, 1);\n    processor.connect(context.destination);\n    context.resume();\n\n\n    var handleSuccess = function (stream) {\n        recButton.classList.add(\"rec-effect\")\n\n        globalStream = stream;\n        input = context.createMediaStreamSource(stream);\n        input.connect(processor);\n\n        processor.onaudioprocess = function (e) {\n            microphoneProcess(e);\n        };\n    };\n\n    navigator.mediaDevices.getUserMedia(constraints)\n        .then(handleSuccess);<\/pre>\n
The micophoneProcess<\/code> inside the handleSuccess<\/code> function receives a stream of an audio buffer with a given buffersize. The micophoneProcess<\/code> does two important things, first it converts the stream from 48000 Hz to 16000 Hz and then uses WebSockets to send it to the NodeJS server close to real time.<\/p>\n
\"\"<\/a>
Our frontend. Mobile first witch a super simple UI. Tap the mic to start the session and add words on the bottom.<\/figcaption><\/figure>\n
Speech Server<\/h2>\n
This server is a lightweight NodeJS app. For using the Google Cloud API you need to have your own user-key and this should not be shared with the frontend, so we created a layer in between them. This server holds in key with the dotenv-library<\/a>. Google Cloud API needs the key to be in the process environment variables, and adding the key on operating system level would be a big pain, so we used this great library. After an audio stream started, the server uses the key to authorize with Google Cloud API, and creates uses speechClient.streamingRecognize(request)<\/code> to open a stream to the cloud. Inside the request parameter is the configuration and encoding information. Our configuration looks like this:<\/p>\n
const request = {\n    config: {\n        encoding: encoding,\n        sampleRateHertz: sampleRateHertz,\n        languageCode: languageCode,\n        profanityFilter: false,\n        enableWordTimeOffsets: true\n    },\n    interimResults: true\n};<\/pre>\n
Please note the last item: interimResults: true<\/code> This tells the Google Cloud API to send unfinished results. Why would we want unfinished results, could you ask yourself? The answer is simple: we do not want to wait until the sentence is finished. As soon as a sentence is finished, Google can calculate the probability more accurate, since the context is closed. That means, when ever a Google detects a finished sentence, we can a more accurate prediction. But we would have to wait until the sentence is finished. Because of that we are using the less accurate results to get faster results, near real time again and might have to correct the displayed results if they change.<\/p>\n
Google Cloud API gives a transcript of all the spoken words it can recognize in the audio stream. So the NodeJS Server sends this transcript, divided in a final and a interim part to the Go Backend where the words are going to be counted – and as soon as the interim part gets final recounted. In that way we have fast results, which is important for a word counter app – nobody would like to wait until the sentence is finished for the counts to update.<\/p>\n
Google Cloud API<\/h2>\n
For this project we used the Google Speech API. To be able to use this, you first have to a Google API key, but this is quite straightforward and described here.<\/a> Next you have to download a library from Google, which mirrors the API functions in your code. With NodeJS the installation is npm install google-cloud\/speech --save<\/code>. Now you can import it and initialise it.<\/p>\n
const speech = require('@google-cloud\/speech');\nconst speechClient = new speech.SpeechClient();<\/pre>\n
After those steps you are ready to go! Just follow the steps described in the Speech server section.<\/p>\n
Another thing to add, with your Google accounts comes 300$ of free credit for the Google Cloud Platform. At first we wondered if this will be enough to develop the project, until we found out that you also have 60 free minutes to analyse with the Speech API. Even after those 60 Minutes it is only 15 ct\/min. What we want to say, just play with it, its easy and the possibility to create a nice prototype is really awesome. Thanks Google!<\/p>\n
Go-Backend<\/b><\/h2>\n
At the beginning, this project was fork of a project we did in the same semester. In the base project people can count words of a person doing a presentation by clicking on a button. For this former project I decided to try language in the backend which was new to me: GO. This language is more low-level than Java or JavaScript, but more high-level than C or C++. So if you come from a Java\/JavaScript like we did, your mind will struggle probably from time to time. There some kind Pointer like in C but fortunately no pointer arithmetic. The main advantage of this language shall be, that it can be almost as fast as C++ but takes way less time to compile. Additionally it has some pure functional constructs in it.<\/p>\n
So long story short: What we wanted to do is to extend this server that it uses the google speech-to-text to get back text form of someone\u2019s presentation in real time. During the development process we found out that the requirements didn\u2019t really meet with the previous ones. The server had to update the frontend in real time like it did before but now we wanted the server also to correct the frontends data. Why correcting? From the google API you get back a text in real time if you do stream processing. More or less word for word the sentence you get back is extended. But this sentence isn\u2019t fix, as well as the words in it as long as the sentence hasn\u2019t finished. This is because google always tries to return a reasonable sentence.<\/p>\n
These facts and we still needed to count the words on every update made us to almost rewrite the whole web server code.<\/p>\n
Docker<\/b><\/h2>\n
Both of us Simon and me Marius we still don\u2019t know what we think about this tool. The general idea of having container deencapsulation purpose is great but working with docker often isn\u2019t.<\/p>\n
There is no real debug tool, at least we haven\u2019t found one. Then it caches that much that you have to rebuild your container almost every time\u00a0 from scratch disabling the cache. This can cause a very long build-step depending on what you have to install during the startup process.<\/p>\n
But now to the advantages and what we did with it. Both of our servers, the node and the GO one are running its own alpine container. Docker-Compose is managing the they are in a docker network with a DNS, so the can communicate with their respective names.<\/p>\n
This make it possible to deploy the application anywhere a docker daemon is available. What is quite nice. The whole software infrastructure comes with docker and the definitions in the Dockerfiles.<\/p>\n
\"\"<\/a><\/p>\n
Summary<\/b><\/h2>\n
Working with cloud service is easier than we thought. Using the technology offered by google was very straight forward, but the docker and the GO thing wasn\u2019t.<\/p>\n
This was our first project using the cloud. As a result we just took a simple functionality it offers and created something it. Regarding the effort from an engineering point of view, I might have been easier to use more out of the box functionalities. For example you could have deployed the server code directly to a go or node service a google. There you can create containers by simply drag and drop modules to it.<\/p>\n
Finally we can say, that it is worth to look deeper into the cloud topic. Not only because you needn\u2019t to buy hardware, but also because it can save you a lot time to develop an application.<\/p>\n
 <\/p>\n
Written by: Simon Deussen & Marius Hahn<\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
For the Dev4Cloud lecture at HdM Stuttgart, we created a simple Go\/NodeJS\/React App, which helps people to keep track of often used words during presentations. In a presentation setting, most people tend to use too many fill words and to train against this, we want to introduce our presentation counter to you.<\/p>\n","protected":false},"author":893,"featured_media":4151,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1,120,650,22],"tags":[],"ppma_author":[766],"class_list":["post-4147","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-allgemein","category-cloud-technologies","category-scalable-systems","category-student-projects"],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/08\/aufbau.jpg","jetpack-related-posts":[{"id":2560,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/17\/build-a-serverless-google-home-app\/","url_meta":{"origin":4147,"position":0},"title":"Build a Serverless Google Home App","author":"mg166@hdm-stuttgart.de","date":"17. August 2017","format":false,"excerpt":"In this blog article, I want to show you how to build your own Google Voice app. For Natural language processing, we will use API.AI. Our backend will run on a Google Cloud function, also called serverless functions, written in nodejs. Github You can find the whole project here on\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/archetecture-300x169.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/archetecture-300x169.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/archetecture-300x169.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/archetecture-300x169.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":12751,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2021\/02\/28\/industry-4-0-real-time-data-visualization\/","url_meta":{"origin":4147,"position":1},"title":"Industry 4.0 – Real time data visualization","author":"Philip Betzler","date":"28. February 2021","format":false,"excerpt":"As part of the lecture \"System Engineering and Management\", we worked on a project in cooperation with IBM to visualize Industry 4.0 data in real time using a configuration-based approach. This project aims to avoid needing a web developer every time there is a modification in the data.","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2021\/03\/config_type_harting_mica.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2021\/03\/config_type_harting_mica.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2021\/03\/config_type_harting_mica.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":12060,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2020\/09\/30\/a-beginners-approach-at-a-cloud-backed-browser-game\/","url_meta":{"origin":4147,"position":2},"title":"A beginners approach at a cloud backed browser game","author":"mk321","date":"30. September 2020","format":false,"excerpt":"Foreword: This article reflects my experiences while developing a real time browser-based game. The game of choice was Tic-Tac-Toe as it is straight forward to implement and does not have complex game mechanics. The following paragraphs explain my experiences I got while developing this game with a cloud-based infrastructure in\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/09\/image-28.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/09\/image-28.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/09\/image-28.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/09\/image-28.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":11460,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2020\/09\/29\/get-car-location-using-raspberrypi-and-google-cloud-iot-core\/","url_meta":{"origin":4147,"position":3},"title":"Get car location using Raspberry Pi and Google Cloud IoT Core","author":"Simon L\u00f6bert","date":"29. September 2020","format":false,"excerpt":"Project idea Have you ever been in the situation, that you parked your car somewhere in the city and some hours later, you couldn't remember where you parked it? You may wish to have an application on your smartphone, which is able to locate your car. From this consideration, the\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2020\/09\/grafik.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":22530,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2022\/02\/28\/discord-monitoring-system-with-amplify-and-ec2\/","url_meta":{"origin":4147,"position":4},"title":"Discord Monitoring System with Amplify and EC2","author":"mk322","date":"28. February 2022","format":false,"excerpt":"Abstract Discord was once just a tool for gamers to communicate and socialize with each other, but since the pandemic started, discord gained a lot of popularity and is now used by so many other people, me included, who don't necessarily have any interest in video gaming. So after exploring\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2022\/02\/data-workflow.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2022\/02\/data-workflow.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2022\/02\/data-workflow.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":25999,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2023\/09\/15\/automate-pdf-a-cloud-driven-workflow-tool-with-cloud-functions-and-kubernetes\/","url_meta":{"origin":4147,"position":5},"title":"Automate PDF – A Cloud-Driven Workflow Tool with Cloud Functions and Kubernetes","author":"fb089","date":"15. September 2023","format":false,"excerpt":"Gitlab You can find the Project under this link https:\/\/gitlab.mi.hdm-stuttgart.de\/fb089\/automatecloud Wiki You can find all the Infos in our Gitlab Wiki (https:\/\/gitlab.mi.hdm-stuttgart.de\/fb089\/automatecloud\/-\/wikis\/AutomateCloud). You can even try it urself. Feel free Short Description Automate PDF is a workflow automation tool created in the course \u201cSoftware Development for Cloud Computing\u201d. The application\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/09\/Bildschirmfoto-2023-09-15-um-19.23.39.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/09\/Bildschirmfoto-2023-09-15-um-19.23.39.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/09\/Bildschirmfoto-2023-09-15-um-19.23.39.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":766,"user_id":893,"is_guest":0,"slug":"sd092","display_name":"sd092","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/4dfd915ec097bb60951b924f5eb09948c4df3c9d12b84f182ed754f793b25ec0?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/4147","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/893"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=4147"}],"version-history":[{"count":11,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/4147\/revisions"}],"predecessor-version":[{"id":4359,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/4147\/revisions\/4359"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media\/4151"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=4147"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=4147"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=4147"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=4147"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}