{"id":7125,"date":"2019-08-21T11:22:06","date_gmt":"2019-08-21T09:22:06","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=7125"},"modified":"2023-06-18T18:23:58","modified_gmt":"2023-06-18T16:23:58","slug":"evaluating-and-visualizing-statistical-bot-data","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2019\/08\/21\/evaluating-and-visualizing-statistical-bot-data\/","title":{"rendered":"Evaluating and Visualizing Statistical Bot Data"},"content":{"rendered":"\n
\"\"
STAB Website Homepage<\/figcaption><\/figure>\n\n\n\n

The idea<\/h2>\n\n\n\n

Video game streaming has taken over a big part of the commercial video game scene with Twitch being its biggest platform. Viewers are able to communicate with other viewers and streamers through the chat, or watch their previous streams or highlights. These highlights are made by streamers creating short clips of their favourite moments either by looking through all of their footage, or marking the time of a potential highlight by using the chat command \/mark.<\/p>\n\n\n\n

Since a streamer marking a highlight in the chat himself while playing a game is highly improbable and not every streamer has a chat moderator to take that responsibility into his or her hands, there is mostly only the possibility of looking through hours of video footage, to find a few highlights to cut out.<\/p>\n\n\n\n

Because of this issue, I had the idea of creating a Twitch Chat Bot, which analyzes and evaluates the chat of a Twitch channel, taking message frequency, length and emotes into account, which is able to detect sudden surges in chat activity and sets a highlight marker accordingly. Also it would be nice to be able to actively track the collected data in a way, which makes it easy and intuitive to understand.<\/p>\n\n\n\n

Implementation<\/h2>\n\n\n\n

Java Bot<\/h3>\n\n\n\n

I decided to implement the Bot first, worrying about everything that comes after later. I took a look at different Twitch Bots to help me start things and created a simple connection to my Twitch Channel through the Java Twitch API, integrated into my project by Gradle. Soon I was able to receive messages I had in my chat and relay them to my console. The next issue I had was to determine the way I detected highlights in my chat. I had to find a way to reliably detect surges in chat activity, without counting and marking isolated cases. Research suggested using the exponentially weighted moving average<\/strong> (EWMA)<\/strong>, allowing my bot to rule out the occasional nonstarter.<\/p>\n\n\n\n

\"\"\/
Equation EWMA<\/figcaption><\/figure>\n\n\n\n

The EWMA allows the detection of exceptional deviation by comparing a value to a history, but adding more weight on recent instead of old values through adding exponentially decreasing weights. This added reliable analysis of my received data, so finally all that was left to do was setting a threshold deviation value and giving my bot access to set a marker command in the chat as soon as this threshold was exceeded.<\/p>\n\n\n\n

\"\"
Highlight detection in Twitch Chat
<\/figcaption><\/figure><\/div>\n\n\n\n

Database and Visualization<\/h2>\n\n\n\n

Throughout my research for a appropriate and comfortable interface between the database to implement and the visualization tool, I came by countless options. The original plan was to turn my Bot into a scheduled task as a service in my very own Wildfly application server. This should be integrated into my IntelliJ IDE via Gradle, or tested out through Eclipse via Maven. Additionally I wanted to save my data in a SQLite database which would save data with JPA\/Hibernate and which would communicate with my Chartjs visualization tool on my website through a REST API. Chartjs would take the data and show a graph, describing the EWMA value of the chat’s activity in realtime.<\/p>\n\n\n\n

Rethinking the Architecture<\/h3>\n\n\n\n

After some research for the best way to set up the Wildfly Server I came across an article, describing .NET core and its web and database features. It seemed like there was everything I needed to realise my project in one package, so I decided to leave the comfort of my Java-only experience and dive into uncharted territory. Setting up the bot again in a .NET core web application using C# was no issue, since I was able to take over a lot of my Java logic into my new project. NuGet proved itself to be an intuitive tool, to integrate external libraries into my workspace and through the integration of TwitchLib, the .NET library for Twitch related projects, my bot was able to do everything, the Java version was able to do. <\/p>\n\n\n\n

Next was the creation and storage of the received data in a database. For this I used the combination of .NET’s provided library for databases of webapplications and Microsoft’s Entity Framework. These allowed me to create a SQLite database model which could be initialized by NuGet and filled by connecting my Bot with the database context.<\/p>\n\n\n\n

Through the NuGet Package Manager I also found a convenient interface to connect the ASP.NET MVC framework to chartjs. By writing a ChartController class, I created the means to send a JSON format result from my database to the visualization website, which I configured to show chat data from every second in a historical line graph.<\/p>\n\n\n\n

\"\"
Live-Chart for Bot Data<\/figcaption><\/figure>\n\n\n\n

Additionally to just being able to run the WebApplication through my IDE, the .NET Web Application allowed me to create a Docker script, which, when running Docker in Linux mode, could simply be started through the Visual Studio IDE, running my application as well.<\/p>\n\n\n\n

Issues and Conclusion<\/h2>\n\n\n\n

My biggest issue was to find and configure the right tools, which would connect every service I needed. This, of course, was also made difficult by the fact, that I have never done a project like this, so initially I didn’t even know what to look for. It was eventually rectified by research and deciding to use .NET core as my preferred platform, which however only happened after a lot of trial and error episodes, trying to get Wildfly working in my IntelliJ, als well as Eclipse IDE. As a developer who has first and foremost worked with Eclipse, Maven and Java I first tried to try something new with the IntelliJ IDE and Gradle. Of course I needed a little time to get used to this change, even though the difference to my usual workflow was not too big.<\/p>\n\n\n\n

By deciding to switch everything up, I challenged myself to take on something completely new. Though I worked with C# once, as part of a Unity project where I was in charge of the enemy AI in a video game, the scope of the project was vitally different. So first I decided which libraries, frameworks and interfaces I would need and use, to afterwards research and understand how I needed to use them in order to set up the base for my project.<\/p>\n\n\n\n

In conclusion this was a really interesting and fun project to work on. By combining Twitch as my private interest and the cloud technologies, I gained valuable knowledge and insights into the subject matter. It also eased my fear in trying new things and technologies. Sometimes it’s necessary to jump over your own shadow in order for things to get eventually easier.<\/p>\n\n\n\n

If you would like to take a look at the source code, or try the bot yourself, click here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

The idea Video game streaming has taken over a big part of the commercial video game scene with Twitch being its biggest platform. Viewers are able to communicate with other viewers and streamers through the chat, or watch their previous streams or highlights. These highlights are made by streamers creating short clips of their favourite […]<\/p>\n","protected":false},"author":934,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[120,262],"tags":[276,278,79,277,275],"ppma_author":[792],"class_list":["post-7125","post","type-post","status-publish","format-standard","hentry","category-cloud-technologies","category-rich-media-systems","tag-net","tag-chartjs","tag-chatbot","tag-nuget","tag-twitch"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":25613,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2023\/08\/15\/the-hardest-boss-in-dark-souls-a-secure-multiplayer\/","url_meta":{"origin":7125,"position":0},"title":"The hardest boss in Dark Souls: A secure multiplayer","author":"Fabian Rei\u00dfer","date":"15. August 2023","format":false,"excerpt":"Over the course of the last decades, video games have continuously risen in popularity. Today, hundrets of thousands of people play video games every day. Many of these games have multiplayer, where an enormous amount of people play together at the same time. When thinking about 'hacking' in multiplayer games,\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/DS3.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/DS3.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/DS3.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/DS3.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/DS3.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/DS3.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":21651,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2021\/09\/18\/deploy-random-chat-application-on-aws-ec2-with-kubernetes\/","url_meta":{"origin":7125,"position":1},"title":"Deploying Random Chat Application on AWS EC2 with Kubernetes","author":"dv029","date":"18. September 2021","format":false,"excerpt":"1. Introduction For the examination of the lecture \u201cSoftware Development for Cloud Computing\u201d, I want to build a simple Random Chat Application. The idea of this application is based on the famous chat application called Omegle. Omegle is where people can meet random people in the world and can have\u2026","rel":"","context":"In "Cloud Technologies"","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2021\/09\/image-19.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":2748,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/08\/31\/developing-a-chat-server-and-client-in-the-cloud\/","url_meta":{"origin":7125,"position":2},"title":"Developing a Chat Server and Client in the Cloud","author":"Jan-Niklas Keck","date":"31. August 2017","format":false,"excerpt":"Introduction During the Lecture \"Software Development for Cloud Computing\" I decided to develop a Cloud based Chat Application with the help of IBM's Bluemix. The Application consists of 3 separate Applications: Chat Server: Allows Clients to connect to it, manages the Chat-Channels\/Users and relays messages sent from a client to\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/toolchain-deploy.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/toolchain-deploy.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/toolchain-deploy.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/08\/toolchain-deploy.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":22923,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2022\/03\/20\/games-aus-der-cloud-wo-sind-wir-und-wohin-geht-die-reise\/","url_meta":{"origin":7125,"position":3},"title":"Games aus der Cloud, wo sind wir und wohin geht die Reise?","author":"Sven Kirsch","date":"20. March 2022","format":false,"excerpt":"Cloud Gaming - fl\u00fcssiges Zocken auch mit schlechter Grafikkarte? (esports.com) Was genau ist Cloud Gaming? Cloud Gaming l\u00e4sst sich mit Remote Desktops, Cloud Computing und Video on Demand Diensten vergleichen. Im Grunde beinhaltet Cloud Gaming das Streamen von Videospielen aus der Cloud zum Endkunden. Dabei erfasst und \u00fcbertr\u00e4gt der Client\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/Cloud-Gaming.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/Cloud-Gaming.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/Cloud-Gaming.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/Cloud-Gaming.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/Cloud-Gaming.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/Cloud-Gaming.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":27440,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2025\/02\/27\/how-i-improved-the-network-security-of-my-live-chat-application-architecture-on-aws\/","url_meta":{"origin":7125,"position":4},"title":"How I Improved the Network Security of My Live Chat Application Architecture on AWS","author":"Jannik Scheider","date":"27. February 2025","format":false,"excerpt":"In an increasingly connected world, the need for robust security measures for cloud infrastructures is constantly growing. Applications that are accessible over the internet must be secured in a way that prevents unnecessary exposure of sensitive backend components. A fully public Virtual Private Cloud (VPC) architecture may be sufficient for\u2026","rel":"","context":"In "Cloud Technologies"","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":27618,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2025\/02\/28\/crowdconnect-developing-a-scalable-live-chat-application-with-aws-cloud-services\/","url_meta":{"origin":7125,"position":5},"title":"CrowdConnect – Developing a Scalable Live Chat Application with AWS Cloud Services","author":"Jannik Scheider","date":"28. February 2025","format":false,"excerpt":"Imagine you're developing a live chat application in the cloud that needs to serve a growing number of users simultaneously and in real time across multiple chat rooms. Sounds like a challenge? It is. But with proven approaches and valuable insights from real-world experience, this task can be successfully and\u2026","rel":"","context":"In "Cloud Technologies"","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2025\/02\/image-14.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2025\/02\/image-14.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2025\/02\/image-14.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":792,"user_id":934,"is_guest":0,"slug":"zc001","display_name":"zc001","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/596029c47af21641a3e519c3446fccf90c38eb3f8604778d6fd2b3e7520cee9a?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/7125","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/934"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=7125"}],"version-history":[{"count":5,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/7125\/revisions"}],"predecessor-version":[{"id":7380,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/7125\/revisions\/7380"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=7125"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=7125"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=7125"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=7125"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}