Category: Ultra Large Scale Systems
How to Scale Jitsi Meet
In today’s world, video conferencing is getting more and more important – be it for learning, business events or social interaction in general. Most people use one of the big players like Zoom or Microsoft Teams, which both have their share of privacy issues. However, there is an alternative approach: self-hosting open-source software like Jitsi…
How to Scale: Real-time Tweet Delivery Architecture at Twitter
There is a lot to say about Twitters infrastructure, storage and design decisions. Starting as a Ruby-on-Rails website Twitter has grown significantly over the years. With 145 million monetizable daily users (Q3 2019), 500 million tweets (2014) and almost 40 billion US dollar market capitalization (Q4 2020) Twitter is clearly high scale. The microblogging platform,…
Energieversorgung als Ultra Large Scale System
Das Versorgernetz innerhalb von Deutschland ist mit dem europaweiten Stromnetz gekoppelt. Das europaweite Verbundnetz gilt als eine der größten Maschinen Europas und versorgt insgesamt rund 530 Millionen Menschen mit einer Stromfrequenz von 50 Hz. Die Energie, die dabei eingespeist wird, ist ein Technologiemix aus erneuerbaren Energiequellen und fossilen Brennstoffen. Derzeit besteht der Energiemix im Stromnetz zu…
Open Source Batch and Stream Processing: Realtime Analysis of Big Data
Abstract Since the beginning of Big Data, batch processing was the most popular choice for processing large amounts of generated data. These existing processing technologies are not suitable to process the large amount of data we face today. Research works developed a variety of technologies that focus on stream processing. Stream processing technologies bring significant…
Distributed stream processing frameworks – what they are and how they perform
An overview on stream processing, common frameworks as well as some insights on performance based on benchmarking data
Isolation and Consistency in Databases
by Samuel Hack and Sebastian Wachter. Most people assume that the data coming from a database is correct. For most applications this is true, but when the databases are used in systems where the database is at its limit, this is no longer always the case. What happens if during a query of a value…
IPFS: the InterPlanetary File System demystified
In this article we will explore IPFS, the InterPlanetary File System. IPFS is a system for storing and accessing files, websites and other kinds of data — just as the Web we enjoy using every day — but unlike the Web, IPFS is peer-to-peer based and automatically distributes its content across the network.
How to increase robustness of a large scale system by testing
When a distributed software system grows bigger and bigger, one will end up with a big amount of various components which all need to scale independently. In order to achieve these components working smooth together, it is necessary to figure out at which time a component needs to be scaled, to avoid having one component…
Convenient internet voting using blockchain technology
Within this century the use of digital technology has probably never been as high and as convenient as of today. People use the internet to access encyclopedias, look up food recipes and share pictures of their pets. It doesn’t matter whether you are at home, standing in an aisle at the grocery store or even…
How internet giants deliver their data to the world
In the course of attending the lecture “Ultra Large Scale Systems” I was intrigued by the subject of traffic load balancing in ultra-large-scale systems. Out of this large topic I decided to look at traffic distribution at the frontend in detail and held a presentation about it as part of this lecture. As this subject…