{"id":26436,"date":"2024-08-26T23:35:39","date_gmt":"2024-08-26T21:35:39","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=26436"},"modified":"2024-08-28T13:25:43","modified_gmt":"2024-08-28T11:25:43","slug":"welcome-to-the-future-of-government-tech-meet-guppyai","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2024\/08\/26\/welcome-to-the-future-of-government-tech-meet-guppyai\/","title":{"rendered":"Welcome to the Future of Government Tech – Meet GuppyAI!"},"content":{"rendered":"\n

If you\u2019ve ever attempted to accomplish anything in a German government office, you might feel like you’ve stepped into a time capsule. Fax machines? They’re still very much in use. Digital workflows? Not so much. But what if we told you there’s a new game-changer ready to revolutionize even the most tech-resistant corners of bureaucracy? Meet GuppyAI – your friendly, SMS-powered AI chatbot designed to help government workers upgrade their skills without the hassle.<\/p>\n\n\n\n

Let’s be honest – technology and government don\u2019t always mesh well. That’s where GuppyAI steps in to rewrite the story by putting cutting-edge AI within easy reach, using something as simple and familiar as SMS. Yes, you read that right – there\u2019s no need for fancy apps or confusing interfaces. If you can send a text, you can use GuppyAI.<\/p>\n\n\n\n

Behind the scenes, GuppyAI runs on state-of-the-art technology. We’ve integrated an on-premise SMS-Gateway with the Azure Cloud Platform via Service Bus, interacting with the ChatGPT 3.5 model through a sleek Node.js function app. Your conversations? They\u2019re securely stored in CosmosDB\u2019s NoSQL database for future reference. But don\u2019t worry, we\u2019ll delve into the technical details, the challenges we faced and all the other fun stuff later – just know that some serious innovation is at play.<\/p>\n\n\n\n

If you want to go on a deep dive into our code, feel free take a look at our GitHub organization<\/a>!<\/p>\n\n\n\n

Architecture<\/h2>\n\n\n\n
\"Chart<\/a><\/figure>\n\n\n\n

At the heart of GuppyAI lies a flexible and adaptable architectural style known as hexagonal architecture, also referred to as the ports and adapters pattern. This approach is designed to keep the core business logic of an application isolated from the external systems it interacts with, making it highly modular and easy to extend. The idea is simple: the core (or the “hexagon”) represents the central business logic, while the “ports” and “adapters” are the external systems or interfaces that interact with this core.<\/p>\n\n\n\n

In the context of GuppyAI, the Azure Function represents the core business logic. It handles the flow of data and decision-making, while all the connected systems – like the SMS gateway, Azure Service Bus, CosmosDB, and the AI API – act as adapters. These adapters can be swapped out or extended without significantly altering the core logic. This flexibility is a significant advantage for a project like GuppyAI, where the ability to easily add new message channels, change the persistence layer, or even replace the AI API is crucial for future development.<\/p>\n\n\n\n

The hexagonal architecture naturally supports these needs, making it easier to evolve the system as requirements change or new technologies emerge. Whether it’s integrating an additional messaging platform like instant messengers or email, or transitioning to a different database or AI service, the architecture allows for seamless adaptation.<\/p>\n\n\n\n

Information Flow<\/h3>\n\n\n\n

Guppy-AI’s information flow is made to be both simple and reliable. This is how it works:<\/p>\n\n\n\n

    \n
  1. SMS Reception:<\/strong> An SMS message is sent by the user and received by the on-premise SMS gateway<\/li>\n\n\n\n
  2. Message Queuing:<\/strong> The gateway forwards the message to the Azure Service Bus, where it is queued for processing.<\/li>\n\n\n\n
  3. Function Trigger:<\/strong> The incoming message in the Service Bus triggers the Azure Function App. The function begins by persisting the user\u2019s message in CosmosDB and retrieving the entire conversation history associated with that user.<\/li>\n\n\n\n
  4. AI Processing:<\/strong> The complete conversation is then sent to the OpenAI API via the official Node.js library. The AI processes the input and generates a response.<\/li>\n\n\n\n
  5. Response Handling:<\/strong> The AI’s response is stored in CosmosDB alongside the conversation history. The function then sends this response back through the Azure Service Bus to the SMS gateway.<\/li>\n\n\n\n
  6. SMS Delivery:<\/strong> The SMS gateway receives the response from the Service Bus and sends it back to the user via SMS.<\/li>\n<\/ol>\n\n\n\n

    Scalability and Performance<\/h3>\n\n\n\n

    GuppyAI’s architecture leverages the scalability of cloud-native services to handle a variety of workloads while remaining cost effective. Services such as the Azure Function App, CosmosDB, Azure Cognitive Services, and Azure Service Bus are designed to scale automatically based on demand. This ensures that the system can handle more traffic without requiring manual inputs.<\/p>\n\n\n\n

    The SMS-Gateway, which is constrained by the throughput of its GSM modem, is the bottleneck in this configuration. Despite this limitation, using zero-scaling services reduces operating expenses, making it a perfect solution for a proof-of-concept project.<\/p>\n\n\n\n

    One notable performance consideration is the cold start latency of the Azure Function App. When the function is inactive for a period, it can take 20-30 seconds to start up, introducing a delay in processing the first message. Once the function is active, the main performance constraint is the throughput of the GSM modem. Latency between cloud components is minimal, with most being located in the same geographical region except for the ChatGPT API, which is located in France.<\/p>\n\n\n\n

    Future Expansion and Upcoming Challenges<\/h3>\n\n\n\n

    Looking ahead, GuppyAI’s architecture is well-positioned for expansion. The use of a pub-sub pattern in the Azure Service Bus makes it easy to add new communication channels, such as instant messengers or email, with minimal changes to the existing codebase. The architecture\u2019s flexibility also allows for other potential improvements, such as the introduction of a logging system and the creation of a development environment to facilitate testing and debugging.<\/p>\n\n\n\n

    In terms of security, while it wasn’t a primary focus during the proof-of-concept phase, future iterations could benefit from placing components that don’t need internet access in a virtual private cloud (VPC) for enhanced security.<\/p>\n\n\n\n

    Overall, the GuppyAI architecture is robust, flexible, and ready to evolve as the project grows. While there are some areas for improvement, the current setup provides a solid foundation for bringing AI-powered chat services to even the most tech-resistant corners of the German government.<\/p>\n\n\n\n

    SMS-Gateway<\/h2>\n\n\n\n

    The SMS-Gateway is the critical entry and exit point for all communications within GuppyAI, acting as the bridge between the traditional SMS protocol and our cloud-based AI system. Developed to run on a Raspberry Pi 3B with a GSM modem, this custom gateway manages SMS transmission and reception while connecting with the Azure cloud. We’ll get into the technical specifics of the SMS-Gateway’s development, the difficulties we encountered while implementing it, and its function within the larger GuppyAI architecture in this chapter.<\/p>\n\n\n\n

    The Elephant in the Room: Why not use Cloud Services for SMS?<\/h3>\n\n\n\n

    When looking at our architecture, one might rightfully ask why we have used cloud services for any task in our use case but the reception and sending of SMS messages. This is were we will explain our partly dissatisfactory explanation on why we had to decide to not base this part of our application on cloud services.<\/p>\n\n\n\n

    Upon our initial research on how to send and receive SMS, we stumbled over numerous cloud services offering exactly the functionality we needed. Among them were services from the three big cloud providers AWS, Google or Microsoft Azure and many different specialized SaaS (Software-as-a-Service) products. However, with all of them there were a few caveats we as a group of three students could not overcome. Some only operated in the US, some required us to use a so called short code, which are the short phone numbers you often see when you’re getting 2FA messages from PayPal and the like – way too expensive for us! Others would require a waiting time of up to six months just to get the phone number assigned.<\/p>\n\n\n\n

    It was not too long until we figured, that what we wanted to achieve is not possible in the cloud with the resources we have at hands. So, there was only one possibility remaining: Running the service on premise and using a modem to send the SMS messages.<\/p>\n\n\n\n

    We knew this would not be easy as there were many unanswered questions:<\/p>\n\n\n\n

    What are modems we can consider for this? How do we communicate with the modem? Where do we get a mobile plan that offers what we need?<\/p>\n\n\n\n

    After some research, we found out about the AT standard. The AT standard is a command set we could use to communicate with the modem. As almost any generic USB Stick used for mobile internet reception contains a modem that supports this standard, we just bought the cheapest LTE USB Stick we could find, the Brovi\/Huawei E3372-325.<\/p>\n\n\n\n

    While we waited for our LTE USB Stick to arrive we researched about mobile plans for IoT appliances. In this regard we had the same devastating outcome as with our research about SMS cloud services. IoT mobile plans are expensive, sometimes reserved for businesses and most of the time they only offer data connectivity.<\/p>\n\n\n\n

    We soon realized our best option would be to opt for a standard consumer mobile plan. The issue with this approach is that many mobile providers prohibit Machine-to-machine communication and similar use cases. Therefore, we chose a plan that would not prohibit our exact use case.<\/p>\n\n\n\n

    However, we cannot recommend anyone to implement our approach for a non-research work without first consulting the respective mobile phone company or a lawyer.<\/strong> It is only our personal, inexpert assumption that our use case is not covered by the actions prohibited by the terms of service of the mobile provider we chose.<\/p>\n\n\n\n

    What’s this Funny Dialect<\/h3>\n\n\n\n

    Legal warnings aside, the next step on our journey was to learn about the AT commands we would need to use to send and receive SMS messages.<\/p>\n\n\n\n

    Some very useful resources for this topic were the following websites:<\/p>\n\n\n\n