{"id":1907,"date":"2017-02-28T20:06:16","date_gmt":"2017-02-28T19:06:16","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=1907"},"modified":"2023-06-07T15:25:14","modified_gmt":"2023-06-07T13:25:14","slug":"microservices-legolizing-software-development-1","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-1\/","title":{"rendered":"Microservices – Legolizing Software Development I"},"content":{"rendered":"

Welcome to our five-part series about microservices and a legolized software development. We\u2019d like to share our lessons learned about architecture, development environment and security considerations with you. We will also explain some issues we stumbled over and what solutions we chose to solve them.<\/p>\n

I) In the first part, we present an example microservice structure, with multiple services, a foreign API interface and a reverse proxy that also allows load balancing.<\/p>\n

II) Part two will take a closer look on how caching improves the heavy and frequent communication within our setup. [read]<\/a><\/p>\n

III) Security is a topic that always occurs with microservices. We\u2019ll present our solution for managing both, authentication and authorization at one single point. [read]<\/a><\/p>\n

IV) An automated development environment will save you. We explain how we set up Jenkins, Docker and Git to work seamlessly together. [read]<\/a><\/p>\n

V) We finish with a concluding review about the use of microservices in small projects and give an overview about our top stumbling blocks.\u00a0<\/span>[read]<\/a><\/p>\n

<\/p>\n

Example Architecture<\/h1>\n

Let\u2019s start with an example generic microservices architecture, containing a user website and a monitoring page for developers. The authentication and authorization is controlled by the Auth service. Furthermore, we have the four services A-D and one for monitoring. The red External Service could be a foreign API, which is connected by the External Service Mapper. The connection lines are purple, because all requests will be send over the Reverse Proxy, e.g. a Nginx server, which maps IPs and ports to static URLs.<\/p>\n

The questions we want to answer in this section:<\/p>\n

    \n
  1. How do the services communicate with each other?<\/li>\n
  2. How do we integrate external services into our system?<\/li>\n
  3. Can we protect our services from unauthorized requests?<\/li>\n
  4. How to handle all the monitoring?<\/li>\n<\/ol>\n
    \"\"<\/a>
    Generic Microservice Architecture<\/figcaption><\/figure>\n

    Reverse Proxy and Load Balancer<\/h1>\n

    By letting the services communicate via the reverse proxy we gain helpful advantages.<\/p>\n

      \n
    1. All information about servers, IP addresses and ports are only held by the reverse proxy. We only need to\u00a0store static URLs in each service to their dependents.<\/li>\n
    2. The services can easily be spread over various servers, because they aren’t referring to the localhost or a fixed IP address.<\/li>\n
    3. The reverse proxy can also load balance the traffic, if multiple instances are running (e.g. service A v1).<\/li>\n
    4. It\u2019s easy to run different versions of a service in parallel (e.g. service A v1 and v2). In some scenarios, it might be useful to initially route only a part of the traffic to the newer version, before migrating to it completely.<\/li>\n<\/ol>\n
      \"\"<\/a>
      Reverse Proxy and Load Balancer<\/figcaption><\/figure>\n

      Now we have a scalable and easy controllable setup for the routing. Managing all IP addresses and ports within each service would rapidly create a mess of configuration and handling flexible or automated changes would seem to be nearly impossible. However, the downside of this configuration is that the reverse proxy is the single point of failure in the system. We think this is a fair trade off, because the Nginx server doesn’t contain a lot of application logic and at some point, you need an entry point for the URL of your services. It would take a lot more time to update reconfiguration in the DNS\u00a0service.<\/p>\n

      Down the road, we plan\u00a0to evaluate if we gain performance by adding the URLs to the hosts file at least while\u00a0the reverse proxy is running on the same server as some services.<\/p>\n

      External Services<\/h2>\n

      It\u2019s always bad to depend on sources you can\u2019t control, but in many cases, you won\u2019t have a choice. The key goal is to build a reliable service even if foreign systems fail. We tried to build a setup that is as flexible and reliable as possible. If a service needs such data it connects an own service (service D), which is connected to the external interface, with another service (External Service Mapper).<\/p>\n

      There are two options to provide data, depending on the type:<\/p>\n

        \n
      1. If you can store the data in your own database, do it. Collect the data from the foreign API (push or pull) within the best interval (e.g. store weather data every hour). This guarantees availability, even if the external service is offline.<\/li>\n
      2. If you can\u2019t store the data due to the size or complexity (e.g. Google Maps), you\u2019ll have to live with it. Build the best possible fallback.<\/li>\n<\/ol>\n

        Why do we need an additional service (External Service Mapper) and don’t implement its logic into our service D?<\/p>\n

        This modularization helps us if we add a second foreign interface (e.g. OpenStreetMap) or the external API provides an updated version. So we will not have to change anything in our internal connected service (Service D) if anything outside of our system changes.<\/p>\n

        Auth Service<\/h2>\n

        Within the microservice setup, connections between all\u00a0services are possible. Due to security in our example setup, every service will always be connected to the auth service. Every request, no matter if it was send by a user outside of the system or an interservice connection, will be checked by the requested service asking the Auth service. We will take a deeper look in the authentication and authorization process in our third part<\/a> of the blog series. Just keep in mind that these are necessary connections we implement in every service. Even if we hide some services from the public, before processing a request we make sure that it\u2019s valid. We don’t build a wall and once someone passed it, he would get access to the whole system. As a side effect, we will be able to\u00a0make a service public anytime in the future, without having to implement any additional security functions.<\/p>\n

        \"\"<\/a>
        Auth Service<\/figcaption><\/figure>\n

        Monitoring<\/h2>\n

        If you have lots of\u00a0services, monitoring their activities is very important. Similar to the auth service, every service has implemented a connection to the monitoring service, to push or get data pulled. There are two different kinds of data to monitor:<\/p>\n

          \n
        1. Heartbeat: To monitor the running services, their system resources and availability, we need a heartbeat that is send or requested in a specified interval. The monitoring service can then provide an overview for the operator\/developer, about the whole system.<\/li>\n
        2. Logs: We need to collect the logs of all services, to process them and gather additional information. The services should at least log all incoming requests and outgoing responses with a timestamp. More detailed logging, e.g. inside the application logic, could be stored in a service specific log file.<\/li>\n<\/ol>\n
          \"\"<\/a>
          Monitoring<\/figcaption><\/figure>\n

          In the next blog post we will take a closer look on how caching improves the heavy and frequent communication within our setup.
          \n          Continue with Part II – Caching<\/a><\/p>\n

          \n

          Kost, Christof [ck154@hdm-stuttgart.de<\/a>]
          \nKuhn, Korbinian [          kk129@hdm-stuttgart.de<\/a>]
          \nSchelling, Marc [          ms467@hdm-stuttgart.de<\/a>]
          \nMauser, Steffen [          sm182@hdm-stuttgart.de<\/a>]
          \nVaratharajah, Calieston\u00a0[          cv015@hdm-stuttgart.de<\/a>]<\/p>\n","protected":false},"excerpt":{"rendered":"

          In the first part, we present an example microservice structure, with multiple services, a foreign API interface and a reverse proxy that also allows load balancing.<\/p>\n","protected":false},"author":193,"featured_media":1927,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[651,2],"tags":[49,11,91,12,93],"ppma_author":[718],"class_list":["post-1907","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-system-designs","category-system-engineering","tag-architecture","tag-loadbalancing","tag-microservices","tag-nginx","tag-reverse-proxy"],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/Architecture-01.png","jetpack-related-posts":[{"id":1915,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-5\/","url_meta":{"origin":1907,"position":0},"title":"Microservices – Legolizing Software Development V","author":"Korbinian Kuhn, Steffen Mauser","date":"28. February 2017","format":false,"excerpt":"We finish with a concluding review about the use of microservices in small projects and give an overview about our top stumbling blocks.","rel":"","context":"In "System Designs"","block_context":{"text":"System Designs","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":23067,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2022\/03\/15\/security-strategies-and-best-practices-for-microservices-architecture\/","url_meta":{"origin":1907,"position":1},"title":"Security Strategies and Best Practices for Microservices Architecture","author":"Larissa Schmauss","date":"15. March 2022","format":false,"excerpt":"Microservices architectures seem to be the new trend in the approach to application development. However, one should always keep in mind that microservices architectures are always closely associated with a specific environment:\u00a0Companies want to develop faster and faster, but resources are also becoming more limited, so they now want to\u2026","rel":"","context":"In "Scalable Systems"","block_context":{"text":"Scalable Systems","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/"},"img":{"alt_text":"","src":"https:\/\/lh6.googleusercontent.com\/LbFspPRY1BxRBdAVjQwWXeJ6UOoxl6JWsRYrxboF5ObXlNNgy3uZikcGkc3cgzI0mr_ZlbWPxvdp0FoJC1k-odh7mRc2lCPXaMSq8TudjfoZ7e5HKstaMHmLpH319jCym6vQRo1a","width":350,"height":200,"srcset":"https:\/\/lh6.googleusercontent.com\/LbFspPRY1BxRBdAVjQwWXeJ6UOoxl6JWsRYrxboF5ObXlNNgy3uZikcGkc3cgzI0mr_ZlbWPxvdp0FoJC1k-odh7mRc2lCPXaMSq8TudjfoZ7e5HKstaMHmLpH319jCym6vQRo1a 1x, https:\/\/lh6.googleusercontent.com\/LbFspPRY1BxRBdAVjQwWXeJ6UOoxl6JWsRYrxboF5ObXlNNgy3uZikcGkc3cgzI0mr_ZlbWPxvdp0FoJC1k-odh7mRc2lCPXaMSq8TudjfoZ7e5HKstaMHmLpH319jCym6vQRo1a 1.5x"},"classes":[]},{"id":23961,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2023\/02\/10\/microservices-any-good\/","url_meta":{"origin":1907,"position":2},"title":"Microservices – any good?","author":"Kim Bastiaanse","date":"10. February 2023","format":false,"excerpt":"As software solutions continue to evolve and grow in size and complexity, the effort required to manage, maintain and update them increases. To address this issue, a modular and manageable approach to software development is required.\u00a0Microservices architecture provides a solution by breaking down applications into smaller, independent services that can\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/02\/Microservice.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/02\/Microservice.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/02\/Microservice.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/02\/Microservice.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/02\/Microservice.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/02\/Microservice.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":1912,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-2\/","url_meta":{"origin":1907,"position":3},"title":"Microservices – Legolizing Software Development II","author":"Korbinian Kuhn, Steffen Mauser","date":"28. February 2017","format":false,"excerpt":"Part two will take a closer look on how caching improves the heavy and frequent communication within our setup.","rel":"","context":"In "System Designs"","block_context":{"text":"System Designs","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/Caching-01.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/Caching-01.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/Caching-01.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/Caching-01.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/Caching-01.png?resize=1050%2C600&ssl=1 3x"},"classes":[]},{"id":1967,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-3\/","url_meta":{"origin":1907,"position":4},"title":"Microservices \u2013 Legolizing Software Development III","author":"Calieston Varatharajah, Christof Kost, Korbinian Kuhn, Marc Schelling, Steffen Mauser","date":"28. February 2017","format":false,"excerpt":"Security is a topic that always occurs with microservices. We\u2019ll present our solution for managing both, authentication and authorization at one single point.","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/auth_login_gesamt03.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/auth_login_gesamt03.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/auth_login_gesamt03.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/auth_login_gesamt03.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/auth_login_gesamt03.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/auth_login_gesamt03.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":1924,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-4\/","url_meta":{"origin":1907,"position":5},"title":"Microservices \u2013 Legolizing Software Development IV","author":"Calieston Varatharajah, Christof Kost, Korbinian Kuhn, Marc Schelling, Steffen Mauser","date":"28. February 2017","format":false,"excerpt":"An automated development environment will save you. We explain how we set up Jenkins, Docker and Git to work seamlessly together.","rel":"","context":"In "System Designs"","block_context":{"text":"System Designs","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":718,"user_id":193,"is_guest":0,"slug":"kk129","display_name":"Korbinian Kuhn, Steffen Mauser","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/e55f12f0048e76d004529e6da31f8f5fe030917ab31166bd5df95af51bc7e234?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/1907","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/193"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=1907"}],"version-history":[{"count":35,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/1907\/revisions"}],"predecessor-version":[{"id":2254,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/1907\/revisions\/2254"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media\/1927"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=1907"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=1907"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=1907"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=1907"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}