{"id":27,"date":"2015-12-17T12:34:28","date_gmt":"2015-12-17T11:34:28","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=27"},"modified":"2023-06-07T11:28:44","modified_gmt":"2023-06-07T09:28:44","slug":"27","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2015\/12\/17\/27\/","title":{"rendered":"Docker- dive into its foundations"},"content":{"rendered":"<p><a href=\"https:\/\/www.docker.com\/\" target=\"_blank\" rel=\"noopener\">Docker<\/a> has gained a lot of attention over the past several years.&nbsp;But not only because of its cool logo or it being&nbsp;the top buzzword of managers, but also because of its useful features.&nbsp;We talked about Docker quite a bit without really&nbsp;understanding why it&#8217;s so&nbsp;great to use. So we decided to take a closer look on how Docker actually works.<\/p>\n<p>In this article, we want to shed some light on a few technologies used by Docker enabling it to be so lightweight and fast in startup compared to &#8220;traditional&#8221; <a href=\"https:\/\/en.wikipedia.org\/wiki\/Virtual_machine\" target=\"_blank\" rel=\"noopener\">virtual machines<\/a> (VMs). Docker itself serves us as an example, you could replace it with any&nbsp;other container technology, for example <a href=\"https:\/\/linuxcontainers.org\/\" target=\"_blank\" rel=\"noopener\">LXC<\/a>.<\/p>\n<p>Reading this article requires some profound knowledge of virtualization. Terms like &#8220;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Guest_operating_system\" target=\"_blank\" rel=\"noopener\">guest system<\/a>&#8221; or &#8220;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Hypervisor\" target=\"_blank\" rel=\"noopener\">hypervisor<\/a>&#8221; should ring a bell. Also you should have heard of an operating system called Linux (it is&nbsp;probably running on your smartphone and you are waiting for an update).<\/p>\n<p>So let&#8217;s go!<\/p>\n<p><!--more--><\/p>\n<h2>Don&#8217;t own &#8211; share<\/h2>\n<p>When you look at the fundamental&nbsp;differences between containers and VMs, you will notice two outstanding characteristics:<\/p>\n<ul>\n<li>VMs rarely share resources with other VMs running on the same host. This applies to almost every resource thinkable on a system. This is the reason why they are isolated so well against each other.&nbsp;But if you want to run several copies of one VM on one host, each VM will take up resources for itself. Take the kernel of the guest system for example: each guest has to&nbsp;burn a lot of resources by shipping and running its own kernel.<\/li>\n<li>Containers almost always share resources with other containers running on the same host. Therefore, the isolation against other containers is worse compared to that of VMs. But running several copies of the same container will cost you just a little more than running just one container because they share almost all their resources. Looking at the kernel again, you will notice, that each container is using the kernel of the host system, reducing its&nbsp;overhead compared to a traditional VM.<\/li>\n<\/ul>\n<p>And with that said, you already understood the key principle of container technology: sharing resources.&nbsp;The only question remaining is: How can this be achieved in a fast, secure and lightweight way?<\/p>\n<h2>Creating a safe space<\/h2>\n<p>Traditional virtual machines run a complete guest operation system on the host&nbsp;operating system. The host system&nbsp;can be either a bare metal hypervisor&nbsp;abstracting the hardware, or it can be something like <a href=\"https:\/\/www.virtualbox.org\/\" target=\"_blank\" rel=\"noopener\">VirtualBox<\/a>, where you run the guest system inside a fully functional&nbsp;host operating system. The execution of the guest system can be accelerated to a certain point using virtualization extensions on modern processors, but basically you have to run the entire guest operation system all the time.<\/p>\n<p>With Docker, you use the host system as basis for your&nbsp;containers. The container itself is nothing more than a process running the specified application. The container processes running all your applications&nbsp;are separated against each other using Linux kernel features like&nbsp;<a href=\"http:\/\/man7.org\/linux\/man-pages\/man7\/namespaces.7.html\" target=\"_blank\" rel=\"noopener\">namespaces<\/a> and <a href=\"https:\/\/wiki.archlinux.org\/index.php\/Cgroups\" target=\"_blank\" rel=\"noopener\">cgroups<\/a>:<\/p>\n<ul>\n<li><a href=\"http:\/\/man7.org\/linux\/man-pages\/man7\/namespaces.7.html\" target=\"_blank\" rel=\"noopener\">Namespaces<\/a> behave&nbsp;identically to variable scopes in programming languages.&nbsp;Everything created inside a namespace is visible to the namespace itself, but you can&#8217;t access resources created outside of your namespace. So, if you put your container process inside a namespace, it&nbsp;can use resources &nbsp;inside its own space, but it can&#8217;t access other containers. These resources can be network devices, mounted file systems&nbsp;or even users.<\/li>\n<li><a href=\"https:\/\/wiki.archlinux.org\/index.php\/Cgroups\" target=\"_blank\" rel=\"noopener\">Cgroups<\/a> allow Docker to control the&nbsp;access to certain resources that need to&nbsp;be used by all containers: the CPU, memory and disk-IO. They ensure&nbsp;equal access to these&nbsp;resources for all containers.<\/li>\n<\/ul>\n<p>This explains, how the different containers can be isolated against each other.&nbsp;However, the containers running&nbsp;on the same host are separated only by the host system kernel. So if your contained application gets attacked, it will be easier for the attacker to also get control over the host system,&nbsp;than having to hack the VM first and then&nbsp;the host.<\/p>\n<p>Nevertheless it also explains why you can start&nbsp;a Docker container&nbsp;so much faster than a VM. If you start a VM, you have to boot an operation system, which takes some time (especially when you are in a hurry). If you start a container, you only need to start a process. This is as fast as starting any process which is almost instantly.<\/p>\n<p>This explains why you can start Containers so fast. Next we will&nbsp;explain why containers need so little of your disk space.<\/p>\n<h2>File System Layering<\/h2>\n<p>If you currently installed a Linux system, for example <a href=\"https:\/\/www.debian.org\/\" target=\"_blank\" rel=\"noopener\">Debian<\/a>, it took probably 1 GB of your hard disk just for the minimal installation. But if you look at the size of a minimal <a href=\"https:\/\/hub.docker.com\/_\/debian\/\" target=\"_blank\" rel=\"noopener\">Debian Docker container<\/a>, it is&nbsp;only&nbsp;150&nbsp;MB.<\/p>\n<p>But why is the Docker container so much smaller? The answer is, of course, because the container shares its files&nbsp;with your host.<\/p>\n<p>This is achieved by using a layered file system like <a href=\"http:\/\/unionfs.filesystems.org\/\" target=\"_blank\" rel=\"noopener\">Unionfs<\/a>.&nbsp;Layered file systems use a tree data structure to allow copy-on-write-operations on files. Whenever you change a file, a new branch of the file system is created, that contains the modified&nbsp;file. The old file is kept in the existing&nbsp;branch.<\/p>\n<p>When you run&nbsp;a new Docker container, you create a new layer on the file system. All the files differing from those&nbsp;of the host system are copied and stored in this layer. So you only need storage capacity&nbsp;for the files you changed.&nbsp;All the unchanged default vanilla files, are read from your host. If you stop your container, all changes are potentially lost&nbsp;because you also shut down your branch of the file system. If you want to preserve this branch, you need to explicitly do so.<\/p>\n<p>So if you are running a Linux based Docker container on a Linux host, you could benefit&nbsp;from the huge overlap of the two systems resulting in very small images.&nbsp;Most of the base images published on <a href=\"https:\/\/hub.docker.com\/\" target=\"_blank\" rel=\"noopener\">Docker Hub<\/a> however overwrite a lot of your host system files, resulting in larger&nbsp;images again. Here you have to choose between having this&nbsp;convenient way to get your images and investing time to&nbsp;build&nbsp;your own, very small images.<\/p>\n<h2>A word on Windows<\/h2>\n<p>We have now talked at length about the technological background of Docker. But you never heard the word &#8220;Windows&#8221;. This is because Windows and Docker don&#8217;t go well together. All the presented techniques are based on the Linux kernel and its API. So far, there are no Windows-based Containers in the <a href=\"https:\/\/hub.docker.com\/\" target=\"_blank\" rel=\"noopener\">Docker Hub<\/a>.&nbsp;They may also exist in the Windows kernel, but we can&#8217;t know for sure. <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/containers-docker-windows-and-trends\/\" target=\"_blank\" rel=\"noopener\">Microsoft has announced they will also support Docker containers<\/a>, but even so, it won&#8217;t be possible to run Linux containers on Windows or the other way around.<\/p>\n<p>This brings us to our conclusion on the techniques used by Docker.<\/p>\n<h2>Final thoughts<\/h2>\n<p>Hopefully, after reading this article you&nbsp;know why Docker containers are so fast and lightweight. Mainly because they share a lot. This is very good from a resource management perspective, but can also be a hazard from a security point of view. You will have to decide for yourself, which of the two is more important.&nbsp;In an other article, we will talk about the security implications of Docker, and how you can protect yourself from bad containers.<\/p>\n<p>If this article didn&#8217;t contain enough information for you or you are just very interested in this topic, here are some links for further reading:<\/p>\n<ul>\n<li><a href=\"https:\/\/docs.docker.com\/engine\/introduction\/understanding-docker\/\" target=\"_blank\" rel=\"noopener\">The official Docker documentation<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Docker has gained a lot of attention over the past several years.&nbsp;But not only because of its cool logo or it being&nbsp;the top buzzword of managers, but also because of its useful features.&nbsp;We talked about Docker quite a bit without really&nbsp;understanding why it&#8217;s so&nbsp;great to use. So we decided to take a closer look on [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[656,650,2,223],"tags":[3,4],"ppma_author":[681],"class_list":["post-27","post","type-post","status-publish","format-standard","hentry","category-databases","category-scalable-systems","category-system-engineering","category-ultra-large-scale-systems","tag-docker","tag-linux"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":1924,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-4\/","url_meta":{"origin":27,"position":0},"title":"Microservices \u2013 Legolizing Software Development IV","author":"Calieston Varatharajah, Christof Kost, Korbinian Kuhn, Marc Schelling, Steffen Mauser","date":"28. February 2017","format":false,"excerpt":"An automated development environment will save you. We explain how we set up Jenkins, Docker and Git to work seamlessly together.","rel":"","context":"In &quot;System Designs&quot;","block_context":{"text":"System Designs","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":7154,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2019\/08\/31\/setting-up-a-ci-cd-pipeline-in-gitlab\/","url_meta":{"origin":27,"position":1},"title":"Setting up a CI\/CD pipeline in Gitlab","author":"nr037","date":"31. August 2019","format":false,"excerpt":"Introduction For all my university software projects, I use the HdM Gitlab instance for version control. But Gitlab offers much more such as easy and good ways to operate a pipeline. In this article, I will show how we can use the CI\/CD functionality in a university project to perform\u2026","rel":"","context":"In &quot;Cloud Technologies&quot;","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2019\/08\/Screenshot-2019-08-26-at-09.53.13.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":282,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2016\/03\/10\/docker-running-on-a-raspberry-pi-hypriot\/","url_meta":{"origin":27,"position":2},"title":"Docker on a Raspberry Pi: Hypriot","author":"Jonathan Peter","date":"10. March 2016","format":false,"excerpt":"Raspberry Pis are small, cheap\u00a0and easy to come by. But what if you want to use Docker on them? Our goal was to run Docker on several Raspberry Pis and combine them to a cluster with Docker Swarm. To achieve this, we first\u00a0needed to get Docker running on the Pi.\u2026","rel":"","context":"In &quot;System Designs&quot;","block_context":{"text":"System Designs","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":21064,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2021\/09\/11\/how-do-you-get-a-web-application-into-the-cloud\/","url_meta":{"origin":27,"position":3},"title":"How do you get a web application into the cloud?","author":"af094","date":"11. September 2021","format":false,"excerpt":"by Dominik Ratzel (dr079) and Alischa Fritzsche (af094) For the lecture \"Software Development for Cloud Computing\", we set ourselves the goal of exploring new things and gaining experience. We focused on one topic: \"How do you get a web application into the cloud?\". In doing so, we took a closer\u2026","rel":"","context":"In &quot;Cloud Technologies&quot;","block_context":{"text":"Cloud Technologies","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/cloud-technologies\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2021\/09\/availableRunners-150x118.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":308,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2016\/03\/10\/more-is-always-better-building-a-cluster-with-pies\/","url_meta":{"origin":27,"position":4},"title":"More is always better: building a cluster with Pies","author":"Benjamin Binder","date":"10. March 2016","format":false,"excerpt":"So you have written the uber-pro-web-application with a bazillion of active users. But your requests start to get out of hand and the Raspberry Pi under your desk can't handle all the pressure on its own. Finally,\u00a0the time for rapid expansion has come! If you have already containerized your application,\u2026","rel":"","context":"In &quot;Scalable Systems&quot;","block_context":{"text":"Scalable Systems","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/"},"img":{"alt_text":"Raspberry Pi 2","src":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/3\/3d\/Raspberry_PI.jpeg","width":350,"height":200,"srcset":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/3\/3d\/Raspberry_PI.jpeg 1x, https:\/\/upload.wikimedia.org\/wikipedia\/commons\/3\/3d\/Raspberry_PI.jpeg 1.5x, https:\/\/upload.wikimedia.org\/wikipedia\/commons\/3\/3d\/Raspberry_PI.jpeg 2x, https:\/\/upload.wikimedia.org\/wikipedia\/commons\/3\/3d\/Raspberry_PI.jpeg 3x, https:\/\/upload.wikimedia.org\/wikipedia\/commons\/3\/3d\/Raspberry_PI.jpeg 4x"},"classes":[]},{"id":1299,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2016\/08\/16\/exploring-docker-security-part-2-container-flaws\/","url_meta":{"origin":27,"position":5},"title":"Exploring Docker Security &#8211; Part 2: Container flaws","author":"Patrick Kleindienst","date":"16. August 2016","format":false,"excerpt":"Now that we've understood the basics, this\u00a0second part will\u00a0cover the most relevant container threats, their possible impact as well as\u00a0existent countermeasures. Beyond that, a short overview\u00a0of the most important sources for container threats will be provided. I'm pretty sure you're not counting on most\u00a0of them. Want to know more? Container\u2026","rel":"","context":"In &quot;Secure Systems&quot;","block_context":{"text":"Secure Systems","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/secure-systems\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/article-1301858-0ABD7881000005DC-365_964x543.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/article-1301858-0ABD7881000005DC-365_964x543.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/article-1301858-0ABD7881000005DC-365_964x543.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2023\/08\/article-1301858-0ABD7881000005DC-365_964x543.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":681,"user_id":5,"is_guest":0,"slug":"bb074","display_name":"Benjamin Binder","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/b39750be005f19ce71d3af93115f9d5f02d18769c36bfa750ca4de423b20d981?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/27","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=27"}],"version-history":[{"count":30,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/27\/revisions"}],"predecessor-version":[{"id":24650,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/27\/revisions\/24650"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=27"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=27"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=27"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=27"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}