{"id":366,"date":"2016-02-20T11:49:06","date_gmt":"2016-02-20T10:49:06","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=366"},"modified":"2023-06-09T14:14:51","modified_gmt":"2023-06-09T12:14:51","slug":"jenkbird-quality-gates-testing-part-4","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2016\/02\/20\/jenkbird-quality-gates-testing-part-4\/","title":{"rendered":"Jenkbird \u2013\u00a0Quality & Testing \u2013\u00a0Part 4"},"content":{"rendered":"
\n

Love and testing can be cruel.<\/p>\n

– Bert on the difficulties in life<\/em><\/p><\/blockquote>\n

\"bert\"<\/a><\/p>\n

Quality and Testing \u2013 one of the most discussed and valuable topics software engineering has to offer. This blog post will cover all the relevant stuff related to quality and testing in regard to Continuous Integration and Jenkins. We will show you in detail, how you can automate your testing with Jenkins to ensure best possible software quality.<\/p>\n

<\/p>\n

99% != 100%<\/h2>\n

We want to start this blog series with a thought-experiment given in the book \u201eContinuous Integration\u201c by Paul M. Duvall [1]:
\nImagine you have a software with three components<\/strong> and the reliability<\/strong> of these software components is 90%<\/strong> each (regardless on how this is measured). At the first glance you might say that the overall reliability of the software is 90% – but that\u2019s wrong. Actually the reliability is 0,99^3=0.729 and therefore only 72,9%<\/strong>.<\/p>\n

\"reliability_pic1\"<\/a><\/p>\n

Now imagine having 100 components<\/strong> with 99% reliability<\/strong>, that\u2019s 0,99^100=0,366 which equals a reliability <\/strong>of 36,6% <\/strong><\/strong>.<\/p>\n

\"reliability_pic2\"<\/a><\/p>\n

What would you do with thinks like a bridge with 72,9% reliability? You probably would not consider using it\u2026 except you are one of those Red Bull guys. And that\u2019s why we need to ensure quality in every little component, service or whatever we develop for our system. We want to dedicate this blog post to Quality Gates or Continuous Testing, because it\u2019s an important part of Continuous Integration and the Agile Methodology. Why would someone develop agile without using the faster feedback to fix the bugs? That\u2019s what it\u2019s all about, right? Let\u2019s test!<\/p>\n

Talking about Quality and Testing in CI\u2026<\/h2>\n

At first we want to mention the two most important points when talking about CI and Testing:<\/p>\n

    \n
  1. Integrate code often and let the build test itself as well as<\/li>\n
  2. do not test on the production environment, but on a clone (like TEST or DEV)<\/li>\n<\/ol>\n

    The first argument has the obvious advantage of automating quality assurance. The more often you deploy and test your code, the faster you can react to failures.  How often have you heard about \u201ebreaking down complex tasks into manageable junks\u201c? It\u2019s better to fix a few bugs immediately than trying to restructure a whole system some time later, isn\u2019t it? Jeff Sutherland, the \u201efather\u201c of Scrum and author of the Book \u201eScrum: The Art of Doing Twice the Work in Half the Time\u201c gives an example: statistics of a project by Palm a few year ago state, that fixing bugs on the same day is 24 times faster than fixing them three weeks later<\/strong> [2] (in our opinion one of the best books about SCRUM – very entertaining read about the history, benefits and other areas of use for SCRUM e.g. a pre-school). A bugfix which would take one hour on the same day will take three complete work days according to this statistic – and we think it\u2019s way cooler to write some new stuff in the additional 23 hours you have than fixing something just to get the system back to work.<\/p>\n

    \u201eTesting is for wimps, real men and women test in production\u2026 Oh, well\u2026\u201c<\/em> – it\u2019s pretty much O.K. if it\u2019s only your student project environment you just crashed, but a customer probably wouldn\u2019t be satisfied. We don\u2019t want to step any further in this topic, because we already talked about it in part 2 of our blog series, anyway: use different stages to improve quality! It is definitely a good habit  to ensure that you always have a stable version of your code.<\/p>\n

    Automating tests with Jenkins<\/h2>\n

    Now that we have covered some basic theories of testing, we will create our first job to test our application! As we are using a small Laravel project as an exemplary project for deployment, we are going to create a PHPUnit test and execute it with our beloved Jenkins.<\/p>\n

    First we need to get PHPUnit. You have two possibilities to do so. Either you install it globally on your instance or you install it locally within your Laravel application. We are going to show you the second option.  You need to edit your composer.json <\/strong>file which is located at the root folder of your application. Insert the following line:<\/p>\n

    \"require-dev\": {\n    ...\n    \"phpunit\/phpunit\": \"5.0.*\"\n},<\/pre>\n
    Now that you have listed PHPUnit in your dependency configuration you simply need to update composer and you’ve successfully installed PHPUnit. Now we get to the actual test. As you can see in the repository we created a basic test case:<\/p>\n
    class ExampleTest extends TestCase\n{\n    \/**\n     * A basic functional test example.\n     *\n     * @return void\n     *\/\n    public function testBasicExample()\n    {\n        $this->visit('\/')\n             ->see('Sesame Deployment Street');\n    \n    }  \n}<\/pre>\n
    It basically is a functional test which visits our application site and checks if the pattern “Sesame Deployment Street” matches any string within the page. If this is the case the test passes, otherwise it will fail. We want this test to be executed every time we deploy a version of our application to the server. For this we need to configure a new Job in Jenkins. Add a new item to your Jenkins as you’ve already seen in the previous post. In this job we act on the assumption that the application code is already in place and we can test it.<\/p>\n
    Just add a shell script to the job that switches to the directory which contains our code and executes PHPUnit. This should be enough for our small example.<\/p>\n
    #!\/bin\/sh\ncd \/var\/www\/html\nvendor\/bin\/phpunit  --log-junit report.xml<\/pre>\n
    As we are really lazy and do not want to manually click on a button to test our application, we schedule the test job every time one of our deploy jobs has been run. In order to do this we configure the test as a post action for the other build jobs. To do so select deploy_demo_branch<\/em> and deploy_dev_branch
    \n<\/em>as Projects to watch<\/em> under Build Triggers<\/em>. Hit the save button and now every time one of the other builds is successfully completed Jenkins will run our test.<\/p>\n
    \"build_triggers\"<\/a><\/p>\n
    On our dev<\/em> branch we manipulated our page to fail the test whereas in our demo branch the test should pass. You can play around with the deploy jobs and watch how the test job is automatically triggered. Feel like a super professional system administrator yet?<\/p>\n
    What’s coming up in the next episode?<\/strong><\/p>\n
    Sadly our journey into the world of continuous integration and deployment comes to an end. The next post will be our last one, giving you a wrap up about everything we have discussed so far and presenting  some alternatives. See you soon!<\/p>\n
    Blogpost by Thomas Derleth (td036), J\u00f6rg Einfeldt (je051), Marc Stauffer (ms445)<\/em><\/p>\n
    [1] Continuous Integration: Improving Software Quality and Reducing Risk<\/a>, Paul M.Duvall, 2007.<\/em>
    \n [2]     Scrum: The Art of Doing Twice the Work in Half the Time<\/a>, Jeff Sutherland, 2014.<\/em><\/p>\n
     <\/p>\n
     <\/p>\n
    \n
    Read more about Testing in our Tutorial part 5<\/p>\n","protected":false},"excerpt":{"rendered":"
    Love and testing can be cruel. – Bert on the difficulties in life Quality and Testing \u2013 one of the most discussed and valuable topics software engineering has to offer. This blog post will cover all the relevant stuff related to quality and testing in regard to Continuous Integration and Jenkins. We will show you in detail, how you […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1,659,650,651,2],"tags":[],"ppma_author":[692],"class_list":["post-366","post","type-post","status-publish","format-standard","hentry","category-allgemein","category-devops","category-scalable-systems","category-system-designs","category-system-engineering"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":1758,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/01\/12\/snakes-exploring-pipelines-a-system-engineering-and-management-project-5\/","url_meta":{"origin":366,"position":0},"title":"Snakes exploring Pipelines – A \u201cSystem Engineering and Management\u201d Project","author":"Yann Loic Philippczyk","date":"12. January 2017","format":false,"excerpt":"Part 4: Jenkins and Wrap Up This series of blog entries describes a student project focused on developing an application by using methods like pair programming, test driven development and deployment pipelines. Our first blog entry for this year will at the same time be the final one for this\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/01\/jenkins_code-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/01\/jenkins_code-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/01\/jenkins_code-1.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3421,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/28\/take-me-home-project-overview\/","url_meta":{"origin":366,"position":1},"title":"Take Me Home – Project Overview","author":"cp054","date":"28. March 2018","format":false,"excerpt":"Related articles:\u00a0\u25baCI\/CD infrastructure: Choosing and setting up a server with Jenkins as Docker image\u00a0\u25baDockerizing Android SDK and Emulator for testing\u00a0 \u25baAutomated Unit- and GUI-Testing for Android in Jenkins\u00a0 \u25baTesting a MongoDB with NodeJS, Mocha and Mongoose During the winter term 2017\/2018, we created an app called Take Me Home. The\u2026","rel":"","context":"In "Mobile Apps"","block_context":{"text":"Mobile Apps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/interactive-media\/mobile-apps\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/tmh_admin_usermanagement_bearbeitet.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/tmh_admin_usermanagement_bearbeitet.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/tmh_admin_usermanagement_bearbeitet.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/tmh_admin_usermanagement_bearbeitet.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/tmh_admin_usermanagement_bearbeitet.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/tmh_admin_usermanagement_bearbeitet.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":3357,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/03\/28\/continuous-integration-move-fast-and-dont-break-things\/","url_meta":{"origin":366,"position":2},"title":"Continuous Integration \u2013 Move fast and don\u2019t break things","author":"Alexander Wallrabenstein","date":"28. March 2018","format":false,"excerpt":"Continuous Integration is an increasingly popular topic in modern software development. Across many industries the companies acknowledging the importance of IT and delivering value to their customers through great software prevail against their competitors. Many reports indicate that Continuous Integration is one of the major contributing factors to developing high\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CIPipeline.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CIPipeline.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2018\/03\/CIPipeline.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":1924,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2017\/02\/28\/microservices-legolizing-software-development-4\/","url_meta":{"origin":366,"position":3},"title":"Microservices \u2013 Legolizing Software Development IV","author":"Calieston Varatharajah, Christof Kost, Korbinian Kuhn, Marc Schelling, Steffen Mauser","date":"28. February 2017","format":false,"excerpt":"An automated development environment will save you. We explain how we set up Jenkins, Docker and Git to work seamlessly together.","rel":"","context":"In "System Designs"","block_context":{"text":"System Designs","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/system-designs\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2017\/02\/draw_io_docker_small-1024x439.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":538,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2016\/04\/27\/an-emotinal-sneak-peak-into-continuous-integration\/","url_meta":{"origin":366,"position":4},"title":"A sneak peak into Continuous Integration","author":"Mario Erazo","date":"27. April 2016","format":false,"excerpt":"What does the development of a software product involves? Is there more than programmers coding at their PCs and managers telling them what they want? What is the real meaning of \u2018release\u2019? In the old age of software development, a development team would have to deal with incredible amounts of\u2026","rel":"","context":"In "DevOps"","block_context":{"text":"DevOps","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/scalable-systems\/devops\/"},"img":{"alt_text":"Software development concept in tag cloud","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2016\/04\/stock-photo-25708797-software-development-concept-in-tag-cloud-1024x709.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2016\/04\/stock-photo-25708797-software-development-concept-in-tag-cloud-1024x709.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2016\/04\/stock-photo-25708797-software-development-concept-in-tag-cloud-1024x709.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":382,"url":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2016\/02\/21\/wrap-up-seeing-the-bigger-picture-part-5\/","url_meta":{"origin":366,"position":5},"title":"Jenkbird – Wrap up & seeing the bigger picture – Part 5","author":"J\u00f6rg Einfeldt","date":"21. February 2016","format":false,"excerpt":"Kchhhhhh, Kchhhhhh, Kchhhhhhh... - Ernie on companies without CI Hello internet and welcome to the last part of our\u00a0tutorial series about\u00a0Continuous Integration, Code Deployment and Automated Testing with Jenkins. If you arrived at this post and have read all the others we are very proud of you, hope you enjoyed\u2026","rel":"","context":"In "Allgemein"","block_context":{"text":"Allgemein","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/category\/allgemein\/"},"img":{"alt_text":"bert","src":"https:\/\/i0.wp.com\/blog.mi.hdm-stuttgart.de\/wp-content\/uploads\/2016\/02\/bert-186x300.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"jetpack_sharing_enabled":true,"authors":[{"term_id":692,"user_id":12,"is_guest":0,"slug":"td036","display_name":"Thomas Derleth","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/96baa052482491040e7a411673fb99628fa7dab61246ae854471e24ad43feb81?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/366","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/comments?post=366"}],"version-history":[{"count":24,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/366\/revisions"}],"predecessor-version":[{"id":24659,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/posts\/366\/revisions\/24659"}],"wp:attachment":[{"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/media?parent=366"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/categories?post=366"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/tags?post=366"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/wp-json\/wp\/v2\/ppma_author?post=366"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}