, , ,

Security in a SaaS startup and today’s security issues with DevOps

cp054

Motivation

Facing security in a company nowadays is a big job: it starts with a backup strategy ensuring the business continuation, plans for recovery after major breakdowns, ensuring physical security (entrance control, lock-pads, safes), screening of potential employees, monitoring servers, applications and workstations, training the employees in security issues and policies and does not even end with a proper patch management or in case of software development secure coding. As we see using a safe password ([10] explains safe passwords) and the hope of not being caught by the next ransom attack is not enough. Security is a combination of organizational, technical and physical measurements.

Another view on security is the technical one, looking at algorithms like RSA, secure hash functions, protocols like tsl/ssl and the PKI etc. We have good tools, but if we look at our technology stack it is still impossible to create a 100% secure system nowadays. There are security breaches in each layer, starting down at the hardware, e.g. a Qualcomm Chip [11], going up to the operating system (e.g. PtH [12]) up to the application using a lot of third party code. A secure system must not use any unsecure third party or legacy code and must not rely on any unsecure other systems – so we would have to start from scratch.

We cannot assure a 100% secure system or even a 100% secure company, but we can keep the risk down at an acceptable minimum. This is where economics plays in – taking security measurements costs money and we must decide how much money (or time – but time is money again) we spend on security.

 “Basically, you’re either dealing with Mossad or not-Mossad [the Israeli CIA]. If your [opponent] is not-Mossad, then you’ll probably be fine if you pick a good password and don’t respond to emails from ChEaPestPAiNPi11s@virus-basket.biz.ru. If your [opponent] is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT.”

James Mickens [4]

A good overview of what can be done today gives Google’s whitepaper on their security management [13] – they do a lot, starting with dedicated hardware encryption and signing. The security in a company is also addressed by the ISO 2700X standard, consisting of a lot of requirements and over a hundred measurements to be taken. But where to start?

In this article, we look at security from a startups point of view: what do we need to do in an early stage, developing in our garage, without too much effort. We will take a special look at the DevOps toolchain, as its automation leads to new security issues.

DevOps? DevOps is the merge of development and operations with the goal to test, build and release software fast, frequently and reliable. This leads to an automated toolchain, in which a single change in code by a programmer can result in a new production release (Continuous Integration, Continuous Delivery). A programmer now has impact on a productive system and needs to think about configuring and running the software securely.

Scenario: Working in the living room

To take a closer look at the security in a small company our scenario will be a SaaS startup, just founded, without any customers yet. Figure 1 gives an overview – making no claim to be exhaustive – of the minimal infrastructure we will need.

Example of an early stage infrastructure – maybe more tools and systems than one would think

As we can see we need a whole bunch of software to communicate, document and organize our work and code. Looking at this infrastructure from a network view we can see a lot of interaction between the different systems. We must handle a lot of accounts, secure communication and especially take care of the infrastructure running in our own cloud (back office, continuous integration, continuous delivery, our own API service), see figure 2.

Example on an early stage infrastructure network view – there are more systems we interact with than one would probably think. | CI – Continuous Integration, VPC – Virtual Private Cloud, PW – Password

Keep track of your tools and systems

To manage all your tools and systems I suggest to introduce something like a company board or system overview at a central place, reachable by every employee. It could be a word document, but better a wiki page, a one-pager-website only available to employees. It should contain a list of all systems and tools with a description what there are for and when to be used. This information is something like a small policy one would write in a larger company, e.g. ‘place documentation in this wiki’ and ‘place invoices in that cloud’. Some security experts might argue that an attacker could get to much information out of such a system overview, but I think that at this stage it’s more important to keep track of all systems.

Enforcing basic security

There are many easy and effective measurements to start with. Some will even sound too easy / boring / common sense to bother with, but they are still highly effective.

We will not cover all of them in detail, but there are worth to be mentioned:

  • Don’t visit unsecure websites (porn, torrents) à Use a private machine
  • No sensitive data via Email (it’s not encrypted in most cases!)
    -> use a shared filesystem and link it or a password manager or an encrypted zip
  • No email attachments (still one of the most popular phishing attacks)
    -> same as above
  • Encrypt your hard drives (one click in most OS – just think of what would happen if you lost your device, you would not be able to sleep anymore)
  • Screen lock you PC and smartphone
  • Use secure (and different) passwords (you can’t remember them? See next bullet)
  • Use a password manager with share function
    -> There will always be some company accounts to share, even if every employee should have his own account
  • Use multiple users when setting up a system/server and even better: use roles and groups

This were some basics, no we will look at some eye-opening, maybe even entertaining examples.

Human failure

A technically secure system is not secure out of the box. Most of the times we need to configure the security and remember or store tokens and passwords. This is, where the human factor plays in – which can lead to fatal security breaches.

Push to public

Looking at public available code, configuration files (.dot files) are of major interest. Those files potentially contain secret server keys and logins, database users and passwords etc. to automate the build and delivery process. Thanks to search functions you can easily look for those secrets, e.g. https://github.com/search?p=98&q=aws_secret_access_key&type=Code&utf8=%E2%9C%93

Example of a secret key published on github. In this case the AWS config was probably pushed by mistake together with some other file. Fun fact: The author added a gitignore entry not to push some other secret vars.

A story of a person publishing his AWS secret key to github and getting a bill over $50 0000 from amazon can be found here [8]. AWS keys are only one example of sensitive data – any secret tokes, e.g. for Slack or a Google API, logins to databases, credentials to access a server to deploy a new version of the software – all this information must not be pushed to a public repository.

The easiest way to avoid this kind of security breaches is to use a nonpublic organizational repository. In addition, there are tools to scan repositories for confidential data, e.g. TruffleHog [6] or GitRob [7]. GitMonitor [5] offers even an online service which will scan your repository and notify you on security breaches.

This is a typical DevOps security issue, as the programmer oversees operating the software as well.

Default configuration

A ransack attack of 27 000 MongoDB databases hit the new lately (January 2017 [3]). What happened? All those MongoDB Instances were on the internet with their default configuration. There is only few software with actually secure defaults – but why don’t we ship software with secure defaults? It’s just not handy setting up secure user and password combination for every database during development. One could think of a tool, managing new installations with a wizard creating, storing and reentering safe passwords all the time. This would help developers to keep their workflow despite secure defaults. But this would have to be a god-like tool, knowing all software and how to configure it. One could even think of a standard, supplying an interface to securely setup new software. Again, a DevOps sourced issue, as programmers probably didn’t think much about setting up a secure operation environment.

Backup

Maybe you have heard of CodeSpaces, maybe not, but one you should know about them: they didn’t do they homework. The SaaS provider is out of business after an AWS security breach, underscoring the need for proper cloud storage backup [9]. In short: CodeSpaces’ AWS (root) account got hacked and blackmailed by the attacker. They changed all passwords but the attacker was watching them. In the end, the attacker wiped out all data – including all backups, as they were on AWS too. In the first place, a proper 2-factor-authorization and IAM would have helped to protect the account. Second, placing backups in the same cloud and even in the same account is just ridiculous.

There is a practical backup strategy: 3-2-1. 3 backups on 2 technologies (cd, cloud, tape, NAS), 1 off site (geographically as well as technically).

Summing Up

Don’t underestimate security! You should be a little bit paranoid but you still want to get you job done. There are a handful of small measurements you can take without too much effort to introduce a basic security. You cannot afford to arm for an attack by a ‘Mossad’, but keep script-kiddies out, prevent known attacks and prepare for unknown attacks.

Don’t bother doing anything that’s not your key-business. We didn’t touch it but patch and vulnerability management is of high importance and for every software or server you run you must do it. This is a very good argument to use cloud services where some else must worry about this.


cp054@hdm-stuttgart.de

Sources

[1] https://github.com/forter/security-101-for-saas-startups, viewed on 24.05.2017
[2] RSA Conference 2017, DevOOPS: Attacks And Defenses For DevOps Toolchains https://www.rsaconference.com/writable/presentations/file_upload/hta-w02-devoops-attacks-and-defenses-for-devops-toolchains.pdf, viewed on 24.05.2017
[3] http://www.zdnet.com/article/mongodb-ransacked-now-27000-databases-hit-in-mass-ransom-attacks/, viewed on 22.04.2017
[4] James Mickens, ;login:logout, USENIX January 2014, https://www.usenix.org/system/files/1401_08-12_mickens.pdf
[5] https://gitmonitor.com/
[6] https://github.com/dxa4481/truffleHog
[7] https://github.com/michenriksen/gitrob
[8] https://www.quora.com/My-AWS-account-was-hacked-and-I-have-a-50-000-bill-how-can-I-reduce-the-amount-I-need-to-pay
[9] http://www.networkcomputing.com/cloud-infrastructure/code-spaces-lesson-cloud-backup/314805651
[10] https://nakedsecurity.sophos.com/2014/10/01/how-to-pick-a-proper-password
[11] https://www.heise.de/security/meldung/QuadRooter-Verwundbare-LTE-Chips-sollen-ueber-900-Millionen-Android-Geraete-gefaehrden-3289647.html
[12] https://bogs.technet.microsoft.com/austria/2015/09/16/pass-the-hash-angriffe-pth-kurz-zusammengefasst
[13] https://.cloud.google.com/security/whitepaper


by

cp054

Comments

Leave a Reply