Are Passwords for Web Authentication Obsolete? Leakage of Passwords and API-Keys and Possible Solutions

Hardly any service today works without an API that allows users to log in and then use features that are not available to unregistered users. To do this, the user can create an account that is password protected. Services such as the Google Maps API also provide access interfaces to allow application developers to easily develop very helpful features and make them available to their users.

Basics

Passwords in general

In order to register with a web service, a user name or e-mail address is usually used in combination with a password. Some services do not give the user any instructions as to what the password should look like, while other services impose strict password rules. These rules can include the following:

Minimum password length
The password must contain (at least one) capital letter
The password must contain (at least one) special character.

These specifications increase the number of possible combinations. For example, an eight-character password consisting only of lowercase letters has 26^8 = 208,827,064,576 possible combinations. If upper and lower case letters are used, the number increases to over 218 trillion possible combinations. [1]

In addition, password policies prevent easy to guess passwords from being used. The following list [2] shows the most frequently used passwords, which would not have been possible without the above rules:

123456
123456789
qwerty
password
111111
12345678
abc123
1234567
password1
12345

Password Leaks

Passwords are leaked over and over again. There are also large sites with many millions of users. The following chart shows how many password leaks there were until 2016. [3]

Figure 1: Leakage of Credentials. Source: [3]

The most important leaks at a glance:

Twitter, 2012: 68 million encrypted passwords [4]
last.fm, 2012: 43 million user names, e-mail addresses and passwords [5]
Adobe, 2013: 38 million IDs and passwords [6]
Sony, 2014: thousands of passwords [7]
Tumbl, 2016: 65 million encrypted passwords [8]
Yahoo, 2013: 3 billion passwords [9]

The service “have I been pwned” can be used to check whether the user’s own access data has been leaked. To do this, enter your e-mail address and find out from which pages the leaked access data originates.

Figure 2: have i been pwned? (https://haveibeenpwned.com/)

How to Save Passwords

Passwords are usually not stored in plain text, but a hash is created from the password using an algorithm. No conclusions about the original password can be drawn from this hash. If the user enters his password, the entered password is hashed with the same algorithm and then compared with the stored hash. If these match, authentication is successful.

Figure 3: Password “1234” as SHA-512 Hash. (Generated on https://hashgenerator.de/)

If hashed passwords are stolen from a database, the attacker does not have the plaintext password and cannot log in with the stolen identity.

However, since the attacker himself can generate the hashes from the password combinations (so-called rainbow tables), hashing alone is not sufficient. To additionally secure passwords, a so-called salt can be attached to the password (see Figure 4). Especially a long and complex salt minimizes the probability that the combination (1234somerandomsalt) is in the attacker’s rainbow table.

Figure 4: Password “1234” as SHA-512 Hash with a salt. (Generated on https://hashgenerator.de/)

Tokens

In most applications, the user only has to enter his password once and can then stay on the site for a very long time without having to authenticate again. In order to guarantee this good experience from the user’s point of view, so-called tokens are often created after the login, which are sent back to the user as a result of the login. With these tokens, the user can make any further request to the server without having to re-enter his password.

Figure 5: JSON-Web Token. (Generated on https://jwt.io/)

Figure 5 shows a JSON Web token. A JWT consists of 3 parts: Header, Payload and Signature. In addition to the type, the header contains the algorithm used for encrypting and decrypting. The payload contains the Expiration Time (exp), the Issued at (iat) and user-specific data such as name, user ID and role. But here, too, the question arises how the server can ensure that the token is valid and that the user actually has the roles that are in the token. As a potential attacker, the user can of course try to change the role in the token, e.g. to ‘ADMIN’. To prevent this, there is the 3rd part of the JWT: the signature. The signature is a hash, which is generated from the coded header, the coded payload and a secret key known exclusively to the server. All 3 parts of the token are Base64 encoded. Each separated by a dot they result in the JWT. If the potential attacker now tries to give himself the role ‘ADMIN’, the signature is invalid and the server will respond to a request by the attacker with the status code 401 (Unauthorized) or 403 (Forbidden) depending on the implementation. The use of tokens is particularly helpful in distributed systems (microservices) to reduce the communication between the authentication server and the microservices. JWT’s also result in a Single Sign On System because the user only needs to authenticate once to access different services (servers).

API’s and their (access) Keys

API is the abbreviation for Application Programming Interface. With the help of API’s systems like Facebook or Google offer other programmers the possibility to communicate with a service like Google Maps, e.g. to query the coordinates of an address. [10]

API keys are often used to restrict access to the interface, since e.g. certain API’s can only be accessed when the service is paid for. Similar to passwords, these keys should therefore be stored securely and encrypted. In contrast to passwords, however, it is not the end user, but the administrator or programmer of an application who has to take care of secure use.

Problem

The number of password leaks shows that even large companies are not protected against the theft of access data. In the following, possibilities are discussed for securing a service against attacks.

Password Rules

As explained earlier, applications should have password policies that are as strict as possible to protect the user from a leak. The password should not include the name, date of birth or email address, either in full or in part.

Disadvantages of strict password rules:

The usability suffers under the strict guidelines, because the user has to consider and remember a very strict password. This can be remedied by a password manager that not only generates such passwords, but also stores them securely. Figure 6 shows how users deal with passwords and how annoying they are.

Figure 6: Usage of passwords of internet users. Source: [11]

Two Factor Authentication (2FA)

While authentication is sufficient for login via e-mail or user name in combination with a password, Two Factor Authentication (2FA) adds an additional security layer. For web services, it is often implemented in such a way that the user first enters his access data and is then prompted to enter a unique token, which is transmitted by e-mail, SMS or in an app. Only after the correct token has been entered is the user fully authenticated.

Even if an attacker gets access to the credentials, he cannot log on to the service with them alone because he does not have the token. This gives the attacker additional hurdles such as access to the e-mail account or the smartphone. This is only possible with considerable additional effort and targeted attacks.

Disadvantages of 2FA:

Often the use of 2FA is optional and must be set up manually by the user. For developers, the integration of 2FA means additional effort in the development of authentication.

Authentication via third-party accounts and authentication providers

Many users already have an account with major providers such as Google or Facebook. They also offer authentication options that can be integrated into your web service. Libraries for different programming languages are offered for this purpose.

Google Sign in: https://developers.google.com/identity/sign-in/web/sign-in
Facebook Login: https://developers.facebook.com/docs/facebook-login/

Figure 7: Google and Facebook Login. Source: [12]

The disadvantage is that the user must have such an account. For this reason, third-party account authentication is usually only used as an additional authentication option.

Authentication providers offer a remedy for this. As with authentication via third-party accounts, the access data is no longer stored on the user’s own servers and responsibility is outsourced. The difference compared to third-party accounts is that the user logs on to the service and does not notice any difference to the fact that it is an external service, since the registration and login interface can be individually designed.

Here, too, the authentication providers offer libraries for various programming languages in which the service can be easily integrated into the user’s own application.

Some known providers are listed below:

Auth0 (https://auth0.com/)
Okta (https://www.okta.com/products/customer-identity/authentication/)
Google Firebase Auth (https://firebase.google.com/docs/auth)
AWS Mobile Hub User Sign In (https://docs.aws.amazon.com/aws-mobile/latest/developerguide/User-Sign-in.html)

Disadvantages of outsourcing authentication:

The disadvantage of outsourcing user authentication is that data is passed on to an external provider, which of course has to be implemented in compliance with data protection regulations – especially with regard to the DSGVO.

Github Leakage and AWS Credential Manager

As already mentioned, many applications use API’s from third parties. In the open source age, the secrets for these API’s are often checked into repositories that everyone has access to. A well-known network for such repositories is GitHub, the largest platform for open source projects. If an API key is leaked there, potentially every Internet user has access to the API. According to [13], leaks from keys happen very often. There are 100k repositories affected and every day 2000 new unique keys are leaked. Of these, 89% are not test keys and 81% stay online for more than 16 days. Once the key has been checked into a public repository, attackers take an average of 20 seconds to discover the key. The GitHub API allows attackers to automatically search for patterns that are present in many API keys. For example, Google API Secrets start with “AIza”. For this concrete example, the endpoint “https://api.github.com/search/code?q=AIza” is called (anyone with a GitHub account can do this).

Figure 8: Call the Github API with postman. Source: own

This can of course be done very easily and automatically, but for demonstration purposes it is sufficient to use Postman to carry out the requests. The query returns 2,590,021 results. So more than 2 million potential API keys have been found, which can be used e.g. the Google Maps API. Therefore, the next step opens the links returned by the search:

Figure 9: Part of the response of the call of Figure 8. Source: own

The page that opens will then be searched again for “AIza” – because this is the API key whose validity is to be tested. The search returns the following API key:

AIzaSyBAqbCqznL_L-B6s9WPNTkaAgP6NJgZ22k

The validity of the found keys can of course be tested automatically – but here the keys are tested manually with the help of demo tools for demonstration purposes. For the Webframework Angular, for example, there are libraries that facilitate the integration of Google Services: only the API key has to be exchanged. Such a demo can be found at https://stackblitz.com/edit/angular-google-maps-demo.

Figure 10: Testing a valid key. Source: own

Figure 11: Testing an invalid key. Source: own

In the file app.module.ts only the found API-Key has to be exchanged. If the card shows no errors and returns a result, the secret is valid – and can be used theoretically. The two following figures show the result for the found key and for a non-valid key: “test”.

To prevent this, API keys should never be located directly in the source code, even if it is a private repository to which only a few users have access. This is because the API keys could also be used by employees for private purposes. A remedy for this problem can be found in the AWS Secrets Manager, which securely stores the credentials in encrypted form and even renews them using a lambda function if the user so wishes. The following explains how the Secrets Manager can be used in an application. It is assumed that an AWS account exists and the Secrets Manager has been activated.

Creating a new secret

Figure 12: Creating a secret on AWS. Source: own

Assign an ID

Figure 13: Assign an ID for a secret on AWS. Source: own

Create AWS Credentials for Access

Figure 14: Create AWS credentials. Source: own

Storing credentials locally

Figure 15: Store AWS credentials locally.. Source: own

Accessing the Secret

SDKs
CLI

Now the credentials are no longer stored directly in the source code, but are queried by AWS as soon as they are needed. Since the credentials file (or generally the AWS credentials) are not checked into the repository, the secrets can be accessed securely.

Face ID, Fingerprint, FIDO

In addition to authentication with a user name and password, there are other ways to authenticate with a service. Face ID or fingerprint unlocking is particularly popular for smartphones. With Apple, passwords stored in the keychain can be accessed using the Face ID or fingerprint, if that is wished by the user and/or the app or service supports it. Google has recently introduced FIDO, which is a way for users to authenticate themselves to a service using their Android smartphone. To do this, the login data must be linked once with the fingerprint. This should make it easier and safer for users to handle passwords [14].

Conclusion

Even hashed passwords can be accessed via rainbow tables if they appear there. Therefore it is necessary that the passwords consist of as many combinations as possible, so that these do not occur in the tables. However, the problem is that it is difficult for users to remember these passwords. Therefore tools like Apple’s Keychain or FIDO increase the security, because even with complicated passwords a login is easy to handle. Password managers can also help, but are usually not available for free and are protected by a master password, which give attackers access to all other passwords if it is leaked. API keys must also be kept safe. Services like the AWS Secrets Manager can help developers to store them securely and even renew them regularly. If an API key is leaked in a public repository at Github, this is usually noticed very quickly either by Github or by the service that issued the credential. The services are then usually very accommodating, but which a developer should not rely on.