{"id":4024,"date":"2018-08-22T12:02:43","date_gmt":"2018-08-22T10:02:43","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=4024"},"modified":"2023-08-06T21:47:52","modified_gmt":"2023-08-06T19:47:52","slug":"why-ai-is-a-threat-for-our-digital-security","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2018\/08\/22\/why-ai-is-a-threat-for-our-digital-security\/","title":{"rendered":"Why AI is a Threat for our Digital Security"},"content":{"rendered":"

Artificial intelligence has a great potential to improve many areas of our lives in the future. But what happens when these AI technologies are used maliciously?<\/span><\/p>\n

Sure, a big topic may be autonomous weapons or so called \u201ckiller robots\u201d. But beside our physical security – what about our digital one? How the malicious use of artificial intelligence will threaten and is already threatening our digital security, will be discussed in this blog post. <\/span><\/p>\n

<\/p>\n

Artificial intelligence is currently one of the most developing technologies and some might also say it\u2019s the \u2018next era of computing\u2019. And that\u2019s for a reason: at least since 2011 when deep learning was introduced as a concept for machine learning, the performance of applications using AI, e.g., for image recognition increased to a super-human-level. Easier access to big data and more computing power reinforced this trend as well.<\/p>\n

\"Computer
Source: https:\/\/medium.com\/mmc-writes\/the-fourth-industrial-revolution-a-primer-on-artificial-intelligence-ai-ff5e7fffcae1<\/figcaption><\/figure>\n

Still, AI didn\u2019t evolve it\u2019s real potential yet and there are lots of research areas that still need to be covered. One of the topics that has not gotten enough attention yet is security. And to express it even more specific: digital security.<\/p>\n

In this blog post I would like to give you a short overview about this unnoted but still important topic, i.e., how malicious use of AI already threatens our digital security and what we can expect within the next years.<\/p>\n

Artificial Intelligence and the Curse of it\u2019s Dual Use Nature<\/h3>\n

There are plenty of projects using AI that are really made and used for the common good: AI helps to prevent cancer<\/a>, detects criminal activities,<\/a> helps to rescue drowning people<\/a>, etc. and there is hope that we will see a lot of more similar projects in the future as artificial intelligence offers the potential to perform almost any task that requires human (or animal) intelligence.<\/p>\n

But this potential is also the central issue. It\u2019s both a curse and a blessing as, sadly, we all know that human intelligence is not used for the common good only.<\/p>\n

Just like the human brain, Artificial intelligence is a dual-use tool and there are and will always be people that abuse AI technologies for malicious use.<\/p><\/blockquote>\n

<\/h2>\n

 <\/p>\n

An Already Existing Threat:
\nMalicious Use of AI in Social Media<\/h2>\n

There are a lot of AI technologies that are already used maliciously and threatens our digital and political security. In the past years we experienced this trend especially in social media. Social media platforms give attackers both access to big data and a possibility to address people personally which might become a \u201ctoxic\u201d combination and creates an ideal base for an efficient attack.<\/p>\n

1. Use of Data Mining \/  Profiling<\/h3>\n

It\u2019s no big news that social media platforms, particularly Facebook, earn money by using their users\u2019 data to generate and display, e.g., customized advertising with the help of intelligent algorithms. Whether this already counts as malicious use or not, this will not be discussed here. But a famous example how these AI technologies have already been abused to threaten the political security of the U.S. citizens (and not only them) is the case of Cambridge Analytica and its influence on the presidential elections 2016<\/a>. With the help of data mining for microtargeting Cambridge Analytica was able to possibly influence millions of american voters.<\/p>\n

\"Between
Source: https:\/\/www.businessinsider.de\/facebook-cambridge-analytica-affected-us-states-graphic-2018-6?r=US&IR=T<\/figcaption><\/figure>\n

The work of Cambridge Analytica describes only one example how the malicious use of data mining or profiling can have a negative impact on security. Fake news or customized content (e.g., customized prices depending on user data) could be increased by it in the future as well.<\/p>\n

2. Use of Natural Language Processing\/ Generation<\/h3>\n

Natural language processing and generation shows many different areas of application, e.g., translation and sentiment analysis on social media platforms. Where once expensive and time-consuming human work was required to synthesize or analyse texts can now be mastered within seconds by one intelligent algorithm. What may appear as a beneficial technology for many people can unfortunately also be abused for instance to automize SPAM or phishing attacks.<\/p>\n

The american security company ZeroFox already showed in their case study #SNAP_R how easy and lucrative this could be for attackers who abuse these technologies. By using a neural network for data mining and natural language generation they automatically generated customized \u201c@mention\u201d-Tweets containing phishing links.<\/p>\n

Details of the study can be found in their presentation<\/a> but in essence #SNAP_R works as follows:<\/p>\n

    \n
  1. Set up fake profiles<\/li>\n
  2. Take a list of random Twitter users<\/li>\n
  3. Evaluate users\u2019 vulnerability<\/li>\n
  4. Collect data about most vulnerable users (topics, bag of words etc.)<\/li>\n
  5. Feed the neural network with data<\/li>\n
  6. Generate and post \u201c@mention\u201d Tweets with phishing link<\/li>\n<\/ol>\n
    \"\"
    Source: https:\/\/www.blackhat.com\/docs\/us-16\/materials\/us-16-Seymour-Tully-Weaponizing-Data-Science-For-Social-Engineering-Automated-E2E-Spear-Phishing-On-Twitter-wp.pdf<\/figcaption><\/figure>\n

    The result is impressive: after 2 days they achieved a 30 – 66% click-through-rate (the percentage is vage as they can\u2019t be sure whether all clicks came from real persons or from other bots).<\/p>\n

    Using a neural network to generate phishing attacks will be both time and (probably) money saving combined with a high efficiency – a dream for every attacker.<\/p><\/blockquote>\n

    Very probably we will see a lot of intelligent and automated phishing, SPAM or social engineering attacks using artificial intelligence in the future.<\/p>\n

     <\/p>\n

    New Threats:
    \nMultimedia Generation and Manipulation<\/h2>\n

    Audio, image and video data have become an essential part of our digital communication. Artificial intelligence has already been used very early to help us dealing with media data in different kinds of ways, e.g., object recognition or media interpretation in general. Recently, also media manipulation and generation technologies have shown up rising new possibilities but also questions for our digital security.<\/p>\n

    1. Audio and Speech Generation<\/h3>\n

    The possibility to let Siri, Alexa and Co. read every possible text is nothing new to us but how great would it be to let your favorite voice actor read all your files for you instead? The current AI technology is not as far from that! In 2016 Adobe already presented their software Voco<\/a> that might be capable to do exactly this (however in the moment there are no news about when or whether the software will be released). But also the canadian start up Lyrebird<\/a> makes it possible to create a vocal avatar –  \u201ca digital voice that sounds like you with only one minute of audio<\/em>\u201d. Although the results still sound kind of artificial it\u2019s just a matter of time when the technology will have developed to a deceptively realistic level.<\/p>\n

    Not only speech generation is already possible yet but also generation of sounds. A research group of the MIT shows how \u201cVisually-Indicated Sounds<\/a>\u201d can be created with the help of machine learning. It \u201cgenerates\u201d appropriate sounds of a drum stick just by giving a video as input to a neural network (for more information I really recommend watching their video<\/a> or reading the paper<\/a> of the project).<\/p>\n

    No doubt, these are impressive examples, how AI can be used for audio generation in the future. But what does this mean for our digital security?<\/p>\n

    Unfortunately a technology that is capable of putting any possible words in the mouth of your own digital voice is also capable of doing this for any other person’s voice. So with just a small amount of original audio recordings of a targeted person, it will be easy to abuse this technology to spread audio-based fake news or to use it for social engineering (especially phone frauds).
    \nAt the Black Hat Conference 2018, John Seymour und Azeem Aqil showed in their presentation \u201c    Your Voice is My Password<\/a>\u201d that it\u2019s already possible to use AI based technologies to outsmart voice authentication. Examples like this put the future of biometric authentication into question.<\/p>\n

    In general it will be almost impossible for humans to differ real from fake recordings in the future and it will be a big challenge for the IT security to find ways how at least machines can detect the difference.<\/p>\n

    2. Image Generation<\/h3>\n

    In times of Photoshop, etc. image manipulation is nothing new to us but machine learning based algorithms now enables us to generate completely new images. Here is an example: could you tell which one of these images is a photo of a real existing person and which one is just generated with the help of a Generative Adversarial Network by Ledig et al.<\/a>?<\/p>\n

    \"\"
    Source:
    https:\/\/arxiv.org\/abs\/1710.10196 ,
    https:\/\/cdn.playbuzz.com\/cdn\/\/9ced2191-d73f-4e3e-9181-824046fac4ed\/2bda590b-182b-4c67-8793-8ffc1bd8b9ea.jpg<\/figcaption><\/figure>\n

    Here is the answer:<\/p>\n

    The left one is a photo of the mexican singer Luis Coronel and the other one is just a fictive person.<\/p>\n

    It\u2019s most probable that in the next years it will be not only possible to generate high resolution images of portraits but also of places, objects, animals and everything one can imagine. What might be great for the stock photo industry or image rights in general, might be used maliciously to create content for fake profiles and to make fake news look more plausible. In the end we will have to face the question: How can we differ real from fake images in the future? Again, a big challenge for the IT security.<\/p>\n

    3. Video Generation and Manipulation<\/h3>\n

    If we can already generate and manipulate audio and image data it\u2019s not surprising that also new AI based video manipulating technologies have come up in the last years. The research project Face2Face<\/a> by the University of Erlangen-Nuremberg, the Max-Planck-Institute for Informatics and the Stanford University shows how easy it is to manipulate a person\u2019s facial expression in a video, even in real time (here<\/a> is their demonstration video).<\/p>\n

    \"\"<\/a>
    Source: https:\/\/www.youtube.com\/watch?v=jI6H-0YWkSc&feature=youtu.be<\/figcaption><\/figure>\n

    Also face swapping in videos, so called Deep Fakes, <\/a>have gone viral and can be created easily just by using a software called FakeApp (however it seems their Website has been turned down since June 18).<\/p>\n

     <\/p>\n

    Especially in combination with speech generation, these technologies can become really dangerous for our security when used maliciously. They open doors for highly deceptive fake news videos: either to change only some significant details, e.g., in a politician interview with the help of Face2Face and voice generation, or to create a whole new video and map a target person\u2019s face into it with Deep Fakes. And what will happen with credibility of video evidence?<\/p>\n

    We will have to overthink our \u201cseeing is believing\u201d mindset.<\/p><\/blockquote>\n

    Again it will be a big challenge for the IT to find a way how we could mitigate this problem.<\/p>\n

    In a Nutshell – What can we expect in the next years?<\/h3>\n

    In summary, both existing and new threats will probably lead to the following problems in the next years:<\/p>\n

      \n
    1. Attacks will become: <\/span>\n
        \n
      1. more automated<\/span><\/li>\n
      2. finely targeted<\/span><\/li>\n
      3. more effective<\/span><\/li>\n<\/ol>\n<\/li>\n
      4. Personal data will be used and abused more than ever<\/span><\/li>\n
      5. Distinction between real and fake content will be almost impossible for humans<\/span><\/li>\n
      6. Fake news will be a bigger challenge than ever before – A small group of people will be able to manipulate the public opinion (e.g., case Cambridge Analytica)<\/span><\/li>\n
      7. Biometric authentication might become insecure
        \n<\/span><\/li>\n<\/ol>\n

        How can we mitigate or even prevent the threats?<\/h2>\n

        First of all, when dealing with dual-use technologies there is no real solution how to prevent them from being abused. The only way would be to prohibit the malicious use of it, which might somehow be promising for material or expert-knowledge required technologies like nuclear energy. But in the end a trained neural network is just a small file and can easily be distributed and adapted to be used even by laypersons. Additionally, it\u2019s quite subjective what counts as malicious use of AI and what not, so a prohibition would be hard to draft in the first place.<\/p>\n

        Nevertheless, there is a lot we can do to mitigate the threats. Here are some suggestions depending on the different roles that are involved.<\/p>\n

        1. User<\/h4>\n

        In simple terms: the less people fall for fake news – the less fake news will be \u201cproduced\u201d. So what one can do as a user is:<\/p>\n