SecAdvent
On Artificial General Intelligence, How to Get There and Security – SecAdvent Day 21
December 21, 2020
This post might be a bit more on the philosophical side than you used to. But stay with me. While we are touching the realms where Computer Science and Philosophy interlace (yes, these realms exist), we keep it as technical and real as possible. We are about to explore first the basics of Artificial General Intelligence (AGI) and its implications on software security.
AI and its methods
We are about to talk of Artificial General Intelligence, one of the most discussed topics these days. First, it is important to know, that we are talking about the middle of the three types of artificial intelligence described1:
- Narrow or weak AI (inferior to humans)
- General or strong AI (equal to humans)
- Artificial Superintelligence (superior to humans)
First, we will steer clear of the discussion if there is an Artificial Superintelligence and what it will look like (I hear you, Terminator-fans). Rather, we will focus on the holy grail: Artificial General Intelligence. It has this role since the very first days of AI research because it was selected as a measure in the famous Turing-Test2. It literally says that we achieved real artificial intelligence when we cannot distinguish between a real human and an AI. The AI argues equal to a human.
Today, we are used to narrow or weak AI in everyday devices. We are so used to these services that we hardly notice it anymore. Lots of them make use of machine learning (called sub-symbolic AI3). So much so, that sometimes AI and machine learning are used interchangeably. I guess if you are not working in a different industry or living under a rock, you know at least basically how this works: We start with a basic model which is more or less random. By running the model against training data and tweaking the model so that the outcomes actually predict the right answer, we gradually get the model to make the right predictions given input data. An obvious problem is that you need enormous data sets (the more, the better in quality, the better). Also, too much data of the same brings the risk of what is called overfitting: The model is trained on a type of data but fails to deliver on never seen types of data. On the flip side, normally it is jolly fast to use such systems plus they are rigid towards noise in the input data.
But there is more to AI4. Symbolic AI models the world using facts and rules in order to connect these facts, as well as to argue new facts and rules. By the way, symbolic AI was researched from the 1950s and some call it the Good Ol’ Fashioned AI. Intuitively, we see where it strives: When we have a “world” that is well sorted and logical.
While in the beginning we tried to capture the real, physical world, soon it was discovered … well … that was too big of a bite for now. So, while symbolic AI strived in some applications (blond spoiler – static code analysis is one of it), it did not deliver on the promise of an General Intelligence. One of its strengths is that it is quite easy for a symbolic AI to argue why it came up with a result. It simply has to compile the rules and the facts that led to the outcome to a nice argumentation. But it struggles when we have an uncertain world. Lots of research has gone into teaching symbolic AI to become fuzzier (example is fuzzy Prolog5). But there is another train of thought. It seems the symbolic AI is nicely strong where the sub-symbolic is weak and vice versa, right?
It is argued that the combination of the two will open a door to move the field of AI on (or to prevent another AI-winter)6 or even Artificial General Intelligence. Anyways, the combination of the two seem to attract attention7. To mention a few things, it promises: Explainability, rules without massive amounts of example data, finding rules in massive amounts of example data, robustness against noise and more.
Regarding the Artificial General Intelligence, one school of scholars argue it will never be achieved (the Dreyfus argument) as recently as 20208. Basically, the argument is that computers are logical machines and as such, not in the world of humans. On the other hand, there is quite some criticism on this approach (as early as 19689). It starts with defining what intelligence really mean10. Why do we make humans to the measure of intelligence? Some argue, the internet already achieved an intelligence that is simply different from ours, and we cannot see it. This is where philosophy and computer science interlace. All in all, it seems that computer science streams ahead while the other sciences – philosophy, public policies, psychology – see it as a pre-paradigm field (using Thomas Kuhn’s theory of science history11). Nonetheless researchers are already thinking about the influences of AI on security.
WE WILL ACHIEVE REAL ARTIFICIAL INTELLIGENCE WHEN WE CANNOT DISTINGUISH BETWEEN A REAL HUMAN AND AN AI.
Artificial Intelligence and security
There are several aspects on AI and security (if you have an hour to spend, here is a deeper introduction by Dawn Song12). AI will influence physical security (e.g. face recognition and physical access); it itself will be susceptible to attacks (e.g. glasses that make you famous13); its influence on software security and much more. We will focus for now on the software security side of things.
As mentioned above, traditionally software security tools used symbolic AI (and now we know what that means). As an example, we are talking about static program analysis or fuzz testing. Recently, we used sub-symbolic tools we developed for other tasks such as natural language processing (NLP) used in static code analysis and saw some amazing results14.
On top, during the past decade, the research community has explored the idea of combining symbolic and sub-symbolic AI. If you ask Google Scholar, you will find a myriad of articles that focus on specifics of the implementation. It seems promising as we have access to vast amounts of training data for the sub-symbolic side and it is inherently a logic world our system is asked to act in – which plays into the cards of a symbolic AI.
In Summary
AI is a fascinating field and the two major family of approaches – symbolic and sub-symbolic – come together after decades of research. Security and especially the field of cyber security is already benefiting enormously from the application of each field separately. There seems to be a vast field of gains to be harvested using a combination of the two approaches. The first tools make their way from an academic world into real life applications.
-
Symbolic versus sub-symbolic http://www2.fiit.stuba.sk/~kvasnicka/NeuralNetworks/1.prednaska/symbolic.pdf
Frank Fischer
About Frank Fischer
Frank Fischer is an IT veteran of 20 years and worked in companies such as Microsoft, Google, and Deutsche Telekom as well as startups. He is working inside Product Management for Snyk in the field of applying AI to source code. He holds an MSc in Physics from the University of Karlsruhe, Germany, an MSc in IT and an MSc in Organisational Psychology from the University of Liverpool, UK. Besides work, family, and study – you can find Frank diving in the waters around Malta.
