Schneier on Security

Clive Robinson • November 9, 2024 4:43 AM

Re : In whom we trust.

There are a couple of implicit questions in,

“[Y]et another illustration of why trusted AI is essential.”

The current AI systems we refer to consist of not just LLM and ML code but three types of input. So at a 20,000ft view we have the following parts to consider,

Firstly the base system
1, The hardware system
2, The LLM code and build process
3, The ML code and build process

Secondly the actual learning data
4, Learning corpus
5, Past users questions
6, Current user questions

Thirdly the resulting,
7, Learning tokens (vectors)
8, Learned relations (weights)

Fourthly there is also,
9, The “guide rails” and other “touch up” by humans that is “so essential”.

We know from the need for the guide rails and touch up “mitigation” that the preceding stages are combined “untrustworthy” currently.

Let us assume by some “magic” –yet to be found– that the hardware can either be trusted or mitigated[1].

We know we’ve no way to make “Turing Engines” and similar actually “Trusted”. Kurt Gödel proved this before Church, Turing and others proved their points.

So the foundations that underlying all current AI LLM and ML systems can not actually be trusted.

Likewise the “software” can not be trusted for the same reasons (and many more besides).

As for the input “corpus data” be it taken from the Internet, scientific papers, published works and any other place it can be reaped from lawfully or not we know it’s very much untrusted.

As for the input that is “user questions” from the time of Microsoft’s Tay, making AI systems untrustworthy has been a highly competitive “blood sport” for “fame and giggles”.

In short we currently have no idea how to make any system “trustworthy” let alone an AI system. It would be madness to suggest we could make any system based on such technologies trustworthy as long as “soft errors” and “metastability” exist.

Though few realise it all systems that,

1, Store information
2, Communicate information
3, Process information

Are inherently unreliable and in fact can be modelled in part or whole as “Shannon Channels”.

As any communications engineer can tell you, you can never make a Shannon Channel 100% reliable due to the required redundancy to communicate information and the laws of physics that give us noise and distortion to contend with.

How do communications engineers get improved reliability?

The answer is by making the Shannon channels deliberately inefficient in various ways to get more redundancy. Then using that redundancy in various constructive ways (various types of coding and feedback systems).

So the only way we can make AI systems more trustworthy is by,

1, Make the data trustworthy.
2, Introduce significant constructive redundancy.

But attackers will always be able to make them less trustworthy “by adding noise” in interesting ways.

And the maths says the attackers will always “get one over” on the defenders if the attackers can “reach the system”…

[1] Although known by hardware engineers since before WWII that mechanical, electromechanical and electronic “switching” circuits were implicitly unreliable, it’s not stopped a multi trillion dollar industry being built up upon them. And upon that “house of cards” the western society of the modern world those reading this live in. It is also one in which most of the Western World citizens implicitly entrust essential parts of their daily lives, as well as in many cases actually entrusting their lives even if they don’t wish to. But the desire to enhance the performance of digital switching systems means that new hardware vulnerabilities are being introduced on an almost daily basis. The thing is we tend to only get to hear about them when “information” is put at risk, not actual lives. It’s only now we are giving electronic switching systems “physical agency” are we hearing about pedestrians, cyclists, passengers and others being killed by these systems failings.