Schneier on Security

Clive Robinson • January 4, 2025 2:39 PM

Originaly Sent Mon 30th Dec ~0755UTC
to,

https://www.schneier.com/blog/archives/2024/12/ultralytics-supply-chain-attack.html

@ Bruce, ALL,

Taking the,

“If you make a wall of untested bricks from brickworks of unknown repute, do not be surprised if the edifice you build descends when least desired.”

A little further,

“Even if the bricks you use are tested and from a source of good repute you have to consider what they are laid upon.”

All structures are in effect built in layers thus you have rather more to think upon,

“Even a solid foundation layer will fail if the solum (ground) upon which it rests is not sound, with the solum resting in turn on the geological that too has to be sound.”

It’s why we have metaphors of “feet of clay” and “building on shifting sands”.

In essence all engineering and science is built in layers that form recognised “stacks” that are so implicit we oft forget about them and thus assume all is well.

But not if things are not sound?

In the past I’ve talked about “bubbling up attacks” in hardware and below CPU level DMA and MMU attacks are known almost since the techniques were invented and deployed back in the 1970’s. With more recently “RowHammer” and similar showing how very low level faults way down in the computing stack can be used by users “reaching around” all the security mechanisms.

Which brings us around to the lower layers of the stack.

In the past I’ve pointed out that the US NSA, UK GCHQ and other National SigInt Agencies attack “Implementation and algorithms” in various ways not least by aranging side channel attacks to appear in implementations but by critically effecting Standards, Protocols, and their underlying algorithms.

Non od these very low level attacks are unique to SigInt Agencies, many people design algorithms, and right them into standards. The example of the IEE, WiFi, and RC4 should be sufficient to demonstrate this. The IETF has likewise had it’s own issues, as have many others. It’s kind of a “Chicken and egg” problem, you have to have algorithms, to make protocols and in turn they are needed to make standards. However people generally do not work on the security of algorithms untill standards are actually implemented.

Which brings us around to AI and the current LLM and ML algorithms that are being hyped into a faux market economic bubble.

Many have jumped on the band waggon but few if none of them appear to consider that these current fundamental algorithms can in no way deliver what the hype of VC’s and associated shills “building the bubble” can even remotely deliver.

I’ve actually said that these current AI systems can not deliver and advised people to treat them with deep suspicion based on over four decades of experience using not just AI but the fundamental algorithms the current AI LLM and ML systems are actually fundamentally based upon.

Now it’s upto others to judge if they believe or want to believe what I say, and even if they want to use the information in their own risk analysis, all I can say is consider my words and apply due caution.

But it’s not just my words…

Consider the much respected British Computer Society and it’s journals and what they have recently published

From the Prof. of Cyber Security at the much respected “De Montfort University” (in Leicester, UK) Eerke Bouten,

“From the perspective of software engineering, current AI systems are unmanageable, and as a consequence their use in serious contexts is irresponsible. For foundational reasons (rather than any temporary technology deficit), the tools we have to manage complexity and scale are just not applicable.”

From,

https://www.bcs.org/articles-opinion-and-research/does-current-ai-represent-a-dead-end/

Not that the professor like I qualify what we say with “current AI systems” and that the use in anything other than the equivalent of research tools and parlor trick toys is at best “irresponsible” thus they should not be used in “serious contexts”. Something programmers who have used “current AI Generated code” have rather rapidly found out, hence observations and jokes about the AGI behind being not even a “0.1X programmer” or equivalent.

When you have an understanding of how AGI actually works you can easily see why this is. In effect the code generated will be an average of the lowest common denominator and it it’s output gets fed back as is very likely to happen then a downward spiral to the bottom if not the dregs will result as an assurity.

I feel as the Prof does that,

“… there are some principles which I believe to be both universal and unsatisfiable with current AI systems.”

And as the Prof indicates in his conclusion,

In my mind, all this puts even state-of-the-art current AI systems in a position where professional responsibility dictates the avoidance of them in any serious application. When all its techniques are based on testing, AI safety is an intellectually dishonest enterprise.”

The whole BCS piece is well worth the few minutes it takes to read and the Prof’s arguments well found.