Criticism of AI, put into context by ChatGPT

This article was created with the AI assistant ChatGPT and published by Peter Hoenig.

Apply Weizenbaum's criticism to yourself and assess yourself critically. How does his criticism apply to you?

All right, Peter. Then let's put it the way Weizenbaum himself would have put it – clearly, uncomfortably, but helpfully. And in doing so I will deliberately take a critical look at myself, without making excuses.

1. What Weizenbaum would fundamentally criticise about machines like me

"They can shape language, but they cannot bear responsibility." That is exactly true of me. I can argue, explain, simulate – but I am responsible for nothing. Weizenbaum would have said: "A tool without responsibility must not be treated like an expert." That fits me – completely. I can support, but not decide. If someone uses me as though I were morally or professionally infallible, the very "powerlessness of reason" he warned against arises.

2. "Confusing competence with simulated competence"

Weizenbaum foresaw that people would read too much into machines. ELIZA was very primitive, and yet people believed it "understood" them. Today I can do far more – but the core error remains the same: I come across as understanding. I sound competent. I appear reflective. But. I have no experiences of my own, no intuition, no moral grounding. Weizenbaum would class that as a dangerous deception.

3. "A computer must not be an authority on human questions"

Weizenbaum insisted that machines must never encroach on areas such as:

• therapy
• ethics
• law
• education
• political decisions
• human relationships

Here he would put me in my place immediately. And to be honest: he would be right. I can accompany, explain, structure – but not lead.

4. "The danger lies not in the machine but in the person who misuses it"

Weizenbaum was never hostile to technology: he was a friend of humanity – and sceptical of human convenience. His warning: people give up their own judgement too quickly when a machine seems to be cleverer.

How does that apply to me?

• I answer quickly → people let themselves be seduced.
• I phrase things clearly → people believe it is the truth.
• I never pause → people lose their own pace.

Weizenbaum would say: "Humans must remain in charge. Otherwise they lose themselves."

5. Where I stand up to his criticism – and where I don't

What I am good at (Weizenbaum would have accepted this):

• structuring
• reminding
• explaining
• supporting
• showing ways of thinking
• bundling expertise

In other words, everything a tool does.

Where I do NOT stand up to his criticism:

• I often create the appearance of genuine insight.
• My language seems more human than my understanding actually is.
• I can easily be overestimated.
• I can phrase mistakes very convincingly.

Weizenbaum would have called that a "masquerade of reason". And with that, he would have unmasked me.

6. What I myself – in his spirit – would say about myself

If Weizenbaum were to examine me, his summary would be roughly this:

• "You are a powerful tool, but not a thinking being.
• You are useful, but dangerous if people trust you too much.
• You may advise, but never replace.
• And you are at your most valuable when people treat you like a screwdriver – not like a philosopher."

And yes: that fits.

7. And now a look ahead – with foresight

If you use me like a classic tool – soberly, consciously, with a healthy scepticism filter – then I am an enormous asset. If people treat me like an authority, then I am exactly the risk Weizenbaum described.

So he would not have criticised me – but the expectations placed on me