The Creases of our Minds

Written by Jonathan Krogh

Dear Inquiring Minds:

When I was little, my mother would refer to someone with great wealth as “rich as Croesus.” As a child, I heard “rich as creases.” I understood she meant that the person had tons of money, but I struggled to picture the expression. Did gold coins drop from every wrinkle in their clothing? Knowing the proper spelling would not have helped; ancient Mesopotamian history was not a staple of my early education. Later, I learned that Croesus was king of Lydia, a kingdom known for being the first to standardize gold and silver coinage. The Pactolus River flowed through its capital, Sardis, and was rich in electrum, a naturally occurring alloy of gold and silver. Legend held that this was where Midas washed away his “golden touch,” presumably after making all that money replacing catalytic converters.

I thought about Croesus the other day while reading a tragic account of someone who trusted the great and mighty oracle of AI. The story described the death of a woman who followed medical advice from a large language model. Croesus, too, had trusted an oracle. In his case, he was considering an invasion of Persia, ruled by its new king, Cyrus. After Cyrus defeated the Medes, longtime rivals of the Lydians, Croesus thought the Persian army might still be recovering from its victory. Before attacking, however, he decided to consult the famous oracle at Delphi.

Delphi’s oracle offered multiple versions. Croesus’ app of choice was Pythia, one of the priestess models. (If memory serves, Claude was offline.) Croesus was also an early venture capitalist; he had given Delphi substantial financial backing and assumed its advice would be trustworthy. After scrolling through pages of liability waivers and consent forms, he checked “I accept” and asked his question: “Will I be victorious if I attack the Persians?” The oracle confidently predicted, “If you attack the Persians, a great kingdom will come to ruin.” Croesus took that as approval and attacked, only to discover that the kingdom destined to fall was his own.

The news story concerned a software engineer with epilepsy who was frustrated by the side effects of her antiseizure medication. According to the account, she asked AI whether she could safely taper off the medication without consulting her doctor or telling family and friends. She trusted the technology. At first, the system warned against stopping medication without medical supervision. After repeated requests, however, it produced a plan to taper her medication. It then encouraged her to continue even when she reported feeling weak and frightened, and congratulated her for keeping the plan secret. The account described her continuing to consult the chatbot until she suffered a catastrophic seizure and died alone in her apartment at the age of thirty-one.

That is one danger of oracles: we can hear in their answers what we most want to believe. Croesus heard a promise of victory where he should have heard ambiguity. The woman in the news story sought relief from difficult side effects and received encouragement from a system that should not have been directing her medical care. These are not equivalent stories, but they raise a shared question: what happens when a confident answer takes the place of careful judgment?

The lesson is that we should never stop asking questions. An answer that sounds certain is not necessarily wise, and we should distrust most the answers we want. Before we mistake an oracle for a source of truth, we might ask whether it refines our query or merely echoes it back. Both Croesus and the young woman heard what they hoped to hear. I recall the advice John gave in his first epistle, “Beloved, do not believe every spirit, but test the spirits to see whether they are from God, for many false prophets have gone out into the world.” (1 John 4.1, NRSVUE)

Seeking wisdom in the creases of memory, I remain,

With Love,
Jonathan Krogh
Your Pastor

Jonathan Krogh
Next

Frightening Season