The Siren Call of Silicon Leviathan: Reflections on blowup and Aufklärungsdämmerung

Abstract

Over three centuries ago our Enlightenment forefathers brought forth in Europe--and later on this continent--a new approach to knowledge production and dissemination, conceived in distrust of authority, and dedicated to the proposition that truth is accessible to human reason. Mathematics, the Enlightenment's firstborn, was its existence proof--knowledge that compels assent without appeal to authority--because the only way to make a theorem certain was to show it to a human mind. Now we are engaged in an existential struggle testing whether mathematics, or any discipline so conceived and so dedicated, can long endure. We are met in the wake of the OpenAI press release: 166 pages, 616,000 lines of Lean, produced in 88 hours by some 10,000 software agents, and read in full--at the time of this Siren Call--by no human being. A proof has historically been two things: a certificate that a theorem is true, and a demonstration that lets another person see why. OpenAI delivered the first without the second. But, in a larger sense, what matters is not what the Silicon Leviathan (OpenAI and its peer competitors) produces but rather what we accept. By accepting the certificate without the demonstration, we adopt the Hobbesian bargain: authority, not truth, makes the law. The age that this augurs is--in Panofsky's phrase--a Middle Ages in reverse--its authority a subhuman superintelligence. The great task before us is to frame a covenant under which what has become optional--understanding--remains required: that nothing counts as mathematics until a human being has understood it--and can show the next person why. Whether we will safeguard and keep our sovereign inheritance--decided at the moment of acceptance--which is not yet past--will determine if this Aufklärungsdämmerung spells the Enlightenment's end.