Ted Chiang on Artificial Intelligence

Grant OtsukiLeave a comment

Ted Chiang is a science fiction author who wrote the story that became Arrival (via Metafilter). Recently in The New Yorker:


How much can you optimize for generality? To what extent can you simultaneously optimize a system for every possible situation, including situations never encountered before? Presumably, some improvement is possible, but the idea of an intelligence explosion implies that there is essentially no limit to the extent of optimization that can be achieved. This is a very strong claim. If someone is asserting that infinite optimization for generality is possible, I’d like to see some arguments besides citing examples of optimization for specialized tasks.

With Ezra Klein for the New York Times:

But another aspect in which [superheroes] can be problematic is, how is it that these special individuals are using their power? Because one of the things that I’m always interested in, when thinking about stories, is, is a story about reinforcing the status quo, or is it about overturning the status quo? And most of the most popular superhero stories, they are always about maintaining the status quo. Superheroes, they supposedly stand for justice. They further the cause of justice. But they always stick to your very limited idea of what constitutes a crime, basically the government idea of what constitutes a crime.

In the same vein, the very idea of an internally self-consistent “cinematic universe,” and, more generally of a fictional “canon,” tends to be politically conservative (as most canons tend to be.) They give rise to strong guardians of the canon. Such universes also naturalize time as “big H History,” that tends to be self-limiting in terms of imagination.

I tend to think that most fears about A.I. are best understood as fears about capitalism. And I think that this is actually true of most fears of technology, too. Most of our fears or anxieties about technology are best understood as fears or anxiety about how capitalism will use technology against us. And technology and capitalism have been so closely intertwined that it’s hard to distinguish the two.

(Previously on AI and capitalism and Marvel superheroes.)

GPT-Neo: An open source alternative to GPT-3

Grant OtsukiLeave a comment

Access to GPT-3 remains limited, but “EleutherAI,” a “grassroots collective of researchers working to open source AI research,” has released GPT-Neo:

GPT⁠-⁠Neo is the code name for a family of transformer-based language models loosely styled around the GPT architecture. Our primary goal is to replicate a GPT⁠-⁠3 DaVinci-sized model and open-source it to the public, for free.

How to write a thesis, according to Umberto Eco

Grant OtsukiLeave a comment

Umberto Eco, in How to Write a Thesis (from this excerpt):

The language of the thesis is a metalanguage, that is, a language that speaks of other languages. A psychiatrist who describes the mentally ill does not express himself in the manner of his patients. I am not saying that it is wrong to express oneself in the manner of the so-called mentally ill. In fact, you could reasonably argue that they are the only ones who express themselves the way one should. But here you have two choices: either you do not write a thesis, and you manifest your desire to break with tradition by refusing to earn your degree, perhaps learning to play the guitar instead; or you write your thesis, but then you must explain to everyone why the language of the mentally ill is not a “crazy” language, and to do it you must use a metalanguage intelligible to all.
[…]
And when Marx wanted to talk about workers, he did not write as a worker of his time, but as a philosopher. Then, when he wrote The Communist Manifesto with Engels in 1848, he used a fragmented journalistic style that was provocative and quite effective. Yet again, The Communist Manifesto is not written in the style of Capital, a text addressed to economists and politicians. Do not pretend to be Dante by saying that the poetic fury “dictates deep within,” and that you cannot surrender to the flat and pedestrian metalanguage of literary criticism. Are you a poet? Then do not pursue a university degree. Twentieth-century Italian poet Eugenio Montale does not have a degree, and he is a great poet nonetheless. His contemporary Carlo Emilio Gadda (who held a degree in engineering) wrote fiction in a unique style, full of dialects and stylistic idiosyncrasies; but when he wrote a manual for radio news writers, he wrote a clever, sharp, and lucid “recipe book” full of clear and accessible prose. And when Montale writes a critical article, he writes so that all can understand him, including those who do not understand his poems.

What Eco writes here seems both current and dated. It is dated because the idea that a thesis must use “a metalanguage intelligible to all” is not a given fact in today’s university, where discursive pluralism is the reality. Its currency is in the idea that a thesis and any other research publication has to strive to connect the inside of a text, community, culture, or era with an outside of diverse readers. Fundamentally, a thesis is not about understanding the inside, but about knowing both the inside and the outside and being able to move between the two. The sociologist of science Harry Collins called this ability “meta-alternation.” Meta-alternation is about knowing science and knowing sociology, and being able to move back and forth between them. The more common expression “hermeneutic circle” (“the idea that one’s understanding of the text as a whole is established by reference to the individual parts and one’s understanding of each individual part by reference to the whole”) refers to something similar.

Even in the pluralist condition, some idea of a universal metalanguage remains as an ideal and as practice (as Hsu implies below.) Eco expresses this by saying that a thesis is “not simply a private letter to the advisor, it is potentially a book meant for humanity.” (147) This flags something very important: perhaps even more than its choice of subject, a thesis necessarily expresses an idea of who matters in its choice of addressee and mode of address. For some students, only the advisor will matter. For others, only themselves. For most, I hope, what matters will be something more expansive. I take what Hsu says below as expressing this sentiment, but adding that in the minutiae of academic writing lies an ideal of community. (Two important corollaries: one is that if one wants to change the community, then one must change the minutiae of its modes of communication. The other is that a person who only ever writes in one mode possesses an impoverished view of humanity.)

Hua Hsu at the New Yorker:

But “How to Write a Thesis” is ultimately about much more than the leisurely pursuits of college students. Writing and research manuals such as “The Elements of Style,” “The Craft of Research,” and Turabian offer a vision of our best selves. They are exacting and exhaustive, full of protocols and standards that might seem pretentious, even strange. Acknowledging these rules, Eco would argue, allows the average person entry into a veritable universe of argument and discussion. “How to Write a Thesis,” then, isn’t just about fulfilling a degree requirement. It’s also about engaging difference and attempting a project that is seemingly impossible, humbly reckoning with “the knowledge that anyone can teach us something.” It models a kind of self-actualization, a belief in the integrity of one’s own voice.

Reviews with interesting insights from the perspectives of teachers and students at Inside Higher Ed, and LSE Review of Books.

That time Norbert Wiener visited Attica Prison

Grant OtsukiLeave a comment

One of the pleasures of doing archival work is of discovering something about the people you are studying that gives insights into who they were as people, that aren’t big enough to get into the published literature.

During my time studying Norbert Wiener’s papers at MIT’s institute archives, I came across some letters that Wiener had exchanged with Frank J. Scimone, prisoner #1158, who was serving time for murder in Attica Prison, a maximum security facility in upstate New York.

Scimone’s first letter to Wiener was dated September 19, 1938, a decade before Wiener would publish his book on cybernetics. Scimone had read about Wiener’s mathematical work in the New York Times, and was compelled to write Wiener to express his own interest in mathematics. This was accompanied with a request for study materials.

[M]ight I prevail upon you to send me something on this new science in the field of Mathematics that deals with “Chaos”?

The next letter from Scimone was from September 25, 1938. In the interim, Wiener seems to have replied to Scimone with enthusiasm (in a letter I did not see, but which is reproduced here) writing,

In the meantime I should like to hear something more about the scope of your reading in Mathematics so that I may be able to suggest and possibly put at your disposal the necessary introductory material.

In an awkward attempt to reassure Mr. Scimone of his seriousness, Wiener closed that letter by writing:

While I have not the facts at hand, I have heard of several cases where war prisoners in concentration camps carried on important research under conditions of confinement, therefore, I am taking your interest in mathematics very seriously and should like to hear something about your studies and to give you any advice in my power as to profitable ways of continuing.

Buoyed by Wiener’s response, Scimone quickly wrote back with information about his studies, and asks for further direction. A few days later (30 September) Wiener responded, saying that he had discussed things with Norman Levinson, one of Wiener’s Ph.D. students and colleagues, and that they had identified a problem area they wanted Scimone to work on. This came with a gift of books.

The following year, Scimone updated Wiener on his progress, and this time Levinson responded with more direction and more books.

Two years after that, Wiener went to the prison to visit Scimone. In a letter dated November 6, 1941, Wiener wrote the chief clerk of Attica Prison, thanking him for the courtesy shown to him during his visit. An accompanying letter was for Scimone, where Wiener described some problems to work on, with the reassurance that “I am giving you something that I really could use and that is not merely to make you work.”

Scattered throughout these letters are details of Scimone’s life. As of 1938, he was 29-years-old and had been in prison for seven years. Scimone says that he attended NYU for a year and a half, but was convicted for murder in 1931 and given a life sentence. In prison, he continued studying math, listing the subjects and books he has studied. He also taught math in the prison school. Scimone speaks French, Spanish, and Italian fluently. Scimone also mentions a brother doing graduate studies in bacteriology at Columbia. In the prison, he earned 10 cents a day, which he was saving to buy books.

In all, I came across nine letters to or from Frank J. Scimone: 5 from Scimone to Wiener, 2 from Wiener to Scimone (with a reference to one more), one from Scimone to Levinson, one from Levinson to Simone. This correspondence began in 1938, and appears to have ended soon after Wiener’s visit to the prison in 1941. There may be other Scimone letters in the MIT collection, but it does not seem likely that they communicated much after that, if at all.

I searched online for “Frank J. Scimone” and variations, and found little except for this reference to a Frank Scimone, born October 25, 1909, died in Brooklyn, on March 22 1993 at the age of 83, which matches the age Scimone mentions in his letter. If he died in Brooklyn, then it seems that Mr. Scimone was able to have his life sentence shortened.

Scimone rarely appears in the voluminous writings about Wiener. Wiener’s student and biographer, Pesi R. Masani, mentions Wiener’s correspondence with Scimone as an illustration of the care that Wiener could show to the people who reached out to him. Conway & Siegelman’s biography of Wiener, Dark Hero of the Information Age, does not refer to Scimone at all. Wiener does not mention Scimone in his autobiographies, to the best of my recollection.

Scimone is almost literally a historical footnote. But I wonder how he must have felt to be exchanging letters and discussing math with a person who would become world famous not long after. One day, he wrote a letter to Wiener, probably without any expectation of a reply. In return, he received dozens of paper reprints and books, guidance in his studies, and the chance to welcome Wiener to his maximum security prison.

AI writes passable college papers

Grant OtsukiLeave a comment

A post by Eduref, a company that deals in information about postsecondary education, has gotten some attention for doing a Turing Test for course papers produced by the AI language model, GPT-3.

We hired a panel of professors to create a writing prompt, gave it to a group of recent grads and undergraduate-level writers, and fed it to GPT-3, and had the panel grade the anonymous submissions and complete a follow up survey for thoughts about the writers. AI may not be at world-dominance level yet, but can the latest artificial intelligence get straight A’s in college?

As the saying goes, “C’s get degrees.” Straight A’s in college, however, are far from common, and with AI being far from perfect, GPT-3 performed in line with our freelance writers. While human writers earned a B and D on their research methods paper on COVID-19 vaccine efficacy, GPT-3 earned a solid C. Performing a bit better in U.S. History, humans received a B and C+ on their American exceptionalism paper, while GPT-3 landed directly in the middle with a B-. Even when it came to writing a policy memo for a law class, GPT-3 passed the assignment with a B-, with only one of three students earning a higher grade.

(More coverage at ZDNet and Inside Higher Education.)

Last year, I wrote something for The Conversation, based on my experience of using GPT-2, an earlier version of the same language model, to produce papers for an anthropology course. At the time, GPT-2 was close but not quite close enough.

I concluded:

While computer writing might never be as original, provocative, or insightful as the work of a skilled human, it will quickly become good enough for such writing jobs, and AIs won’t need health insurance or holidays. 
If we teach students to write things a computer can, then we’re training them for jobs a computer can do, for cheaper. 
Educators need to think creatively about the skills we give our students. In this context, we can treat AI as an enemy, or we can embrace it as a partner that helps us learn more, work smarter, and faster.

From all accounts, GPT-3 seems much more capable as is than GPT-2 was. While GPT-3 is not widely available, it won’t be long before it or something like it is. This means we need to rethink what writing assignments are, and what we want them to do.

John Warner, in Inside Higher Ed, suggests a change to how we approach grading is sorely needed:


In this case the problem is in our well-trodden patterns of how we assess student work in the context of school. [GPT-3’s] response is grammatical, it demonstrates some familiarity with the course and it is not wrong in any significant way.

It is also devoid of any signs that a human being wrote it, which, unfortunately does not distinguish it from the kinds of writing students are often asked to do in school contexts, which is rather distressing to consider, but let’s put that aside for the moment.

When confronted with this kind of work, what if we did something differently?

What if we replaced that … sigh … B with a “not complete, try again”?

Because honestly, isn’t that a more appropriate grade than the polite pat on the head that the B signals in this case?

This seems like an opportunity to put something like Labour-Based Grading into wider use.

(Previously (1), (2) on GPT-3.)