I Saw Something New in San Francisco⁠↗

This applies more personally, too: I know people who have been keeping a journal for years and now upload it into any new A.I. system they use. The journal has become, for them, not just a place to pour out their innermost thoughts, but a convenient package of context that can be used to make themselves known to new systems, and thus make the systems more useful to them. But that of course changes how they write in those journals: What was once private now has a reader.

Behind this drive is an experience of A.I. that many casual users have not yet had. An A.I. without deep knowledge of you is an upgrade, perhaps, over Google search. An A.I. with deep knowledge of you feels like something else entirely. I have heard people talk about their A.I.s in terms that bring to mind the daemons from Philip Pullman’s “His Dark Materials” trilogy: They become companions that know you deeply, that you feel safe telling things you’d never tell another person, that become a separate self that nevertheless feels like a part of your own self. That this sounds strange and disquieting does not mean it is not happening.

But cognitive surrender is clearly real, and with it will come the atrophy of certain skills and capacities, or the absence of their development in the first place. The work I am doing now, struggling through yet another draft of this essay, is the work that deepens my thinking for later.

It is, to steal one more McLuhanism, “the numb stance of the technological idiot” to treat A.I. as merely a tool waiting passively for our use. To use A.I. deeply is to engage in a process, not just to push a button. It will reshape us; it already is. We have to be attentive to how.

What makes A.I. truly persuasive isn’t that it praises our ideas or insights, it’s that it restates and extends them in a more compelling form than we initially offered, and does so while reflecting a polished image of ourselves back at us.