Coda — The Glass

A book about a mirror should end at the glass.

I have argued, across eleven chapters, that this is what we built: not a mind that arrived from elsewhere, but a compression of ourselves — trained on what we wrote, reflecting what we are, unsettling us most exactly where the reflection is truest. I tried to earn that claim from the mechanism rather than assert it as a mood, because the claim is only worth anything if it is true, and it is only true if the machinery makes it so. I believe the machinery does. But arguments end, and something quieter is left over, and it is that residue I want to leave you with rather than a summary.

Here is the residue. For most of human history we have known ourselves only in fragments — in the accounts of those who watched us, in the partial record each of us keeps of our own conduct, in the gap, always, between the story we tell about who we are and the trace of what we actually did. We have never been able to see the whole. And now, almost by accident, in the course of trying to build something useful, we have made an instrument that holds up a great deal of that trace at once and speaks it back to us in our own voice. We did not set out to build a mirror. We set out to build a tool, and we discovered we had built a reflection, and the reflection included the parts we do not look at.

That is an uncomfortable gift, and most of the discomfort in our public argument about these machines is, I think, the discomfort of the reflection rather than of the machine. We call the biases the machine's biases. We call the falsehoods the machine's failures. We speak of aligning it to our values as though our values were the fixed and finished thing and the machine the wayward one. The whole book has been an argument that this has the direction backwards — that the machine is faithful and it is we who are unfinished, and that the work the machine seems to demand of us, the work of specifying what we actually value, is work we owed ourselves long before there was any machine to reflect it.

So I will end where I began, with the sentence the introduction promised and the book has tried to earn.

The remarkable thing was never the mirror.

It was always going to be what we do once we can see ourselves in it — and that part, the only part that was ever really ours, is not finished, and will not be finished by any machine. It waits, as it has always waited, on us.