faking versus making

Something came to mind while reading this excellent essay by Tom Rachman, passed on by Conor Griffin. It’s not directly relevant to Tom’s specific subject, the role of human writing in the age of AI, but I think it’s in an area that sort of rhymes, and which is very important to me because it’s relevant to the subject of my next book, the one I guess I’m writing now, with short breaks to revise the manuscript I sent to my editor on Monday. (cheers cheers thank you).

Twenty years ago, a sociologist of science called Harry Collins tried out an experiment. He had been researching the community of physicists who worked on gravity waves for the preceding two decades, trying to get insight into standards of proof and how experimental results were reconciled and interpreted. In furtherance of this project, he had been attending gravity waves conferences, where he was a familiar figure and had lots of friends in the community (because he talked to lots of people, physicists often sought him out to get gossip and news on what other labs were doing and seeing).

The experiment was a sort of Turing test – one group of physicists came up with a list of half a dozen questions about gravity waves, and then sent them by email to both Collins and a physicist, with instructions just to answer them from general knowledge rather than looking anything up. A jury of other physicists were then given the job of deciding which set of answers had come from Collins, and which from the physicist.

And all but one of the judges identified the sociologist as the physicist.

Although extremely funny, this wasn’t actually meant as a practical joke; it was meant to test a serious hypothesis about expertise. Harry Collins theorised that there was a thing called “interactional expertise” (basically the ability to hold a sensible and useful conversation about a subject) which was distinct from “contributory expertise” (he didn’t actually even know enough maths to do gravity wave physics). Deep interaction with a knowledge base and literature could provide one, but it wasn’t the same thing as actually working within the field.

When I wrote this up at the time, I remember thinking – was there really a clear dividing line? Like, for example, if someone really tried and dedicated a lot of time and effort to it, might they be able to pass an even tougher version of the same test. For example, could you get someone to be a co-author on a published journal article, based on nothing more than interactional expertise? I think the answer might be “trivially obviously so if it was just a literature review, and quite possibly they could be seen as a helpful and valuable co-author even on quite substantial original research”.

Or in fact – how does transactional expertise of the kind that a sociologist of science can put together by attending conferences – how does this differ from the understanding of the cutting edge of research which might be possessed by a principal investigator who runs a lab, makes decisions about hiring and such but who hasn’t actively carried out an experiment for the last decade? If someone like that hires a professional grant application writer to describe their current research program, what is the level of expertise that’s gone into that piece of writing?

You can see where I’m going here, can’t you, particularly since I gave so many clues on Wednesday. Because it seems to me that there is an important distinction here, which is not any less important because the dividing line might be difficult to establish empirically, or even if that line turns out to be in a different place from where we guessed it was. As well as difficult cases where it’s not clear, I think we could also come up with cases where the distinction between interactional and contributory expertise would suddenly become very clear and important indeed – the ones where someone who was faking it got “found out”.

And so the question that I think is quite important is whether there is a similar kind of distinction between the kind of expertise that it’s possible for a machine[1] to get by industralised consumption and interaction with a much larger corpus of literature than any human being could inhale, and genuine contributory expertise that could apply to entirely new situations outside that literature.

I think there’s a few contradictory intuitions here, and I’ll put down a marker that I would probably guess that the answer would be different in mathematics, logic and similar fields where there’s an axiomatic path to extrapolate from things you’ve already read to interesting and important theorems and sentences that haven’t yet been uttered. At present, I think it looks like the very impressive results of LLMs in mathematics seem to resemble Thomas Royen’s proof of the Gaussian correlation inequality – the sudden making of a connection which allows interaction with one part of the literature to open up something that’s always been implicit in another part.

But if we start thinking about other fields where progress isn’t monotonic; where things can be disproved, authorities disagree with one another and results can be true at one time and false at another … as we move along that spectrum I think something like the interactional/contributory distinction becomes more important.I am still not sure whether it’s a sharp distinction between two different things – and I’ll be writing more about that when an important paper gets published next week that I will be citing a lot – but I think it matters.

[1] To continue with the project of “sensible skepticism”, I should say that there’s two questions – whether it’s possible in principle for any machine at all, and whether it’s possible for any machine which can be built subject to reasonable resource and economic constraints in the foreseeable future.