This is a great article. I never thought much about the accessibility aspect of Ai and how it can give rise to voices we may not be as likely to hear. Seems important these days, especially in the bubble of academia where new info often fails to permeate the discourse… Also, when I was a grad student (100 years ago) at the New School, lugging the books from Bobst library wasn’t ideal ;-)
Very interesting! Love this kind of first-principles thinking, and it's especially cool when it leads to unbundling concepts we implicitly treated as the same thing, but only because historically there happened to always be a correlation between them. In times like these of dramatic change of circumstances, it's important to do this kind of analysis of what still makes sense.
A point that worries me about this new world, I think, is that the deep skill that remains once we strip away the superficial things that used to be attached to it — which is, as you wrote, something in the realm of judgment/taste/understanding/depth of knowledge — isn't very tangible. It's certainly real, and it's what actually matters in the big picture, but it's hard for a person to measure it even in themselves (maybe even in some sense *especially* in themselves — AI makes all of us much more prone to Dunning-Kruger).
Like, it was always hard to distinguish statements that are actually true from ones that are merely plausible, but there were proxies — for instance, checking the speaker's credentials, or even the way they speak or write, which on one hand is ad hominem but on the other hand is also a kind of chain of trust; two things might seem equally plausible, but if you know one comes from someone with proven experience and thinking — even if you can't observe that directly — that's a convenient shortcut for raising the probability it's correct.
It's well known that charlatans have always been the people who succeed at sounding authoritative, at passing the pattern-matching for people with expertise and confidence, and at mimicking that style — and we're very bad at telling the difference. So in a sense, we've now lowered the barrier to entry for charlatanism.
And as you wrote, if it's easier to produce an opinion that's hard to *know* is correct, but not necessarily more likely to actually *be* correct, then that creates a lot of noise — "the ratio will be dismal."
And in that context, I'm not sure whether "cheap production, expensive judgment" is actually an efficient model, practically speaking. That is, whether filtering, top-down, from within the space of "everything that could be said" is more efficient than going bottom-up and generating claims from scratch.
Beyond all that, personally, as someone who has to read a ton of AI-written content daily, I just suffer from the prose style it produces. I can't even fully define it, because it goes well beyond the repetitiveness of clichés like em dashes or "It's not X, it's Y" — it's something about the overly "uniform texture" of the writing, or the way every paragraph and section carries the same "weight." Which becomes exhausting when you have to read paragraph after paragraph after paragraph written that way.
This also connects to the fact that the writing simply tends to be much longer (not in terms of micro-level phrasing, which sometimes actually gets tighter, but in terms of how much content gets packed in at the macro level). It creates a kind of inversion, where the effort/cost required to read becomes much more expensive than to write/edit, so nothing forces writers to make the hard choices about what to leave out — even things that are valuable but not fully critical are still better cut for reasons of economy.
Although one could argue that in many kinds of writing this was always a problem. "I would have written a shorter letter, but did not have the time," etc.
By the way, another point you mentioned that I thought was very interesting was why we like eureka stories like that. I think you're touching on something true — that the fact the path of discovery was very non-traditional, and didn't come from orderly academic work, somehow makes it more... accessible, or exceptional, or something. It would be interesting as a topic in itself to understand that further.
And one more point toward the end, which remains partly an open question: what traits do we actually want to reward, in an academia that is in part indeed a race. And not just any race, but one where, to a large extent, whoever wins first place in the early heats also gets to start the next heats ahead of everyone else; presumably, being accepted into/getting accredited by prestigious institutions is a huge part of what not only predicts but actually determines success later on the path — even when this happens arbitrarily among people whose traits/starting point are completely identical.
In a sense this is just the question you address explicitly: if we practically want to maximize the advancement of knowledge/human progress as a whole, then which people have the traits such that, if positioned in the prestigious spots and given the limited resources (like public attention), would achieve the best results (of course, given a world where all these AI tools etc. exist). But I wonder whether there's also a question embedded here of: if we want to reward merit, then what actually is merit...
Thanks, Tom. I think we are actually very close on most of this, and several of the things you raise are worries I tried to build into the argument.
On the first point, I agree entirely that judgment is much harder to observe than the proxies we traditionally used for it. Credentials, fluency, writing quality, command of the literature, etc. were useful partly because they were expensive signals of capacities we could not inspect directly. AI has made some of the observable signals cheaper without making judgment, understanding or actual expertise correspondingly cheap. That is basically the problem I discuss in “The bundle” and “What a credential was pricing.”
Your charlatan point is also right, but I think it is already partly internal to the argument. The distinction I draw in “Kuhn, expertise and apprenticeship” is between having the map and having the tacit judgment acquired by actually living inside a field. AI can give an intelligent outsider the vocabulary, the arguments, the schools of thought and the standard objections remarkably quickly. It cannot thereby give him the accumulated pattern recognition that tells the expert, sometimes before he can even articulate why, that something is wrong. As I put it there, giving someone the map without the judgment can produce somebody who knows how to formulate questions in the right vocabulary but cannot tell which questions are worth asking.
And yes, that makes charlatanism cheaper. I think that is a genuine cost. An articulate person can also create a remarkably persuasive appearance of expertise in person, which is why I make the same concession later in “The return of the oral defense.” I explicitly reject the idea that oral examination somehow solves the proxy problem. It merely gives us a proxy better fitted to the present problem.
I would clarify one thing about “cheap production, expensive judgment,” though. I am not proposing a model in which we should generate the entire space of possible claims and then filter it from the top down. I am describing where scarcity moves when production becomes cheap. If anyone can produce fifty plausible objections in thirty seconds, the scarce task becomes deciding which one deserves five minutes. Cheap critique can bury serious work just as effectively as the absence of critique. Evaluation becomes the bottleneck. That pessimistic consequence is actually the point of that section.
Your writer-reader inversion strikes me as an excellent additional example of exactly that problem. If AI saves the writer ten minutes by producing another three reasonably good paragraphs, but a thousand readers each spend two minutes deciding that those paragraphs were unnecessary, the cost has not vanished but has simply moved from producer to consumer. And because the marginal cost of another paragraph is now so low, one of the old incentives for intellectual discipline, the cost of actually writing the thing, has weakened considerably.
I also know exactly what you mean by the “uniform texture.” It is a better description than most lists of AI tells. There is often a strange equality of emphasis, like every qualification gets its symmetrical answer, and every section seems to deserve roughly the same rhetorical weight. It's pretty exhausting. However, does that really matter to point at hand? Should we not be willing to sacrifice comfort for a higher probability at enhanced human innovation, knowledge, scientific progress?
On the eureka stories, yes, I think there is another essay hiding there. In this piece I use them mainly for the distinction between discovery and justification. The route by which an idea occurred to somebody is different from the grounds on which we should accept it. But your psychological question is different and interesting: why do we positively enjoy stories in which discovery escaped the approved route? Perhaps because contingency makes intellectual discovery feel less like the inevitable output of an institution and more like something that could have happened to someone outside it?..
It remains, to me, an open question. I'll try to think about it more, and maybe write about it in the future.
Your last point is the one I think genuinely pushes the argument further. I think we need to first separate several things that universities have historically bundled together. A doctorate is a contribution to knowledge, a training process, a personal achievement and a competitive credential. Those produce different answers to the AI question.
If our question is instrumental, “Who should receive the laboratory, grant, journal space, academic position or public attention if we want the greatest contribution to knowledge?”, then I think the answer is relatively straightforward. Give scarce resources to whoever is most likely to use them well under the technological conditions that actually exist.
But "who *deserves* the reward?" is subtly different.
Suppose one scholar spends five years acquiring a capacity another person, using better tools, acquires sufficiently in five months, and the second person then produces better work. From the standpoint of advancing knowledge, I see very little reason to reward the first person merely because his route was harder. But if we ask who deserves greater recognition as a personal achievement, intuitions about effort, sacrifice and difficulty suddenly reappear.
And that may be one more thing we have been bundling without noticing. “Who will produce the best result?”, “Who possesses the competence we want certified?”, “Who performed the greater achievement?” and “Who deserves the prize?” are not necessarily the same question.
I end the piece very close to this distinction by asking whether the difficulty we are defending mattered because it actually produced knowledge and judgment, or because difficulty gave us a convenient way of deciding who deserved to be called an expert.
Your formulation (if I understand it correctly) pushes it further to something like 'once those two come apart, what exactly do we mean by merit?'
To that, I'm afraid I cannot offer a complete answer at present.
This is a great article. I never thought much about the accessibility aspect of Ai and how it can give rise to voices we may not be as likely to hear. Seems important these days, especially in the bubble of academia where new info often fails to permeate the discourse… Also, when I was a grad student (100 years ago) at the New School, lugging the books from Bobst library wasn’t ideal ;-)
Thank you, Joanne.
Very interesting! Love this kind of first-principles thinking, and it's especially cool when it leads to unbundling concepts we implicitly treated as the same thing, but only because historically there happened to always be a correlation between them. In times like these of dramatic change of circumstances, it's important to do this kind of analysis of what still makes sense.
A point that worries me about this new world, I think, is that the deep skill that remains once we strip away the superficial things that used to be attached to it — which is, as you wrote, something in the realm of judgment/taste/understanding/depth of knowledge — isn't very tangible. It's certainly real, and it's what actually matters in the big picture, but it's hard for a person to measure it even in themselves (maybe even in some sense *especially* in themselves — AI makes all of us much more prone to Dunning-Kruger).
Like, it was always hard to distinguish statements that are actually true from ones that are merely plausible, but there were proxies — for instance, checking the speaker's credentials, or even the way they speak or write, which on one hand is ad hominem but on the other hand is also a kind of chain of trust; two things might seem equally plausible, but if you know one comes from someone with proven experience and thinking — even if you can't observe that directly — that's a convenient shortcut for raising the probability it's correct.
It's well known that charlatans have always been the people who succeed at sounding authoritative, at passing the pattern-matching for people with expertise and confidence, and at mimicking that style — and we're very bad at telling the difference. So in a sense, we've now lowered the barrier to entry for charlatanism.
And as you wrote, if it's easier to produce an opinion that's hard to *know* is correct, but not necessarily more likely to actually *be* correct, then that creates a lot of noise — "the ratio will be dismal."
And in that context, I'm not sure whether "cheap production, expensive judgment" is actually an efficient model, practically speaking. That is, whether filtering, top-down, from within the space of "everything that could be said" is more efficient than going bottom-up and generating claims from scratch.
Beyond all that, personally, as someone who has to read a ton of AI-written content daily, I just suffer from the prose style it produces. I can't even fully define it, because it goes well beyond the repetitiveness of clichés like em dashes or "It's not X, it's Y" — it's something about the overly "uniform texture" of the writing, or the way every paragraph and section carries the same "weight." Which becomes exhausting when you have to read paragraph after paragraph after paragraph written that way.
This also connects to the fact that the writing simply tends to be much longer (not in terms of micro-level phrasing, which sometimes actually gets tighter, but in terms of how much content gets packed in at the macro level). It creates a kind of inversion, where the effort/cost required to read becomes much more expensive than to write/edit, so nothing forces writers to make the hard choices about what to leave out — even things that are valuable but not fully critical are still better cut for reasons of economy.
Although one could argue that in many kinds of writing this was always a problem. "I would have written a shorter letter, but did not have the time," etc.
By the way, another point you mentioned that I thought was very interesting was why we like eureka stories like that. I think you're touching on something true — that the fact the path of discovery was very non-traditional, and didn't come from orderly academic work, somehow makes it more... accessible, or exceptional, or something. It would be interesting as a topic in itself to understand that further.
And one more point toward the end, which remains partly an open question: what traits do we actually want to reward, in an academia that is in part indeed a race. And not just any race, but one where, to a large extent, whoever wins first place in the early heats also gets to start the next heats ahead of everyone else; presumably, being accepted into/getting accredited by prestigious institutions is a huge part of what not only predicts but actually determines success later on the path — even when this happens arbitrarily among people whose traits/starting point are completely identical.
In a sense this is just the question you address explicitly: if we practically want to maximize the advancement of knowledge/human progress as a whole, then which people have the traits such that, if positioned in the prestigious spots and given the limited resources (like public attention), would achieve the best results (of course, given a world where all these AI tools etc. exist). But I wonder whether there's also a question embedded here of: if we want to reward merit, then what actually is merit...
---
(Translated by AI 🙂)
Thanks, Tom. I think we are actually very close on most of this, and several of the things you raise are worries I tried to build into the argument.
On the first point, I agree entirely that judgment is much harder to observe than the proxies we traditionally used for it. Credentials, fluency, writing quality, command of the literature, etc. were useful partly because they were expensive signals of capacities we could not inspect directly. AI has made some of the observable signals cheaper without making judgment, understanding or actual expertise correspondingly cheap. That is basically the problem I discuss in “The bundle” and “What a credential was pricing.”
Your charlatan point is also right, but I think it is already partly internal to the argument. The distinction I draw in “Kuhn, expertise and apprenticeship” is between having the map and having the tacit judgment acquired by actually living inside a field. AI can give an intelligent outsider the vocabulary, the arguments, the schools of thought and the standard objections remarkably quickly. It cannot thereby give him the accumulated pattern recognition that tells the expert, sometimes before he can even articulate why, that something is wrong. As I put it there, giving someone the map without the judgment can produce somebody who knows how to formulate questions in the right vocabulary but cannot tell which questions are worth asking.
And yes, that makes charlatanism cheaper. I think that is a genuine cost. An articulate person can also create a remarkably persuasive appearance of expertise in person, which is why I make the same concession later in “The return of the oral defense.” I explicitly reject the idea that oral examination somehow solves the proxy problem. It merely gives us a proxy better fitted to the present problem.
I would clarify one thing about “cheap production, expensive judgment,” though. I am not proposing a model in which we should generate the entire space of possible claims and then filter it from the top down. I am describing where scarcity moves when production becomes cheap. If anyone can produce fifty plausible objections in thirty seconds, the scarce task becomes deciding which one deserves five minutes. Cheap critique can bury serious work just as effectively as the absence of critique. Evaluation becomes the bottleneck. That pessimistic consequence is actually the point of that section.
Your writer-reader inversion strikes me as an excellent additional example of exactly that problem. If AI saves the writer ten minutes by producing another three reasonably good paragraphs, but a thousand readers each spend two minutes deciding that those paragraphs were unnecessary, the cost has not vanished but has simply moved from producer to consumer. And because the marginal cost of another paragraph is now so low, one of the old incentives for intellectual discipline, the cost of actually writing the thing, has weakened considerably.
I also know exactly what you mean by the “uniform texture.” It is a better description than most lists of AI tells. There is often a strange equality of emphasis, like every qualification gets its symmetrical answer, and every section seems to deserve roughly the same rhetorical weight. It's pretty exhausting. However, does that really matter to point at hand? Should we not be willing to sacrifice comfort for a higher probability at enhanced human innovation, knowledge, scientific progress?
On the eureka stories, yes, I think there is another essay hiding there. In this piece I use them mainly for the distinction between discovery and justification. The route by which an idea occurred to somebody is different from the grounds on which we should accept it. But your psychological question is different and interesting: why do we positively enjoy stories in which discovery escaped the approved route? Perhaps because contingency makes intellectual discovery feel less like the inevitable output of an institution and more like something that could have happened to someone outside it?..
It remains, to me, an open question. I'll try to think about it more, and maybe write about it in the future.
Your last point is the one I think genuinely pushes the argument further. I think we need to first separate several things that universities have historically bundled together. A doctorate is a contribution to knowledge, a training process, a personal achievement and a competitive credential. Those produce different answers to the AI question.
If our question is instrumental, “Who should receive the laboratory, grant, journal space, academic position or public attention if we want the greatest contribution to knowledge?”, then I think the answer is relatively straightforward. Give scarce resources to whoever is most likely to use them well under the technological conditions that actually exist.
But "who *deserves* the reward?" is subtly different.
Suppose one scholar spends five years acquiring a capacity another person, using better tools, acquires sufficiently in five months, and the second person then produces better work. From the standpoint of advancing knowledge, I see very little reason to reward the first person merely because his route was harder. But if we ask who deserves greater recognition as a personal achievement, intuitions about effort, sacrifice and difficulty suddenly reappear.
And that may be one more thing we have been bundling without noticing. “Who will produce the best result?”, “Who possesses the competence we want certified?”, “Who performed the greater achievement?” and “Who deserves the prize?” are not necessarily the same question.
I end the piece very close to this distinction by asking whether the difficulty we are defending mattered because it actually produced knowledge and judgment, or because difficulty gave us a convenient way of deciding who deserved to be called an expert.
Your formulation (if I understand it correctly) pushes it further to something like 'once those two come apart, what exactly do we mean by merit?'
To that, I'm afraid I cannot offer a complete answer at present.