One thing it seems to me we need to be careful about when talking about AI detection software is the difference between being told that a piece of writing is "84% written by AI" and being told that there's an "84% probability" it was written by AI.
These are completely different things. The first is telling you how much of a piece (how many of the words) were written by AI (without any indication of the statistical confidence of that statement—which should make anyone suspicious of the analysis). The second is giving you that statistical confidence (presumably that the piece was *substantially* written by AI)—which, all else being equal, would incline me to take that analysis a little more seriously.
I've seen writers protest that an AI analysis accused them (the writer) of being "40% AI", when it was pretty clear that the actual conclusion was that there was a 40% probability that the piece in question was written by AI—in other words, that it most likely *wasn't*.
Very good point. I think people are looking at it with absolutes. They disregard “probability,” and go straight for, “This was written by AI.” The truth is, they can’t know that for certain. If people continue making public accusations without 100% certainty, it will backfire.
Maybe speaking up for my experience will offer some hope: both of my books were put through grueling, extensive legal editing and fact checking. Every claim has a receipt or source or I wasn’t allowed to keep it. I spent so much time with my publisher-provided legal editor that I bonded with her, and I credit her sharp and protective eye as giving me an ultimate layer of confidence to speak about the trauma and accusations in my story with bravery. A Well-Trained Wife is memoir and I Belong to Me is self-help. My publisher is St Martins Press.
I'm still unclear on how publishers (or agents) using another AI program to verify if a writer used AI is any better. Can we trust a third party software to capture all of that data and then use it for other purposes than what it was intended for? It's all very messy and I don't see any quick resolutions.
Would a potential solution be a more proactive one? Rather than trying to figure out afterwards if AI was used, would authors be open to using a program or app that just monitors AI activity (including anything happening in a browser — logging what tools were used, when, and for how long) and send some kind of certificate with the results (which can be signed/timestamped/blockchain-verified so no alterations can be made)? Or could publishing companies/agents ask their authors to use something like this, so it removes any doubt as to whether a manuscript is AI-generated?
This feels like a much cleaner standard, and it protects everyone — authors with nothing to hide, editors who shouldn't have to become AI detectors, and publishers who don’t want to pull books after the fact.
"...the book’s “facts,” if one can call memories retrieved during an MDMA therapy session as such." 😂
Excellent article. Thank you. The trust issue, I feel, has forever been a part of publishing and storytelling, and is now amplified by AI and social media. It used to be scandals (or later finding out, long after some author had died) of appropriating someone else's story or a culture. And outright plagiarism has always been an issue. And passing "memoir" off when it is largely fictionalized. Trust, especially form those presenting the book to the market / audience, is at the root of it. People generally don't like to be lied to. Go figure.
Does someone have a good chance at winning a case of "this memoirist's false memories are actually inspired by actual events that happened to me?" Wild.
Well, the one good thing that I take away from this is that I will give Margo a chance that I might not have otherwise. 😏 The rest of that's pretty depressing. You sharing your perspective is appreciated nonetheless.
Excellent essay. I've written two non-fiction books and, coming from the news business, I was surprised how little editing was done by the editor. (Let alone fact checking lol!) For one book, the Big 5 editor had one big idea -- removing all "I" references -- but otherwise made virtually no line edits in a 100k manuscript. I received more feedback from friends and colleagues who read it, and so I relied on them to set me straight. Some of my friends have had more hands-on editors, but in general I sense that editors take a hands-off, trust-the-author approach. I'm surprised there haven't been more blow-ups from a lack of fact-checking and due diligence.
There’s a real philosophical discussion we all need to have. Right now, as a society and profession we’re whistling past it. Where do we draw the line? On the one side we have the fully AI-generated novel. On the other we have the traditional human-crafted novel. In the middle there are a lot of gradations, though. The human-crafted novel is edited by human editors; they go through drafts with a bunch of feedback. Are we going to accept an AI providing editorial feedback? That's not exactly AI writing but it is AI involvement. And then on the other end of it we've got AI drafting and then human involvement, going towards this muddy middle where it’s just hybrid. Most of us are acting as though this is a light switch that's on or off but it's a far more complicated scenario than I think we are prepared to really deal with.
Thank you for your thoughtful take, as always! One important piece of context for the SHY GIRL situation is that the author has stated that someone used generative AI on her book. As she told the NYT, “an acquaintance she hired to edit the self-published version of the novel had used A.I.” Elsewhere online, Ballard mentioned hiring someone from her writing group to edit the novel and suspecting that they had used AI. So it could be that Hachette is basing their decision on the author's own admission--and/or her failure to disclose the use of AI upfront--rather than on an AI detection program.
I thought about putting that in, but I really wanted the onus to be on the publisher here because they are also responsible. Nothing has been said about the editorial process. At all.
I agree--a thorough editorial process could have made a world of difference. I don't think Ballard's statement changes that responsibility, but a publisher reacting to an author's statements is a different scenario than reacting to an AI checking software. I wrote about this as well, and said, "I can’t help but wonder: what could have been done differently? If Shy Girl had benefited from a longer publishing timeline, would the use of generative AI have been caught before publication? Would the editorial team have been able to notice any inconsistencies across multiple drafts and conversations with the author? Would the author have been given the opportunity to address any concerns and revise the manuscript accordingly? In hindsight, all of these possibilities seem vastly preferable to Shy Girl’s high-profile cancellation—even if a lengthier publication process resulted in some lost sales."
AI detection software is already out of date. It is now almost impossible to tell if something was written using AI because AI has gotten that good. In a year it will be actually impossible to tell. Authors warrant (meaning they promise) by signing a contract, that their work is original and created by them. Very soon, we will not be able to prove otherwise unless the author admits to it.
Really interesting discussion.
One thing it seems to me we need to be careful about when talking about AI detection software is the difference between being told that a piece of writing is "84% written by AI" and being told that there's an "84% probability" it was written by AI.
These are completely different things. The first is telling you how much of a piece (how many of the words) were written by AI (without any indication of the statistical confidence of that statement—which should make anyone suspicious of the analysis). The second is giving you that statistical confidence (presumably that the piece was *substantially* written by AI)—which, all else being equal, would incline me to take that analysis a little more seriously.
I've seen writers protest that an AI analysis accused them (the writer) of being "40% AI", when it was pretty clear that the actual conclusion was that there was a 40% probability that the piece in question was written by AI—in other words, that it most likely *wasn't*.
Very good point. I think people are looking at it with absolutes. They disregard “probability,” and go straight for, “This was written by AI.” The truth is, they can’t know that for certain. If people continue making public accusations without 100% certainty, it will backfire.
I’m sure you’re right. We’re inclined to black-and-white thinking.
Maybe speaking up for my experience will offer some hope: both of my books were put through grueling, extensive legal editing and fact checking. Every claim has a receipt or source or I wasn’t allowed to keep it. I spent so much time with my publisher-provided legal editor that I bonded with her, and I credit her sharp and protective eye as giving me an ultimate layer of confidence to speak about the trauma and accusations in my story with bravery. A Well-Trained Wife is memoir and I Belong to Me is self-help. My publisher is St Martins Press.
I'm still unclear on how publishers (or agents) using another AI program to verify if a writer used AI is any better. Can we trust a third party software to capture all of that data and then use it for other purposes than what it was intended for? It's all very messy and I don't see any quick resolutions.
You’re correct.
And Authors Guild has entered the chat: https://authorsguild.org/news/use-of-ai-in-publishing-and-new-model-contract-clause/
Would a potential solution be a more proactive one? Rather than trying to figure out afterwards if AI was used, would authors be open to using a program or app that just monitors AI activity (including anything happening in a browser — logging what tools were used, when, and for how long) and send some kind of certificate with the results (which can be signed/timestamped/blockchain-verified so no alterations can be made)? Or could publishing companies/agents ask their authors to use something like this, so it removes any doubt as to whether a manuscript is AI-generated?
This feels like a much cleaner standard, and it protects everyone — authors with nothing to hide, editors who shouldn't have to become AI detectors, and publishers who don’t want to pull books after the fact.
"...the book’s “facts,” if one can call memories retrieved during an MDMA therapy session as such." 😂
Excellent article. Thank you. The trust issue, I feel, has forever been a part of publishing and storytelling, and is now amplified by AI and social media. It used to be scandals (or later finding out, long after some author had died) of appropriating someone else's story or a culture. And outright plagiarism has always been an issue. And passing "memoir" off when it is largely fictionalized. Trust, especially form those presenting the book to the market / audience, is at the root of it. People generally don't like to be lied to. Go figure.
Does someone have a good chance at winning a case of "this memoirist's false memories are actually inspired by actual events that happened to me?" Wild.
Actually, yes, they do. I have a feeling this will get settled out of court.
This is a good example of how your intent/feelings (if the events are false memories and not blatantly stolen) doesn’t affect the law.
Well, the one good thing that I take away from this is that I will give Margo a chance that I might not have otherwise. 😏 The rest of that's pretty depressing. You sharing your perspective is appreciated nonetheless.
Margo is so good!
Excellent essay. I've written two non-fiction books and, coming from the news business, I was surprised how little editing was done by the editor. (Let alone fact checking lol!) For one book, the Big 5 editor had one big idea -- removing all "I" references -- but otherwise made virtually no line edits in a 100k manuscript. I received more feedback from friends and colleagues who read it, and so I relied on them to set me straight. Some of my friends have had more hands-on editors, but in general I sense that editors take a hands-off, trust-the-author approach. I'm surprised there haven't been more blow-ups from a lack of fact-checking and due diligence.
There’s a real philosophical discussion we all need to have. Right now, as a society and profession we’re whistling past it. Where do we draw the line? On the one side we have the fully AI-generated novel. On the other we have the traditional human-crafted novel. In the middle there are a lot of gradations, though. The human-crafted novel is edited by human editors; they go through drafts with a bunch of feedback. Are we going to accept an AI providing editorial feedback? That's not exactly AI writing but it is AI involvement. And then on the other end of it we've got AI drafting and then human involvement, going towards this muddy middle where it’s just hybrid. Most of us are acting as though this is a light switch that's on or off but it's a far more complicated scenario than I think we are prepared to really deal with.
Thank you for a such a thoughtful comment.
Thank you for your thoughtful take, as always! One important piece of context for the SHY GIRL situation is that the author has stated that someone used generative AI on her book. As she told the NYT, “an acquaintance she hired to edit the self-published version of the novel had used A.I.” Elsewhere online, Ballard mentioned hiring someone from her writing group to edit the novel and suspecting that they had used AI. So it could be that Hachette is basing their decision on the author's own admission--and/or her failure to disclose the use of AI upfront--rather than on an AI detection program.
I thought about putting that in, but I really wanted the onus to be on the publisher here because they are also responsible. Nothing has been said about the editorial process. At all.
I agree--a thorough editorial process could have made a world of difference. I don't think Ballard's statement changes that responsibility, but a publisher reacting to an author's statements is a different scenario than reacting to an AI checking software. I wrote about this as well, and said, "I can’t help but wonder: what could have been done differently? If Shy Girl had benefited from a longer publishing timeline, would the use of generative AI have been caught before publication? Would the editorial team have been able to notice any inconsistencies across multiple drafts and conversations with the author? Would the author have been given the opportunity to address any concerns and revise the manuscript accordingly? In hindsight, all of these possibilities seem vastly preferable to Shy Girl’s high-profile cancellation—even if a lengthier publication process resulted in some lost sales."
Everything is a rush to make sure imprints hit their numbers. I don’t necessarily believe that an editor even touched that book.
AI detection software is already out of date. It is now almost impossible to tell if something was written using AI because AI has gotten that good. In a year it will be actually impossible to tell. Authors warrant (meaning they promise) by signing a contract, that their work is original and created by them. Very soon, we will not be able to prove otherwise unless the author admits to it.
So interesting! Thank you for these summaries. I recently attended an event with Oliver James, a man who learned to read in his 30s.
He basically said he "wrote" the book by talking to a ghost writer as he's still not fully literate.
Did he type the words? No. Did he "write" the book. Maybe?