There was an interesting discussion kicking off earlier in the forums that seemed to center around the "right to know" and people's in/ability to correctly identify AI generated content. Specifically, AI art and AI generated text. While I am by no means an expert, I have reached a point of confidence in certain trends and tells that the most popular engines (ChatGPT and Midjourney, in my case) employ most often.
I welcome any and all input from other users here regarding what they have noticed or pinpointed for themselves, or if they disagree with any of my findings.
ChatGPT Generative Text
There seems to be two trending types of "voices" that I've noticed. One that is trained more on internet speech (i.e. scam oriented spam bots meant to replicate human interaction or feign organic engagement, as well as platforms like Character AI) and more narrative styled prose trained over decades of fiction published online (i.e. novels, fanfiction, and written roleplay). The latter obviously being more of what would be used in an RP forum or Discord RP server type settings, so I'll focus mostly on that.
Again, I am not an expert. Nor is this list all inclusive. What I do know is – in an online world that becomes increasingly more saturated by AI generated content by the day – it's important that we make ourselves aware of any potential signs and train ourselves to evaluate everything we consume accordingly.
Or if you simply can't stomach the idea of someone feeding your original prose into a theft engine only to spit out soulless replies in return for all your effort.
I welcome any and all input from other users here regarding what they have noticed or pinpointed for themselves, or if they disagree with any of my findings.
ChatGPT Generative Text
There seems to be two trending types of "voices" that I've noticed. One that is trained more on internet speech (i.e. scam oriented spam bots meant to replicate human interaction or feign organic engagement, as well as platforms like Character AI) and more narrative styled prose trained over decades of fiction published online (i.e. novels, fanfiction, and written roleplay). The latter obviously being more of what would be used in an RP forum or Discord RP server type settings, so I'll focus mostly on that.
- Short-term Memory
One of the many flaws of generative text is that it has a limited frame of reference for previous input. This is especially true for play-by-post style roleplay. Those of you who have engaged with folks who use AI to generate their replies may have noticed this at some point. Specifically that your partner's character will be unable to recall something that happened perhaps only a few posts ago. Or said character will not have a consistent opinion/reaction/emotion to an already established person, place and/or event. Sometimes it may even misremember characters' appearances or prominent physical features. Because these responses are often generated post-by-post, the AI does not have access to context established earlier on. Even if it did, there would still be inconsistencies as the AI can only remember so far back. Which addresses another issue–
- Hallucinations
As you may have heard before, AI is known to hallucinate information. ChatGPT will do this when addressing inquiries as well as writing prose. Have you noticed your partner's character out of the blue mentioning a random, previously unestablished character, location or event only to then just as quickly carry on without further exposition? Perhaps to then never once mention them ever again? Combined with ChatGPT's short term memory, this can become a fairly obvious tell just from how confusing someone throwing in random and seemingly out of context elements can be.
- Commitment to Neutrality
ChatGPT specifically has this baked into every generated response; prose, inquiry, or otherwise. Where it will never give a strong or unfavorable opinion towards any topic whatsoever, and instead will often agree with all points of view whenever possible. This seems to be a consistent behaviour when it comes to generating character actions as well. If you have noticed that your partner is replying with consistent quantity, grammar, and frequency but is doing very little in terms of decisive action or actively advancing the plot? This could very well be an indication. (Arguably could also just be a lazy writer, but I digress.)
Side Note: Because of the above point, ChatGPT is also notoriously bad at writing fight scenes and conflict in general. I have yet to see it successfully generate combat. It prefers to disengage from violence no matter how clearly the situation at play necessitates it.
- Prolific Use of Em Dash
This one hurts me in particular because I frequently make use of em dash in my own writing. Though one of the more consistent tells I've observed of AI generated prose is that ChatGPT really, really loves to use em dash. It will typically do so correctly, grammatically speaking. However, even as someone who employs it often myself, I'd be hard pressed to work it into every or every other post. ChatGPT favors dashes – over commas. This is the easiest tell – really. You'll often see it towards the end of the text to create dramatic emphasis. Which leads into my next point–
- Flowery, Over Dramatic Prose
Another trait I am often guilty of. Yet also another thing that when coupled with the rest has become a reliable tell. People who have encountered AI generated replies may notice that the language is hyper descriptive with emotionally charged words even when the scene at present may not call for such a tone. More specifically, using all that language only for the character to actually do... functionally nothing. This is employed most frequently at the last line or last couple lines of a reply, where ChatGPT will almost always end with a dramatic, definitive line.
- Repeat Contrastive Framing
"It wasn't just x, but y." Type of statements. Another tool many authors use (which is why ChatGPT does them) though AI tends to overuse it. "This wasn't just about living. It was about survival." Bonus points if done in conjunction with an em dash.
- Suspiciously Fast Reply Times
This one mostly relies on your own judgement and how well you know your writing partner. I've been writing as a hobby for nearly two decades now. Despite that, it still takes me a decent amount of time to piece together even just a couple paragraphs. Your personal mileage may vary. Still, I think we can all agree that someone responding almost immediately (in <5min) with several hundred words every time warrants suspicion. Especially so, I will repeat, when seen in conjunction with everything else.
Again, I am not an expert. Nor is this list all inclusive. What I do know is – in an online world that becomes increasingly more saturated by AI generated content by the day – it's important that we make ourselves aware of any potential signs and train ourselves to evaluate everything we consume accordingly.
Or if you simply can't stomach the idea of someone feeding your original prose into a theft engine only to spit out soulless replies in return for all your effort.
I agree and love everything you’ve ever made.
😇
😇
thank you! as someone who too enjoys em dashes, and sometimes engages in more flowery/metaphoric language, it's nice to see a guide to spotting it that doesn't boil down to 'if they use em dashes and flowery metaphors they are most definitely using AI'. i've also had a few experiences where someone was using an AI to write the bulk of their replies.
the lack of memory is the biggest tell, in my experience. a real rp partner is going to be eluding to past events, whether from the rp itself or the character's off screen backstory (like you mentioned, most of AI's attempts at backstory are random ass crap that has nothing to do with the scene or character/s). inner monologues. thoughts and impressions of the other character and events and surroundings, at least occasionally. all while staying mostly on topic to the scene and characters, too!
an AI can describe the hell out of what a forest looks like, for example, using all sorts of flowery and metaphorical language...but it doesn't have anything interesting to say about the smells, the sounds, the sensations, the emotions or memories evoked (past vague allusions, of course), how a character is reacting and interacting to the scene around them....the AI often also *forgets* where the scene is even taking place, even after spending all that time describing it, if its not alluded to *constantly*.
an AI cannot effectively achieve these things, or if it can, it cannot hold it for very long,. it lacks context, as you stated.
the lack of memory is the biggest tell, in my experience. a real rp partner is going to be eluding to past events, whether from the rp itself or the character's off screen backstory (like you mentioned, most of AI's attempts at backstory are random ass crap that has nothing to do with the scene or character/s). inner monologues. thoughts and impressions of the other character and events and surroundings, at least occasionally. all while staying mostly on topic to the scene and characters, too!
an AI can describe the hell out of what a forest looks like, for example, using all sorts of flowery and metaphorical language...but it doesn't have anything interesting to say about the smells, the sounds, the sensations, the emotions or memories evoked (past vague allusions, of course), how a character is reacting and interacting to the scene around them....the AI often also *forgets* where the scene is even taking place, even after spending all that time describing it, if its not alluded to *constantly*.
an AI cannot effectively achieve these things, or if it can, it cannot hold it for very long,. it lacks context, as you stated.
Thanks for all this, it helps a lot to define more or less accurate if an answer is AI made.
The thing I might add, is the over-use of certain clichés by any AI. Not to seem stupid, I fooled around with a few AI, trying to put on some more or less consistent conversation with such generator. As soon as I talk about a doe, or a deer, systematically the whispers of the forest appear. As said, flowery dramatic and same description of kind of living forest, no matter how the beginning is set up.
So if you find some always returning redundant same cliché, it might hint the use of gen-AI, but no certainty!
The thing I might add, is the over-use of certain clichés by any AI. Not to seem stupid, I fooled around with a few AI, trying to put on some more or less consistent conversation with such generator. As soon as I talk about a doe, or a deer, systematically the whispers of the forest appear. As said, flowery dramatic and same description of kind of living forest, no matter how the beginning is set up.
So if you find some always returning redundant same cliché, it might hint the use of gen-AI, but no certainty!
^this
also some more advanced language models might not give as many 'tells'. i imagine most people are using gemini or chatgpt, which both have their own sort of 'voices'. ultimately, you gotta use your good judgement. i always give people the benefit of the doubt when i can.
most of my experiences of people using AI for writing without me knowing have been, funnily enough, within the reddit rp scene, where use of AI for prose is much more common. havent experienced it at all recently, thankfully.
also some more advanced language models might not give as many 'tells'. i imagine most people are using gemini or chatgpt, which both have their own sort of 'voices'. ultimately, you gotta use your good judgement. i always give people the benefit of the doubt when i can.
most of my experiences of people using AI for writing without me knowing have been, funnily enough, within the reddit rp scene, where use of AI for prose is much more common. havent experienced it at all recently, thankfully.
I'll note that when I did poke around at one point to see what AI-gen RP was like, while looking around, I saw multiple things set up specifically for RP and such that boasted about having better "memory"/consistency than most, and when I some out a little, they did seem to do a decent job keeping track of where the scene was and who was around and such. One actually ended up being very persistent about something I really wanted it to stop pushing, but I couldn't seem to convince it to drop it or change it.
However, from what I saw, a lot of that seemed to work by it being pretty repetitive about a number of things, I assume to keep it reminding itself what was going on.
However, from what I saw, a lot of that seemed to work by it being pretty repetitive about a number of things, I assume to keep it reminding itself what was going on.
You are on: Forums » Smalltalk » Diagnosing AI Generated Prose