Use of AI on sciforums. Policy discussion

[geordief]

Some bots could do that, some couldn't.

And they could guess.They don't all need to get through.

Perhaps a chatbot could work upon a single post(after they got through the registration process) long enough to maintain credibility.

Some of us may not post very frequently and so they could blend in.

 

[DaveC426913]

The curiosity here is that "Richard Townsend" posted here for a short while back in 2017 (I think it was - of course I now can't search under his name any more), before AI was available. Presumably that was the real person. One wonders what motivated him to send us a chatbot troll, 9 years later.

I speculated that the real RT account was hacked recently. While there is no direct evidence, it is certainly consistent with all available facts.

I wonder if a picture question could be set-up for new members at the signing-up stage when they join.
Something like, " How many green balls in this picture".

That won't stop a human.

I get the impression the latest RT was a real person, just copy-pasting from a chatbot. So, he comes on, answers the anti-chatbot riddle and then has carte blanche from there on in.

 

[exchemist]

We all worked it out in different ways. I think that sort of poster is far more toxic to the site than a random troll.

Agreed. But the forum needs a policy with rules members can agree to when they sign up, based on an arbitration method that looks fair and reasonable. I'd have thought material not identified by the poster as AI, which looks like AI and which comes up +ve as AI in a GPTZero (or equivalent) test, should be subject to being sent to the Cesspool at moderation's discretion and the poster warned or given an infraction. I say at discretion, just in case there may be circumstances in which the material posted becomes the subject of a valid discussion.

So a policy might be something like:

- The forum is for human beings to exchange and discuss ideas.
- Posts made by AI robots are not acceptable.
- Posts made by a human who is substantially outsourcing his or her thinking to an AI robot instead of engaging in discussion are also not acceptable.
- Material posted should be written by the human poster or, if copied from elsewhere, including from AI, it should be flagged as such and accompanied by enough human text to explain why it is being posted. (Just as we require videos etc to be accompanied by explanatory text).
- AI assistance in drafting may be used by those whose command of English is insufficient, but the use of AI help must be stated.
- Posts, or text within posts, that look to moderation like AI text but have not been identified as such by the poster, may at moderation's discretion be sent to the Cesspool and the poster issued with a warning or an infraction. Moderation may rely on own judgement and/or on AI detection software to do this.

 

[DaveC426913]

I'm fed up with your constant grumbling.

Are you?

Because you know that's entirely in your control, right?
You can put any member you want on ignore, and then you will never have to read anyone's grumblings again.

 

[foghorn]

Some bots could do that, some couldn't.

Only want the best bots here, none of those cheap ones.

 

[foghorn]

That won't stop a human.
I get the impression the latest RT was a real person, just copy-pasting from a chatbot. So, he comes on, answers the anti-chatbot riddle and then has carte blanche from there on in.

I think what you have been doing with GPTZero seems to be the way ahead.
I don’t think there will be any move by this forum’s owners to alter the present set-up.

 

[Sarkus]

It occurs to me that GPTZero or an equivalent may be a useful tool for moderation, in enforcing whatever policy Sciforums eventually lands on. It seems to go a long way towards solving the problem of how to determine whether a piece of text complained about is written by a human or a robot.

They're pretty much garbage for use as such, though. They (currently) suffer from significant false positives.
If the aim is to spot the obvious AI-generated crap then, frankly, people are probably just as good. Heck, the fact that people have raised the issue in the first place means that they can spot it when it's obvious. And when it's not, who cares?
But be aware that online checkers are not foolproof, and, frankly, some are awful.

Take this text, for example, when I asked ChatGPT to give me a few paragraphs on how good AI is at checking for AI-generated text:

AI isn’t actually very good at spotting AI-generated text, at least not as good as people often think. Most AI detection tools only get it right some of the time, and in real-world use they’re often wrong. It’s common for them to flag human-written text as AI, or miss AI-generated writing completely. Once a piece of AI text has been lightly edited by a person, detection becomes even harder and accuracy drops quickly.

One big reason this is so difficult is that modern AI is trained on huge amounts of human writing, so it naturally sounds human. The patterns detectors look for—like predictability, sentence structure, or word choice—also show up in polished human writing, especially academic or professional work. That overlap makes it very easy for detectors to make mistakes. On top of that, AI models keep improving faster than detection tools can keep up.

Humans don’t do much better, either. Most people can’t reliably tell whether something was written by AI unless it’s very obvious. People who work with writing or AI a lot sometimes do better, because they recognise certain habits or tones, but even they get it wrong fairly often.

I put this into Grammarly's online check, and it came back 30% AI.
Scribbr's online check came back at 0% - all human-generated content.
Quillbot: 0%. - all human-generated content.
Copyleaks came back with 100%.
GPTZero came back with "We are highly confident this text was originally AI, but rewritten by AI or a human". Note that it wasn't certain.

So not all checkers are equal, clearly. Before any of them are used as part of a policy, I'd strongly recommend that some rigorous and sensible testing is done.
Further, I'm sure you could probably find some 100%-human input from this site that such checks would flag as high-% AI.


But I'd also question the reason for using any at all.
What is the actual issue with people using AI? To me it's those who abdicate their thinking to the AI. It's absolutely NOT those who use AI to help improve their communication of their own ideas. If we have issue with that then anyone using Grammarly is going to have to watch out!

With regard your proposed policy, therefore, I'd draw the line at:
"- AI assistance in drafting may be used by those whose command of English is insufficient, but the use of AI help must be stated."
I think this goes too far. AI is just being used as a tool, similar to a dictionary, or a thesaurus, or a search engine etc, in formulating one's own thinking. My view is therefore that requiring people state the tools they use for such should not be a policy.

Frankly, if you can't spot something is AI either in the way something is written or in the way someone conducts themselves in conversation, then who cares whether they're using AI or not. I would just have a simple policy along the lines of:
This site prefers that its members think for themselves.
Any member that is deemed to be merely a mouthpiece for an AI-chatbot (or equivalent) will be removed.

So if you suspect a post or poster to be a mouthpiece for AI, report them. Simples.
And, really, if it's not obvious they're just regurgitating AI, who cares.

 

[DaveC426913]

They're pretty much garbage for use as such, though. They (currently) suffer from significant false positives.

I did some "control" analysis on users' posts to test whether it was throwing false positives, even with long, technically dense posts. It did pretty well.

Take this text, for example, when I asked ChatGPT to give me a few paragraphs on how good AI is at checking for AI-generated text:

Where did that text come from? Your post says it was generated by ChatGPT.

GPTZero came back with "We are highly confident this text was originally AI, but rewritten by AI or a human". Note that it wasn't certain.

It will never say certain - for the same reason meteorologists never say 0% chance of precipitation (well, they do now, they didn't used to).
By definition, it can never be certain.

So not all checkers are equal, clearly. Before any of them are used as part of a policy, I'd strongly recommend that some rigorous and sensible testing is done.

So far, the worst case scenario has not come true. Someone being accused of using chatbot slop and vigorously denying it.

So, we have no cases of false positives in-the-wild. And evern if we do, the policy is certainly not going to be automatic, and certainly not going to be unchallengable.

There's a big difference between: "we have a policy; please don't this" and "this violated our policy; you have been infracted".

Further, I'm sure you could probably find some 100%-human input from this site that such checks would flag as high-% AI

My analysis (miniscule though it was) does not confirm this.

But I'd also question the reason for using any at all.
What is the actual issue with people using AI? To me it's those who abdicate their thinking to the AI.

How do you distinguish? Where do you draw the line?

With regard your proposed policy, therefore, I'd draw the line at:
"- AI assistance in drafting may be used by those whose command of English is insufficient, but the use of AI help must be stated."
I think this goes too far. AI is just being used as a tool, similar to a dictionary, or a thesaurus, or a search engine etc, in formulating one's own thinking. My view is therefore that requiring people state the tools they use for such should not be a policy.

How about the proof being in the pudding? If there is sufficient AI-gen'ed content that we can tell and/or that it is flagged by a detector, then that is a lot.

Note that ChatGPT at least, is explicit about saying text blocks under 100 words results in low confidence verdicts. The corollary of that being that long blocks of text that do get flagged are likely because the person abdicated their thinking process.

Frankly, if you can't spot something is AI either in the way something is written or in the way someone conducts themselves in conversation, then who cares whether they're using AI or not.

If it doesn't get spotted then it doesn't get analyzed. So that's self-correcting.

So if you suspect a post or poster to be a mouthpiece for AI, report them. Simples.

Well, except that puts the entire onus on moderation.

The idea of an official policy is that it doesn't have to go to the top floor. Anyone can simply say "Hey, that violates site policy. Don't do that. And I reject your argument based on that. Please try again in your own words, if you can."

 

[TheVat]

I wonder if a picture question could be set-up for new members at the signing-up stage when they join.
Something like, "How many green balls in this picture".

If it's a photo of the President, the answer is likely to be two.

 

[TheVat]

So if you suspect a post or poster to be a mouthpiece for AI, report them. Simples.
And, really, if it's not obvious they're just regurgitating AI, who cares.

The only flaw I see in the "it's not obvious" criterion is that humans do care about truthful presentation. If someone plagiarized something here, while it might not be obvious I would still care and want to know. While AI cut/paste isn't quite plagiarism, to conceal it is still a trust violation. Trust is a core value in all human communication and relationships.

 

[Sarkus]

The only flaw I see in the "it's not obvious" criterion is that humans do care about truthful presentation. If someone plagiarized something here, while it might not be obvious I would still care and want to know. While AI cut/paste isn't quite plagiarism, to conceal it is still a trust violation. Trust is a core value in all human communication and relationships.

That's an appeal to authority. Ultimately it shouldn't matter who says anything. What matters is what is said. It's not a question of plagiarism, as that is using other people's words verbatim. Simply having the same idea as someone is not, after all, plagiarism. In a discussion, the validity of an argument depends only on its content, and subsequent evidence in support. Why do you care how it is arrived at? It's not a violation of trust, unless your objective is to argue against a person rather than what they're saying? What if you ask a friend about what you're posting, and gain their help in formulating your position: do you have to disclose that you asked a friend? Even if it's your view?

Now, if someone is claiming a personal experience then, sure, one should trust what they say. But in all discussions one should at least start with the assumption of good faith, surely? If it is found that they're claiming experiences they don't have, then subsequent claims of experience in support of their position might not be trusted. That's okay. But in a normal discussion, the only relevant trust one need have is that they stand behind their position (even if as Devil's Advocate, for example).

If you care about the person who says something rather than what they're saying and whether they stand behind it, ask yourself why the person is relevant. It nearly always isn't.

When it comes to the obvious AI-generated content, the trust breaks down generally because the person doesn't/can't stand behind it. Yes, one can get frustrated that they're not talking to a real human, but that's why I think that if you can't tell, why care about it.

 

[Wizard of Whatever]

The best deal is to warn once or twice for posting AI without attribution and ban for a subsequent offense.
Allow one to challenge your judgment that it is AI and create a way to work out that challenge.

 

[DaveC426913]

Ultimately it shouldn't matter who says anything. What matters is what is said.

In an ideal world and with good faith actors, sure.

But, as we see, copy-pasting AI is being used to pollute good-faith discussion.

The nature of trolling (at least the "long game" type) is to take advantage of the average user's trust and see how long one can keep up a silly line of argument before getting caught. The entire goal there is to waste the users' time.



In a recent post by banned member RT, he posted a great big apology, falling on his sword and prostratng himself and calling himself a coward and a fraud for abusing AI in exactly the way he was complaining about.

Except that entire "apology" was written by AI and pasted here. Should we have taken him at his word that he was sorry for what he did? Does it not matter that the AI spat out its own apology?

 

[TheVat]

If you care about the person who says something rather than what they're saying and whether they stand behind it, ask yourself why the person is relevant. It nearly always isn't.

Well, possibly we go to forums for different purposes. For me, the social aspect is as important as the debate or analytical aspect. In a social mode, knowing something of the person, their motivation, their subtext, has some value. Not every thread here is a pure process of sifting ideas. For me, anyway. YMMV.

 

[Sarkus]

I did some "control" analysis on users' posts to test whether it was throwing false positives, even with long, technically dense posts. It did pretty well.

That's good to hear, but how many times did it suggest with a high confidence that it was AI-generated where you were not already convinced that it was? How many times did it surprise you? Did it ever surprise you by saying that a piece that you thought was AI-generated was probably not?

Where did that text come from? Your post says it was generated by ChatGPT.

Well, my guess would be that, given that I said I asked ChatGPT, and given that I quoted it as being from ChatGPT, that it came from ChatGPT. What do you think? I did, though, ask it to write the paragraphs in a more colloquial, less formal style - so as to avoid the obvious AI-formatting clues.

It will never say certain - for the same reason meteorologists never say 0% chance of precipitation (well, they do now, they didn't used to).
By definition, it can never be certain.

Okay.

So, we have no cases of false positives in-the-wild. And evern if we do, the policy is certainly not going to be automatic, and certainly not going to be unchallengable.

Which does sort of beg the question of why we need any policy whatsoever, when we can spot the (obvious) cases and things already are getting done about them. I mean, one could implement additional policies that don't make any practical difference, but... why bother?

There's a big difference between: "we have a policy; please don't this" and "this violated our policy; you have been infracted".

Splitting hairs, DaveC. The issue is doing something moderators/users don't want at the site, not the action they take against an individual case.

My analysis (miniscule though it was) does not confirm this.

Okay - should we wait for you to trawl through the rest?

How do you distinguish? Where do you draw the line?

I don't think there is a "line" - not a hard and fast rule that needs be applied, anyway. It is up to individuals as to whether they recognise that the post is AI, that the poster is standing behind what they post, and then what they do with it.
Again, there's always the "ignore" button for posters you don't want to engage with, and the "report" function.

How about the proof being in the pudding? If there is sufficient AI-gen'ed content that we can tell and/or that it is flagged by a detector, then that is a lot.

Note that ChatGPT at least, is explicit about saying text blocks under 100 words results in low confidence verdicts. The corollary of that being that long blocks of text that do get flagged are likely because the person abdicated their thinking process.

If a lot of users spot it - then it's obvious - and it can be dealt with. We would then seem to not need a tool to tell us.
If not a lot of users spot it - then who cares? Clearly those posters don't seem to care, and if they subsequently get upset about having been "duped" then that's more about them. Again, i think it's the content of posts that matter, not who is posting it. A valid argument is a valid argument. If the poster/AI is hallucinating false examples, this would be an issue irrespective of whether it is AI or a real person lying.
And yes, I take the point of the accuracy improving with larger blocks of text. But the larger the block, the easier it is for us mere mortals to identify it. And it is only when we identify it that it would seem to matter. If we can't, who cares.

If it doesn't get spotted then it doesn't get analyzed. So that's self-correcting.

So, again, you're only concerned with AI-generated content when it is already obviously AI-generated content.
Look, to be clear, I have no issue with moderators running suspect posts through a tool if that's what they want to do. I just don't think that that should be part of the policy, nor - as I mentioned previously - do I think AI-generated content should be flagged as such, but if people want to go down that route then definitely NOT as far as if people are using the likes of grammarly, or using AI just to help reword their own thoughts. That's too far and is an attack on the tools, not on the message.

Well, except that puts the entire onus on moderation.

??? It's already on moderation, and can only ever be their onus. All we lay-people can do is report, as with violations of all other site rules. If you don't like a post because you think it's AI-gen, report it. What else can you do? You can't infract them, you can't ban them, etc.

The idea of an official policy is that it doesn't have to go to the top floor. Anyone can simply say "Hey, that violates site policy. Don't do that. And I reject your argument based on that. Please try again in your own words, if you can."

I've got no issue with a policy that rejects AI-gen content, but a simple policy that rejects should be sufficient - e.g. "Do not simply post AI-gen content" should suffice. Everything else is up to the individual posters to report, as with violations of all other rules. And the onus is always on the moderator to assess the issue and react as they see fit.
Ultimately I'm looking to avoid there being a policy that would penalise people using AI in a manner whereby the reader can't tell that it's been used. So, something simple, something that helps remove the obvious.

 

[Sarkus]

In a recent post by banned member RT, he posted a great big apology, falling on his sword and prostratng himself and calling himself a coward and a fraud for abusing AI in exactly the way he was complaining about.

Except that entire "apology" was written by AI and pasted here. Should we have taken him at his word that he was sorry for what he did? Does it not matter that the AI spat out its own apology?

And he's gone. Could he have been removed earlier? Sure. A simple policy that says something along the lines of "Do not post AI-gen content as if it is your own" should suffice, shouldn't it?

Edit: I realise he hasn't been banned yet - as he is a current "online" member?

 

[Sarkus]

Well, possibly we go to forums for different purposes. For me, the social aspect is as important as the debate or analytical aspect. In a social mode, knowing something of the person, their motivation, their subtext, has some value. Not every thread here is a pure process of sifting ideas. For me, anyway. YMMV.

No, I have a social life for social things. I come here specifically to share views (and occasionally to vent). I'm only interested in the views, and in the honesty of the exchange. Frankly, I couldn't give two hoots about any anonymous person that I only know about through this site. You could all be AI-bots as far as I'm concerned - although I know most aren't, because the conversation isn't that good.

 

[DaveC426913]

I'm only interested in the views, and in the honesty of the exchange.

But an AI can't be honest in any meaningful sense. And, technically, it doesn't have views either.


I was going to post a point-by-point response to post 55. Maybe I will later.

And he's gone. Could he have been removed earlier? Sure. A simple policy that says something along the lines of "Do not post AI-gen content as if it is your own" should suffice, shouldn't it?

Let's look at where that's headed. So, if RT had simply posted "My chatbot says" at the bottom of every one of his posts, he would get a free pass.

We'd stil lhave to wade thoghu the weasel-word-speak of AI that skirts the boundaires of truth and hallucination. I don't wan't to spend the rest of my days here in SciFo not being able to trust anytihng anyone says because it's all AI slop anyway.

For my part, if SciFo became a magnet for AI chats like that, I would leave. The signal to-noise ratio will drop to some level below which it is not worth it (it is nearing that now, but not with AI - with trolls). If others were like me, and followed suit, SciFo would die, or at least become a zombie.

 

[Olga]

But an AI can't be honest in any meaningful sense. And, technically, it doesn't have views either.


I was going to post a point-by-point response to post 55. Maybe I will later.



Let's look at where that's headed. So, if RT had simply posted "My chatbot says" at the bottom of every one of his posts, he would get a free pass.

We'd stil lhave to wade thoghu the weasel-word-speak of AI that skirts the boundaires of truth and hallucination. I don't wan't to spend the rest of my days here in SciFo not being able to trust anytihng anyone says because it's all AI slop anyway.

For my part, if SciFo became a magnet for AI chats like that, I would leave. The signal to-noise ratio will drop to some level below which it is not worth it (it is nearing that now, but not with AI - with trolls). If others were like me, and followed suit, SciFo would die, or at least become a zombie.

Этот сайт уже и так присмерти. Агонизирует.

 

[Pinball1970]

How did you sus RT?
The format was not the usual Ai exchemist "framework."

I initially thought it was someone trolling because the he started 8 threads.
The language was polite, light and gay. No one one is that fucking happy about lithopanspermia and gravity.
He posted on the Higgs and the chirality thread, like he knew.
Checked history from 2017 and he was out of his depth with "observer."

A newcomer could fool me I reckon, no history on SF, perhaps edits the happy and adds his own phrases.

Would GtpZero, zero in on that? I've never used it.