117
u/Serious_Engin33r 1d ago
Doesn’t using AI during the review process break the principle of confidentiality?
Like, it’s ok if you need to polish the grammar of the comments, but feeding an unpublished manuscript to an AI should be unethical (not considered the other many unethical practices involving AI in research)
65
u/Sensitive-Control49 1d ago
yeah the confidentiality angle doesnt get talked about nearly enough imo
6
u/evagarde 1d ago
Yes, CNS have specific instructions to reviewers not to use AI for any part of the process and you have to declare that you did not when you submit.
4
u/Stock-Self-4028 1d ago
I think you can do that with no issues to a local LLM.
So basically you need quite a lot of RAM to run at a reasonable speed, but it works without issues if you "download the AI" (typically tens to hundreds of gigabytes) instead of uploading the manuscript to an inference provider.
4
u/Serious_Engin33r 23h ago
Yes exactly, my organization hosts some older GPT models as well as a few open-source European models. Nothing groundbreaking performance wise, but at least the data stays in our hands.
Still, even with selfhosting, if an LLM is more expert than I am on a particular topic, maybe I shouldn’t be the reviewer in the first place.
72
u/Urusander 1d ago
This is why I have little confidence in "AI scientist" models. If you train it on some internal dataset taken in consistent conditions it can work fine, but as soon as you start training it on the corpus of published papers it will just get glutted with noise.
13
u/TheTopNacho 1d ago
In my experiences the AI models bias towards high impact journals and don't give credit to moderate or low impact findings.
It gives perspective only to the words in the paper as well.
Both of these are problems because 1: saying something is different doesn't provide context into the magnitude of effects so AI reporting seems to ignore context and experimental conditions. And 2: Nature papers are usually heavily influenced by shady practices. We all know the people in our field who always get high impact work but run on N=3, or have a gossip circle that validates their bad research conduct such as picking and choosing data points, redoing experiments until they work, using methods that are not disclosed, or blatantly making up data etc.
I know my field extremely well. It's easy for me to spot the literature AI uses when I probe it with questions. And it largely overlooks vital context in papers and can't keep up with literature that I know to be ethically problematic. If it's making decisions on the rare examples of data from Nature that aren't reproducible, it's conclusions will likely be idealistic at best, and lacking vital context that we can think critically about.
Kinda scary that PhD students will grow up with these biased and shallow learning tools. I think the field of science will be significantly impaired because of this. Despite the usefulness in other areas.
6
u/hummus4u Translational Science, U.S. 1d ago
Not just this, it hallucinates citations. It will cite papers that don't exist, papers that make an entirely different conclusion to what the paper says. If editors use AI to verify citations, we're going to be left with incorrect or misleading information backed by a nonexistent or fabricated source! It's ridiculous
19
10
u/Odd-Landscape-9418 1d ago
After endless blood, sweat and tears which are required to produce even a semi-good paper, I can assure you it is VERY frustrating when I receive the reviewer's comments and I instantly recognise blatant use of AI. Things like "we would want you to also use method X/do experiment Y" which I have on the paper and literally mention. Or "please justify why you have done A and not B" and I have mentioned it loud and clear on manuscript. Or the best, "it is widely known that X is not true/method A is better than method B" which is the opposite of reality - essentially a hallucinating LLM.
Yeah not really fun having to revise my paper based on all this.
3
u/Ill-Refuse-7834 15h ago
You all are using LLM for reviews? I literally spent 6 hours last week as a peer reviewer :(
1
u/ElijahNSRose Military History, Liberty University 1d ago
But people still get angry when you say most of the published research is useless.
-5
u/mmmtrees 1d ago
eh I think it's fine as long as funding agencies don't start relying on AI for grant reviews. Then at least the sloppers will get filtered out from lack of funds.
3
u/SkyFullOfWisteria 1d ago
Several facaulty in my department use ai to do like everything for them, from "the boring lit review and research" to "imagining images" of CADs that they didnt actually make and dont exist, and if you actually look at the image for 2+ seconds, the design shown literally cannot function. And they're putting ts on grant purposals. Its bad out here, and when ai is inevitably no longer as free as it is currently, a lot of people are gonna either be completely cooked or be coughing up a tonnn of money.
-33
u/IchDunno 1d ago
What do you mean? Is this not allowed? 🫣 I am now submitting my thesis and Claude and I partnered on most of the thesis. I definitely made it add the published papers as chapters, extend it with the appendix, and write the intros of the chapters. I mean, I went over every word of it of course, but it's still Claude's wording. P.S. I am in Germany.
26
8
9
9
4
u/Ok_Support_4750 1d ago
what is the value of this? beyond getting your checkmark ✔️ “thesis done” if the machine is doing this work for you, was it valuable? it’s all perspective and what you personally get out of the journey. doing the thesis is hard, writing these things out yourself is hard, finding the proper way to convey your thoughts in a way it’s understandable, is the challenge. if all you care is spitting out results, then yes the AI will do it. don’t need a human for that.
1
u/mmmtrees 1d ago
Contrary to the other opinions, I'd say this is ok. As long as you are confirming the validity of Claude outputs it isn't really consequential to the actual quality or impact of your work. Researchers hire scientific writers and editors all the time, this essentially the same. Disclose AI usage in your acknowledgements and move on to the next project while the AI haters clutch their pearls.
-7
u/IchDunno 1d ago
Finally 🙄 thanks! I was feeling crazy! Also, we don't have the luxury of ignoring this technology. We need to learn how to use it without compromising correctness, or we will fall behind and it's scary.
15
u/Cyphomeris 1d ago
After asking for people's evaluation, you received more responses saying this isn't okay, and your reaction as an academic is to pick the one response you agree with to feel validated?
That doesn't exactly bode well for an academic career, lmao.
-13
u/IchDunno 1d ago
Duuuuuudes 😡 Why am I getting so much hate!?!
What do you think Claude does when it writes a thesis chapter intro? It takes your paper's intro, removes the citations that were already mentioned in the thesis intro and gives cross references to other thesis chapters. And it connects this and that better than me with nice english connectors. How is this so outrageous? 😒😒
-3
-8
u/Sgt_Radiohead 1d ago
You didn’t do anything wrong. Genuinely these people downvoting you are sanctimonious virtue signalers that pretend like they are too good for using LLMs in their work.
Nobody bullshits like academics. You’re comparing your behind the scenes to their stage show. The only mistake you did was to openly admit that you use LLMs, which is heavily stigmatized. It’s everyone’s dirty little secret nobody wants to admit to. So now people are throwing stones at you from glass houses
4
u/alexb4you 1d ago
Nobody bullshits like an LLM.
1
u/Sgt_Radiohead 21h ago edited 20h ago
You’re using the LLM wrong and you’re shocked that it hallucinates and gives you the wrong answer
This all reminds me of when optimisers were beginning to be introduced to the conventional RF simulation softwares and people were ranting about how «they tried it once and gave it zero restrictions and it gave him a terrible result!». Yeah no shit Sherlock
166
u/Alert-Translator2590 PhD, Science, World 1d ago
Peer review has been in shambles for quite some.
Many Authors know the editors, reviewers. They cite each others papers and they have it easy to publish in so called ‘respected’ and ‘high IF’ journals.
So many academics know it already.
Many cases where the phds know that things are wrong in figs but the reviewers can’t see it. Or maybe they can but it’s their friends.