Really mad about this Linux kernel - academic research kerfuffle. See: https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah.com/
TL;DR: researchers at UMN introduced patches to Linux kernel that contained known-buggy code TO WRITE A PAPER and the UMN IRB didn't consider it human subjects research
Wtaf were these researchers thinking? How did the IRB not consider this human subjects resaerch? How did someone think this was an ethical thing to do?
UPDATE: the research has been suspended and inquiries are being made: https://cse.umn.edu/cs/statement-cse-linux-kernel-research-april-21-2021
@deejoe it's in the authors' response to the "outrage" - https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.pdf
IDK if anyone has written + requested from the IRB their letter or if they've made it available tho
@deejoe IRB has no idea how to handle ethics for digital communities. Social media studies are often wholesale made exempt bc it's not "human subjects research"
industry practice around A/B testing has never sat well with me, to be sure, however widespread it is.
it's also in the paper, a link for the PDF on github to which was posted in the LKML thread but which I only just now got around to looking at, my bad.
"The IRB of University of Minnesota reviewed the procedures of the experiment and determined that this is not human research. We obtained a formal IRB-exempt letter."
@VickyRampin the fucking fuck:
In the paper, we provide our suggestions to improve the patching process.
* OSS projects would be suggested to update the code of conduct, something like “By submitting the patch, I agree to not intend to introduce bugs”.
Fuck you and your entitled attitude.
@VickyRampin statement from the UMN CS department: https://twitter.com/lorenterveen/status/1384954220705722369
@VickyRampin @deejoe probably the IRB didn't understand that this was actually an experiment on developers, and thought it was just an experiment on code. IRBs have huge blind spots for research where it's not blindingly obvious that there are human subjects involved :(
And it's not unheard of for tech-related researchers to exploit that blind spot
@be i’m glad you mentioned this. it’s important to not let institutions act like each of their ethical decision-making opportunities is separate and self-contained
@VickyRampin someone should contact a news outlet that specializes in this kind of thing—The Register maybe
as noted, Phoronix has it. I'll be interested to see what, given a little time to actually make some calls or have some email/social media exchanges, other outlets dig up about it (including the institutional review board [IRB]) question
@VickyRampin wowwwww that's no good. I saw a kernel update last round so I guess I'd better to run them again.
But the IRB also needs to face some consequences for this, not just UMN. I wonder if there's a way to contact them about it... hmmm...
@VickyRampin here's a place to submit complaints to UMN's IRB : https://research.umn.edu/units/hrpp/research-participants/questions-concerns
Based on a 2019 article, they're prioritizing fast approvals rather than thoroughly understanding implications : https://research.umn.edu/units/irb/news/new-model-managing-irb-submissions-leads-faster-approvals
@bouncinglime I haven't seen any IRB that evaluates digital community work well TBH. There definitely needs to be a reckoning
@VickyRampin Imagine someone in any other field intentionally fucking up something and then calling it research.
I'm wondering who the "Linux maintainers" mentioned in the acknowledgements section are:
hey @VickyRampin ! just wanted to point out for people citing this thread that, according to https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.pdf , they did not apply for IRB *before* the study.
(Which is exemplary of how there is a severe lack of ethics considerations in empirical software engineering research.)
Then, when they did apply, IRB did not consider it human subjects research, as you point out. But there was another major flaw in the process before that one (= not seeking approval).
@zacchiro @VickyRampin I've been an academic social science researcher. It's essential to understand that the core competence of a university IRB is the ethics of _medical_ research. That's what they were created to handle.
The kinds of unethical research they understand, therefore, are the kind where the experiment directly causes bodily harm to participants, and the kind where the researchers' files contain embarrassing secrets about the participants (think "Patient J has syphilis").
Making Greg K-H do a bunch of extra work to weed the bad patches back out of the kernel _is_, I would argue, harm to a person, but not the kind of harm they're institutionally set up to recognize.
@zacchiro @VickyRampin The other angle the IRB ought to have picked up on, though, is the deception. I think there's a strong case that this is isomorphic to a psychology experiment involving lying to participants, which you're only supposed to do if there's no other way to get the data you need, and you're expected to tell people the truth after the experiment.
@zwol @zacchiro @VickyRampin For those following, it is important to clarify terms. "Exempt" human subjects research is still human subjects research, but does not require "informed consent" by the study subjects.
Any university researcher should have had an introduction to IRB, and should know to contact the IRB whenever humans are involved. "Exempt" status is common for collecting survey data or other responses from people.
@zwol @zacchiro @VickyRampin
Since their IRB provided an "Exempt" status determination (which should have been done prior to conducting the research), the IRB review did not think that there was a level of risk to individuals that required informed consent.
The email thread poses an additional twist. I doubt that the IRB could have considered the ill-will generated by the research toward both the researchers and the institution: "ban all future contributions from your University"
@VickyRampin @zwol @zacchiro FYI University of Minnesota has suspended the research, and is reviewing https://cse.umn.edu/cs/statement-cse-linux-kernel-research-april-21-2021
@zwol @zacchiro @VickyRampin In other threads about this there was talk about trust. And that's the thing here: This research is manipulating the kernel development community by misusing trust, and while the product in question is code, there's a social process built around that. The attack is not targeted at technology, it's targeted at people and the conventions of their community.
I'm little surprised some of those working in "computer" sciences might not want to see the difference though.
I think that what these researcher did was noble and highly ethical: they proved an (obvious and) dangerous operational issue that was likely exploited before without anybody noticing.
All this drama is just the king that, fooled by thieves, is crying loud that the kids pointing at his nudity must be executed.
@Shamar @galaxis @zacchiro @VickyRampin The experiment may have been worth doing, but the execution was botched. They've both invalidated their own results, and poisoned the well for anyone wanting to do similar research in the future. This in turn means that genuinely malicious actors will probably find it *easier* to get their changes into Linux now.
Why you talk at future tense?
If a bunch of University students got their bugs in the #Linux's stable tree, it's plain obvious they were not the first.
Also I do not follow your reasoning: why they invalidated their result? How that would facilitate malicious attackers?
To be honest I think that the Linux developers should thank them for showing that such obvious risks were not just theoretical.
Instead, they stop accepting patches from that university.
It's like when a led in your car shows that your engine needs oil... and you cover the led.
@zwol Is this true, though, that there's no harm to people? A quick search indicates that Linux is installed in medical devices. It's used in most web sites last I checked, and I expect that includes many that secure private medical information, various PII, etc. I haven't done a thorough code review, but at a glance it looked like some of the reverted commits introduced security bugs that could reasonably harm these.
Sure, when IRBs were established in the US in the 70s, they were established for medical research. But in 1991 the common rule was revised for social science research. 30 years of IRB to get it together. And the Internet existed then.
I don't resolve IRB of their duties in this case or other web-based or social-science based research.
The linked PDF credits US National Science Foundation (NSF) support for the research. Primary US federal public funding for health related research usually comes from the National Institutes of Health (NIH). NSF tends to fund other, non-health, basic science work. NSF requires IRB review too.
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!