Science should be more like this, in current times, yes.

But until much of academia is burned to the ground, or until science can be properly separated from modern academia, this will never be so. The current academic incentives are all wrong: low-quality research is rewarded and results in publications, whereas high-quality research (that takes time, and usually reveals that most exciting publications depend on p-hacking or other highly data-dependent analyses and selective presentations) is not published or actively blocked during peer review.

So instead you get BS arguments about how data can't be released for various privacy concerns (when in reality the vast majority of most datasets are trivial to scrub of identifying factors, and even in more complex datasets where you need to consider k-anonymity, it is still trivial to release data that allows replication of core analyses), and academic science is increasingly irrelevant unless it is tied to tech and industry, where producing junk actually has real negative economic and personal consequences.

I don't know what world this article / post lives in, but it isn't the messy world of actual reality.

> Every result is instantly reproducible. When you read a paper claiming that a new drug reduces symptoms by 30%, you click a link and watch the exact analysis run in your browser. The data processing, statistical tests, and visualizations execute in seconds using the same environment the authors used—preserved perfectly through reproducible containers.

At least some journals have this as a stipulation e.g. https://www.nature.com/nature-portfolio/editorial-policies/r...

Particularly the "data availability" and "Availability and peer review of computer code and algorithm".

However, in my limited experience, of trying to reproduce certain scRNA-seq processing pipelines, in practice it's never available as just a Github link. I can understand that some/many researcher's code is not in good shape, so I think it'll be quite a stretch to have this available.

I do think it's laudable though, to try and make it available. It would certainly have been very useful for me in the past.

> modern science is synonymous with open source software.

Another problem with reproducibility is the openness of the underlying data. Many academics are terrified of giving away the golden goose and the software is often useless without the data.

However many scientists do work openly, e.g. The Journal of Open Source Software:

https://joss.theoj.org/

This article is correct but it bummed me out. It made me think of how much of science has been perverted into other goals, like medical science for example. The advancement of human health largely depends on corporate interests, and that is so insane to my brain it hurts to think about. very few independent scientists can research anything because everything costs money so you need a corporate interest to even do research. And the corporation gets the "rights" to those findings? It's absolutely insane to discover something natural and claim it as your own, and science is natural nobody is inventing it or being creative and writing it themselves, they're essentially walking up to a mountain and saying "Ok it's mine now I'll charge you 100$ to walk on the mountain". It genuinely makes my brain do somersaults in my skull that we've somehow backed SCIENCE of all things into this weird gatekept scenario its in now. It wouldn't bum me out so much but it clearly does nothing but hinder progress

The point at the end "The scientific revolution succeeded because it insisted on transparency, reproducibility, and constant scrutiny." is a good one. But the real kicker that makes the situation so bleak is it's not the scientists who get to decide whether or not these things get applied to science or not, it's government policies and corporate interests.

Science is open, but science is not software and software definitely not science.

100% disagree! Do not mix and cherry pick terms and definitions to make your point. That's not scientific -) Take e.g. a look in https://opensciencemooc.eu/ and check a nice accepted handbook like "The Turing Way" [1]

[1] https://book.the-turing-way.org/

Nice read. I believe that traditional software is a great way to showcase the proofs when it comes to physics and mathematics. You can easily code up a theorem in a language of your choice and justify that 'Okay, the output matches the expected value'.

I am particularly fascinated by labs like DeepMind [https://deepmind.google/science/]. The recent advances in their frontier models that are able to predict diseases before they're diagnosed is incredible. This is what AI should be built for and actually do!

The title reminds me of this, which is arguing the opposite direction: https://softpanorama.org/Articles/oss_as_academic_research.s...

We really need to ban tech people from using the word "open".

> NixOS is quickly becomming the biggest and best tool there is. It will guarantee that your code will run exactly the same way, even 100 years in the future. Docker, Conda, and similar tools are better, but NixOS gives more comprehensive guarantees.

I like how this is dropped as a fact. Dare to explain why though? Especially vs Docker. NixOS is not even standardized. No guarantees it will not be superseded by some descedant or eg. Guix in a near decade.

I agree with the general sentiment, but there's one major caveat. We should implement reproducible programs on top of a virtual machine spec like JVM or WebAssembly rather than replicate the entire environment. It's more practical to do and doesn't push software towards further centralization. Let people use whatever OS and VM implementation they want, or even write their own.

One can decide to dedicate their own time to creating open SW. Typically someone (like tax payers or private companies) pay for the creation of scientific discoveries.

Thus there is indeed a difference in the monetary intensives.

Replace "Open" with free as in "freedom" gnu.org

When all you have is a hammer, everything looks like a nail.

I refer to what I think the author is asking for, as "push button reproducibility," i.e., the idea that the results will reproduce themselves at the push of a button, anywhere, at any time in the future. I have a couple of misgivings about this.

First, the whole idea of "open" research predates computer technology. Forcing science to keep up with the latest ideas in software distribution is too much of a burden, when science is already too risky and slow. I had the odd privilege of learning the scientific method from my mom, before there was widespread access to computers. Her version was that a study should be reproducible by a reasonably skilled person. This is a greatly relaxed standard, but is realistic for a discipline that spans decades if not centuries.

I supplied all of the data and code for my thesis research (and a sufficient number of mechanical and electrical drawings). But nobody has Turbo Pascal today, and some of the commercial instruments such as specialized lasers were already obsolete by the time I finished. Also, the experiment was dangerous, and might not pass safety review today. It required about $500k of equipment and a dedicated lab.

Today I have the luxury of saying that if my code fails upon loading a new version of a dependency, the person who discovers that failure is probably skilled enough to fix it, and my work rarely hinges on the idiosyncracies of dependency versions. If it goes into a product, they'll totally rewrite it anyway.

Second, science is still at its core an experimental discipline. Even in physics, there are more experimentalists than theoreticians. Reproducibility means roll up your shirt sleeves and head for the lab. To this day, some processes have not been mechanized, and you still need to spend years developing "lab hands" which not all people succeed at.

I think there's a clue in the fact that the social and medical sciences seem to be the most deeply embroiled in the reproducibility crisis. It's because the quality of results depends on the the quality of measurements, and it's just harder when dealing with living subjects or one-of-a-kind specimens (such as the earth's climate). In fact, not much more than a century ago, it was believed that studying those things was beyond the reach of scientific methodology.

Third, we're not going to stop doing science in areas where it's hard, particularly in medicine, but we're also not going to staff up in areas such as software development, to make science work better. People are suffering from disease right now so there's always an urgency to finding cures, plus an obvious profit motive.

Disclosure: Experimental physicist, developing better measurement equipment.

GPTZero says: "We are highly confident this text was AI generated (100%)"

Am I the only one that's tired of "my random shower thought turned into full article with AI" articles?

> modern science is synonymous with open source software

But why does the public have to pay for e. g. Elsevier? We pay for research of scientists already via taxpayers money (at the least in a civilized country), then we have to pay again for a private entity. If science is really open then it also needs to require public publishing. Gangsters such as Elsevier and others should not be able to drain the public here. Taxpayers financing something should also require public access to findings, at all times. Instead, Elsevier, Springer etc... get more public money while keeping things private. That's the antithesis to science.

[flagged]

[dead]

TL;DR I claim that modern science is synonymous with open source software. This post explains why, why it matters, and what you can (and should) do next.