<commit_before>---
title: "Becoming a better scientist with open and reproducible research (2)"
tags: ["open science", "open research", "reproducible research", "talks"]
comments: true
---

This is an modified version of the [Becoming a better scientist with
open and reproducible research](https://lgatto.github.io/rr-publ/)
talk that I gave at TU Delft in May 2019. This version is modified and
was prepared for the [Virtual Bioinformatics Student
Symposium](http://www.rsg-belgium.iscbsc.org/event/student-symposium/student-symposium-04-12-2020/)
on the 4 December 2020. The slides are available
[here](https://lgatto.github.io/2020_12_04_RSG_online/#1).


{% include toc %}

## Introduction

I am always particularly keen to discuss open and reproducible
research, which is a topic that is very close to my heart. Interacting
with early career researchers makes it even more thrilling because
those are the one that will (hopefully) make a bigger change for the
best. Believe it or not, ERCs are unlikely to be able to rely on their
senior peers to drive that change.

**Disclaimer:** I wanted to my personal experiences and opinions. I am
not speaking from authority here. Authority generally comes from
seniority, and in most cases senior academics aren't those that have
much experience in open and reproducible research.

Another thing I don't plan doing here is listing technical solutions
on how to implement open and reproducible research. I would be very
happy to answer your technical questions if you have any, or follow up
by email or twitter. You'll also have the oportunity to discuss with
your peers during the social gathering, many of which will be able to
help very efficiently. The main reason I won't focus on technical
aspects of open and reproducible research is that I very much doubt
that technical aspects are the real barrier. We have many solutions at
hand. The real challenges are the (academic) environment we are in,
the inertia of academia and the vested interests of many senior
actors.

For a start, here are three quick remarks about open and reproducible
research in relation with the title of the talk/post, and I will of
course define/discuss these terms in more details later - I will just
assume for now that you know, at least vaguely, what I am referring
to.

> Open != good (by default)

- A piece of open research doesn't automatically make it good, where
  good is defined as of high academic quality. A piece of closed
  research doesn't make it bad, where bad here is defined of low
  academic quality. So openness doesn't equate to academic
  quality. But openness provides some desired quality (i.e. desirable
  property) independent from academic excellent. Openness leads to
  trust (more details later).


> Reproducible != good (by default)

- A piece of reproducible research doesn't automatically make it good,
  where good is defined as of high academic quality. A piece of non
  reproducible research doesn't make it bad, where bad here is defined
  of low academic quality. So reproducible doesn't equate to academic
  quality. But reproducibility provides some desired quality
  (i.e. desirable property) independent from academic
  excellent. Reproducibility leads (among other things) to trust (see
  details below).

And 

> Open != reproducible 

<blockquote class="twitter-tweet" data-cards="hidden" data-lang="en"><p lang="en" dir="ltr">Many people seem to think that <a href="https://twitter.com/hashtag/OpenScience?src=hash&amp;ref_src=twsrc%5Etfw">#OpenScience</a> &amp; the reproducibility crisis in psychology are somehow causally related. They are not. Open science is decades old &amp; did not focus on reproducibility as a single issue — more here: <a href="https://t.co/KpJHIEqPj3">https://t.co/KpJHIEqPj3</a> &amp; here: <a href="https://t.co/KdMeK6PCUT">https://t.co/KdMeK6PCUT</a> <a href="https://t.co/qF5yPTqNqu">pic.twitter.com/qF5yPTqNqu</a></p>&mdash; <NAME> | Ολίβι<NAME> (@o_guest) <a href="https://twitter.com/o_guest/status/1068791012481470464?ref_src=twsrc%5Etfw">December 1, 2018</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

- Open research and reproducible research aren't the same thing, and
  one doesn't imply the other. Even though in our modern understanding
  of these terms and concepts, they are intimately linked,
  historically, they are very different.

While a lot of the things I will talk about are relatively recent (I
consider myself a *modern* open researcher), note that the principles,
implicit or explicity of open research/science aren't new. I was
pleasantly surprised when learning about the **[Mertonian
norms](https://en.wikipedia.org/wiki/Mertonian_norms)** from 1942.

- **Communism**: all scientists should have common ownership of scientific
  goods (intellectual property), to promote collective collaboration;
  secrecy is the opposite of this norm.

- **Universalism**: scientific validity is independent of the
  socio-political status/personal attributes of its participants.

- **Disinterestedness**: scientific institutions act for the benefit of a
  common scientific enterprise, rather than for the personal gain of
  individuals within them
  
- **Organised scepticism**: scientific claims should be exposed to
  critical scrutiny before being accepted: both in methodology and
  institutional codes of conduct.


## So what is open research?

Open science has seen a continuous evolution since the 17th century,
with the advent of dissemination of research in scientific journals
and the societal demand to access scientific knowledge at
large. Technology and communication has further accelerated this
evolution, and put it in the spot light among researchers and
academics (for examples funder mandates) and more widely in the press
with the cost of publications (see for example this Guardian *long
read* article [Is the staggeringly profitable business of scientific
publishing bad for
science?](https://www.theguardian.com/science/2017/jun/27/profitable-business-scientific-publishing-bad-for-science)
or the [Paywall](https://paywallthemovie.com/) movie).

So what is open research? What characterises a modern open researcher?

Let's start with a definition

> Open science/research is the process of transparent dissemination
> and access to knowledge, that can be applied to various scientific
> practices (image below from Wikipedia):

and a figure

![The six principles of open science](https://upload.wikimedia.org/wikipedia/commons/9/9c/Open_Science_-_Prinzipien.png)


What I dislike about the previous figure is that it can give the
misleading impression that open research is about collecting badges,
and that the more badges you possess, the better an open researcher
you are. And reciprocally, not having any badge to display
**excludes** one from being an open researcher. And as soon as people
start to believe this, we will stop practicing open research and will
start doing stamp collection.

Incidently, here's an ironic tweet that offers tips to do exactly that:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">21 Tips on
how to sound <a
href="https://twitter.com/hashtag/openscience?src=hash&amp;ref_src=twsrc%5Etfw">#openscience</a>:
in the last three weeks before x-mas, I will tweet on tip each day on
how to sound like an open scientist, without actually doing open
science. Enjoy! <a
href="https://t.co/K7TXb9UmHM">pic.twitter.com/K7TXb9UmHM</a></p>&mdash;
<NAME>ⓐgen (@egonwillighagen) <a
href="https://twitter.com/egonwillighagen/status/1334411812973015040?ref_src=twsrc%5Etfw">December
3, 2020</a></blockquote> <script async
src="https://platform.twitter.com/widgets.js"
charset="utf-8"></script>

On a side note, I very much prefer the Martonian norms shown above
that address more fundamental principles intimately related to the
principles of open research.

> Open science/research can mean different things to different people,
> in particular when declined along its many technical and
> philosophical attributes.

An important word above is **excludes**: as thriving open researchers,
we need to understand that it isn't only the distance towards
better/open research that we have travelled that is relevant, but that
the starting point matters a lot. The way somebody practices open
research, whether that person has the possibility to implement this or
that open (and reproducible) research practice, or whether they can be
vocal about it mostly depends on their environment and the support or
push back they get.


Embrace open and reproducible research to the extend you want and you
can. Seek allies and support around you, but do not feel pressured. It
isn't open or close. It is certainly not the same open or close for
everybody.

So my very first take-home messages are:


> Open isn't binary, it's a gradient, it's multidisciplinary, it's
> multidimensional.

How to be an open scientist:

> Let's be open and understanding of different situations and
> constraints, including our own.

## Perverse side-effect: open research as a business

Everything that is labelled as open research isn't open of good. An
easy example is ~~Open Access~~ **Open Ace$$**, a very lucrative
business model set up by commerical publishers... and supported,
directly or indirectly by most researchers.

- **Green open access**: post your **pre-print** (on<commit_msg>fix typos<commit_after>---
title: "Becoming a better scientist with open and reproducible research (2)"
tags: ["open science", "open research", "reproducible research", "talks"]
comments: true
---

This is an modified version of the [Becoming a better scientist with
open and reproducible research](https://lgatto.github.io/rr-publ/)
talk that I gave at TU Delft in May 2019. This version is modified and
was prepared for the [Virtual Bioinformatics Student
Symposium](http://www.rsg-belgium.iscbsc.org/event/student-symposium/student-symposium-04-12-2020/)
on the 4 December 2020. The slides are available
[here](https://lgatto.github.io/2020_12_04_RSG_online/#1).


{% include toc %}

## Introduction

I am always particularly keen to discuss open and reproducible
research, which is a topic that is very close to my heart. Interacting
with early career researchers makes it even more thrilling because
those are the ones that will (hopefully) make a bigger change for the
best. Believe it or not, ERCs are unlikely to be able to rely on their
senior peers to drive that change.

**Disclaimer:** I wanted to my personal experiences and opinions. I am
not speaking from authority here. Authority generally comes from
seniority, and in most cases senior academics aren't those that have
much experience in open and reproducible research.

Another thing I don't plan doing here is listing technical solutions
on how to implement open and reproducible research. I would be very
happy to answer your technical questions if you have any, or follow up
by email or twitter. You'll also have the oportunity to discuss with
your peers during the social gathering, many of which will be able to
help very efficiently. The main reason I won't focus on technical
aspects of open and reproducible research is that I very much doubt
that technical aspects are the real barrier. We have many solutions at
hand. The real challenges are the (academic) environment we are in,
the inertia of academia and the vested interests of many senior
actors.

For a start, here are three quick remarks about open and reproducible
research in relation with the title of the talk/post, and I will of
course define/discuss these terms in more details later - I will just
assume for now that you know, at least vaguely, what I am referring
to.

> Open != good (by default)

- A piece of open research doesn't automatically make it good, where
  good is defined as of high academic quality. A piece of closed
  research doesn't make it bad, where bad here is defined of low
  academic quality. So openness doesn't equate to academic
  quality. But openness provides some desired quality (i.e. desirable
  property) independent from academic excellent. Openness leads to
  trust (more details later).


> Reproducible != good (by default)

- A piece of reproducible research doesn't automatically make it good,
  where good is defined as of high academic quality. A piece of non
  reproducible research doesn't make it bad, where bad here is defined
  of low academic quality. So reproducible doesn't equate to academic
  quality. But reproducibility provides some desired quality
  (i.e. desirable property) independent from academic
  excellent. Reproducibility leads (among other things) to trust (see
  details below).

And 

> Open != reproducible 

<blockquote class="twitter-tweet" data-cards="hidden" data-lang="en"><p lang="en" dir="ltr">Many people seem to think that <a href="https://twitter.com/hashtag/OpenScience?src=hash&amp;ref_src=twsrc%5Etfw">#OpenScience</a> &amp; the reproducibility crisis in psychology are somehow causally related. They are not. Open science is decades old &amp; did not focus on reproducibility as a single issue — more here: <a href="https://t.co/KpJHIEqPj3">https://t.co/KpJHIEqPj3</a> &amp; here: <a href="https://t.co/KdMeK6PCUT">https://t.co/KdMeK6PCUT</a> <a href="https://t.co/qF5yPTqNqu">pic.twitter.com/qF5yPTqNqu</a></p>&mdash; <NAME> | Ολί<NAME> (@o_guest) <a href="https://twitter.com/o_guest/status/1068791012481470464?ref_src=twsrc%5Etfw">December 1, 2018</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

- Open research and reproducible research aren't the same thing, and
  one doesn't imply the other. Even though in our modern understanding
  of these terms and concepts, they are intimately linked,
  historically, they are very different.

While a lot of the things I will talk about are relatively recent (I
consider myself a *modern* open researcher), note that the principles,
implicit or explicity of open research/science aren't new. I was
pleasantly surprised when learning about the **[Mertonian
norms](https://en.wikipedia.org/wiki/Mertonian_norms)** from 1942.

- **Communism**: all scientists should have common ownership of scientific
  goods (intellectual property), to promote collective collaboration;
  secrecy is the opposite of this norm.

- **Universalism**: scientific validity is independent of the
  socio-political status/personal attributes of its participants.

- **Disinterestedness**: scientific institutions act for the benefit of a
  common scientific enterprise, rather than for the personal gain of
  individuals within them
  
- **Organised scepticism**: scientific claims should be exposed to
  critical scrutiny before being accepted: both in methodology and
  institutional codes of conduct.


## So what is open research?

Open science has seen a continuous evolution since the 17th century,
with the advent of dissemination of research in scientific journals
and the societal demand to access scientific knowledge at
large. Technology and communication has further accelerated this
evolution, and put it in the spot light among researchers and
academics (for examples funder mandates) and more widely in the press
with the cost of publications (see for example this Guardian *long
read* article [Is the staggeringly profitable business of scientific
publishing bad for
science?](https://www.theguardian.com/science/2017/jun/27/profitable-business-scientific-publishing-bad-for-science)
or the [Paywall](https://paywallthemovie.com/) movie).

So what is open research? What characterises a modern open researcher?

Let's start with a definition

> Open science/research is the process of transparent dissemination
> and access to knowledge, that can be applied to various scientific
> practices (image below from Wikipedia):

and a figure

![The six principles of open science](https://upload.wikimedia.org/wikipedia/commons/9/9c/Open_Science_-_Prinzipien.png)


What I dislike about the previous figure is that it can give the
misleading impression that open research is about collecting badges,
and that the more badges you possess, the better an open researcher
you are. And reciprocally, not having any badge to display
**excludes** one from being an open researcher. And as soon as people
start to believe this, we will stop practicing open research and will
start doing stamp collection.

Incidently, here's an ironic tweet that offers tips to do exactly that:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">21 Tips on
how to sound <a
href="https://twitter.com/hashtag/openscience?src=hash&amp;ref_src=twsrc%5Etfw">#openscience</a>:
in the last three weeks before x-mas, I will tweet on tip each day on
how to sound like an open scientist, without actually doing open
science. Enjoy! <a
href="https://t.co/K7TXb9UmHM">pic.twitter.com/K7TXb9UmHM</a></p>&mdash;
<NAME> (@egonwillighagen) <a
href="https://twitter.com/egonwillighagen/status/1334411812973015040?ref_src=twsrc%5Etfw">December
3, 2020</a></blockquote> <script async
src="https://platform.twitter.com/widgets.js"
charset="utf-8"></script>

On a side note, I very much prefer the Martonian norms shown above
that address more fundamental principles intimately related to the
principles of open research.

> Open science/research can mean different things to different people,
> in particular when declined along its many technical and
> philosophical attributes.

An important word above is **excludes**: as thriving open researchers,
we need to understand that it isn't only the distance towards
better/open research that we have travelled that is relevant, but that
the starting point matters a lot. The way somebody practices open
research, whether that person has the possibility to implement this or
that open (and reproducible) research practice, or whether they can be
vocal about it mostly depends on their environment and the support or
push back they get.


Embrace open and reproducible research to the extend you want and you
can. Seek allies and support around you, but do not feel pressured. It
isn't open or close. It is certainly not the same open or close for
everybody.

So my very first take-home messages are:


> Open isn't binary, it's a gradient, it's multidisciplinary, it's
> multidimensional.

How to be an open scientist:

> Let's be open and understanding of different situations and
> constraints, including our own.

## Perverse side-effect: open research as a business

Everything that is labelled as open research isn't open of good. An
easy example is ~~Open Access~~ **Open Acce$$**, a very lucrative
business model set up by commerical publishers... and supported,
directly or indirectly by most researchers.

- **Green open access**: post your **pre-print** (on
{'fraction_non_alphanumeric': 0.07169469874387907, 'fraction_numerical': 0.011390249095167128, 'mean_word_length': 4.845986309894212, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 91, 'https://': 36, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6699355', 'n_tokens_mistral': 5347, 'n_tokens_neox': 4835, 'n_words': 2486}