Ongoing peer review – #dreambig?

What’s the best way to have a scientific debate? I think, especially when it comes to public health research, the answer is not “through memes”.

When this happened to me the other day, I didn’t think much of it. I consider myself a pretty hip researcher. I study video games and hang out with other game researchers. I feel confident wearing a t-shirt and blazer to conferences. I talk openly about playing video games (but also about what it means to have problems related to playing too much). I understand a lot of pop culture references and I like a good GIF[1] as much as the next scientist.

However, I did find it the above meme a little insulting when it was used as a critique of a recent paper. I was also surprised because I really liked this scientist the couple of times I hung out with him, despite our fundamental disagreement on whether the current evidence for problematic gaming supports a having a specific diagnosis yet. It almost seemed like a cyberbullying tactic (“Posting a mean or hurtful picture or video“). Those slight negative feelings were mostly overshadowed by being super impressed with my co-author Daniel Kardefelt-Winther’s reasoned response, and I was glad that this encouraged a good discussion with another scientist later. I replied to the original “critique” in a clever (I thought) but reasoned way, pointed out that his argument was inconsistent with his own work, corrected a link (man, if you’re going to draw attention to our article, please at least link the URL correctly), and moved on.

But then it was pointed out to me that seeing such an unprofessional jab from one scientist to another was discouraging to students who were thinking of becoming game studies researchers. It doesn’t look good for the field. And that was a really good point, because video game research has been the subject of debates for decades, and what we don’t need to do is scare away reasonable people who could make good contributions.

After being part of an early commentary on the lack of consensus for Internet gaming disorder criteria, I had high hopes for the issue of video game “addiction” being a way to bring together people who have differing opinions about gaming. My personal (and scientific) belief is that these types of debates will only be resolved by systematically and transparently gathering all evidence and all stakeholder perspectives and using formal consensus development techniques. There is just too much potential bias on the part of people who are afraid of games and people who love games. After the initial collaboration, other papers sparked out and some people wrote other commentaries, started an open definition of behavioral addiction and invited input, which also sent a lot of web traffic to the journal that published a second debate paper.

Sadly, the last round of commentaries to the Scholar’s Open Debate paper (2271 reads and 35 citations, by the way, according to ResearchGate) seemed to devolve a bit into some ad hominem attacks[2], and it really is discouraging to see. It’s not a good way to make a point. I don’t have formal training in debate or critical thinking (just the regular this-is-how-you-critique-a-scientific-article kind), but it seems pretty clear that challenging rationale, methods, logic and conclusions are better ways to critique science than disparaging people or, in the case of the meme-jab, peer review.

I’m not sure what would be good to fix this, though. I still like that researcher, but it’s harder to think about trusting him enough to work with him now, which is a shame, because I would have loved to have his unique perspective on future studies. Also, he was trained by another collaborator who I would really like to work with more, so that’s a little concerning too. Not to say I’m perfect; God knows I’ve made mistakes and will continue to make them. I just hope that we can all keep moving forward in a way that legitimizes video games and video game research rather than mocking it.


[1] Pronounced “JIF”, according to the creator of this format – thanks to Sanjay Srivastava (@hardsci ) for the reference

[2] (a) “The notion is implied that the authors perceive themselves as the only saviors of good scientific practice” and (b)“…a number of the coauthors of the Aarseth et al.’s (2016) study have not published any empirical data in this area and may therefore not be sufficiently familiar with the debates in this field.”