[Event] The new practices of scientific evaluation : 22th of may, 6-8 p.m. Paris

Peer-review has come under an unprecedented crisis. The rising rejection rate of academic articles have casted a widespread doubt on the main regulatory process of scientific publishing. Simultaneously, the web 2.0 allows the emergence of uncostly, feasible alternatives. Some people goes as far as talking about an academic spring.

Could daily research work without traditional peer-review? Will the assessment of scholar works require some radically different rules? Or could peer-review survive and evolve into something more compliant with current researchers’ practices?

In order to adress theses significant issues, HackYourPhd welcomes you at the maison de la Recherche. on Wednesday, the 22th of May from 6 to 8 p.m.

What is the future of peer-review?

 Peer-review is far from being a recent invention. This social device seems actually quite essential, both as a way to control scientific activity and as a way to enhance a mutual knowledge of each other’s work within the scientific community.

Peer-review involves a re-reading of a submitted article by fellow researchers belonging to the same disciplines, and therefore apt to bring helpful criticism. Anonymity of the evaluators (or, even, of both the evaluators and the authors) is one of the main guarantees of an efficient and uncompromising peer-review.

As soon as the XVIIth century, several bristish journals set up informal committe whose main mission was to verify the accuracy and interest of the published information. Yet, peer-review really establishes itself as an unconditionnal standard during the XXth century. By then scientific publishing is more and more dominated by wide economic corporation, like Elsevier. The accuracy of a publication is no longer a relative virtue, evolving on a long terme basis. It becomes a selling argument. The peer-review process requires therefore some very specific stages and formal interaction. The drawing below describe the main procedures of “standardized” peer-review:

Schéma Peer-Review simplifié

The generalisation of open access publishing has significantly contributed to reveal the limitations of peer-review. Since the beginning of the 00s, the rejection rate (i. e. the proportional number of articles dismissed by a journal after being published) has experienced an exponential growth. This does not mean that science is less accurate than before but that fraud can be more easily detected. For instance, the psychologist Diedrick Stapel has mystifyed dozens of peer-review comittees for twenty years till several young researchers became increasingly doubtful of the reliability of his work.

Simultaneously, alternatives gain some widespread support. Back in 1982, Perloff and Perloff proposed in some sort of humoristic way a well-established model nowadays: automatic self-archiving, frequently occuring before the peer-review process (the article is therefore called a preprint). In this outlook, every publication is kept, disregarding its quality. In physic and mathematic research, self-archiving (in ArXiv for instance) has become a very current practice.

For few years, we can note the emergence of autonomous peer-review communities, disconnected of a specific journal. Peer-Evaluation.org or F1000 collect the feedbacks and suggestions of fellow researchers, beyond the classic editorial process.

Out of the academic world, the online collaborative projects adapt the main principles peer-review to their specific needs. Wikipedia depends on a continuous collective assessment : « Taken individually, wikipedians are much less clever than experts, yet as each one has to be the ignorant master of the others […] they create a stronger form of publishing knowledge than the one actually used by experts »

With theses rising new uses, is the very concept of academic article still meaningful? Preprint, postprint, blog posts: the growing number of alternative versions make it all the more vain to have a unique and ponctual assessment.

HackYourPhd Workshop on the new practices of scientific publishing

In order to discuss this significant issue, HackYourPhd welcomes you at the maison de la Recherche on Wednesday, the 22th of May from 6 to 8 p.m. The workshop will open with a debate led by Pierre-Carl Langlais (our “hacker” PhD student — see the bio…), that will help to summarize the current state and ongoing evolutions of peer-review.

Then, all these thoughts will materialize into a more concrete case : the conception of an alternative journal, applying new means of assessment. Most aspects of its activities will be questionned, whether it be the editorial format, the economic organisation or the involvement of the reader.

This case study is not a pure conceptual experience. It will enhance a project incepted a few months ago on the wiki@research.com mailing list, under the code name « Knowledge Now » : a periodic academic publication on the new scientific mediations. This aims of the journal are note merely theoretical. Following the advice of the creator of the wiki software, wiki, Ward Cunningham, the journal will implement some of the very mediations it claims to study.

A survey of the workshop will be published in the main French newsletter on wiki studies Nouvelles du Wikilab.

We’re hoping to see you come in numbers !

Our “hacker” PhD student : Pierre-Carl Langlais

Pierre Carl langlais





I am a French PhD student at Paris-Sorbonne in information science. My doctoral thesis bears upon the introduction of the stock exchange section in the daily press. Besides financial information, my main research fields include datajournalism (participation in the Jourdain project, funded by the CNRS) and Wikipedia. A regular contributor to the French Wikipedia since 2006, I have become one of its sysop in october 2012. I regularly publish articles on Wikipedia and open knowledge topics on Rue89.



Ce message est également disponible en : French