Nature Tech Feature: A toolkit for data transparency takes shape

In an excellent article on tools to enhance reproducibility in research, A toolkit for data transparency takes shape, Jeffrey M. Perkel talks of the need for researchers to share code and data, and describes a 'toolkit' (and specific tools) for how to do it properly:

• Use code. 
• Go open-source. 
• Track your versions. 
• Document your analyses. 
• Archive your data. 
• Replicate your environment. 
• Automate. 
• Get help. 

Wiley announces new Data Sharing and Citation policies to improve transparency in research

HOBOKEN, NJ—September 14, 2017— John Wiley and Sons, Inc. (NYSE:JWa) (NYSE:JWb), today announced the launch of new data sharing and citation policies that will be implemented across all participating Wiley journals.  Sharing data enables researchers to reuse experimental results and supports the creation of new work built on previous findings. These new policies will improve the efficiencies of the research process supporting the critical goals of transparency and reproducibility.

http://newsroom.wiley.com/press-release/all-corporate-news/wiley-announces-new-data-sharing-and-citation-policies-improve-tran 

Archaeologists and reproducibility: "Here’s the three-pronged approach we’re using in our own research to tackle the reproducibility issue"

https://theconversation.com/heres-the-three-pronged-approach-were-using-in-our-own-research-to-tackle-the-reproducibility-issue-80997

"Reproducibility is not just a minor technicality for specialists; it’s a pressing issue that affects the role of modern science in society.

Once we’ve identified that reproducibility is a big problem, the question becomes: How do we tackle it? Part of the answer has to do with changing incentives for researchers. But there are plenty of things we in the research community can do right now in the course of our scientific work."

"Another advantage of our choice to use scripts is that we can share them with the scientific community and the public. We follow standard practices by making our script files and main data files freely available online so anyone can inspect the details of our analysis, or explore new ideas using our data."

"Should Journals Be Responsible for Reproducibility?"

"One of the top journals in political science makes data-sharing and replication part of the publication process."

https://www.insidehighered.com/blogs/rethinking-research/should-journals-be-responsible-reproducibility

Reproducible and reusable research: are journal data sharing policies meeting the mark?

Vasilevsky NA, Minnier J, Haendel MA, Champieux RE. (2017) Reproducible and reusable research: are journal data sharing policies meeting the mark? PeerJ 5:e3208 

 https://doi.org/10.7717/peerj.3208

 

Background

There is wide agreement in the biomedical research community that research data sharing is a primary ingredient for ensuring that science is more transparent and reproducible. Publishers could play an important role in facilitating and enforcing data sharing; however, many journals have not yet implemented data sharing policies and the requirements vary widely across journals. This study set out to analyze the pervasiveness and quality of data sharing policies in the biomedical literature.

Methods

The online author’s instructions and editorial policies for 318 biomedical journals were manually reviewed to analyze the journal’s data sharing requirements and characteristics. The data sharing policies were ranked using a rubric to determine if data sharing was required, recommended, required only for omics data, or not addressed at all. The data sharing method and licensing recommendations were examined, as well any mention of reproducibility or similar concepts. The data was analyzed for patterns relating to publishing volume, Journal Impact Factor, and the publishing model (open access or subscription) of each journal.

Results

A total of 11.9% of journals analyzed explicitly stated that data sharing was required as a condition of publication. A total of 9.1% of journals required data sharing, but did not state that it would affect publication decisions. 23.3% of journals had a statement encouraging authors to share their data but did not require it. A total of 9.1% of journals mentioned data sharing indirectly, and only 14.8% addressed protein, proteomic, and/or genomic data sharing. There was no mention of data sharing in 31.8% of journals. Impact factors were significantly higher for journals with the strongest data sharing policies compared to all other data sharing criteria. Open access journals were not more likely to require data sharing than subscription journals.

Discussion

Our study confirmed earlier investigations which observed that only a minority of biomedical journals require data sharing, and a significant association between higher Impact Factors and journals with a data sharing requirement. Moreover, while 65.7% of the journals in our study that required data sharing addressed the concept of reproducibility, as with earlier investigations, we found that most data sharing policies did not provide specific guidance on the practices that ensure data is maximally available and reusable.

New tools for PLOS Authors to support reproducibility and recognition

http://blogs.plos.org/plos/2017/04/protocols-io-tools-for-reproducibility/

PLOS now partners directly with protocols.io to provide authors better ways to share methodological details about their work, practical tools to reduce wasted research efforts and persistent, citable identifiers for laboratory methods. For PLOS, this is a step forward on commitments to address reproducibility and provide improved recognition and credit for all contributions to a work.

This new offering complements PLOS’ already robust data availability policy, requiring that data underlying the conclusions of an article be made available at the time of publication. Since the strengthening of this policy in 2014, about 60,000 articles have been published that contain a Data Availability Statement.

 

Cell Press transforms article methods section to improve transparency and accessibility

Amid discussions around scientific reproducibility, the leading biomedical journal Cell will introduce a redesigned methods section to help authors clearly communicate how experiments are conducted. The first papers using Structured, Transparent, Accessible Reporting (STAR) Methods, which promotes guidelines encouraged by reagent labelling and animal experimentation initiatives, appear in Cell on August 25. The format will then be adopted by other Cell Press journals over the next year, starting with Cell Systems in the fall.


STAR Methods promotes rigor and robustness with an intuitive, consistent framework that integrates seamlessly into the scientific information flow--making reporting easier for the author and replication easier for the reader. The focal point is the "Key Resources Table," which offers an overview of the key reagents and resources (e.g., antibodies, animal models, or software) used to produce the results in the paper.


More at: http://www.eurekalert.org/pub_releases/2016-08/cp-cpt082516.php


Note: unfortunately, while 'easy access' to related data is included in the recommendations, a formal data citation is not.

1,500 scientists lift the lid on reproducibility : Nature News & Comment

1,500 scientists lift the lid on reproducibility

Survey sheds light on the ‘crisis’ rocking research.

• Monya Baker

25 May 2016

A fascinating article on a survey of scientists' experiences on reproducibility

http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970

Research Software Sustainability report

Report on Knowledge Exchange Research Software Sustainability workshop 1-2 October:

http://www.knowledge-exchange.info/event/software-sustainability

Without software, modern research would not be possible. This report recommends practices of software sustainability to minimise the risks of reliability and reproducibility of research.

Nature post: Repetitive flaws: Strict guidelines to improve the reproducibility of experiments are a welcome move.

From next week, scientists who submit grant applications to the US National Institutes of Health (NIH) will be asked to take a little more care. As part of an increasing drive to boost the reliability of research, the NIH will require applicants to explain the scientific premise behind their proposals and defend the quality of their experimental designs. They must also account for biological variables (for example, by including both male and female mice in planned studies) and describe how they will authenticate experimental materials such as cell lines and antibodies.

Read the full post at:

http://www.nature.com/news/repetitive-flaws-1.19192?WT.ec_id=NATURE-20160121&spMailingID=50507456&spUserID=NDg3Mjc3MjM4MzYS1&spJobID=842660933&spReportId=ODQyNjYwOTMzS0

Nature News Feature: How scientists fool themselves – and how they can stop

Humans are remarkably good at self-deception. But growing concern about reproducibility is driving many researchers to seek ways to fight their own worst instincts.

Failure to understand our own biases has helped to create a crisis of confidence about the reproducibility of published results, says statistician John Ioannidis, co-director of the Meta-Research Innovation Center at Stanford University in Palo Alto, California. The issue goes well beyond cases of fraud. Earlier this year, a large project that attempted to replicate 100 psychology studies managed to reproduce only slightly more than one-third². In 2012, researchers at biotechnology firm Amgen in Thousand Oaks, California, reported that they could replicate only 6 out of 53 landmark studies in oncology and haematology³. And in 2009, Ioannidis and his colleagues described how they had been able to fully reproduce only 2 out of 18 microarray-based gene-expression studies⁴.

Read the fully story at:

http://www.nature.com/news/how-scientists-fool-themselves-and-how-they-can-stop-1.18517?WT.ec_id=NEWS-20151008&spMailingID=49730097&spUserID=NDg3Mjc3MjM4MzcS1&spJobID=781102586&spReportId=NzgxMTAyNTg2S0

Nevermind the Data, Where are the Protocols?

Reproducibility of published research results is increasingly coming into question. Efforts from funding agencies and publishers are calling for more and more transparency around research data, but so far, little attention seems to have been paid to a crucial aspect of experimental reproducibility: publication of detailed methodologies. A new NIH-led effort seems a first step in correcting this oversight.
Read more: http://scholarlykitchen.sspnet.org/2014/11/18/nevermind-the-data-where-are-the-protocols/