Eleven tips for working with large data sets
“It’s a mindset,” says Teal, “treating data as a first-class citizen.
https://www.nature.com/articles/d41586-020-00062-z
Showing posts with label data management. Show all posts
Showing posts with label data management. Show all posts
Tuesday, 14 January 2020
Data management for data intensive research (i.e. eResearch) in a nutshell
https://www.nature.com/articles/d41586-020-00062-z
Thursday, 12 September 2019
Everything a Data Scientist Should Know About Data Management
From towardsdatascience.com:
https://towardsdatascience.com/everything-a-data-scientist-should-know-about-data-management-6877788c6a42
By Phoebe Wong & Robert Bennett
To be a real “full-stack” data scientist, or what many bloggers and employers call a “unicorn,” you’ve to master every step of the data science process — all the way from storing your data, to putting your finished product (typically a predictive model) in production. But the bulk of data science training focuses on machine/deep learning techniques; data management knowledge is often treated as an afterthought. Data science students usually learn modeling skills with processed and cleaned data in text files stored on their laptop, ignoring how the data sausage is made. Students often don’t realize that in industry settings, getting the raw data from various sources to be ready for modeling is usually 80% of the work ...
https://towardsdatascience.com/everything-a-data-scientist-should-know-about-data-management-6877788c6a42
By Phoebe Wong & Robert Bennett
To be a real “full-stack” data scientist, or what many bloggers and employers call a “unicorn,” you’ve to master every step of the data science process — all the way from storing your data, to putting your finished product (typically a predictive model) in production. But the bulk of data science training focuses on machine/deep learning techniques; data management knowledge is often treated as an afterthought. Data science students usually learn modeling skills with processed and cleaned data in text files stored on their laptop, ignoring how the data sausage is made. Students often don’t realize that in industry settings, getting the raw data from various sources to be ready for modeling is usually 80% of the work ...
Wednesday, 13 February 2019
Life sciences data steward function matrix - article
Life sciences data steward function matrix
Salome Scholtens; Petronella Anbeek; Jasmin Böhmer; Mirjam Brullemans-Spansier; Marije van der Geest; Mijke Jetten; Christine Staiger; Inge Slouwerhof; Celia W G van Gelder
Sufficient, high quality data steward expertise and capacity in projects and institutes is one of the necessities for FAIR data management in life- sciences and personalised medicine research. In a ZonMw funded project of UMCG, UMCU, Radboudumc, Radboud University and DTL, supported by the relevant national stakeholders, we are working to make the data steward function concrete, to create consensus on the function and required competencies and to develop tailored education. The overall project aim is to professionalise the data steward function within the life-sciences domain, with a special focus on the implementation of the FAIR data principles. All documents related to this project can be found on the Zenodo Collection “Towards a community-endorsed data steward profession description for life-science research”.
This publication contains the first project deliverable: a matrix, that may function as the basis for a common job description of a data steward that is broadly supported within the Dutch life-sciences community. In the next phase of the project, this matrix will be complemented by knowledge, skills and competencies of a data steward, which will be translated into concrete learning objectives. These in turn will be used to develop an education line and training material for data stewards (including a design for an eLearning module). Sustainable implementation and alignment with existing education will be ensured.
from: https://zenodo.org/record/2561723#.XGNSD88zauM
Salome Scholtens; Petronella Anbeek; Jasmin Böhmer; Mirjam Brullemans-Spansier; Marije van der Geest; Mijke Jetten; Christine Staiger; Inge Slouwerhof; Celia W G van GelderMonday, 23 July 2018
"FAIR in practice" - Jisc report now available
To provide a better understanding of current practice with regard to the use of FAIR principles in the UK and the potential these have to enhance research operations and output, Jisc assigned work to Robert Allen and David Hartland of Hapsis Innovation Ltd. They worked in close collaboration with esteemed experts in the UK such as Simon Coles, Susanna Sansone and Melissa Terras and investigated practices and views in various research disciplines.
The results have now been published in the report ‘FAIR in practice - Jisc report on the Findable Accessible Interoperable and Reuseable Data Principles’, that can be found using DOI 10.5281/zenodo.1245567
A blog on the report, titled ‘Open science is all very well but how do you make it FAIR in practice?’ by Rachel Bruce and Bas Cordewener, is available here.
Friday, 16 March 2018
Nature Career feature - Data management made simple
Nature magazine has a Career feature article focused on data management:
Data management made simple
Keeping your research data freely available is crucial for open science — and your funding could depend on it.
It includes:
Data management made simple
Keeping your research data freely available is crucial for open science — and your funding could depend on it.
It includes:
- 12 tips for writing a data management plan
- Who needs them?
- Where do I get help?
- Do plans vary across disciplines?
- Will they improve my science?
Monday, 26 February 2018
Friday, 16 February 2018
International Survey of Research University Faculty: Data Management and Archiving Needs - Published
(From an email sent to the JISC-REPOSITORIES list)
Primary Research Group Inc. has published the International Survey of Research University Faculty: Data Management and Archiving Needs, ISBN 978-157440-496-8
This report, based on responses from 104 faculty from research universities in the USA, Canada, the UK, Australia, and New Zealand, looks closely at the data archiving, curation and management needs of faculty in data intensive fields such as medicine, chemistry, biology, physics, astronomy, statistics and mathematics, engineering, economics and finance, sociology and political science and other data-intensive fields.
The 114-page report reveals the extent to which faculty are impacted by data management and archiving requirements from co-authors, funding sources, academic departments, scholarly publications, data repositories, regulatory bodies, and others with data management requirements.
Data in the report is broken out by many personal and institutional criteria including: age, gender, tenure status, teaching load, subject specialty and academic title of the survey participant, and country of origin, public/private status, and world university ranking of the university affiliated with the survey participant, among other variables. The study looks at how many scholars have imposed data access restrictions on their archived data, and for what reasons or following the strictures of what kind of organization or circumstance. The study also looks at which individuals or offices handle data archiving needs: the individual scholar, academic library, academic department, office of research or other venue.
Just a few of the report’s many findings are that:
Nearly two thirds of post docs and fellows said that the data requirements of journal publishers impacted their work, the highest percentage saying so among all academic titles.
Scholars in physics and astronomy were the most likely to say that the data requirements of funding sources impacted their work.
Researchers based in the UK/Ireland were the most likely to say that there is a great or good deal of need for data management and archiving services (30.77%), followed by those in the U.S./Canada (22.66%) and those in Australia/New Zealand (20.00%).
For 2.91% of those sampled, the university office of research handles data archiving needs.
11.65% of those surveyed have had to impose data access restrictions to their data to meet terms of an intellectual property agreement.
To view a table of contents, an excerpt, the survey questionnaire and a list of the institutional affiliations of the survey participants – or to place an order – view the product page for this report at:
http://www.primaryresearch.
Or call us at 212-736-2316 or email your order to: jmoses@primaryresearch.com.
Primary Research Group Inc. has published the International Survey of Research University Faculty: Data Management and Archiving Needs, ISBN 978-157440-496-8
This report, based on responses from 104 faculty from research universities in the USA, Canada, the UK, Australia, and New Zealand, looks closely at the data archiving, curation and management needs of faculty in data intensive fields such as medicine, chemistry, biology, physics, astronomy, statistics and mathematics, engineering, economics and finance, sociology and political science and other data-intensive fields.
The 114-page report reveals the extent to which faculty are impacted by data management and archiving requirements from co-authors, funding sources, academic departments, scholarly publications, data repositories, regulatory bodies, and others with data management requirements.
Data in the report is broken out by many personal and institutional criteria including: age, gender, tenure status, teaching load, subject specialty and academic title of the survey participant, and country of origin, public/private status, and world university ranking of the university affiliated with the survey participant, among other variables. The study looks at how many scholars have imposed data access restrictions on their archived data, and for what reasons or following the strictures of what kind of organization or circumstance. The study also looks at which individuals or offices handle data archiving needs: the individual scholar, academic library, academic department, office of research or other venue.
Just a few of the report’s many findings are that:
Nearly two thirds of post docs and fellows said that the data requirements of journal publishers impacted their work, the highest percentage saying so among all academic titles.
Scholars in physics and astronomy were the most likely to say that the data requirements of funding sources impacted their work.
Researchers based in the UK/Ireland were the most likely to say that there is a great or good deal of need for data management and archiving services (30.77%), followed by those in the U.S./Canada (22.66%) and those in Australia/New Zealand (20.00%).
For 2.91% of those sampled, the university office of research handles data archiving needs.
11.65% of those surveyed have had to impose data access restrictions to their data to meet terms of an intellectual property agreement.
To view a table of contents, an excerpt, the survey questionnaire and a list of the institutional affiliations of the survey participants – or to place an order – view the product page for this report at:
http://www.primaryresearch.
Or call us at 212-736-2316 or email your order to: jmoses@primaryresearch.com.
Friday, 23 June 2017
Data Management in the Long Tail: Science, Software, and Service
Data Management in the Long Tail: Science, Software, and Service
Abstract
Scientists in all fields face challenges in managing and sustaining access to their research data. The larger and longer term the research project, the more likely that scientists are to have resources and dedicated staff to manage their technology and data, leaving those scientists whose work is based on smaller and shorter term projects at a disadvantage. The volume and variety of data to be managed varies by many factors, only two of which are the number of collaborators and length of the project. As part of an NSF project to conceptualize the Institute for Empowering Long Tail Research, we explored opportunities offered by Software as a Service (SaaS). These cloud-based services are popular in business because they reduce costs and labor for technology management, and are gaining ground in scientific environments for similar reasons. We studied three settings where scientists conduct research in small and medium-sized laboratories. Two were NSF Science and Technology Centers (CENS and C-DEBI) and the third was a workshop of natural reserve scientists and managers. These laboratories have highly diverse data and practices, make minimal use of standards for data or metadata, and lack resources for data management or sustaining access to their data, despite recognizing the need. We found that SaaS could address technical needs for basic document creation, analysis, and storage, but did not support the diverse and rapidly changing needs for sophisticated domain-specific tools and services. These are much more challenging knowledge infrastructure requirements that require long-term investments by multiple stakeholders.
Developments in research data management in academic libraries
Developments in research data management in academic libraries: Towards an understanding of research data service maturity
Cox, Pinfield, Lyon & Kennan (2017)
http://onlinelibrary.wiley.Sunday, 30 April 2017
Toolkit of Best Practice for Research Data Management
The LEARN Project has just released its free Toolkit of Best Practice for Research Data Management:
The Toolkit expands on the issues outlined in the LERU Roadmap for Research Data (2013). It is nearly 200 pages long, and includes:
- 23 Best-Practice Case Studies from institutions around the world, drawn from issues in the original LERU Roadmap;
- 8 Main Sections, on topics such as Policy and Leadership, Open Data, Advocacy and Costs;
- One Model RDM Policy, produced by the University of Vienna and accompanied by guidance and an overview of 20 RDM policies across Europe;
- An Executive Briefing in six languages, aimed at senior institutional decision makers.
LEARN's Final Conference will take place in London on 5 May:
Friday, 21 April 2017
New Open Toolkit for Teaching File and Data Management
The ETD+ Toolkit (https://educopia.org/
The ETD+ Toolkit provides free introductory training resources on crucial data curation and digital longevity techniques. It has been designed as a training series to help students and faculty identify and offset risks and threats to their digital research footprints.
About the Toolkit
The ETD+ Toolkit is the result of a project (https://educopia.org/research
What it is:
An open set of six modules and evaluation instruments that prepare students to create, store, and maintain their research outputs on durable devices and in durable formats. Each is designed to stand alone; they may also be used as a series.
What each module includes:
Each module includes Learning Objectives, a one-page Handout, a Guidance Brief, a Slideshow with full presenter notes, and an evaluation Survey. Each module is released under a CC-BY license and all elements are openly editable to make reuse as easy as possible.
Who it is for:
Anyone may freely adopt and adapt this toolkit. We especially recommend its use by administrators, faculty, and librarians teaching students and by students seeking practical advice about digital content management.
Give us feedback (Please)
Like the modules? Hate them? Think they’re unique? Redundant? These materials have been produced under a grant-funded project, and we are requesting feedback that we can share with our funder and that we can use to improve the Toolkit. Please help us to refine the workshops and report back to our funder about how and where they are being used. Contact katherine@educopia.org
Tuesday, 22 November 2016
http://eprints.whiterose.ac.uk/101389/
Developments in research data management in academic libraries: Towards an understanding of research data service maturity
Cox, A.M. 
, Kennan, M.A., Lyon, L. et al. (1 more author) (2017) Developments in research data management in academic libraries: Towards an understanding of research data service maturity. Journal of the Association for Information Science and Technology. ISSN 2330-1635 (In Press)
, Kennan, M.A., Lyon, L. et al. (1 more author) (2017) Developments in research data management in academic libraries: Towards an understanding of research data service maturity. Journal of the Association for Information Science and Technology. ISSN 2330-1635 (In Press)Thursday, 10 September 2015
Data sharing – build it and they will come
From the University of Cambridge Blog: – case study about development of RDM services in Cambridge
If a tree falls in the forest and no one was there to hear it, did it happen? You could ask the same philosophical question of research – if no-one can see the research results, what was the point in the first place?
Moving science forward and increasing the knowledge of the world around implies exchange of findings. Society cannot benefit from research if there is no awareness of what has been done. Managing and sharing research data is a fundamentally important part of the research process. Yet researchers are often reluctant to share their data, and some are openly hostile to the idea.
This blog describes the research data services provided at Cambridge University which are attempting to encourage and assist researchers manage and share their data.
A tough start Read more:
Friday, 14 August 2015
DataQ: Q & A forum for all things data management
DataQ is live! DataQ is a collaborative platform and community aimed at
addressing research data questions in academic libraries. Check out http://ResearchDataQ.org to find answers to
questions about data management plans, DOIs, ARKs, repositories, and more!
DataQ is a living resource, so continue to submit your questions and our
editorial team of experts will craft responses to share with the community.
Friday, 7 August 2015
Open Data Maturity Model
The Open Data Institute from the U.K. has released the Open Data Maturity Model in March 2015. The model is very similar to the one developed by ANDS in 2011 for research data. This is a good cross domain validation of such an assessment model for organisations.
http://www.ands.org.au/guides/dmframework/dmf-capability-maturity-guide.html
"The Open Data Maturity Model is a way to assess how well an organisation publishes and consumes open data, and identifies actions for improvement.
The model is based around five themes and five progress levels. Each theme represents a broad area of operations within an organisation. Each theme is broken into areas of activity, which can then be used to assess progress."http://opendatainstitute.org/guides/maturity-model
http://www.ands.org.au/guides/dmframework/dmf-capability-maturity-guide.html
Tuesday, 13 January 2015
J of eScience Librarianship - issue on data
Dear Colleagues,
The latest issue of the Journal of eScience Librarianship (JESLIB) has just been published! It is available at http://escholarship.umassmed.
Table of Contents
Volume 3, Issue 1 (2014)
Editorial
What is Data Literacy?
Elaine R. Martin
http://dx.doi.org/10.7191/
Full-Length Papers
Planning Data Management Education Initiatives: Process, Feedback, and Future Directions Christopher Eaker
http://dx.doi.org/10.7191/
A Spider, an Octopus, or an Animal Just Coming into Existence? Designing a Curriculum for Librarians to Support Research Data Management Andrew M. Cox, Eddy Verbaan, and Barbara Sen
http://dx.doi.org/10.7191/
An Analysis of Data Management Plans in University of Illinois National Science Foundation Grant Proposals William H. Mischo, Mary C. Schlembach, and Megan N. O'Donnell
http://dx.doi.org/10.7191/
Initiating Data Management Instruction to Graduate Students at the University of Houston Using the New England Collaborative Data Management Curriculum Christie Peters and Porcia Vaughn
http://dx.doi.org/10.7191/
EScience in Action
Research Data MANTRA: A Labour of Love
Robin Rice
http://dx.doi.org/10.7191/
Building Data Services From the Ground Up: Strategies and Resources Heather L. Coates
http://dx.doi.org/10.7191/
Building the New England Collaborative Data Management Curriculum Donna Kafel, Andrew T. Creamer, and Elaine R. Martin
http://dx.doi.org/10.7191/
Lessons Learned From a Research Data Management Pilot Course at an Academic Library Jennifer Muilenburg, Mahria Lebow, and Joanne Rich
http://dx.doi.org/10.7191/
Gaining Traction in Research Data Management Support: A Case Study Donna L. O'Malley
http://dx.doi.org/10.7191/
The New England Collaborative Data Management Curriculum Pilot at the University of Manitoba: A Canadian Experience Mayu Ishida
http://dx.doi.org/10.7191/
Are you interested in submitting to JESLIB? Please refer to author guidelines at http://escholarship.umassmed.
Regards,
Elaine R. Martin, Editor, and the Editorial Team, Journal of eScience Librarianship
Start your own SelectedWorks to get more readership http://works.bepress.com/sw_
If you do not wish to receive these notifications, please click the link below:
http://escholarship.umassmed.
The latest issue of the Journal of eScience Librarianship (JESLIB) has just been published! It is available at http://escholarship.umassmed.
Table of Contents
Volume 3, Issue 1 (2014)
Editorial
What is Data Literacy?
Elaine R. Martin
http://dx.doi.org/10.7191/
Full-Length Papers
Planning Data Management Education Initiatives: Process, Feedback, and Future Directions Christopher Eaker
http://dx.doi.org/10.7191/
A Spider, an Octopus, or an Animal Just Coming into Existence? Designing a Curriculum for Librarians to Support Research Data Management Andrew M. Cox, Eddy Verbaan, and Barbara Sen
http://dx.doi.org/10.7191/
An Analysis of Data Management Plans in University of Illinois National Science Foundation Grant Proposals William H. Mischo, Mary C. Schlembach, and Megan N. O'Donnell
http://dx.doi.org/10.7191/
Initiating Data Management Instruction to Graduate Students at the University of Houston Using the New England Collaborative Data Management Curriculum Christie Peters and Porcia Vaughn
http://dx.doi.org/10.7191/
EScience in Action
Research Data MANTRA: A Labour of Love
Robin Rice
http://dx.doi.org/10.7191/
Building Data Services From the Ground Up: Strategies and Resources Heather L. Coates
http://dx.doi.org/10.7191/
Building the New England Collaborative Data Management Curriculum Donna Kafel, Andrew T. Creamer, and Elaine R. Martin
http://dx.doi.org/10.7191/
Lessons Learned From a Research Data Management Pilot Course at an Academic Library Jennifer Muilenburg, Mahria Lebow, and Joanne Rich
http://dx.doi.org/10.7191/
Gaining Traction in Research Data Management Support: A Case Study Donna L. O'Malley
http://dx.doi.org/10.7191/
The New England Collaborative Data Management Curriculum Pilot at the University of Manitoba: A Canadian Experience Mayu Ishida
http://dx.doi.org/10.7191/
Are you interested in submitting to JESLIB? Please refer to author guidelines at http://escholarship.umassmed.
Regards,
Elaine R. Martin, Editor, and the Editorial Team, Journal of eScience Librarianship
Start your own SelectedWorks to get more readership http://works.bepress.com/sw_
If you do not wish to receive these notifications, please click the link below:
http://escholarship.umassmed.
Monday, 24 November 2014
Check list available for social science researchers; - assume it could also be used to identify relevant areas that should be addressed in a policy.
http://www.gesis.org/
http://www.gesis.org/
Monday, 17 November 2014
Data management stories: good and bad
Wednesday, 22 October 2014
Elsevier Rise of the Databrarian webinar
A recording of the recent Elsevier Rise of the Databrarian webinar is available at
http://libraryconnect.
“With
the increased pressure in the research community to make research data
available, librarians are becoming more involved in the process.
Librarians are well suited to these new
tasks with their experience in curation, description, preservation and
access. In this one-hour webinar, learn how librarians are transitioning
and enhancing current skill sets, and what new skills are required.”
Topics:
- What is in the data librarian’s toolbox
- You don’t need to be an informaticist
- Training and networking opportunities
- Pitching new data services to library leaders
- Balancing RDM with other duties – what’s realistic
Friday, 26 September 2014
Short, interesting video resources for data management
DCC: Managing Research Data
In short: a range of fast moving viewpoints highlighting various aspects of research data management including its benefit to institutional reputation, advancement of knowledge etc. Acknowledges researcher fears for sharing data.
On the Clock: Open Data Executive Order
about 2 mins
In short: this is direct from the White House and the Obama Administration and covers why open data is good for America. Kinda like speed dating: 2 guys making the points in 2 minutes!
Whose data is it anyway?
From Yale School of Medicine
about 2 mins
In short: researchers need to share their data for the advancement of medicine
Sharing data: good for science, good for you
From DANS, Netherlands
4.10mins
In short: starting with the inspiring Neelie Kroes from the European Commission, covers why open data is important and how researchers can benefit (including getting cited). It’s a mix of English and Dutch with English subtitles.
https://www.youtube.com/watch?
Subscribe to:
Posts (Atom)