Everything a Data Scientist Should Know About Data Management

From towardsdatascience.com:

https://towardsdatascience.com/everything-a-data-scientist-should-know-about-data-management-6877788c6a42

By Phoebe Wong & Robert Bennett

To be a real “full-stack” data scientist, or what many bloggers and employers call a “unicorn,” you’ve to master every step of the data science process — all the way from storing your data, to putting your finished product (typically a predictive model) in production. But the bulk of data science training focuses on machine/deep learning techniques; data management knowledge is often treated as an afterthought. Data science students usually learn modeling skills with processed and cleaned data in text files stored on their laptop, ignoring how the data sausage is made. Students often don’t realize that in industry settings, getting the raw data from various sources to be ready for modeling is usually 80% of the work ...

Microsoft releases data use agreements for open data

Microsoft has released three data use agreements for open data, and is asking for public comments/feedback before the end of September 2019:

• Open Use of Data Agreement (O-UDA)
• "Designed for use with open datasets which don't include personal data or data owned by a data provider. It is the most open and least restricted of the three first proposals."
• Computational Use of Data Agreement (C-UDA)
• "The goal of the C-UDA is to define a use of data sets for AI training purposes that contain third party materials, in a manner consistent with law."
• Data Use Agreement for Open AI Model Development (DUA-OAI)
• "The goal of the DUA-OAI is to create a template agreement that parties might use to share data to train an artificial intelligence (AI) model and then to make that trained AI model publicly available through an open source licensing structure."

From ZDNet:

Microsoft looks to 'do for data sharing what open source did for code'
https://www.zdnet.com/article/microsoft-looks-to-do-for-data-sharing-what-open-source-did-for-code/

The agreements are listed here:
https://news.microsoft.com/datainnovation/#data-use-agreements

Decoding ‘Game of Thrones’ by way of data science

“With the final season of the television series ‘Game of Thrones’ upon us it is a good opportunity to take a closer look at the books that the series is based on. We will discover how a numerical processing of the books can help us reveal patterns that lie hidden in ‘A Song of Ice and Fire’.”

https://blog.usejournal.com/decoding-a-game-of-thrones-by-way-of-data-science-fd81e66d1255

Technical debt for data scientists

“We take shortcuts in developing a solution without an understanding of the risks and costs of those shortcuts, and without a realistic plan for how we’re going to pay back the debt ... The result is that data science projects become expensive or impossible to maintain as time goes on.”

This blog post explains how to tell if your projects are too deeply in debt and what to do about it:

https://blog.shotwell.ca/posts/2019-04-19-technical-debt-in-data-science/