Using Big-Data Analytics to Improve Quality

This post addresses the expanding use of Big-Data Analytics to improve quality in many organizations.   Davenport (2013) defines Big Data as data that is either too unstructured, too voluminous or from too many different sources to be analyzed by traditional approaches.   The label analytics describes the use of big data to drive decisions and actions.   An informative definition is “Extensive use of data, statistical and quantitative analysis, explanatory and predictive models, and fact-based management to drive decisions and actions” (Davenport and Harris, 2007).

Is use of Big-Data Analytics a significant factor in helping companies perform more effectively?   The MIT Sloan Management Review and the IBM Institute for Business Value conducted a survey of more than 3000 business executives to help answer that question (LaValle, Lesser, et al. 2011).  The respondents were located in 108 countries and in more than 30 industries.   The results indicated a widespread belief that analytics offers value.

  • Half of them stated that improvement of information and analytics was a top priority in their organizations.
  • More than one if five stated that they were under intense or significant pressure to adopt advanced information and analytic approaches.
  • Respondents specified whether their organizations substantially outperformed (top performers) or underperformed industry peers.
    • Organizations that strongly agreed the use of business information and analytics differentiates them within their industry were twice as likely to be top performers.

Interest in big-data analytics is rapidly increasing (Davenport 2013).   Data from Google Trends shows that searches using the term “analytics” 2012 is more than 20 times greater than it was in 2005.   Starting in 2010, searches using the term “big data” increased at a higher rate than searches using “analytics”.

Netflix uses big-data analytics to recommend movies to customers using a system called Cinematch (Davenport and Harris 2007).   Cinematch defines clusters of movies, connects customers to the clusters, and recommends movies the customers will like.   It also recommends movies based on customer evaluations of movies they viewed.  I personally use Netflix, and I value these recommendations since they significantly reduce the time and energy required to identify desirable movies.  Netflix’s revenue increased from $5 million in 1999 to about $1 billion in 2006.   A major reason for its success is that it uses analytics.

Brigham Hospital in the Boston area uses big-data analytics to reduce Adverse Drug Events (Davenport 2013).    Brigham hospital established a Computerized Order Entry (CPOE) system for doctors to input online orders for drugs, tests and other treatments.   This system could also check whether a particular order made sense for individual patients.   For example, the CPOE could check whether:

  • The prescribed drug is consistent with best-known medical practices.
  • Did the patient have past adverse reactions to it?
  • Had the same test been prescribed multiple times before with no apparent benefit?

To prevent dangerous errors, Brigham set up its CPOE system in 1989. They also setup an outpatient electronic medical record system (EMR) at Brigham in 1989.  The EMR contributed outpatient data to the CPOE.  The combination of the EMR and CPOE was very effective in helping to prevent medical errors called Adverse Drug Events.    In the U.S. 14 out of 1,000 inpatients experience Adverse Drug Events.   Before the EMR and CPOE, at Brigham patients experienced about 11 events per 1,000 inpatients.   After implementation of the EMR and CPOE, patients at Brigham experienced about 5 Adverse Drug Events per 1,000 inpatients.   That is a 55% reduction.

However, Snee (2015) points out potential problems with big data.   The data could be observational data not collected from a statistically designed experiment or survey.   The next posting will address this problem and how to reduce its effects.

This posting has been adapted from “Quality Improvement Using Big-Data Analytics“ in the Statistics for Quality Improvement Column appearing in the ASQ Statistics Division Statistics Digest, October 2015.


  1. Davenport, T. (2013). Enterprise Analytics: Optimize Performance, Process and Decisions Through Big Data. FT Press, Upper Saddle River, NJ.
  2. Davenport, Thomas H., Harris, Jeanne, G. (2007). Competing on Analytics: The New Science of Winning . Harvard Business School Publishing Corporation, Boston, Massachusetts.
  3. LaValle, S., E. Lesser, et al. (2011). “Big Data, Analytics and the Path from Insights to Value.” MIT Sloan Management Review 52(2): 21-31.
  4. Snee, R. D. (2015). “A Practical Approach to Data Mining: I Have All These Data; Now What Should I Do?” Quality Engineering 27(4): 477-487.

Leave a Reply

Your email address will not be published. Required fields are marked *