Data has been leveraged by large organizations for a very long time and the evolution of data driven decisions can be described in three distinct stages.
The era of On-Line Analytical Processing Systems (OLAPs), 1980-90
During the 1980s, there used to be two types of applications – Online transaction processing systems (OLTPs) and online analytical processing systems (OLAPs). It was recognized that data, if used for operational reporting requirements, can cause considerable load on the transaction processing systems, and it made sense to build separate systems for reporting requirements. The nature of reporting was mainly operational in nature.
The era of Data Warehouses and Data Marts, 1990-2010
With the evolution of database technologies, a lot of data warehouses were built in the decade of 1990s, to support both strategic as well as tactical decision making. During this time, Ralph Kimball introduced the concept of dimensional modelling, which is a specific way to organize data such that, querying data from relational database systems was most efficient.
Further, in the decade of year 2000, companies like Teradata introduced the concept of massively parallel databases, where how the data is organized becomes irrelevant, as querying data efficiently is handled by the underlying infrastructure.
For many years, these technologies flourished and got widely adopted. These technologies were powerful, but expensive to implement, and only very large organizations could afford to build truly enterprise-wide decision support systems.
The era of big data analytics, post 2010
BIG data technologies were adopted by companies like google and facebook in the decade of 2010, and it caused a revolution in the space of data science. For the first time, open source frameworks like hadoop made it possible to distribute processing of data to several low cost computers, making it feasible to process enormous amounts of data in real time at astonishingly low costs.
This new found ability to process huge volumes of data at low costs, combined with the progress of various machine learning algorithms, resulted into the coining of the term – data science, an intersection of statistics and information technology.