Configuring Data

Many people have argued about data lakes and/or data warehouses. Here is another one on this never ending debate

- Ashish Dubey

All data-driven organisations use data in three ways – to report on the past, to understand the present and to predict the future. Data storage isn’t as simple as it once seemed. Intricate machines and technologies now collect an incredible breadth of data – over 2.5 quintillion bytes every day – from equipment sensors, logs, users, consumers and elsewhere.

Considering the volume and variety of data available today, quite a few misconceptions exist about the ways in which data can be stored. The common argument is around the two predominant types of data storage – data lakes and data warehouses. Data warehouses support reporting and analytics on historical data while data lakes support newer use cases that leverage data for machine learning, predictions, and real-time analysis.

These are the typical arguments one hears in favour of either of them:

Argument #1: You need either a data lake or a data warehouse

This gives the impression that data leaders need to choose one over the other. But the reality is that data lakes and data warehouses serve two different purposes. While both provide storage of data, they do so, using different structures, support different formats and are optimised for different uses. A company may benefit from using a data warehouse as well as a data lake.