Data lake

What it means

A data lake is a central repository that stores data from various sources in their original form, including unstructured data. It may hold ERP tables, website logs, documents, images or sensor readings, which are processed only when someone needs them.

Unlike a data warehouse, where data are cleaned and structured before loading, a data lake stores them unchanged. That makes it more flexible and cheaper for large volumes, which suits big data analysis and machine learning.

Modern solutions combine both approaches in a so-called lakehouse, where raw data are stored cheaply and curated layers for reporting are built on top. It typically runs on cloud storage, paid for by volume and usage.

How it works in practice

Data reach the lake through ETL or ELT processes that regularly extract them from source systems. Analysts and data scientists take what they need from the lake and prepare inputs for business intelligence tools and dashboards.

For smaller firms a data lake is often too big a step. If the company mainly needs financial and sales reports, a data warehouse or a well-configured BI tool is enough. A data lake pays off when large volumes of varied data are combined and the firm has people who can use them.

Cataloguing is key: every dataset should have a description, an owner, a source and a last-updated date. Without these nobody knows which data are reliable, and analysts spend more time searching than analysing.

What to watch out for

Without data governance a lake easily becomes a data swamp full of duplicate, outdated and incomprehensible files. Rules for naming, quality and deletion must be set from the start, not once the lake has become impossible to navigate.

Storing everything just in case conflicts with the GDPR data minimisation principle. If the lake holds personal data, you need to know the purpose, who has access and when the data will be deleted, and sensitive data should be pseudonymised.