Database, Data Warehouse, and DataLake

https://miro.medium.com/max/1200/0*lpTyvlOtgJGScfRb

Original Source Here

Database, Data Warehouse, and DataLake

Photo by Kirill Sh on Unsplash

Any business system in the world needs a database to save the daily transactional data, when users process the daily routines of the business, they are silently linked with the database in the background, the normal transactional events like swiping a card or withdrawing money, the database in the backend is running or validating the transactions. The main purpose of such databases to support transaction processing and any other information related to the business event like deposits and withdrawals, that need to be saved in the database for future retrieval.

The stored data in the database is usually required by business analysts, and by decision-makers in order to analyze the patterns and correlation from a large number of business transactions and many business activities. The real concern for the business analyst is to extract useful information from the business data, there are many types of business patterns that exist in the database which will be examined by the business decision-makers like declining profit state, the shortage of inventory, the sales of the products, and many more.

The database serves great in respect to storing the transactional data but not good at analytical work, the front desk executives also not interested in finding the best selling product or the fall of profit, but these types of reports are more important for higher management of the company, therefore a need for the data warehouse has been evolved, the need to merge the daily transaction data as well as the analytical or accumulative data, the data in the warehouse support large scale analysis and insights of the business activities. The data warehouse is a data management platform, where data extracted from multiple sources of transactional data into the data warehouse, the data converted and transformed into the shape of summarized data and give its users business intelligence and deep insights into the business activities.

The analytical processing requires a lot of reading operations from the database, the data warehouse is designed with optimization in mind, the structure of tables and interconnection is specially designed for fast data retrieval, the column space of the tables in the data warehouse keep optimized for faster query and reduction in computation overhead. The data warehouse schema is designed with respect to analysis and business intelligence, the data warehouse computes the analytical events in a highly efficient and comprehensive manner.

The database is responsible for transaction processing, the data warehouse is mostly responsible for business analysis-related matters, in some situations, a business needs both transaction processing and business analysis at the same time, the requirement of such a system is to gain the advantage with the power of transaction processing and analytical processing, therefore, the data lake has been born.

If the company is constantly growing then data becomes more important for finding out the historical, real-time, online, offline, internal, external, structured, and unstructured business activities, the data in the transactions processing database or in the data warehouse is not enough to handle the business decision making but the data in the data lake serves great, only you need to pour the data in, from multiple sources and hoard the data for further business intelligence.

The data lake provides on-time intelligence of the data rather than historical business intelligence insights, the storage architecture of the data lake has sufficient scalability and reliability so the data can be carried for a long period of time, many times before committing any transactions, the insights of the past and present of the data are important therefore data lake provide extended services which data warehouse unable to give its users.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: