An enterprise data architecture is a blueprint of various architectural components that are typically used to manage enterprise data and leverage it for various business objectives.
These are some of the important components of a typical enterprise data architecture. Note that companies can choose some or all of them depending on their specific needs and plans.
Data Streaming Solutions
Various applications within the enterprise hold similar data. These applications need to ensure that they are synchronised, and data within these applications need to be integrated. There should be clear policies around de-duplication of data and synchronisation of data across various applications. Data needs to be synchronised across these applications in real time, and enterprise bus service is the most recommended solution for achieving the same.
Master Data Management Solutions
Master data includes data about the customers, products, employees, vendors and business data. These need to be managed properly, so that their is a common view of master data across the enterprise. This also involves managing reference data, which is used to standardise master data within the organization.
Data Quality Management Solutions
Data quality needs to be measured, monitored and improved on a continuous basis. There should be dedicated efforts towards this goal, and solutions that help achieve the goals. These solutions comprise of defining business rules to measure data quality across the enterprise, and ensure quick remediations on critical data quality and data integrity issues.
Metadata Management Solutions
Data should be easily accessible to business users, but still secure. Business users should be able to quickly search data elements, view their definitions, refresh times, source, owner, context, contents, profiles etc in a very easy and intuitive manner. Metadata management solutions should combines business, technical, operational and usage metadata together and present it to authorised users so that they can use data effectively to solve business problems.
Data Access Layers
Data stored inside a various applications, data warehouses, data lakes or data marts can be cryptic because it is modelled to optimize storage and retrieval. A semantic layer converts this data into business friendly structure, so that it can be consumed by business analysts. It also provides the necessary tools like data catalog and traceability for users to make sense of data. It also enables authorised data access, thereby ensuring data confidentiality, security and protection. Data reports and visualizations provide data insights to business users to help them take various decisions.
Analytical models are algorithms that use data to predict outcomes and take actions. They are used to derive automated insights from data and help businesses in taking crucial decisions in real time. They are also used to improve various products and services of the organization.
Enterprise Data Warehouses
An Data Warehouse is a collection of enterprise data in a structured form that represents the single instance of truth. It is a highly trusted source of enterprise data available in the lowest grain that can be used by various data and business analysts across the organization to achieve various data driven business objectives. Building and managing an enterprise data warehouse is an extremely costly process, and hence, it is used for highly critical business requirements.
Data Lake is a big data platform for storing data that is not structured or extremely large in volume. Is is also an ideal repository to store enterprise data that is currently not used by any business users, but has the potential of being useful sometime in the future. It requires minimal cost to build and manage a data lake and store data in it. Hence is it a popular choice for exploring enterprise data for potential use cases, or for business requirements that dont justify the costs associated with the enterprise data warehouse.
Data Marts are relatively small repositories of data that generally stores data pertaining to a specific use-case. Data inside a data mart can be highly processed and aggregated, and generally used for reporting and analytics purposes. Data marts are sometimes more convenient to use than enterprise data warehouse, as they store data that is pre-processed for a specific use-case, thereby isolating that use-case from the rest of the noise.
Data labs provide an opportunity to data analysts and data scientists to explore enterprise data to create new use cases of leveraging data for business insights. They source data from both data warehouse as well as data lake, and do further processing to data to build data sets that help them to support their hypothesis.