Data is a reflection of things and events of the real world in digital form, and good quality data is nothing but a more accurate reflection of the real world.
Data Quality is an extremely important aspect of enterprise data management, and one of the key factors for promoting the usage of enterprise data for business use. It is necessary to ensure that the quality of data within the enterprise continues to improve overtime, enabling business users to trust data.
Building a data quality framework involves the following key steps
Data quality rules
Data quality rules should be consistent across the organization. This activity should happen in conjunction with data cataloging, where data elements defined within the enterprise data catalogue should include data quality checks and acceptable value range. Data quality checks should be built as micro services and deployed across the organization.
Data quality monitoring
Data quality should be measurable and data quality monitoring mechanisms across the enterprise should be consistent, with clearly defined roles and responsibilities for monitoring and remediating data quality issues. This should happen in conjunction with overall enterprise data governance, where data owners and stewards should be published for all data elements within the data catalogue and their responsibilities should be clearly defined.
Data quality improvements
Dedicated efforts should be made to ensure that data quality within the enterprise improves over time. Data quality rules and monitoring mechanisms help in identifying data issues within the enterprise, and facilitate continuous improvement of data quality when supported with dedicated focus and rigour.
- Data enrichment libraries and data integration techniques should be used to improve the quality of enterprise data.
- Data quality improvements need to happen as early as possible in the data lifecycle. Data collection processes should be monitored and improved, applications that collect data need to be upgraded to ensure better data quality.
- With digitisation of the world, a variety of data is now available and accessible. External data should be combined with internal data to get an enriched view of enterprise data and a true picture of data.