How can you improve data quality maturity across domains and industries?

One has to own the understanding around the objective of data. The more accurate the understanding, the better is the data outcome. When we say data it primarily comprises three prominent phases and one has to own and drive it all to integrate quality and integrity in data. The three phases are-understand, observe and analyse. To make a great start towards building quality data, the first and essential step one has to take along with understanding or owning a data is, knowing your objective to map your data. Once this phase is achieved, key is to observe how your data is being collected to shape towards your desired outcome. Analyse your data in reference to the objective while you collect and shape your data. Makes it a continuous cycle.

Assessing your current state is vital, but TRICKY. You need to answer some important questions before you go crazy with the initial assessments: 1. What are the important data sets and critical data elements for business operations and decision-making 2. Who are the consumers of the critical data, and how would they define fit-for-purpose When you have this direction, you need to stay focussed on understanding the last 200 to 300 records that have been added. Try not to get bogged down in details and full-blown profiling and assessment. This will come later. You are trying to get a high-level view of: 1. THe state of your CDEs 2. The cost impact 3. The behaviour towards "high-quality data"

Assessing the current state of data quality is critical and it has to start with what aspect is important for your usage. Defining metrics to measure, target goals, and tolerance levels is very importance to measure if the steps taken to improve it are effective. The important quality aspects could include data loss in collection or curation, timestamp or typology variations in data, variations between channels or sources, data type variations, missing required attributes, variations from architecture or delays beyond service level agreements.

1. Define the Critical Data Elements (CDE) and Critical Risk Measures (CRM). 2. Define the tiering and threshold for the CDE and agree with upstream data providers. 3. Work with upstream on the data quality observations and have a treatment plan for fixes (if required). 4. Socialise with the process owner on the data gaps and agree on risk acceptance for the process. 5. Transparent on the Data Quality Outcome.

This initial step in the journey is absolutely crucial. Often, data teams find themselves in a reactive mode, addressing data-related issues as they arise. However, it's imperative to proactively establish a robust monitoring system that allows for an accurate assessment of the current state of DQ. By doing so, data teams gain a comprehensive understanding of their data quality landscape. This strategic approach ensures that their efforts lead to meaningful and impactful improvements. In the dynamic realm of data, constant change is the norm. Therefore, maintaining a clear and precise view of data quality becomes instrumental in helping the entire organization gauge whether DQ is on an upward trajectory or, in some cases, regressing.

It is crucial to assess the current state using a data quality maturity model such as DAMA's. It's a comprehensive framework that helps organizations understand their role in the data quality process. Identifying the five dimensions—awareness, people, process, tools, and metrics—is essential for a holistic approach. Identifying strengths and weaknesses does not only guide targeted improvements, but also fosters a culture of continuous improvement in data management. I'm eager to see more organizations embrace this approach!

I feel Data Health is a better word to use ,for better understanding. Now , if we want to monitor, track and maintain the healthiness of our data; we must define predefined standards/matrices to measure the health or quality, and set thresholds in context of the domain/industry knowledge . Consider it similar to our blood report , we have several features/tests showing the blood health and then a defined range, which indicates if everything is good or needs some action.

One of the most beneficial aspects of a current state assessment is to inform the baseline for quality metrics. Once a baseline is available, it's easy to measure and compare against change initiatives (before vs after). With a baseline, it's easy to gather and present a factual view of the differences (between the old and the new world). It's also a great way to compare against industry benchmarks for data quality - at the start of change initiatives and on an on-going basis.

Governance: Policies, responsibilities, & processes. DQ Assessment: Metrics, data sources, & tools to measure accuracy, completeness, & consistency. Data Standardization: Practices, formats, naming conventions, & definitions. Cleaning & Enrichment: Tools to eliminate errors, duplicates, & inconsistencies. Training/Awareness: Enhances data literacy, ensures accuracy & consistency. Tech Integration: Leveraging tools like AI, machine learning, & automation. Collaboration: stakeholders, i.e. IT, analysts, & business units. Cont. Improvement: DQ processes on feedback, business needs, & advancements. Regulatory Compliance: Adherence to data privacy regulations & standards Establishing Metrics: KPIs to measure DQ initiatives.

Governança de dados não pode ser negligenciada. Uma empresa data-driven tem em seu core, a qualidade dos dados e metadados. Definições de ciclo de vida dos dados, indicadores chave para gestão de metadados são ferramentas para que permitem alcançar uma melhor qualidade e confiança na informação gerada.

This involves setting goals for data quality maturity and pinpointing critical issues. Leveraging frameworks like DAMA's assists in goal setting and prioritization. Such frameworks align data quality goals with business strategy, governance, and management principles. They also help in understanding initiative scope, scale, and complexity. This step ensures that data quality improvements align with broader business objectives and the intricacies of managing and governing data effectively.

It is a crucial step in data quality management. Using frameworks like the one by DAMA facilitates organizations to align their data quality objectives with overarching business strategies. This alignment is crucial for ensuring that data quality efforts provide tangible business benefits. Additionally, such frameworks provide guidance on the scope and scale of data quality initiatives, which is essential for effective resource allocation and management. Setting the right priorities helps in tackling the most critical data quality issues first, making the journey towards data maturity both efficient and beneficial. #DataQualityGoals

We align data quality initiatives with the client’s business strategy by setting clear, prioritized goals. Using MDM software, we streamline this with frameworks that connect quality objectives to governance practices. For instance, a client in retail needed immediate improvement in product data accuracy. Our approach: define a step-by-step plan that balanced urgency with scalable, strategic enhancements across domains.

From many personal conversations with companies, it turns out that the role in the company has been clarified as to who is responsible for the topic. Unfortunately, these issues do not receive the management attention they need. Having the right data should be a strategic goal of companies - growth with the wrong customer data is not possible or only possible with difficulty. That's why there should be people at VP or board level who are responsible for the accuracy and completeness of their own data within the company and who declare it to be part of their own board agenda.

Roles and responsibilities are mainly defined through the Data Governance Framework. Some of the key roles necessary for the DQ process to function well are: - Data Owner (responsible for ensuring adequate data quality in his domain) - Data Analyst (knows the Data Governance process and provides support to the Data Owner) - Application expert (Knows applications and systems in which data is stored)

Someone needs to be responsible for data quality. It doesn't happen on its own. You will need: 1) Data program manager 2) Data analysts and potentially: 3) Data solution developer

This is a bit like litter on the streets, it is easy to notice poor data quality and agree something needs to be done about, but few actually take any action. It needs real drive from a senior level to free up people's time from their operational responsibilities to do something about this across the organisation. In practice, it is easier to start small, particularly in areas where business change is already happening. Attempting to do an upfront organisation-wide change is difficult. When starting small, the organisation can learn from the smaller initiatives before scaling up.

The Data Quality Pro model is an excellent tool for this task. By defining who is responsible and accountable for various aspects of data quality, organizations can ensure effective management and stewardship of their data assets. This clarity does not only facilitate collaboration among data producers, consumers, and stewards, but also facilitates the allocation of resources and budget. Remember, data quality is a team effort, and having a well-structured organization model is essential to success. #DataQualityManagement

This focuses on defining roles for data quality improvements. Utilizing models like Data Quality Pro's helps in assigning accountability and authority. These models establish clarity in roles, encouraging collaboration among data stakeholders. They also streamline resource allocation and budgeting for data quality activities. By defining these roles, organizations ensure a cohesive approach for managing data quality, fostering collaboration, and optimizing resource utilization toward quality improvement initiatives.

To add to this, personnel training is a key here, they must understand the importance of the data being collected and why they can't compromise on quality

Data quality success hinges on accountability. We structure roles using a data stewardship model, integrated into our MDM system. One client, for instance, benefited from clear responsibilities, with data stewards empowered to make critical quality decisions. This proactive organization fosters collaboration and ensures data management is a shared, well-defined effort across teams

Implementing the proper processes and tools is a crucial step in enhancing data quality. The Data Quality Campaign's Lifecycle Model provides a robust framework for this. It is about creating a cycle that encompasses planning, collection, storage, processing, analysis, distribution, and utilization of data. This comprehensive approach ensures consistency and reliability in data processing. Furthermore, it is essential to not only identify and resolve data quality issues, but also to prevent their recurrence. The effective implementation of such models transforms data into a valuable asset that drives informed decision making. #DataQualityLifecycle

This involves implementing processes and tools for data quality enhancement. Models like the Data Quality Campaign's lifecycle model structure these efforts. They help in ensuring a consistent, reliable data flow across its lifecycle - from collection to analysis and utilization. These models facilitate issue identification, resolution, and prevention. By implementing these processes and tools, organizations establish a systematic approach to manage data, ensuring its consistency, reliability, and usability across various operations and analysis.

For sustainable quality, we automate data validation and monitoring using sophisticated MDM tools. Take our work with a financial services client: by embedding automated checks within their data pipelines, we not only enhanced quality but also reduced manual errors. The right processes and tools aren't just operational—they’re foundational to achieving reliable, scalable data management.

Some organisations assume that if there is no noise, there is no need to measure quality. You don't know what you don't know, so you miss out on opportunities such as cost reduction, consistency or optimisation.

This emphasizes monitoring data quality progress and outcomes. Designing tools like the Data Quality Journal's dashboard helps in visualizing performance. Dashboards track indicators like accuracy, completeness, and timeliness, showcasing improvements. They also facilitate communication with stakeholders, demonstrating the value of data quality. This step ensures ongoing assessment, enabling organizations to adapt strategies and showcase the tangible benefits of data quality improvements to stakeholders, fostering continued support and investment in these initiatives.

Utilizing a data quality dashboard, as recommended by the Data Quality Journal, is a smart move. It's not just about tracking progress; it's about visualizing and understanding the impact of your efforts. A well-designed dashboard offers insights into key data quality indicators like accuracy, completeness, timeliness, consistency, and relevance. This level of measurement and reporting is invaluable in communicating the value and benefits of data quality initiatives to stakeholders, ensuring continued support and understanding of their importance.

Timely monitoring needs to be set in place for data quality management. Data quality dashboards does provide an insight on the current status of data quality and areas of improvement can be identified. Once the improvement measures are deployed those need to be monitored too. In short maintaining the quality of data would help in the long run to understand underlying issues with business and make data driven decisions.

Monitor all the traditional metrics, but don't forget that these are just sign-posts. How does it relate back to actual value. This is not always the bottom line, but also the intangible benefits of people being able to focus on asking new and worthwhile questions of the data, rather than "just" data wrangling. Don't get stuck in looking at that Dashboard, but rather use that as one of the insights that you gather from all relevant sources: users, data owners/stewards, analysts, SMEs, managers, execs, applications/systems, as well as customers and suppliers.

Monitoring our data quality performance and outcomes is an essential practice that enables us to measure the effectiveness of our data quality initiatives. By using a data quality dashboard, we can visually track and analyze key indicators such as accuracy, completeness, and timeliness. This visibility allows us to identify areas for improvement and to demonstrate the tangible benefits of our data quality efforts to stakeholders. It's crucial for validating the impact of our strategies on business operations and for guiding informed decisions on future data quality enhancements. This ongoing assessment ensures that our data management practices continue to align with our strategic objectives and adapt to new challenges.

Continuous monitoring is essential. We set up dynamic dashboards using our MDM solutions, giving clients a transparent view of data quality metrics like accuracy and completeness. In one project, real-time tracking revealed inconsistencies in customer data, enabling swift corrective action. Such insights make quality measurable and the business impact tangible.

I love the idea of using a PDSA for improving data quality. The PDSAs can be be used by those who generate the data to bring about new ways of pushing the data to users, for instance, how much or which data the users can/should make use to make decisions that improve processes or outcomes; which users need which data and so on. PDSAs can also be used by data users to understand their reliability on data to make decisions that are based on facts. PDSAs can also be used to improve ability of data collectors to come up with innovative of data collection, verification, validation, and timeliness

Accepting feedback of the measures and learning from mistakes is one of the best ways to help you maintain the quality of data. PDCA method does provide a positive impact on data quality management. New methods and practices need to be researched and studied for effectiveness. Requirments of data keeps changing, and

Improving data quality doesn't always allow for advanced planning. Data scientists and analysts often uncover issues unique to the organizations. Once the issues are gathered, it will then be important to find out common trends of Data Quality issues. Building and automating Data profiling on these provide highest ROI. As an example, if Data analysts find out problems in customer address data commonly it makes sense to include address in profiling. Needless to say it requires a support framework to correct the issues identified by profilers. I would say the framework which is most effective in reality for Data Quality is CPDAC (Check, plan, Do, Act & Check)

Data quality isn’t a one-time effort. We promote a culture of continuous improvement, facilitated by feedback loops in our MDM software. For example, after launching a data initiative, we conduct quarterly reviews to adapt to new challenges. This iterative model keeps quality evolving, ensuring our clients stay ahead of data governance demands.

Link the Data Quality to metrics which demonstrate clear business value and which are agreed by the business owners and who are influential in the prioritisation of Data Quality changes (people, process and technology)

Leverage data contracts. You can easily monitor how data quality is defined in the contracts, the number of rules as well as various ratios. Check out Data Quality in Open Data Contract Standard from the Bitol project at the Linux Foundation.

Data quality should be part of your strategic goals and objectives and hence there should be an explicit call out inside your data strategy around data quality. This is applicable to enterprises working with sensitive data and also organisations who acts as a authoritative sources of data.