Data Quality - Pre-Migration

[MUSIC PLAYING] [WHISTLING] Hey, Danny. How are you? How you doing, Mike? I'm studying up on data quality for that assignment. And is there anything that data modeling can do with data quality and remediation during a migration? Absolutely. Absolutely, in fact, it's at the core of it. Really there's a couple of things that you want to do from a modeling perspective that's really going to accelerate your ability to get the level of quality up pre-migration. Because you don't want to just take bad stuff and put it into the new one and then hope for the best. So the first thing you want to do is have a good logical model. And you can get that logical model by reverse engineering the schema and then bringing that into a logical model. Give you a real clear picture of what the business requirements and what the data should be. Because just because the data is in a database, it doesn't mean it's what it should be. So a clear picture and a clear definition of what high quality is is absolutely essential to getting it right before you move all of this data over into a new target or a new platform. And then from that can then look at the physical schema, again, by pulling that in through reverse engineering, and understanding what kind of mechanisms have been implemented in the database to address quality. Because in a lot of cases, they don't have good controls. And that's where bad quality data comes from, is not controlling how data and what data can get into the database. So looking at that, from there you're going to get a clear picture of what quality data looks like. So it gives you a goal to move towards and something to measure against. And then on the physical side, understand where the mechanisms are in place and where they're not, how they need to be adjusted, so that you can actually implement those in the database here and remediate the quality prior to migrating it to the new platform. And then as we migrate that schema using the data modeling tool, you'll have all of those mechanisms in place so that you maintain that in your new platform and you go from a go forward perspective, you're going to be able to address data quality issues much easier at the source, and have a clear map of how to address data quality issues that may come up post migration. Yeah. So for example, one of our customers had an old CRM system, and they were migrating to a new CRM system and that involved migrating the database that sat underneath that. And one of the problems with the old CRM system it was Wild West. They had no control and the level of quality of data was really, really, really, really poor. So they were able to understand what was missing in the old system, then use that as part of their guide to what the new system should have. They were able to deal with the vendor on that new system, to understand which quality pieces were in place, which weren't. And then we're able to do that gap analysis and understand what they needed to do from the perspective of building into their own data store a backstop for data quality. And again, what this allowed them to do was clearly get quality data into the new system, understand what the new system was going to do for them, and what they needed to do for themselves to ensure that level of quality maintained and continue to rise as they use the new system. I got to say, that sounds like a gap analysis. Well, it is. What they call a gap analysis. Yeah, with data modeling, you're going to find a common theme of always understanding what you have, where you want to be and understanding the difference between those and then using that modeling tool to actually plan and design that out and design good into the new system. And you've said this many, many times where data modeling facilitates a lot of these discoveries, this project work, these learnings, these planning, these designs. Yeah, I was trying to keep it concise, but you're absolutely right. There's so many places where it touches that journey. Starting out with, what is the perceived quality from the end user perspective? That data model because it's visual, because it's in common everyday language, it's a great place to deal with your stakeholders and drive and garner feedback by walking them through the data model and say, this is what we would expect to happen. What's happening in reality in your world as you've used this application and database in the past? And they'll say, yeah, well this happens, but this doesn't, and then that'll actually take you down it to be able to formulate a plan in terms of where to look, what to look for and then how to remediate not just the data itself, but the system that's allowing that data to reflect poor data quality. Gotcha. Gotcha. OK thanks, Danny. Appreciate it. Yeah, my pleasure.