Data Empowerment in a Diverse Data Ecosystem

Hi, everyone. This is Tim Fritz from Quest Software. I'm a Database Technologist and Market Strategist a Quest. Today we're going to talk about data empowerment in a diverse data ecosystem. So to start this conversation, let's talk about digital transformations. They're happening everywhere, probably no surprise to anyone on this call. But I wanted to just give you a feel for the breadth of the issues or transformations that we're seeing out there, and why we think data empowerment is so meaningful-- even those of us who work with databases every day. Right? So you might be wondering, well, I manage MongoDB, I manage different flavors of databases. Why do I need to be concerned about something called data empowerment? OK. So anyway, data transformations include things like movement to cloud, application modernization, shifting things to the cloud, pandemic-driven changes, more people working from home means even more devices out there that can connect to your corporate data, and keeping that data private from all those new endpoints out there is critical. Compliance is not going away, certainly. Right? Data breaches-- there have been some very high profile examples of that lately within in the last few months, and continues to be a huge challenge for organizations to make sure that data breaches are not happening. Data is proliferating around organizations. A vast majority of data has begun to accumulate in organizations, especially over the last few years. And it's growing exponentially in a lot of companies, and that's an issue-- just knowing where all that data is and what it's useful for things. And then data democratization is really where we're heading as far as making sure that the data is useful for business decisions. The thing is, democratization-- one aspect of that is that so many of us now are being considered data analysts. We're being asked to create reports, go after different data sources, on the same reports visualize the data in certain ways. So the democratization of data raises many issues in organizations for how to make that happen safely. Make sure that decisions are being made soundly on data that is being used the correct way. So all those things taken together are challenges that our customers are facing and that you are probably facing to some extent as well. And we think data empowerment is really the answer here as far as making sure that the challenges brought by these transformations are not keeping businesses from making use of a lot of data that is probably useful in organizations, and they may not even know it yet. So in summary, we're seeing reinvention of businesses from the data perspective, or from data out. And so the data is really at the center of all this, and it's woven throughout all these transformations. All these parts of the businesses are touched by the transformations. So few things that we've heard from our customers. I'm going to address Quest's platform for data empowerment here. Right? So what we've heard from customers-- first of all, people who work in data governance actually want visibility into more context around the data that they're governing, including the systems, the business processes, and the security and protection of the data, all those aspects of data. And building data literacy, making sure, again, that the data is useful, understood by everyone-- it's all comes down to data governance. It's all part of that. Now, another pillar of the state empowerment framework is data protection. And our definition of data protection is broader than a lot of people's definitions of data protection, but we think our definition is where the market is heading and where more and more people's heads are thinking about data protection in a broader sense. Right? It's not just backup and recovery, although that's still a critical part of it-- making sure that the data is stored somewhere safe and it's accessible. But audit and compliance fall into this category too. And access management-- making sure that only people who need access get access, and keep malicious use of data at a minimum. Security endpoint management, like I mentioned before-- making sure that you know all the devices that can access your data and make sure that, again, only people who should be accessing your data are. All right. And then data operations is the first column here, and that's what we're going to spend the majority of this 30 minutes on. But first, let's dive into a few details about data protection and data governance. So data protection is all the things listed on the screen. Like I said, our definition is broader than some people's definition of data protection. And here are some specifics. Proactive and defensive methodologies-- so you need to be defensive, as far as protecting your data, but you also need to have offensive tactics in place to identify and protect against those endpoints out there that might be able to access your data. Visibility, securing assets, and encrypting-- OK, all those things are data protection. And here are some of the aspects of it that kind of translate to what your business is probably doing with data right now. And people around the organization are going to really reap the benefits of solid data protection policies in the company-- policies, practices, hopefully tools that can help with all that. It's going to benefit various areas and businesses, as this slide depicts right here. All right. Now data governance-- here is kind of how we define data governance. What are the sorts of things that have to get done in data governance? Well, identifying technical assets. Knowing what data exists. Where it is. Curating the data assets and with business and quality context-- so that's important. That means documenting things a lot, and I'll talk more about that as we go. That's a key aspect to it. Right? Documenting it so everyone knows the truth about the data, where it came from, where it is, what it means. So again, here are some of those things that result from or are impacted by data governance maybe in your organization right now. Maybe it's called something different, but those things are common requirements in organizations now. And again, who benefits? Well, the list is very similar, it might even be exactly the same as the data protection list. People all around the organization are going to reap the benefits of all these data governance procedures. OK, so data operations-- we're going to focus in on this one. So we defined data operations-- that third pillar of data empowerment-- as the information management products at Quest Software. So database DevOps, data modeling, database system performance, infrastructure, monitoring performance, data prep, and optimization of workloads, database workloads, infrastructure workloads, virtual machines. OK? We'll talk more about the products that plug into those things. But those are the sorts of areas that we're talking about when we talk data operations. So data operations-- what do we need from data operations? Well, I just went through a list of things that need to get done-- are part of data operations. The thing I guess I'll hit on the most here is the complexity that the data transformations are creating. So new hybrid cloud, new infrastructure, new database types-- so a polarity builds in companies. This is a management problem, polarities, and the one that really pops out for me is innovation versus improvement. So operations teams are being asked to innovate, to implement new technologies for these transformations. But at the same, time they're being asked to improve what they already have. Customer experience needs to remain high. Certainly the business critical applications need to stay available all the time and perform well. So the two sides of that polarity take different skill sets though. Right? No doubt about it. So how does a polarity get managed? Well it gets managed by giving adequate attention to each side of the polarity. And so just focusing on innovation to the detriment of improving customer experience, systems that are already in place-- can't do that. Right? It just won't work. And looking at it from the other direction-- the improvement. If you spend all your time on the existing systems to the detriment of innovation, the new things aren't going to be put in place that will eventually have a major impact on the company. All right. So to make all this work, to make all this data operations stuff work, what has to happen? How can we simplify the data ecosystems so that these things that we've just talked about can happen? Well, data operations has to happen. So I put the tasks of data operations into four buckets here-- assembling the data that the company needs, moving the data to where it needs to be, managing the data once it's there, and making the data useful. So let's step through each of those. All right. So first, assembling the data. And part of that, by the way, is making sure that requirements in the organization are met by the data, business rules are followed, and the data reflects the business. And that of course is data modeling. So that's where I'm going to start my discussion-- designing the right data structures. So when I was in DBA, we designed database schemas based on what needs to be stored. So the storage optimized model really ensured efficiency and integrity in data storage and relational access. Well, with NoSQL, what we now design data structures based on the questions we want the data to answer. Right? It's a whole different kind of approach to it, but modeling is still important. So, what kind of questions? Well, what do I want to achieve with this information? What kind of queries typically need to execute against this? How frequently do I need to create things? Read and update things, et cetera? How often am I going to do things from the database? So those are important questions that have to be defined, and then data modeling can guide you through this paradigm shift, enabling successful, modern database design into your data management practices. So it takes some of the pain away from adopting NoSQL-- some of the pain that companies might be going through as they, again, have people skilled in other databases perhaps, or the processes for managing NoSQL just haven't been put in place yet. So erwin Data Modeler is the tool I want to talk about here. So erwin, acquired by Quest in late 2020. erwin is the family of products that help enable a lot of these steps I'm talking about here including data modeling. So one of the nice features of erwin Data Modeler, especially since we're talking about MongoDB here today quite a bit, is if you're moving things, if you want to migrate from relational databases over to MongoDB, the erwin Data Modeler by Quest will help you through that process. So another aspect of assembling the data at kind of early stages here is that we need to optimize-- we need to monitor our database, certainly. And like I said before, many companies, DBAs are being asked to manage multiple platforms. And in fact this research citing here, Unisphere Research from last year, said that over 85% of DBAs are supporting more than one database platform now. So it's getting up there, and that's kind of what I would have guessed. I suppose almost all DBAs are being asked to support more than one. So understanding the dials that can be turned to make sure that the database is tuned well-- used to be when you only supported one database, it might have been more straightforward. Right? Tuning it's probably more repeatable. If you know what's on hand, you could follow them again and again and again when certain issues popped up. Well you want to start that monitoring early in the process here. And try to tune your workloads down a little bit. All right? So Foglight for Databases is one of those-- is the database that would monitor your MongoDB environments for sure. And then optimizing workloads is really a two pronged approach, like I said before. So you want to optimize all the virtual machines. So say you have VMware, hyper-v, you are going to be moving those maybe to the cloud or to someplace else. You're going to want to optimize them probably, and get rid of some waste. So why pay-- especially if you move to the cloud, why pay for cloud services for resource consumption? That's really unnecessary. So if you have virtual machines just sitting there, they're using resources but nobody's getting any value from them, it's time to once and for all get rid of those. Rights? Foglight Evolve helps with that. There's a component called Optimizer that will identify those things. And even if you've deleted a virtual machine, things like IPs and storage can stick around. You want to get rid of all that stuff too. So Foglight Evolve can help you save real money there. And it helps in finding cloud alternative, service alternatives, that are going to be good fit based on the resource consumption of your workloads on premises before you move them to the cloud. You get suggestions on service tiers to move to, and you get some estimated cloud costs too based on that. So that can all be a very helpful step before you migrate, so everyone has a view of whether or not this looks like it has some ROI attached to it. Right? This move to the cloud, for example. OK? So there you go. Just talked about that. There's the database side of things. Going to try to tune out any unnecessary resources in the database as well. Right? CPU, memory, whatever. You want to document your data. So data lineage and insights is going to be key here, all the way through the life cycle of data. And that needs to start early on. So early on when you're building your new MongoDB database or you're getting ready to migrate to something else, something new, or you're getting ready to migrate something else to MongoDB, all of that should be documented real well-- transformations that are happening, the rationale for the migrations or the transformations, anything about cloud services that you've chosen based on what I just talked about a couple of slides ago. You get some recommendations, you have some good ideas where you want to put things up in the cloud. Well, what about geographic locations? That can certainly have an effect on performance. Geographic locations of the cloud services-- you want to take a look at that. Right? And you're going to want to document why you chose certain cloud services, because then it might be clues later for others as to why things are performing not as well as expected. Or maybe it's been a super successful migration and people are going to want to know why. So have it all documented. OK, then we're going to get the data on the move. We want to get data to where it needs to be in the organization. All right. So again, we're talking cloud migration here. Now cloud migration isn't the only kind of digital transformation, of course. Right? But it's one that's so common now that I wanted to focus in on it. And it's one that a lot of us can relate to it now-- organizations. So on-premises or cloud? Well, it's certainly not always an easy answer or an obvious answer as far as, hey, is it going to make sense to move my stuff to the cloud. But regardless, we're going to want to optimize our workloads first, like I said. And then do some cost modeling before the cloud migration like I said before. Right? . And we're going to be moving the data. I'm not going to talk about specific tools to migrate the data. We do have some products that can do restores and things. There's a lot of ways to get data actually into the cloud, and it kind of depends on where in the cloud you're moving the data, of course. But this list from Flexera 2021 State of The Cloud Report here-- the very top one there-- 51% of respondents said that understanding application dependencies is a big challenge for them in cloud migration. So looking at dependencies, are VMs, virtual machines, dependent on one another? Which ones do they really need to migrate together? Those kind of things. It can be a real roadblock for a lot of folks who want to move things to the cloud, but they don't have good visibility into those dependencies. So Foglight Evolve it's going to help there as well. All right. So now once the data gets to where we want, say maybe we've migrated those virtual machines up to an Azure cloud service for example, and now we're managing data up there. What does that look like? Well, that is where data protections come in. So data protections actually started way at the beginning of all this. But now that we've got the data where we want it, right, after the migration of the final placement of the data, we need to kind of address our plans again. We have to go back and take a look at our plans and make sure that everything is accounted for, everything's solid. And business resiliency has to remain. Right? We need to make sure that we can always recover our business. And again, once we've moved the data around an organization or up into the cloud, we want to make sure that is all still accounted for. Net Vault is the Quest product that helps with the backup and recovery, automated on-premises and in cloud. Huge amounts of data. Net Vault is our solution for that. And then managing the data across the enterprise also has to do with monitoring. Again, optimizing those data systems-- we have to make sure that service levels remain high. You know, if we've moved the data around and service levels go down, that's exactly the opposite of what we want to happen. So knowing that we have measurements of performance now once things are where we want them, we can compare them back to performance when they were on-premises, for example. Make sure that they're comparing well. And then we want to make sure that we're managing cloud costs too. So I think I have a slide for that coming up. Here's the monitoring of all of our instances, wherever they now reside. Right? We have deep dive work with analytics for some databases. And again, cloud costs are something that we need to take a look at too. Here you go. Here's diagnostics for MongoDB. So take a look at your top operations, things like that. And try to figure out-- maybe there's some new things happening on the database that weren't happening before. Maybe there's some memory consumption issues, CPU consumption issues, things like that. And comparing performance from between times is going to help you see what the effects of changes to the environment have had. Right? And then finally, make the data useful. So we definitely, again, need to make sure that everyone who wants and needs to use the data can, and they understand what the data means, et cetera. Now DevOps-- mentioned DevOps before. Right? So one of the aspects of DevOps is that containers are being used so much now. Right? Kubernetes containers-- Foglight Evlove monitors those so you can keep an handle on those. If any changes that are made in the applications for the database cause performance changes in any of those containers, Kubernetes-- you're going to want to know about that, and Foglight Evolve helps you see those things. You're going to want to increase stakeholder data literacy. Make sure, again, business context is understood, you've accumulated that community data knowledge and spread, and it's available to everyone who wants to use that data. And this all sounds like documented information. Right? So erwin Data Intelligence, data catalog, data literacy, products around all those things is our solution for that. We're trust in the data. Right? So we're going to prepare the data for the data driven organization. Democratize the data, because that's that's what's being demanded of the data. Right? Everyone needs access to the data, but they need to know what data and where it is and that it's trusted, that they know where it came from, and how it's been handled. All right? Once you lose trust in data, It's really hard to really come back from. I wanted to point out that Toad Data Point is our product for that. It helps with data preparation, provisioning, and the workbook interface for business users. It's a really simple to use visual query building tool and workflow automation tool. So there you go. And all of this is leading up to the business gaining insights from their data. And here's a final list of those benefits that I mentioned earlier from data empowerment. And thank you very much.