While it’s no secret that cloud technologies are rapidly changing the way that organizations are running their IT infrastructures, for many organizations that I speak with, databases (and data in general) seem to be the slowest to move. There are very real and valid reasons for this:
- IT security policies have not come up with a set of rules and regulations around how data stored outside of a company’s network should be treated.
- The volume of data stored can be cost prohibitive in a cloud infrastructure that bills by the GB.
- Migrating legacy applications to the cloud can be complex, time consuming, costly, and in many cases impossible.
- And many others….
I think it’s unlikely in the near term that an existing organization is going to be running a high percentage of its database systems in the cloud, however I’m always surprised at how little any cloud technologies are considered amongst database teams. I often ask DBA’s around their strategy for embracing the cloud, and commonly hear “we have none”. Granted, while asking this same question to executive management often leads a different response.
In these conversations with database administrators, I acknowledge that it’s unlikely that they will be managing the majority of their database systems in the cloud in any near term time range, however I encourage them to consider and embrace cloud technologies when discussing new projects, no matter how seemingly trivial, as the cloud may enable them to do things more quickly, more cost effectively, and with greater success than they would be able to accomplish otherwise. Examples that I’ve had success with:
Leveraging Azure Blob Storage as an Ad Hoc Backup Target
A particular customer had an immediate need to migrate data between different data centers. This customer was standing up a disaster recovery data center, in response to a corporate mandate to lower their RTO (recovery time objective). They had a fairly large volume of data to be moved, but the network connectivity between the data centers was unreliable and highly latent. While there were other options available to them (putting data on hard drives and shipping them), sending data to Azure Blob Storage, where it could be downloaded in the secondary data center, provided an easy, fast, and cost effective way to move data. This particular customer used SQL Servers built in ability to BACKUP to, and RESTORE from Azure Blob Storage, coupled with standard file copy tasks and were able to migrate over a terabyte of data.
Leveraging Azure Virtual Machines as a Sandbox for Testing
Some customers are lucky enough to have a plethora of test servers at their disposal that they can “play around” with. Servers that can be broken, rebooted at will, and generally abused with no ramifications. Most customers I come across do not. It may take days or even weeks to get VM’s built, they may share their demo systems with other users and departments, and occasionally may not have a test server at all. Azure Virtual Machines may provide a customer with an agile way to stand up machines, do whatever they wish to them, and then trash them and start over with few (if any) ramifications. I personally have made heavy use of Azure VM’s when testing Availability Groups, SQL Server in place upgrades, as well as the ability to install, configure, and implement different monitoring solutions.
Temporarily Storing Small Datasets in Azure SQL Database
As a database geek, I often think of data in a tabular format. I may be building a grocery list while considering what items are best purchased at which stores, doing analysis of my Fantasy Football team and who I should pick up on the waiver wire, or possibly even trying to keep track of all of the different houses in Game of Thrones. Side note, Kaggle: https://www.kaggle.com/datasets offers a ton of downloadable data sets that are great for this type of work. Figuring out the best way to store data, how to relate tables together, and interesting ways to query that data is fun (nerd alert!), and why not include the cloud in these side projects? Sure, it’s easy to store small data sets in Excel. Most DBA’s I know of also keep small installations of SQL Express on their laptop, which would be more than sufficient for storing this type of data, but storing it in Azure SQL Database offers a few benefits:
- Data is accessible from any machine with a network connection.
- You don’t need to spend valuable laptop system resources (CPU/Memory) to run SQL Server services.
- You get to learn about the cloud, while playing with data.
Bottom line, you don’t have to move all of your corporate database to the cloud, to embrace the cloud. There are so many services and technologies provided by all of the various cloud platforms that may help you with specific projects, or at least help broaden your skill set and prepare you for that upcoming meeting in two years, when your CTO gives you a 6 month timeline for managing 50% of your data in Azure! Those of you that have active MSDN subscriptions, you likely have monthly credits that you can use to assist in this "tinkering". If anyone reading this has any other “pet projects” involving cloud technologies, I’d love to hear them in the comments!