In a related blog post, I revealed that Quest was named as an Example vendor in multiple categories in the 2022 Gartner® report, “Implement IAM Best Practices for Your Active Directory.”1 In fact, Quest or One Identity was listed as an Example vendor in eleven of the categories for which the report lists vendors.
One of those categories is the vital function of Active Directory backup and recovery. In the other post, I briefly described the key Quest solutions in this area and promised you a deeper dive. Here it is! Today I’m going to walk through some of the core tenets of the Quest AD backup and recovery strategy and explore how they align with recommendations in several recent Gartner reports.
Recovery is a core part of an AD security strategy, not an afterthought.
Quest offers a variety of solutions to help you strengthen your cybersecurity and cyber resilience, and to help you quickly spot and investigate threats across your on-premises, cloud or hybrid IT ecosystem. But we recognize that no organization can completely eliminate the risk that an under-trained or over-caffeinated admin might make an improper change to Active Directory (AD), a careless or distracted business user might be duped into opening a malicious attachment in a phishing email, or a natural disaster could wipe out your data center.
That’s why cybersecurity experts recommend taking an “assume breach” posture, and why the NIST Cybersecurity Framework has five components and not just the first four: Identify, Protect, Detect, Respond and Recover. Indeed, ensuring that you can recover your Active Directory quickly and accurately has never been more important. So I was not surprised that both the Gartner report I just mentioned and the Gartner report, “How to Protect Backup Systems From Ransomware Attacks,” make exactly the same statement:
“Add a dedicated backup and recovery tool for Active Directory and keep a regularly updated backup copy.”2
It’s easy to find statistics that illustrate why this best practice is so critical, with cyberattacks pummeling organizations around the world these days. For example, two in three organizations (66%) suffered a ransomware attack in 2021, up from just 37% the year before. And that’s just ransomware! When you consider all the other types of cyberattacks being launched, it’s clear that no organization is immune, and every organization requires a solid Active Directory backup and recovery strategy.
In an AD recovery scenario, every second counts.
There are many types of Active Directory recovery operations, from reverting a single change to a single AD object to a full forest recovery. What they all have in common is that, because AD is responsible for vital authentication and authorization functionality across the domain, time is of the essence. If an AD account has been altered or deleted, that individual might not be able to do their job or a critical application might not be able to run, disrupting vital business processes. And if Active Directory itself is down, your entire business is down.
That can get expensive quickly. In fact, 40% of enterprises say that a single hour of downtime costs $1 million to over $5 million. In a worst-case scenario, losses can reach millions of dollars per minute. Want a real-world example? A ransomware attack cost business services provider Atento $42.1 million, including $34.8 million in revenue losses and $7.3 million in mitigation costs, even though the company said serves had begun to come back online within 24 hours and data center operations had resumed less than a week later.
To illustrate how that happens, let’s do a quick back-of-the-napkin estimate. Assume a ransomware attack brings down AD for the average amount of time, 21 days; the organization has 10,000 employees with an average annual compensation of $65,000; and total yearly revenue is $100M. The cost of lost productivity and lost revenue alone will exceed $60 million! Then you’ll need to tack on the costs of the AD recovery operation, compliance penalties, lawsuits, and long-term reputation damage and customer loss. (For a handy worksheet to do your own calculations, see the ebook, “Be Prepared for Ransomware Attacks with Active Directory Disaster Recovery Planning.”)
“Just pay the ransom” is not a recovery strategy.
If you’re thinking that it’s cheaper to just pay the ransom, consider this: Just 8% of organizations that paid a ransom got all their data back. Plus, 80% of victims who paid a ransom suffered another ransomware attack — and almost half of them (46%) believe it was launched by the same attackers. Finally, the US Department of the Treasury’s Office of Foreign Assets Control (OFAC) declared it illegal to pay a ransomware demand in some instances, noting that an organization could face penalties even if it did not know it was paying (or facilitating payment to) an entity that is prohibited under OFAC regulations.
Moreover, even if you do get your data back, that is not the same thing as getting your domain controllers (DCs) — and therefore your IT ecosystem — up and running again! Remember the infamous NotPetya attack in 2017? Within hours, this malware brought companies around the world to a standstill, including shipping giant Maersk. Although Maersk had backups of much of its data, nobody could locate a single Active Directory backup. While the IT team scrambled, the company had to reroute ships, was unable to unload cargo in dozens of ports and could not process new orders. In the end, Maersk was saved only by a stroke of luck: One DC at a remote office had been offline during the attack. The company painstakingly shuttled that precious machine to its headquarters to enable the AD recovery process.
Clearly, luck is not a strategy. It’s no wonder that the 2022 Garner report states:
“A dedicated backup tool for AD will be of great help in case of a successful ransomware attack.”1
Native tools are not enough — and neither are enterprise backup tools.
Microsoft’s Recycle Bin is a handy way to restore certain kinds of recently deleted objects. But it is not, and was never intended to be, a comprehensive Active Directory backup and disaster recovery strategy. Certain objects are not moved to the Recycle Bin when they are deleted, and even items that would normally go in the Recycle Bin can be hard-deleted, which means they bypass the Recycle Bin and therefore cannot be recovered with native tools. Moreover, the Recycle Bin is for deleted objects only; there is no way to restore specific attributes that have been modified in an object. And it’s hard to figure out what you need to restore, since there is no change log or comparison report to help you determine which objects were changed or deleted.
When we move from granular restore of specific deleted objects to a full disaster, native tools are even less satisfactory. Manual forest recovery is possible, but it is prohibitively slow. Indeed, Microsoft’s AD recovery procedure includes 40+ steps that must be performed on each DC you plan to recover, correctly and in the proper sequence, and the process must be coordinated across the forest. In addition, many of the steps aren’t operations that AD administrators are familiar with; they are tedious, often command-line based steps, so it’s very easy to make mistakes and have to start over.
Because of the limitations of native tools, many organizations have invested in enterprise backup tools or other data protection solutions. These products can be sufficient in some recovery scenarios, such as when the operating system needs to be recovered, and some solutions can even perform a bare metal restore of servers. However, most data protection tools simply do not suffice for Active Directory disaster recovery. In a forest recovery, you must coordinate the configuration effort across multiple DCs. Failure to do so can run the risk of USN rollback, RID bubbles, RID re-use, lingering objects in the Global Catalog and other issues that can cause serious issues with Active Directory functionality. But most traditional data protection solutions simply focus on getting individual DCs to a “healthy” state — and leave all the coordination work to you. In addition, while some of data protection solutions support IFM, you still have to promote each DC one at a time. That’s not exactly a recipe for getting your business fully back on its feet quickly! Finally, on-premises Active Directory backup solutions do not cover cloud-only objects and properties.
It’s no wonder that the Gartner report, “Restore vs. Rebuild — Strategies for Recovering Applications After a Ransomware Attack,” states:
“If possible, invest in dedicated tools for Active Directory recovery as the Microsoft tools and procedures along with the limited capabilities of enterprise backup tools are often not fit for purpose.”3
Quest Active Directory backup and recovery solutions enable you to quickly and accurately restore improperly deleted objects and attributes, individually or in bulk, as well as to automate the entire Active Directory forest recovery process. Indeed, Enterprise Strategy Group (ESG) found that Quest Recovery Manager was more than five times faster than the manual process to recover the test AD forest to bare metal servers. Large environments with hundreds of DCs may require days or even weeks to manually recover AD versus just a few hours when using the Quest Active Directory backup and recovery solution.
Flexibility matters.
The quickest way to get your business on its feet again after a ransomware attack, and the Microsoft recommended best practice for AD forest recovery, is a phased approach. At a high level, the strategy is to get at least one DC in each domain back online quickly, and then turn your attention to the less critical DCs.
To determine which DCs to prioritize, during ransomware recovery planning, collaborate with your business counterparts to identify the applications that are most critical for core business operations, and then determine which DCs are essential for those applications. Create a prioritized list to guide a phased recovery in which the Active Directory team restores the most critical DCs and then moves on to the other domain controllers while the application teams, database teams and others start their recovery work on the critical DCs.
Restoring in phases means the difference between your business being down for minutes, hours, days or even weeks. For example, using the Quest AD disaster recovery solution, you can restore a DC in less than one hour, enabling critical operations to resume while you complete the rest of the recovery.
Still, restore to clean OS may not be the strategy you want in a particular recovery situation. That’s why Quest gives you a full suite of recover options:
- Restore to clean OS — Restore AD onto a new Windows Server while reducing the risk of reinfection.
- Bare metal recovery (BMR) — Recover all volumes of your DC to new or different hardware.
- Install Active Directory — Promote new servers to take the place of DCs you did not restore from backup.
- Uninstall Active Directory — Force-demote a DC and remove all metadata for it from the directory.
- Reinstall Active Directory — Force-demote and re-promote DCs where the operating system is still intact.
- Restore AD from backup — Restore AD onto an otherwise healthy server.
- Repromotion — Promote the remaining DCs in a partially recovered forest.
Your backups are useless if they’re encrypted, corrupted or deleted.
You’ve probably read that cyberattacks, especially ransomware, can move at incredible speed, and that’s certainly true. But it’s also true that in many incidents, adversaries penetrated the network weeks or even months before the final ransomware attack was actually launched. What are they doing during that long stretch of time? The Gartner report, “How to Protect Backup Systems From Ransomware Attacks,” states:
“In that intervening period, attackers are penetrating critical systems such as backup and Active Directory, dropping malware on any server they breach. If attackers successfully breach the backup console or gain write access to the backup data, then they can destroy backups, change retention policies and redefine backup jobs, leaving the backup system in chaos. Organizations without a useful backup system will be left with few options but to pay the ransom.”2
Accordingly, you must not just take regular Active Directory backups but keep them where they can’t be affected by an attack — a place that’s offline, disconnected and inaccessible from both your internal networks and the internet. The old-school option is to write your backups to tape and send them to an off-site storage facility like Iron Mountain. However, that approach is costly and will significantly slow recovery, since retrieving, transporting, mounting and reading tapes requires a great deal of time.
The Gartner report, “Restore vs. Rebuild — Strategies for Recovering Applications After a Ransomware Attack,” states:
“Maintain regular backups of Active Directory in a secure location, such as data vault and practice the recovery process. If the budget allows, add Active Directory-specific backup tools to simplify and speed up the process.”3
Quest Active Directory backup and recovery solutions offer multiple options for secure physical and cloud storage to help ensure your backups are always available without the hassle, expense and delays. In particular, Recovery Manager for Active Directory Disaster Recovery Edition offers Secure Storage, a hardened server that is isolated according to IPSec rules with regular checks to confirm backup integrity; even if you lose your DCs, Tier 1 storage and even your Recovery Manager server, you still have the Secure Storage backup that is hardened and secure to withstand the ransomware attack. Plus, the solution also offers you the flexibility to store backups in secure cloud locations, such as immutable Azure Blob Storage and Amazon Web Services (AWS) S3 storage.
Quick recovery is useless if you restore the malware along with AD.
Modern ransomware and other malware attacks have added a new complexity to Active Directory recovery. The specific challenges are discussed in the Gartner report, “Restore vs. Rebuild — Strategies for Recovering Applications After a Ransomware Attack.”3 It notes, “Recovery from a ransomware attack requires much more than simply restoring applications and data from backups.”
The reason is simple: Simply restoring from backup can lead to reinfection. The Gartner report explains the problem in more detail:
“The malware dropped by attackers and any configuration changes are captured in backups, so organizations can’t simply restore from backup without running the risk of leaving their organization open to repeat attacks. Because the backups may be infected, organizations have to develop ways to restore service as fast as possible, while minimizing the risk of reinfection during the restoration process.”3
The first step in the recovery process is to identify a recovery point, which is the most recent backups in which application data has not been encrypted by the ransomware. As the figure below illustrates, having an Active Directory backup and recovery tool available simplifies the process of identifying a clean recovery point. Specifically, it eliminates the need to either perform forensic analysis of the attack to determine when encryption started or to painstakingly undertake a trial and error process of restoring different backup sets until you find one in which the data isn’t encrypted — either of which is likely to slow the recovery operation significantly.
Quest backup and recovery solutions help you ensure not just a quick Active Directory recovery, but a secure one. With a less comprehensive solution, you might be limited to image-level recovery like bare metal recovery. BMR restores entire volumes (disk partitions), which includes files that are not part of AD, such as the boot sector, the program files directory, and the Windows and WinSXS directories. This leaves a lot of places where malware can hide and reinfect the organization after the recovery operation. With Quest solutions, you also get the option of recovery to a clean OS, which restores only AD components, drastically reducing the places where malware can hide
Gartner names Quest an Example vendor for Active Directory backup and recovery.
The first Gartner report I cite above names Quest as an Example vendor for AD backup. To recap, Quest backup and recovery solutions for on-premises Active Directory, Azure AD and hybrid environments include:
- Recovery Manager for Active Directory Disaster Recovery Edition enables you to back up and restore any AD object, including users, attributes, organizational units (OUs), computers, subnets, sites, configurations and Group Policy objects (GPOs), without restarting your domain controllers. Moreover, Recovery Manager also handles disaster scenarios, slashing AD forest recovery time from days or weeks to just hours, giving you peace of mind that an AD disaster will not become a business disaster. It automates every step of the AD forest recovery process, protects your Active Directory backups from compromise and eliminates the risk of malware reinfection throughout the recovery. You’ll have flexibility to choose the recovery method that works best in a given situation, whether that’s phased recovery, restore to a clean OS or bare metal recovery.
- On Demand Recovery helps ensure the availability and integrity of both on-premises Active Directory and Azure AD from a single recovery dashboard. It protects cloud-only objects not synced by Azure AD Connect, such as Azure AD groups, Azure B2B/B2C accounts and Conditional Access policies. You can run difference reports comparing your backups with live Azure AD to pinpoint specific changes or deletions, and granularly restore exactly what you need or recover multiple users, groups and group memberships in bulk without PowerShell.
1 Gartner,., “Implement IAM Best Practices for Your Active Directory” Paul Rabinovich, 14 March 2022.
2 Gartner,., “How to Protect Backup Systems From Ransomware Attacks”, Nik Simpson, 21 September 2021.
3 Gartner,., “Restore vs. Rebuild — Strategies for Recovering Applications After a Ransomware Attack” , Nik Simpson and Ron Blair, 2 March 2022.
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
