All right. Good morning, everybody and welcome to today's tech talk on Hardening Privileged AD Access. I am your host for today, Jennifer LuPiba, with Quest Software. And I am joined today by our speaker Brian Desmond, principal from Ravens group technology-- Ravenswood Technology Group. He's also a 15-time Microsoft MVP for IAM.
So first of all, I am very glad that you guys were able to join. I want to go through a few housekeeping tips.
For those that had some errors joining, my apologies. We are integrating teams live events into our back end events system. And I'm one of the first folks to do that here. So we're finding a few bugs.
But since you guys are able to join, you did get my updated email while we work those things out in our system in the back. So thank you for your patience and for joining us today.
The Q&A will be open. And we'll hold the questions until the end. And I'll read them off, and Brian can answer those. And I'll type the responses back so we also have the written response.
For those who would like CPE credit or a read receipt and the slides so they can submit them for some kind of continuing education that you applied for, please email me. So if you look in the announcements, you should see my email there. Jennifer.LuPiba@quest.com. And I will send you both of those items so that you can get some credit for this.
So today's talk around hardening privilege AD access, we wanted to bring our tech talks that we've been doing over the last several weeks back to Active Directory. So as your Active Directory has been getting a lot of the attention, a lot of the love right now, because it helps to enable that remote work with Office 365, but we can't take our eyes off of on-premise Active Directory, which is Enterprises still have that on-premise Active Directory and will for some time.
And you know, and AD is where everything begins, both for on-premises security as well as for Azure AD and then replicating it up there. So that's why we have decided to put today's session on to share some helpful reminders around how to make an adversary's job much harder by hardening privilege access.
So today's presentation is a TEC Talk. This is not a Quest product pitch. It's not our typical webcasts where we might give a Quest pitch at the end. This is pure training in the spirit of our Active Directory and Office 365 training conference, which is the experts' conference.
So here, let me advance the slide. The Experts Conference. This is an event that Quest sponsors that we put on each year. Really focuses around bringing that training, the type of training you're going to see today from one of our tech speakers, Brian Desmond is one of our tech speakers, bringing that to the attendees.
And we're really excited because this year we're replicating the hybrid Active Directory security track and the Office 365 track that we had last year. We're adding a third track around migration and modernization because that is a very important topic, and we receive a lot of feedback that our attendees would also like to continue to see more around best practices, lessons learned, around Office 365 and AD migrations and modernization projects.
The Experts Conference is also a great way where we carve out time for networking and interacting with those industry experts and peers, folks like today's speaker, Brian Desmond, other folks like Randy Franklin Smith, Sean Metcalf, Chris McNulty, David Kennedy is a founder of TrustedSec.
You know, we have over 30 speakers ranging from Microsoft employees, Microsoft and BP's industry experts so you're sure to get the technical insights on Microsoft technologies that you're looking for. Right now TEC 2020 is slated for November 17 through the 18 in Atlanta, Georgia. Smack dab in Midtown Atlanta.
You know, we understand that this is a very new situation that we're all going through. November is still a very long ways away. We are optimistic about where we as a country may stand with COVID-19. But we also understand this is a new experience for everyone. So we are planning for every possibility.
We are still planning for an in-person event. You know, we would still like to have that. We are a smaller event, and we have a very large venue. So being able to build in safe and healthy social distancing practices will be possible.
But we are also building contingency plans because we want to make sure that everyone feels very comfortable with the event and some of the different options that we'll provide so that you can you can attend. One of the things we are doing right now is because we're still planning it for in-person, we do have really great early bird rates set up for this. And you know, given that we're not sure how things will roll out, we do offer a full refund as well if something has to change. So you can be comfortable with registering and knowing that if something needs to change, you can easily, easily work with us.
So continuing forward, one other announcement before I introduce Brian. We have another TEC Talks set up on June 3 with Amy Babinchak. She is a Microsoft MVP for Office 365.
She is going to be speaking directly to the US Department Of Homeland Security CISA Security Warning. They put out a security alert a couple of weeks ago around rushed Office 365 deployments given that everyone's trying to enable work from home.
And so she is going to have a very demo-heavy presentation around different things you can do inside of Office 365 that help mitigate some of the issues that he CISA is warning about. And she'll be walking through also what CISA is recommending as mitigation techniques and showing where you can do that within Office 365 natively.
So make sure that you register for that. I will put that link in the announcements here shortly so that you can-- and you'll also get an email from me afterwards so you can register for that.
All right. So I would like to move on to introducing today's speaker. So Brian Desmond has been a tech speaker. So long ago, when we had TEC originally spun up we went on a seven-year hiatus. And then he came back last year and delivered, I think, two sessions for us. He is slated to be a speaker this year.
Brian is the principal at Ravenswood Technology Group. He helps commercial enterprises and higher education customers solve problems around enterprise mobility, Active Directory, identity management, and Office 365. And he's been recognized as a Microsoft MVP for IM for 15 years.
He's also the author of Active Directory, the fifth edition published by O'Reilly. And he's also a frequent contributor in other leading industry publications.
So with that, I would like to hand over the reins to Brian.
All right. Thanks, Jennifer. So thanks to everybody for joining this morning, or this afternoon, wherever you are. And today I want to talk a little bit about on-premises Active Directory. And while it certainly doesn't get the press that it used to, being something that's been around for a long time, it's also foundational to most organizations' IT environment and certainly security. So it's still something that we need to pay attention to.
Oops. There we go.
So like I said, for most large enterprises, AD is effectively the gate to almost every IT asset that you have. If you want to authenticate at some point or another, you're probably going to be using a set of AD credentials. If you want to see whether you have permissions to something, there's a good chance that AD is involved in that somehow.
So obviously, if AD is compromised, then that provides for that adversary or attacker a path to get to pretty much anything. And in many cases, that's exactly the goal is to elevate privilege so that you have complete control of the environment.
So because of that, it's really important that we figure out how to protect AD and make sure that it doesn't become vulnerable to those types of attacks where someone can gain access to virtually anything on the network. And privilege escalation, which is something we're gonna talk about quite a bit in this discussion, is typically that path. I start with a low amount of access. Maybe I've fished an end user for example. And I've used that to gain access to just a regular account or maybe a workstation. But then how can I take that low-level access and proceed to have administrative access maybe to an end user computer and then to member servers and even ultimately, domain admin access, which effectively is the end game from a control perspective.
So we'll talk about how to put controls in place to either prevent or mitigate some of those risks. And another thing that I like to call out, and we've seen this transition with many of our customers over the past five years, at least, and for some organizations they're still starting this type of transition, you know, AD, it's not just infrastructure. It's not a set of servers. It really needs to be managed as an asset that is part of your security posture.
And organizationally for many large organizations, AD has moved under a security organization out of a typical server or Windows support type organization. So we continue to see that transition. And it tends to have a really positive impact towards how we see the service run and how it gets protected.
So with that we'll jump in, and we'll walk through some of the different controls that you can put in place to help mitigate some of the risk we're talking about.
So the first one that's super important and one of the ones that we always start with is talking about lateral movement. And lateral movement is the concept of, I have access to-- I've gained access to a system in a specific tier or bucket, however you want to look at it. And we'll use end user client computers. PCs and laptops are a really good example.
And if I can take the control I have of one end user machine and then I can use that to hop to other end user machines, that's lateral movement. And as someone is able to hop from one to the next, that provides more opportunities to escalate privileges because there's more going on, there's more people touching those workstations.
So it's super important to figure out how we can prevent lateral movement. And that applies both at the endpoint level-- PCs and laptops and so forth-- as well as to member servers. If I gain control of one server, I shouldn't be able to use that controlled hop to hop to another server.
And one of the ways to help prevent that is to make sure that local administrator passwords are A, not the same across all your endpoints, be they client computers or member servers. And B, even if they aren't the same, they can't be set in a programmatic way that someone could actually derive what they are.
Sometimes we see solutions where the local admin password is a function of some properties of the machine, maybe the serial number. The asset tag is an input to it and so forth. That's typically just as bad as having the exact same password set on all these machines because if someone can derive the formula, they can use it to hop from one machine to the next.
So the solution, at least from Microsoft to solve this problem, is something they make available for free. And it's called LAPS. The Local Admin Password Solution. LAPS is a very small package that you deploy typically with Group Policy.
And what it does is on a regular basis it escrows a randomized password for that computer into the computer's account in Active Directory. So now every single machine has a different local admin password. And that local admin password is stored in AD. And you can use traditional AD permissions and delegation to decide who has access to retrieve that password.
So perhaps you have a help desk for each region. Maybe you have operations in North America and some in Europe. You might delegate only the help desk in Europe the ability to access passwords for PCs located in Europe, and likewise for the help desk that's located in North America.
And then also very important and part of LAPS is, once a password is accessed, that password needs to be changed and invalidated so that it's no longer useful. And this is built into LAPS automatically.
There's a free tool that comes with LAPS. It takes a couple extra clips for the end user, the help desk person, whoever is retrieving that password, to trigger that invalidation. Or there are some free solutions that bolt on to LAPS that automatically do this anytime someone retrieves the password.
And then finally, as part of that time stamping mechanism, those passwords change in the background however often you configure it, whether it's every 7, 14, or 30 days, or so forth.
But the bottom line, this is something that's really easy to deploy. Typically the hardest part is not the technology. It's really the workflow changes, especially where you have situations where support personnel are used to either logging in as themselves or knowing a password or being able to quickly derive it to access endpoints.
LAPS changes that so that they have to check out a different password for each machine. But most importantly, it puts a substantial blocker for a lateral movement in place, which is a key defense that you need to have in place from a security and privileged access security perspective.
So the next control that we typically talk about when we talk about securing and hardening AD is implementing the concept of a tiered access model. And this is something that Microsoft publishes and has done so for a long time. But the idea here is that as we think about lateral movement, and then we think about privilege escalation, you want to contain that within the different tiers.
So for example, what we don't want is to have someone that has administrative access to Tier 2, which is our client computers, be able to access Tier 1, which is our member servers. They can certainly access that in a non-privileged way. But we don't want to be able to escalate privileges or control paths, as is the case in the diagram on the screen, upwards in the diagram.
So we typically put these controls in place using group policy to prevent things like logon paths and administrative credentials from being used in the different tiers. We also put in controls to make sure that privileged credentials from higher tiers-- so for example, tier 0, which would typically be doing admin accounts, for example-- those privileged credentials can't be exposed to Tiers 1 and 2 because we assume that the lower tiers are running at lower levels of assurance and might be compromised.
So what we don't want to have is an avenue for a credential theft type attack where the admin account from Tier 0 is exposed to Tier 1 or Tier 2, which might be compromised. And then an adversary can capture that and use that to escalate privileges. So once these controls are in place, it makes it a lot harder both to move laterally and more importantly, to be able to escalate privileges, which is typically how many attacks play out.
And there's a little bit more to this than just putting the group policies and so forth in place. It's also looking at how do we actually segregate these tiers because today, if you haven't done this, AD is effectively one tier. And then all your machines are commingled. And admin access is often commingled across those tiers as well.
So there's a few things to think about here. We talked about preventing lateral movement. We talked about preventing privilege escalation.
And in the context of AD specifically, AD falls into what we call Tier 0, which is typically encompasses any system that could provide complete control of the environment. So if we compromised Tier 0, of course, we have complete control of the environment because we own AD.
There are also things that we often spend a lot of time looking at. The concept of Tier 0 equivalencies. And those are things that while it's not necessarily something like a domain admin account, for example, it might provide the same level of access. And typically those exposures come in the form of agents that run on systems in Tier 0, like domain controllers.
If you have an agent, for example, SCCM-- System Center Configuration Manager is a really good example of that that's running on all your domain controllers. That agent can be used to install packages on those servers. It can be used to run arbitrary code and so forth.
So if I can gain control of the SCCM server, which is typically running in Tier 1 or Tier 2, I can effectively elevate myself to domain admin. So it becomes really important to look at all the different software packages, agents, and so forth that are installed on your Tier 0 systems and understand what exposure they're giving you.
Systems that allow you to run code or really the highest exposure-- SCCM, other endpoint management solutions, some blogs collection and SIMS solutions are another example. Those need to be addressed. And typically that involves removing those agents, those systems, from your Tier 0 systems and either replacing them with an alternative or building an equivalent environment that is just for Tier 0.
So of course, that adds additional overhead that you're going to need to address. But it also closes off a significant point of compromise or vulnerability. And I can tell you from experience, it's one that we see in environments that have been compromised where the attackers use a system like a CCM to elevate access and gain complete control.
So in summary, adding these tiered controls is really important. It can be a significant undertaking. Although if you can focus just on Tier 0, which is you can think of as the crown jewels, having domain admin access to the environment, you can make a significant amount of headway. But you should be prepared for it to be more than just about admin accounts and controls about where those go, but other systems that can provide indirect access that are equivalent to those admin accounts.
So the next step on the journey is the concept of just-in-time access. And with just-in-time access what we can do is we can go from permanent access to AD, which is what most everyone is used to today whereby your admin accounts are permanently in a group like domain admins or server admins or SQL server admins or something like that.
Instead, we can move to a system where those accounts are entitled to that access. But in order to have that access, they need to go through a process to elevate. And that elevation is for a period of time.
So that might be that if I need domain admin access, which chances are, I don't need every single day, I can go to a system or to a portal. And I can elevate. And perhaps that portal prompts me to do a second factor of authentication before I actually receive that access. Then that access becomes time-bound, depending on how it's configured.
Microsoft's model for this actually moves that privileged access out into a separate administrative forest. And then they have a product called Microsoft Identity Manager that provides a end user interface to actually do this elevation. And the way this works is you actually have a trust between what we call your protected forest, which is your production forest-- your production forest that you have today. You move the admin accounts out of that into this administrative forest. And part of the reason that you do that is you're operating under the assumption of breach by default, which assumes that your production or protected forest, as we would call it, is compromised.
So we want to remove the high value targets from there, which are your highly privileged admin accounts. And so what we do is we move them into this new administrative forest that we built cleanly and we built very securely. And we create entitlements for those accounts to have the same access they did before. And we create those entitlements with something that are called shadow groups.
Shadow groups are a construct that was introduced in Windows Server 2016. And the idea with them is that they're actually able to inherit the SID, the security identifier, of a group from another forest. And what that lets us do is bring these admin accounts over and the groups that grant the administrative access without having to [INAUDIBLE] all the resources in the existing production forest.
Because what happens is when that admin account that's now in the administrative forest elevates, Windows adds it to the shadow group. And then that shadow group adds the SIDs to the users' token, which crosses the trust, which in turn means that they're able to access and manage the resources that they were able to manage before.
Like I said, MIM, or Microsoft Identity Manager, is the workflow and automation layer that Microsoft provides on top of this. But all the core functionality is actually built into Active Directory and Windows. And we'll talk about how that works in a minute.
So in some organizations, MIM may make a lot of sense as that orchestration layer. Others may realize that there's a non-trivial amount of overhead that goes with implementing that. And there are other ways to gain effectively the same functionality without deploying an entire new application layer to orchestrate it.
So in terms of how this works, there are a couple new things that were added to Windows Server 2016 Active Directory. They're all bundled into what's called an optional feature in AD called Privileged Identity Management, or PIM. Optional features was something that came quite a long time ago to AD with the idea being that when new functionality was introduced, you could turn it on if you wanted to, but you didn't have to.
In practice, only one other feature ever took advantage of this, which was the AD recycle bin. But as the case may be, if you want to take advantage of this, you have to go through a process in AD called enabling that optional feature. And it's called Privileged Identity Management, or PIM.
And when you enable this you get two different things. One, you get a concept of what are called time bound linked attributes. Linked attributes have been in AD since the beginning, since Windows 2000. The quintessential example being group membership. You have the member and number of attributes. Those are linked together. They're referential.
The manager and managed-by attribute is another common one. You want to see who reports to you or who your manager is in the [INAUDIBLE]. That's stored as a reference. Then the manager attribute and the managed-by attribute.
Of course, those links have always been permanent in the sense that until an administrator removes them, they stay in the AD database forever.
The change that was added in Windows Server 2016 was the concept of a TTL, or a Time To Live, for every singly linked attribute. So now when you add a group member, or in the case of shadow principal-- a member of the shadow principal-- you can in addition to saying who's being added, you can specify a number of seconds that that object will remain there.
So perhaps you decide that your domain admin role, you can only elevate into it for an hour at a time. When you're added to the shadow principal or even domain admins group, but typically the shadow principal, on the end you can say that that's valid for 600 seconds. And AD will keep track of that. When that 600-second time window expires, each domain controller will automatically remove that link as it expires. And at that point, that membership is no longer there.
The other key change that happened is the Kerberos KDT was changed so that when a TGT-- that Ticket Granting Ticket, which has all of your access bundled into it-- is issued rather than expiring by default after 10 hours, it now expires based on your shortest group membership or shadow principal lifetime. So if you're in one shadow principal that's good for one hour, you're in another that's good for four hours, you log on. That TGT is only going to be good for one hour. So that when you renew it, it no longer includes the memberships that have expired.
The second concept of this is there was a change to the way trusts work so that these shadow principals can cross them. You mark a trust. It's what's called a PIM trust. What that does is that changes a function that's been around for virtually forever called SID filtering.
And what that does is it says that now I can have a SID from the trusting domains-- that would be my production forest-- actually be issued in the token from the trust domain, which is my admin forest. Because remember, I've taken my high value groups and I've moved them into shadow principals. But I've kept the SIDs. So in turn what I need to be able to do is have the trusting domain in my production forest, my production domain, honor that token, which includes SIDs that it thinks it should have been responsible for.
So this is native in Windows Server 2016 and of course, Windows Server 2019. If you have older Windows Server 2012 R2 doing chores, there is an update rollout that came out quite a long time ago that introduces the concept of PIM trust to it as well. All right.
And in terms of what the orchestration layer looks like with MIM, like I said, there's quite a bit involved in actually using MIM. And we won't go through all of the-- all this. But know that if you do want to have a highly available MIM deployment that provides the orchestration layer around the just-in-time access built into AD, you're probably looking at least two MIM servers and a SQL server availability group to be able to support that in the administrative forest.
That administrative forest can actually protect lots of different forests. You might have one production forest. We also have customers that have tens or even hundreds of these. And you can have one admin forest protects all those, potentially.
And when you use MIM, just kind of walking through the diagram on the right, the first thing that will happen is the user will sign in with their normal account, if you will, to the MIM portal. They'll select what roles they want to elevate into and complete an MFA challenge. Out of the box, that supports Azure MFA. Although there are mechanisms to extend that to competing solutions.
And once that's done, optionally gone through any approval processes and so forth, MIM actually adds that user to what's called the shadow group or the shadow principal in the administrative forest. And that's the point that the user actually gets time down privileged access. They're added to that shadow principal with whatever TTLs been for that membership. Could be an hour. It could be half a day, a day, whatever you choose.
And then now as the user signs into something in the protected forest, the production forest across that trust, their token includes the SID of the shadow principal, which used to exist in the protected forest. And now when they go to manage something like a server, for example, they have that SID in their token. They're able to be able to perform privileged tasks.
So like I said, MIM as an orchestration layer for this. But fundamentally, all of the actual plumbing under the covers is happening in AD with the concept of the PIM trust and that time [INAUDIBLE] to attributes. So there's multiple ways to deploy this.
So the next thing I want to talk about a little bit is how you actually manage AD or really manage any high value system. And the source that you're managing it from is critically important because if I can control that source, which typically is a laptop or a desktop computer, I can in turn [INAUDIBLE] control whatever that system is being used to manage.
So abstractly, we usually talk about this from the concept of something called the Clean Source Principle. If you think about this as having three different systems where System A controls System B, System B controls System C, then transitively, of course, System A controls System C as well.
So if you think about this in another context, making it a little bit less abstract, if System C is AD, then all the systems upstream-- A and B-- need to be run at the same level of trust that system C, AD or Tier 0, is run at. Because otherwise, if I can compromise one of those upstream systems, then indirectly I've compromised AD, System C.
So the way we solve for this is with something called privileged access workstations. They create a clean source for actually managing our high value systems.
And when we say a clean source, we talk about it being-- typically talk about in the concept of a clean keyboard. We know that that system that the administrator is using, whether they're performing privileged tasks, sensitive tasks, that could be managing AD, that could be managing your PKI, your identity management solution, or in some organizations, even just high value applications-- your financials or something that has core intellectual property or something that's super sensitive.
The only way to manage that is from a PAW-- a Privileged Access Workstation. And that PAW is highly locked down, highly isolated, to remove many of the typical compromised vectors that exist to gain control of a workstation.
So the idea here is that we build it from a clean source all the way from the ground up. We build it from clean media. We look at the supply chain. Where is that machine coming from? How have we acquired it and so forth.
We prevent it from accessing typical threat vectors like the internet, email, productivity tools, and so forth. It's really a purpose-built machine for one thing. And that's managing high value assets. And it's managing them in an isolated way so that an adversary can't potentially take control of that machine and use it to perform tasks or escalate privileges on behalf of the administrator.
There are a number of different ways to deploy PAWS. I'm gonna walk through a few of those different ways that we typically deploy those. And you'll probably realize as we're talking through this that this can be a bit of an undertaking to deploy these. And we'll talk about what some of those things that you really need to think about are once we talk about the different models.
But the important thing to know is that this is a critical control and one that we typically recommend very strongly, even more so than some of the other controls that we talk about in this discussion because privilege escalation through using someone's day-to-day machine is a very real threat. And it's one that, yeah, there are many documented instances where adversaries have used that to elevate privilege and to gain control of an environment.
So the first and the simplest way to deploy this is actually just to give every administrator two machines. They have what we would call on the right their productivity machine. That's that same machine that every other end user in your organization gets issued. It has your standard image, your standard deployment. It has Office. It has Outlook. It's used for browsing the internet, et cetera.
But it's definitely not a clean keyboard. So it's not something that we should be using to manage things like Active Directory or other high risk assets.
So what we do is we give those people that need it a second laptop-- our privileged access workstation. That one is just running Windows 10, just like any other machine. But it's running a separate image, or in newer models, a separate configuration, that's highly locked down, only used for the tasks or for the tasks that they need to be for administering things.
This works well at small scale. It's relatively straightforward to deploy. A lot of the complicated overhead that's involved in some of the other models isn't here.
But as you think about it, now everybody's carrying two laptops. And in many cases two can turn into three or four, depending on how you use this. And that doesn't scale so well. Certainly people that are traveling or what have you, having a backpack full of laptops to put through the TSA doesn't-- it doesn't scale that well. But this does work. And it is a good jumping off point.
The second level that's a little more complicated is to only have one machine. But now what we do is we run HyperV on the base machine. That base machine is running our PAW configuration or PAW image. It's locked down. The base machine is used for all the administrative tasks.
But that productivity image that before was on your typical laptop is now running inside a VM that's hosted on the PAW. This model works well. It still isolates the PAWs from the productivity VM.
It's really important call out that the opposite does not work. You cannot have the PAWs running as a VM on your productivity machine, because if someone gains control of that productivity machine, can transitively they gain control of the VM. And you've just negated all the controls that you've put in place to maintain that clean source or that clean keyboard.
Just a little bit harder to deploy. There's a number of logistics around things like, how do we get our typical image into that productivity VM? How do we provision it and service it, and so forth?
But this model does work. There's still potentially some scale issues with this. And it certainly requires a more robust piece of hardware. But it's something to think about.
You might be thinking in the back of your mind as you're seeing this, well, there's a lot of desktop management-- traditional overhead-- around deploying applications and managing configuration using group policy and so forth to manage these PAWS. And kind of going through that lifecycle on a long-term basis.
So one of the ways to abstract that so that you don't necessarily get into the desktop business is still have that hardened machine that is your PAW. But then actually have it connect to something like a VDI solution or a set of jump service or so forth that are also hardened and only accept connections from the PAWs.
And put out the tooling, all the applications, everything that you need to use on those. So you still maintain a clean keyboard. But rather than trying to manage the configuration of all the endpoints, you can manage that a little bit more centrally with a VDI solution, a Jump Server, or so forth.
And again, I want to call out, the opposite here does not work. You cannot use your productivity machine to connect to a hardened VDI solution to do your administrative tasks or a hardened jump server or anything like that and expect to get the value. Again, you have to have that clean keyboard. And your day-to-day productivity machine is not a clean keyboard, and it never will be. So anything that you use it to connect to by definition violates the clean source principle.
So one more option I put out there and we don't deploy this very often, but it's one that Microsoft talks about in some of their documentation. Part of that is because they use a variant of this within their organization, is the ability to get an even higher level of assurance about the health of the PAW at a very low level about the hardware, the ufi firmware, the boot sequence, and so forth.
And you can do that with a feature that's included in Windows called Shielded VMs. Shielded VMs is built into HyperV. And it uses something called the Host Guardian Service. And what happens here is that those VMs are actually encrypted. So the only way that they can boot is if the HyperV host can attest its health and identity to the host guardian service to retrieve a key to decrypt the-- decrypt the VM.
So that provides a lot of assurance that something hasn't happened and that that PAW isn't being used on another machine. Because the health of the V host starting at the various earliest levels of the boot sequence in terms of the ufi firmware bootup and so forth has to actually be attested to get that key.
And we can also do other things like stack multiple PAWS onto one host machine because they're isolated from one another. And we can stack that productivity machine to do day-to-day tasks on there as well and still maintain our clean keyboard and minimize the number of laptops that are issued to people.
This has a significant amount of overhead just in terms of what's required to manage this. But at larger scales, it can potentially make a fair bit of sense to look at.
So regardless of which of these models you choose, there's a fair bit of things that you need to think about because this is complicated to do. And the number one thing here is that that concept of the clean source principle has to transcend the entire design. But where you cut corners with that is where you introduce vulnerabilities into the design. And as you think about the risk of that, you have to decide whether that risk actually negates the value of the PAWs altogether.
And I put-- there's a few things we'll talk about here. These are typically things that we run into when we do these PAW machines or PAW projects that can add some or a lot of complexity to it. And the first thing starts with supply chain. You have to maintain that clean source, that clean keyboard, you can't just take a laptop off, stack of laptops, put SIDs wherever you provision end user machines.
It's been recycled. You don't know what's happened to it, and so forth. You have to use new machines. They have to be clean. You want to have the minimum number of people involved actually touching those because those are the minimum number of opportunities for that hardware to get compromised and so forth.
Some of the projects that we do, we actually just choose to order that PAW hardware even from a different OEM than your standard laptops so that they don't get mixed in together. Those get shipped ideally directly to provisioning locations where they're built. Where those are built should ideally be secure locations.
Sometimes that doesn't work, especially with kind of global and follow-the-sun delivery models. But it's certainly something to think about.
And then the third one here is the one that hangs up almost every single one of these exercises. And that's the networking layer. You want to isolate the PAWs from the traditional production network. And you don't want them being able to surf the web or go to really any internet property other than the ones that you need them to such as whether that's management tooling or things like management portals like for Azure or AWS or so forth.
So getting that network isolation in place and getting dedicated VPN tunnels so that the PAWs have, like, an always-on VPN. Whether you're at home or you're in the office or you're somewhere else, you should always be riding that dedicated management tunnel. And our experience is typically getting this done is often the hardest part of the project because you're partnering with different teams, you're potentially deploying new network hardware, network appliances, and so forth.
And then the next one is actually managing the PAWs. That clean source supply we talked about agent isolation and Tier 0 equivalency. All the tooling that you have for endpoint management, vulnerability management, log collection, software distribution, patching, et cetera, that needs to be built. And that needs to exist for the PAWs. And it can't be the same tooling that's used for all your other endpoint machines, because otherwise, that becomes the source of privilege escalation to take control of the PAWs.
So if you think about this, now you're getting into the business of building a whole bunch of more infrastructure. And typically that infrastructure needs to be run by an AD team or a dedicated PAW management team so there's overhead involved in that. And then of course, just the life cycle of the machines. Hardware breaks. Things need to be rebuilt. They need to be reissued, et cetera.
There's some-- we say end user support here. But this is really end users in the sense of administrators.
So these are just a few things to think about. But they're important and are typically things that we run into and that can provide significant hangups throughout the PAW project.
Traditionally as we deploy these, they are very much a heavy on-premises infrastructure. That has evolved as Microsoft has evolved their offerings from a cloud service perspective. And as of late, we've really looked at kind of what we would call a modern managed PAW ecosystem.
So the diagram on the screen explains what some of that looks like. But in lieu of traditional disk imaging, on-premises AD managing those group policy management, and other traditional on-prem tools like System Center or Config Manager, we really look to make this run out of a what we would call a Tier 0 or highly locked down Azure AD tenant with some of the Microsoft 365 services.
So we run Windows 10 E5 on the actual PAW using all the security controls that are involved there. So Credential Guard, Device Guard, Bit Locker, Windows Hello for Business, and then Defender, Advance Threat Protection, ATP, for endpoint detection and response. That all hangs off of a dedicated Azure AD premium tenant that houses just the accounts for PAW users.
And then all the actual device provisioning and management is done with Intune, which is connected to that same Azure AD tenant. And then finally, we use Azure Log Analytics and Azure Sentinel to collect log telemetry, not just from the Azure AD tenant and Intune and so forth, but we actually run that agent on every PAW as well to collect all that log data. So we have, you know, one place to go.
And then for management of those traditional on-premises resources, we use always-on VPN built into Windows 10 to have that device be online all the time. It actually uses a short-lived certificate issued by Intune that it only issues if that device is policy compliant. And that's the tunnel that the PAW administrator uses to manage things like AD and so forth.
So the nice thing with this model is that it removes a substantial amount of the on-premises infrastructure that's required in a traditional or kind of legacy PAW deployment. And it also especially in the current climate, lends itself much better to remote work since the majority of the management tooling is actually available from anywhere.
So a couple more things here. Kind of the last step on the journey-- and this is taking it all the way, and we'll look at this in summary in just a moment-- is, do we need to actually go not just from protecting the existing AD, but do we need to think about having a separate forest that has all of those administrative accounts in it?
Some people call this a red forest. Microsoft has had in the past an offering called the SAE. We typically call this a Tier 0 forest. Whatever you call it, the whole idea is that you have very strong credential protection for your domain admin accounts so that they become very, very difficult to attack.
And some people will do this when they feel that their production AD has been compromised and they don't have enough assurance that it is-- it can be brought back into a state where they don't feel that it has been compromised. But it would be far too expensive or far too involved to migrate to a new forest. So let's instead just migrate our high value accounts-- our domain admin accounts-- into an administrative forest.
And to do this, this is an undertaking. But before we do this, you look at all the other things that we've talked about being prerequisites for this, and then layering on that Tier 0 forest or administrative forest on top of all the other controls. That typically also involves things like smart cards and PKI to protect those credentials.
There are some things that have recently come out around FIDO keys and so forth with Windows 10 that may offset the need for some of that smart card or PKI infrastructure. But traditionally, that's how this has been built. And then all of that Tier 0 management infrastructure lives in the tier 0 forest or the admin forest and is similarly protected.
Ultimately, though, we've talked about a bunch of different controls. And as you look along the way, there's a lot of complexity in implementing some of these. And there's a certain amount of assurance or confidence that you get that they're providing additional protection for your environment.
So sometimes when the way we look at this is, you know, what's our complexity? And cost, of course, is a key driver of complexity. And what's our confidence or our reward for doing this?
And the curve that's on the screen is really how we look at prioritizing each of the different controls that we talked about and what the value for them is versus the cost complexity, both in implementation as well as ongoing management.
And if you look at this for each step along the way, you're getting a lot of reward. But until you get higher up on the curve, the overall complexity of implementation is relatively small compared to some of the other things that you can do.
So a kind of takeaway there is you don't have to do everything. But each step that you take is going to buy you quite a bit.
So that's about all I had. I mean, I see there's a bunch of questions in the window. So we're going to go through those. My contact info is on the screen as well.
If this is something that you're thinking about, this is certainly something that I and my team do all the time. And we would love to hear from you as well. But hopefully, this has been useful. And I look forward to answering some of the questions.
All right. We do have a lot of questions, Brian.
So first, I have had a lot of people ask for the CPE credit. So I will be sending out an attendee receipt to those who reached out to me. I think I might just send out the PDF of the slides to all the attendees, though, just because we've had so many questions. I think that might be good to share. But if you need an attendance receipt for CPE submission, in the Q&A you'll see an announcement with my email address.
So I think we should just go through these as they came in. Lot of really good questions here. So let's go to the first one.
This is from TJ. What is LAPS solution for in-one password? For example, complete system restore from prior to last, LAPS password change. Then the current AD LAPS password is invalid. This is real world and has happened multiple times.
So I think the question is, so we use LAPS that password changes periodically. And then somehow we restore the machine to before that password changes, how do we get that password back.
And in terms of how LAPS works and how AD works natively, you can't. However, there are a couple controls I can think of to mitigate that.
One, AD has the concept of what are called snapshot backups. So potentially you could schedule those to run periodically, once a day, or whatever works, and keep that for a period of time. And then an administrator who has permissions to mount that snapshot would then be able to retrieve the previous password.
There are also attribute level backup solutions. Quest makes one of those. That would be another way to potentially get that back.
OK. Excellent. And as you talk to this, I type it out.
All right. Shane asks, does this mean-- I think in regards to your just-in-time access, I noticed that's when the question came in-- does this mean that you would need to deploy separate SCCM or S-C-C-M infrastructure for each tier in the environment?
Yes. That's a really good question. And typically the answer is that certainly with SCCM is yes. What you don't want is an instance of SCCM that, for example, your desktop admins have access to, being able to push packages or code to your servers or domain controllers. And then SCCM potentially as well in the sense that it can push some of that code as well with management hacks.
The flip side is that you can say, well, it's managed out of Tier 0, which means they can also control lower tiers like Tier 1 and Tier 2. But chances are that your AD team isn't going to get into the desktop business. That doesn't always work.
SCCM is a little harder in terms of equivalence. You know, with SCCM we're seeing a lot of our customers at least looking at Azure login analytics, Azure AD Connect Help, and so forth as an alternative to SCCM for their Tier 0 environments. So that may be one of those things also just to look at architecturally is this still the right solution.
Which one is effective and easy going solution to detect PHS and other attacks? The advanced analytics or SIM tool?
Yeah. And PHS testing passed the hash, I think. So Microsoft Advanced Threat Analytics is definitely something that we strongly recommend if you own that to deploy that. If you have the Enterprise Cow or you have EMS for all your users, you own that. Advanced Threat Analytics is a user entity behavior analytics solution that installs an agent on all your demand controllers.
And it looks actually at the network level and is able to analyze things like NTLM and Kerberos and various other things and then compare that to known attack patterns and a machine learning model that's built around typical user behavior to detect anomalies.
And then the cloud version of that, which is really where I think you see the investment going, is something called Azure Advanced Threat Protection, which works very similarly. But rather than doing all of the computation and so forth on premises, those agents stream the data to Azure. And then that becomes available from an Azure portal.
And then you potentially combine that with another product called Cloud App Security from Microsoft, which now gives you very similar insights to user behavior in things like Office 365 and other cloud workloads.
OK. OK. Here's a question. Or I'm not sure if they piggyback on each other. It came from the same person.
How effective is adding an administrative forest, considering that this also needs to be managed, patched, monitored, et cetera, other than securing a Tier 0 on your consolidated corporate domain? I'll ask that one. And then he has another one about PAWs.
Yeah. And then I see the other part is, should the PAWs be joined to AD.
If you don't have an admin forest, should we join those there as well? So the admin forest is very effective in terms of control if it's done right. But the point that's made in the question, we'll now have to manage it, patch it, monitor it, et cetera is also very real and potentially non-trivial.
So I mean, I think our approach typically is that there is a lot of work that you can do to secure your existing AD. And I would start there before I went down the admin forest path. But sometimes, sometimes the admin forest is just what's called forward based on the situation.
And then PAWs, sometimes we join those to AD. I think with the modern manage PAW model that I talked about, that's a really big stepping stone towards not joining those to AD. And that's kind of our default model these days to deploy. And so those machines are just Azure AD joined as opposed to being joined to AD.
OK. All right. Another question. Re: your tier model, regarding that. Can protected forests and the admin forest share common Tier 0 SCCM-- S-C-C-M, SIM services, or do they need two separate instances?
Yeah. Great question. The answer is yes, they can share it. But those services need to be run out of the admin forest so that SCCM server exists in the admin forest. That's where the servers are joined and so forth. So that they're-- the control path is the admin forest, not the protected forest.
OK. All right. Are there recommendations with bastion forest for PSMs?
Um, I'm not sure what a PSM is. So I'm going to ask whoever committed that could maybe clarify that, we can circle back to it.
OK. What are the most common or recommended MFA methods for the administrative forest accounts? Only used for MIMs? Or used for single sign-ons from administrative forest?
Yep. So if you use the just-in-time method, you can use the Azure MFA with MIM. From a pure-- you know, as you take that a little bit further, really natively the only thing that Windows historically has supported as smart cards, which takes on the dependency of a PKI environment, which to do that right is an undertaking.
One of the things that really recently became available is using what are called FIDO tokens to sign in to Windows, not just in an Azure Ad scenario, but also in a hybrid scenario. And I think as that evolves a little bit, we'll look to see if that can offset some of the PKI investment that's historically been needed to issue smart cards.
OK. Can PAWs be administered using group policy?
They absolutely can. That requires them to be joined to a domain, ideally an administrative domain. And then with the hybrid-- or the modern managed approach in tune with offset group policy.
OK. All right. Some of these hardening steps involve considerable investment. This is a good one. Which of these give the biggest bang for buck? And what should be the implementation priority order?
Yep. So that's really this diagram to us. If you think about complexity as being kind of that investment, and then the confidence being your bang for your buck. But if you can get LAPS deployed, if you can isolate Tier 0 both in terms of administrative accounts and also the agents, you're going to be substantially further than you were.
And then the PAWs is often the next thing that we talked about. But it's probably of those initial four, if you will, the largest investment.
OK. Good one. Which one is effective and easygoing solution to detect PTH, Pass the Hash, Brute Force Lateral Movement, and other attacks? The advanced threat analytics or SIM? Did I just ask that one?
Yeah. I think that's a duplicate one we just covered.
I did. Yeah. OK. OK. Here's one that I actually am interested in. I've heard this from customers in our customer advisory board.
Management believes that because there are network tier security layers, there's no urgency on making sure added security is needed to protect AD. Because it usually represents additional costs to have dedicated infrastructure monitoring applications, VMware farms, et cetera. So how can we justify and explain in a way that upper management realizes and understands risk?
Well, I think AD probably transcends every one of those network tiers and layers. So they probably provide very little isolation to AD is where I would start. And as you think about, you know, as we walked through this model, a lot of it was if we start from zero, typically there's an elevation path either directly or very close to starting from an end user machine to get you all the way to a domain admin credential. So I don't know that the network tiering actually provides much in the way of isolation of a system that effectively controls things in every single network tier.
OK. How do you manage transference of data between privileged and non-privileged machines? For example, if I receive an email where I need to perform a massive operation but I need to get the CSV file into the privileged workstation to perform the task?
Yeah. That's a tricky one. We've solved that a couple different ways. In the past we've used file servers that have shares that are either readable by admin accounts and writable by non or vice versa, and then have like a scanning step in between to do some analysis on whether there's potentially something malware or so forth. We've also done things with the modern manage model where we've used one-driver teams in the high tier Azure AD environment as an intermediary point, and then Office Advanced Threat Protection to scan those things.
Ultimately, there's an element of risk moving things back and forth. But obviously, it's-- ultimately it's necessary.
Could you use something like a Windows live USB on a secure flash drive as a PAW?
I am not sure I know what that is. That might be another one to clarify and circle back to.
OK. So if you ask that one, you can maybe submit another Q&A. Are MF secure Azure managed workstations fundamentally the same thing as PAWS? And are they interchangeable? Should separate PAWs be used for cloud admin then for AD admin?
There's a couple questions there. I know there's been some guidance released around like the secure Azure managed workstations in Microsoft's documentation. Yeah, it's effectively PAW. And kind of that modern manage model I talked about is similar to that.
You don't necessarily need a separate PAW for cloud admin versus AD admin. We look at Azure AD, for example, as a tier 0 resource. So managing that from the same PAW that you manage AD from is fine.
All right. And then let me see what else. OK. There's a couple others that just came in. Oh. The PSM-- Privileged Session Manager. So let me go to that question.
Are there recommendations with bastion forest for privileged session managers?
Got it. So I'm assuming that's a solution like a CyberArk or something. Typically what we see is that organizations that deploy these PIM tools, like a CyberArk, and there's a bunch of them, to protect domain admins and other admin accounts don't do it in a way that maintains any of the clean forest principle that we talked about or the tier model. So in many cases, effectively it moves the vulnerability from point A to point B because the PIM or PSM solution is still vulnerable to many of the same attacks. It's not run at the correct level of assurance and so forth. So the value add isn't necessarily there.
All right. And then here's another one. Where do PIM solutions fit into the mix of your chart from earlier? Everything you've mentioned thus far is Microsoft native.
Yeah. So some of the PIM solutions can implement kind of this just-in-time model. But kind of like I just said, around the process and manager side of things, if that solution isn't deployed in a way that it actually protects the credentials and isn't vulnerable to many of the other kind of same privilege escalation and lateral movement, and things we talked about, it doesn't add much in the way of value. So it doesn't help a whole lot in the grand scheme of things.
All right. I think that we've-- that was a lot of questions, which is great. It showed a lot of folks that were very, very much paying attention, very much interested in this topic, Brian.
As I said, I will be sending out these slides afterwards. If you need specifically an attendance receipt, you can email me. And I will send that out.
And we will put this recording live on the ExpertsConference.com. There's a tab called TEC Talks.
Brian, any last words before I end our live event?
No. I really appreciate folks coming. Great questions. As I said, contact info is on the screen. I'm happy to feel free to reach out if there is there's follow-ups or if this is something I can help you with, I'd love to talk about that too.
All right. Thanks, Brian. Thank you, everybody for your time and attention today. Thank you. Have a good day.
All right. Brian, are you there?