Challenges and Solutions for Expiring Service Passwords

Out of curiosity, I always ask a specific question at least once when I’m speaking at a conference: How often does your company expire your users’ logon passwords?

If you’re just as curious, you might like to know that 60 days is the most common answer. I see a lot of 90-day policies, and more than a few 30-day policies. 45 days isn’t unheard-of, but it’s in a minority with 30-day passwords. A lot of companies rely on self-service, intranet-based portals to handle password changes, since those portals often sync passwords between multiple authentication systems or a metadirectory. In addition, those self-service portals can help users unlock a locked user account, or reset a forgotten password, without help desk involvement - saving companies untold help desk overhead, and probably preventing more than a few help desk technicians from going completely insane.

We make users change their passwords because we’re mean. Well, that and it’s more secure: Passwords spend less time “exposed,” so they’re less likely to be compromised. It’s important to understand exactly how passwords can be compromised, so that we can set some sensible ground rules for password expiration. After all, the 30, 45, 60, and 90-day marks are all pretty arbitrary. Is one better than another? We also need to consider service passwords, which many companies don’t expire - at least not as often as user passwords. Are services’ logon passwords subject to the same kind of compromise, and should we be changing them more or less frequently - or at all?

Passwords can really be compromised in two ways. The first way is by accidental disclosure, or what I call the “you wrote what down on a sticky note and attached it to your monitor?!?!” disclosure. Service passwords are pretty immune to that problem, since services don’t need to write their passwords down at all. You might choose to write them down for maintenance purposes, but you’re smart enough to lock that password book in a safe, right? The other way in which passwords become compromised is what I call fatigue. It’s like this: Any password can be brute-force hacked, no matter how long or complex it is, given enough time. That’s one reason we lock out user accounts after too many wrong attempts: To slow down brute-force methods and give ourselves some indication that an attack is underway. The theory is that, by changing passwords frequently, we basically force any brute-force attack to start over, since any combinations it’s tried up to that point will have to be re-tried with the new password. Services’ passwords are just as vulnerable as users’ passwords to this kind of attack, and we should really be changing those passwords just as often as we make users change theirs.

Service Passwords: A Particular Problem

The problem is that changing service passwords is a massive pain in the neck. Each user can be guided to a Web site or dialog box that handles the password change routine; services’ passwords have to be manually changed, and we have to manually update every instance of the service, on however many computers, so that it knows to log in with the new password. Heaven help you if you miss an instance, because the next time that computer restarts for whatever reason, the service won’t be able to log in - and you’re likely to not even notice until users start complaining about inoperative services.

So given that we ought to treat services just like users, and given that services can’t use those dialog boxes and Web sites to handle the change, what should we do?

One Service = One Account = One Password

One approach is to simply create a unique user account, which will obviously have its own password, for each and every service. Don’t ever re-use accounts across different computers, or even on different services on the same computer. This can, of course, create a lot of new user accounts - and the accompanying pain of going through and changing their passwords every cycle. In some cases, there can also be a hard cost: If you’ve licensed any third-party products that charge per active AD user account, then it’s possible this approach will drive up those costs, which would be bad.

I don’t see a ton of companies taking this approach, unless they have relatively few service accounts to deal with. Having a ton of extra user accounts for services means someone (or some group) will spend a decent amount of time each cycle changing those passwords as well as manually reconfiguring the services that use those passwords. Of course, as with many problems involving an excess of manual labor, one solution would be to automate some or all of the process.

Script It

This is where I see a lot of companies heading, or at least trying to head: Scripting their way out of the problem. Windows PowerShell is probably the most popular tool right now, but you could also use VBScript ActivePerl, or many other languages. For example, supposing that you have all of your SQL Server computers in a single Active Directory OU named SQLServers, and you wanted to change the MSSQLSERVER service password on each, then you could run these commands:

Import-module ActiveDirectory
Get-WmiObject Win32_Service -filter "name='MSSQLSERVER'" -comp (Get-ADComputer -filter * -searchbase "ou=SQLServers,dc=company,dc=com" | Select -expand name) | ForEach-Object { $_.Change($null,$null,$null,$null,$null,$null,$null,"P@ssw0rd")}

This assumes that you’re using Windows Server 2008 R2 and its ActiveDirectory PowerShell module. It also assumes that WMI is accessible on all of the servers (not being blocked by a firewall, etc), and that your AD domain name is It also assumes that you want P@ssw0rd to be the new password these services use to log in. Yes, that’s a lot of assumptions - but that’s not really the complete problem with this approach.

One big problem is that, as written, this command has no logging. You’ll get error messages for failures, but you’ll have to manually parse those and manually deal with those exceptions. You could, of course, write this into a more complex script that included error handling and logging.

Another problem is that this approach leaves a lot of room for missed servers. Forgot to drop a SQL Server into the right OU? It’ll be missed - and stop working with the service needs to log in the next time. You might not have your servers grouped this way in AD, and in fact I see very few companies who do group machines this way. In that case, you might instead make a manual list of server names to change, listing them in a text file with one name per line:

Get-WmiObject Win32_Service -filter "name='MSSQLSERVER'" -comp (Get-Content c:\sqlservers.txt) | ForEach-Object { $_.Change($null,$null,$null,$null,$null,$null,$null,"P@ssw0rd")}

Again, this doesn’t offer any error logging, and it won’t help you if you haven’t kept your list updated and accurate. Start repeating the potential for exceptions across however many services you need to administer, and you can see how this approach could quickly become a maintenance nightmare.

This approach has a third problem: It doesn’t help you manage service passwords. Sure, you could still keep your little password book, which you’d lock in a safe after updating it each month, but that’s still manual. More likely, you’re putting passwords into an Excel spreadsheet, which has to be properly-secured or you’ll be releasing those passwords. I know some folks have gone so far as to create password databases, and write scripts that generate passwords, update the database, generate error logs, reconfigure the services, and handle the whole process - but those tend to be complex scripts (I have one here that’s about 500 lines long, and the administrator who sent it to me is still having trouble getting it to work consistently). Besides, when you start getting that complicated, you start putting yourself into the software development business, which might not be where you want to point your career.

The last option, then, is to go with something from someone who does make a career - or at least a serious hobby - out of software development. Find a tool to do the job.

Tool Up

Service password management is increasingly becoming integrated into full-fledged security management systems, although standalone service password management tools are still out there. Here are some of the capabilities I think you should look for - or perhaps even consider writing in that mega-script you’ve always wanted to produce:

Discover the services on your network. Using a combination of IP scanning, Active Directory, and other sources, a tool can inventory the services your servers run, note which accounts they use, and create an inventory database.

Centrally manage passwords, and push changes out to services. In other words, you should be able to click a few buttons, enter a new password for a given service user account, and have every service using that account updated to use the new password.

Logging, so that you can keep track of which services have been properly updated and which haven’t. This log should become part of whatever change management processes and logging that your organization uses, so that there’s a change trail for other administrators to follow. If something breaks the next day, they’ll be able to check the password-change log to see if maybe a service’s password just didn’t get updated.

Optionally, you may want a system that can generate new, random passwords for you (after all, you don’t have to type them in yourself so the more complex, the better), and potentially even keep track of those passwords in some secure database or file, so that you can look up service passwords if necessary.

Some of my colleagues counter-argue that last point: They’re all about random, complex password generation, but they don’t want the service passwords tracked once the services have been reconfigured to use the new password. They feel - and their opinion is not without merit - that there’s no need for a human to know the password, and too much room for compromise if a human does. They’ll just use the tool to change the password to something known and non-random, if needed, and then use the tool to re-randomize the password when they’re done with whatever maintenance task they were completing. I know, however, that some companies just don’t feel safe unless those passwords are documented someplace, so consider your company’s culture when looking into possible tools.

Service Passwords: Worth Your Attention, but Perhaps Not Your Time

I don’t think there’s any question that we should be changing our service passwords petty frequently. They’re just as subject to fatigue as user passwords, and often act as the keys to some of the most highly-privileged accounts in our domains. I’ve met very few administrators who disagree that service passwords ought to be changed - it’s just such a hassle that plenty of them manage to overlook it.

Don’t. Whether you choose to attack the problem with individual accounts, a script, or a third-party tool, managing service account passwords is a worthy task. It just shouldn’t be a time-consuming task. Scripts can certainly help, and task-specific tools can provide even higher levels of automation and management, especially in environments that value change management and change-control processes.