Throttling Stinks (And What You Can Do About It)

[MUSIC PLAYING] Hi. And welcome to the Virtual Experts Conference sponsored by Quest. I'm Paul Robichaux and today I'm going to be talking about how service throttling in Microsoft 365 stinks, but also what you can do about it. So I come to this honestly, I've worked as an Office Apps and Services MVP since 2003. And right now I'm the Senior Director of Product Management at Keepit where I manage our backup products, but before that, I had a stint at Quest managing their SAS application products and before that it Quadratec. So I've been around the 365 ecosystem for quite a while. And you can see Pancake the cat has joined me for this meeting. That's something the virtual folks are getting that the live event attendees in Atlanta did not get because there was no cat. So what are we talking about? Well, very quickly, we're going to talk about what throttling, is how it works, when it happens and what you can do. But I want to sum up at the very beginning by a quote from my former coworker Randy Rempel at Quest, "if you don't want to get throttled? Don't migrate" because throttling is an inextricable part of pretty much every large scale thing that you might do with this service. And so understanding when it happens, when it's likely, and how to respond is really critical because there is no world in which you're not going to get throttled some of the time working with the service. So we talk about throttling, there are two meetings in the dictionary. One is to attack or kill someone by choking them and the other is to control an engine or a vehicle with a throttle. So neither one of these is quite exactly right for what we want, but if we treat throttling like a crime and ask what a real detective or a CSI detective would look for, we're looking for a motive, a means, and an opportunity. Well for throttling, let's define what we actually think the crime is in this case. If it's not choking someone or controlling an engine, it's an intentional restriction of performance to keep the system from getting into a bad state. So if the restriction is accidental because the system designers didn't put enough scale into the system, it's not throttling. If throttling-- If the mechanism of throttling isn't to restrict performance in some way, it's probably also not throttling. And then if you're doing it for some other reason other than the managed performance, generally, most people would say that's not considered to be throttling. For example, Exchange supports a mechanism called tarpitting that slows down incoming connections when the exchange server thinks that someone is using it to spam. That's not exactly throttling. It's not meant to keep the system from going into an undesirable state, it's meant to keep the riffraff out, if I can say it that way. Autoscaling is a little bit different, this is automatically changing the amount of resources that are assigned to something to keep a desired performance level. So these two are sort of opposite. Throttling means I have a certain amount of resources and I'm going to restrict people's ability to use it to maintain performance. Autoscaling is I want to maintain a level of performance and I will throw more resources at the problem. This is what Xbox Live does. When you are playing a multiplayer Xbox Live game the back end of that game will autoscale by adding more Azure VMs or more VMs on whatever platform the game is running on to provide enough capacity to support party chat and the lobby and so on. If a million people are playing halo there are going to be more autoscale VMs running than if 10 people are playing it. Throttling would just say, oops, sorry. Halo is too busy, you can't play right now. Or you would play, but you get one frame per second and you'd be sad. Now that we have a better understanding of what throttling is, let's talk about the motive. Why would Microsoft do this? You know it's not going to make people happy. The truth of the matter is that in modern environments at cloud scale vendors have to do this, if they don't they can't control costs, they can't deliver reliable performance, or reliable service quality. Throttling can help prevent denial-of-service attacks, it prevents a big company or a big tenant from hogging all the capacity that smaller tenants might want, it helps even out spikes in demand during busy periods, and it helps provide more predictable performance for everybody. You would hate it if Microsoft didn't throttle. That doesn't mean that we love it when they do, but things would be much worse without a degree of throttling applied throughout the Microsoft 365 platform. So how does it work? Well, I talked earlier about crimes having means, motive, and opportunities. It turns out throttling has actually got multiple means that can be applied to enforce it. The first thing I want you to understand is that throttling can be applied across all levels of the stack, from the network to the physical server, the mailbox, et cetera. And we'll talk more about what those are in a minute, but it helps if we separate different types of throttling or different layers where throttling is applied and call them something, I'm going to arbitrarily choose the term domains. So most of the time in 365, domains are throttled independently of one another, which means you may get throttled using the SeeSaw API to access SharePoint data, but that same throttling won't necessarily affect you when you are doing operations against the same tenant with Microsoft Graph. And that's important, you'll see why in a little bit. Now, if you think about all of the different domains where Microsoft might potentially throttle some set of operations that you're trying