Quest is excited to release a new workload queuing feature within On Demand Migration that automates migration task orchestration and simplifies migration planning. This functionality was initially introduced for OneDrive Migrations and is now expanding to all workloads.
This new feature allows migration operators to submit migration tasks for any number of objects at once.
- Removes the need to create multiple migration tasks that would have to be submitted and monitored separately
- Maximizes how many concurrent migrations can be processed in parallel
- Reduces the stress of organizing your migration around capacity constraints
- Frees up time to focus on other project tasks
How does the queuing process work?
When you submit a migration task, On Demand Migration places all objects to be migrated into a queue for that workload, confirms available capacity, and then intelligently selects objects from the queue by processing objects with the highest priority first, using a first-in-first-out approach within that priority.
How do I assign priorities?
When submitting a migration task, you will have the option to assign priority levels to help control which objects will be processed from the workload queue first.
How does first-in-first-out (FIFO) work?
When you submit a new migration task, objects are placed at the end of the workload queue for their assigned priority, meaning that tasks previously submitted with the same priority will be processed first.
As On Demand Migration completes a migration job, it will immediately start processing the next available job by selecting objects from the beginning of the highest priority queue.
This respects the FIFO approach while still ensuring that higher priority tasks are always processed before those with lower priorities.
What is an example scenario?
Let’s imagine that a migration operator submits five OneDrive migration tasks throughout the day, with a mixture of priorities, in the order of Task A, Task B, Task C, Task D, and Task E.
If there is available capacity to process all the objects when each task is submitted, the migrations will all immediately switch to “Migrating” and all five tasks will process concurrently with no queue management required.
However, if the available capacity for OneDrive migrations is at its maximum limit when the tasks are submitted, On Demand Migration will place the objects in the OneDrive queue and then process the tasks in order of priority and FIFO as capacity becomes available.
Note: Every On Demand Migration project has its own migration queue and does not compete with workload queues in other projects.
What determines available capacity?
On Demand Migration uses Microsoft APIs to perform migration tasks and must abide by the API limits that Microsoft enforces in each tenant. These limits are in place to ensure that migration activities do not impact your daily business operations. On Demand Migration operates within the available capacity for your source and target tenants.
How do I know which objects are queued?
When you submit a migration task for a workload that is processing at maximum capacity, the newly queued objects will display a migration state of “Queued” in the UI.
You can sort and filter by migration state to view all queued objects.
What’s next for queuing?
Release to all workloads
Queuing in On Demand Migration has been rolling out gradually and is available in all workloads except Mail as of July 2023. Mail queuing is scheduled to be released in Q4.
Ongoing performance monitoring
Quest will periodically adjust processing for each workload to ensure On Demand Migration is utilizing all available resources and performing at peak efficiency.
Releasing PowerShell support
After queuing is deployed for all workloads, Quest will enable full PowerShell support for On Demand Migration, enabling authorized migration operators to manage migrations outside of the user interface. Complete details will be published when this feature is released.
Research for managing individual queued objects
We are researching future enhancements that would allow you to modify the priority of a single object after it has been added to the queue or even remove it from the queue without having to cancel and resubmit the entire migration task.
Research for cross-workload queue priorities
If you submit tasks for OneDrive while concurrently submitting tasks that migrate SharePoint content from Teams and SharePoint Online, you can quickly hit a tenant’s capacity limit since these tasks all utilize the SharePoint API and compete for resources.
Today’s best practices advise scheduling competing workload migrations at different times so you can control which one should take precedence.
We are researching future enhancements that would allow you to assign cross-workload priorities so that you can have On Demand Migration prioritize Teams tasks over OneDrive tasks, for example, even if you submit both at the same time.
Further Information
For more information on this and many other features within On Demand Migration, check out the On Demand Migration User Guide, and visit us at Quest.com.