Just over a year ago I did my last migration before “jumping ship” and leaving behind the day-to-day toil of life as a DBA.
That migration took 56 hours – Friday 10pm to Monday 6am.
Needless to say by 6am on Monday morning I was wrecked, and if anything had gone wrong at that stage only a flux capacitor could have saved me! Luckily, everything went well but 2 weeks before Christmas there are much better things I could have been doing with my weekend. So how could we have done things differently that would have allowed me to put on a stupid Christmas jumper and head to the pub instead?
First of all, this was quite a substantial migration. As the client put it to us – “if we get 56 hours of downtime we need to upgrade everything!” So they did upgrade everything. The main driver for this was that they had to move data centres. The hardware was due an upgrade anyway and some systems were also moving onto different operating systems. The Oracle databases were being upgraded from 10g to 11g. The Oracle application servers were also moving from 10g to 11g. SSO was being replaced by OIM. A new DR site had to be created in a different data centre. An Oracle data warehouse & OBI instance were being migrated and upgraded also. There was an internet-facing portal, originally built on Oracle Portal, moving to Oracle Web Tier and several applications were being upgraded as well.
So a lot of moving parts – data centre, hardware, operating systems, Oracle database & app server versions, and application upgrades.
The application server & OIM parts were pretty easy. They were all built and tested in the weeks leading up to the migration weekend. The need for the extended period of downtime was the export, copy and import of all the databases, plus a little wiggle room built-in. This, as you can imagine, took time and a lot of bandwidth! As I mentioned before everything went well, but it could have been a lot easier and stress free.
In the past few months I’ve started working with SharePlex. A CDC tool that replicates data changes from database A to database B. You may want to do this for a variety of reasons, one of which is migrations. One of the issues I had was that I was going from 10g to 11g, so setting up the new database as a standby was not an option – with SharePlex you can replicate between versions. That means we could have created the 11g databases days or weeks beforehand, during office hours, and had the data replicating from 10g to 11g. Then to migrate, all we would have had to do was log the users out and ensure the 11g was in-sync with the 10g. A saving of over 40 hours of downtime right there!
Another plus of using SharePlex is that you have a solid fail-back plan if something does go wrong. For example, if all the users logged into the new system after the migration and later that day we had hit a serious issue, we couldn’t have gone back to the old system as there was several hours of transactions inputted into the 11g environment. With SharePlex we could have set the replication from 11g back to 10g during our downtime window, so when the users logged into the 11g database their transactions would be replicated back to the 10g database. Then, if there was an issue on 11g we could flip back to 10g with no loss of data and a few seconds of downtime.
I estimate that with SharePlex we would have completed the migration and testing in under 5 hours (and have had a solid fail-back plan) – saving over 50 hours of downtime, with far less risk.
To be honest I would still probably have been wrecked on Monday morning – but for a very different reason!