• State Not Answered
  • Locked Locked
  • Replies 4 replies
  • Subscribers 28 subscribers
  • Views 5666 views
  • Users 0 members are here
Options
Related
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Poor Replication Speeds. High disk queue and processor usage.

timothym
over 7 years ago

Just wondering if there are some tweaks to get our replication speeds up. We have three Quest Rapid Recovery servers doing our backups. One main backup, one offsite replicated backup, and the last one we unplug to use as an offline replication backup which we plug back in once a week to update then unplug it when it is finished.  I want to speed up the replication of our offline backup.  It seems we have a major disk queue issue on our main core backup server and it spikes to well over 100 while performing replications.  CPU is struggling as well.  Network link is full 10Gb but never goes above 1.5Gb/sec.

Main Core: 

  • Poweredge R740xd.
  • Intel Gold 5122 3.66Ghz, 4 core/8 logical.  Stays above 80% usage with constant spikes to 100% during replication.
  • Perc H730P raid controller.
  • 128GB RAM.
  • Raid1 SSD for OS.
  • 10x 12TB spinning disks in RAID 10.  Disk queue highest I have ever seen. 40+ is the minimum. 100 average. Spikes to 800+.
  • Dell PERC settings: Read ahead, write back, 64Kb stripe, Disck Cache Policy Enabled.
  • 10Gb Network card(link between servers confirmed full 10Gb with a file transfer hitting over 8Gb/sec). Jumbo frames enabled and confirmed valid between servers.
  • I have turned off compression to try and alleviate the CPU usage but I have not taken fresh base images since then.
  • Replication settings: Cache lifetime 60 seconds. Max Parallel Streams 100.

Target replication core:(no bottlenecks it seems)

  • Poweredge R720xd.
  • Intel E5-2640x2 2.0Ghz, 16 core/32 logical. (stays below 20% usage, as one would hope with 16 cores)
  • Perc H710P. 12x4TB Raid 6. Perc H810 7x4TB RAID6.
  • 64GB RAM.
  • 10Gb Network Card.

Is our new server seriously not fast enough to just transfer data? I know quest uses a database but really why is it so demanding? Main Core server was setup initially as RAID6, but I read since I am using insanely large 12TB disks that RAID6 is not recommended anymore and changed to RAID10. We have more than enough space so the penalty for raid 10 was not a problem.  I do remember this issue was still there with RAID6 but I did not take benchmarks to see which one was better.

Anyhow, just curious if there is anything I can do to help or do I just have to roll with it as is (cannot buy more upgrades, server was already $20k).

Thanks!

-Tim