• State Not Answered
  • Locked Locked
  • Replies 15 replies
  • Subscribers 29 subscribers
  • Views 15286 views
  • Users 0 members are here
Options
Related
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Virtual Standby to ESXI are slow

network.systex
over 6 years ago

Appliance: DL4300

Software Version: RapidRecovery 6.1.2.115

ESXI Version: 6.0

ESXI Server Spec: Dell R730x, 24 15k SAS Drives in RAID 6 - Adaptive Read Ahead - Write Back, 64GB RAM, Dual Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz

Problem: Exports from RR to ESXI run below 10MB/s on a 10Gb/s network. Many of our standby's are in the TB per day range, meaning most take approx 20-30 hours to complete. During this time backups of that agent do not take place, leaving the server vulnerable.

 

Troubleshooting taken with Support:

 - All software and hardware updates completed

 - Direct 10Gb/s fibre between Core and ESXI

 - Pausing all other tasks on the Core and restarting the server

 - Change RAID policy to Adaptive Read Ahead and Write Back

 

Performing an export of an agentless backup last night seemed to run in the 100MB/s+ range, however after checking the logs this morning it dropped back down to 18MB/s after an hour.

 

Any help is appreciated, and even a comment to say you are experiencing the same and info about your setup will assist.

 

James

  • 0 over 6 years ago

    i'd dream of speeds like 3-8 MB/s since the upgrade. I currently have a 3.6GB incremental update (yes 3.6 Gigabytes) that has been running for 5h13 minutes and estimates another 18 hours to complete.

    EDIT:

    I am however wondering if i'm also impacted by https://support.quest.com/appassure/kb/119686 since all my free memory has been consumed and allocated to standby.

    Am waiting for these exports to finish and some other jobs before I stop the core and change the registry entry, just takes a while to get to that situation.

  • 0 over 6 years ago
    That is the exact fix that I have just applied which has resulted in the dramatic increase in our speeds. Our RAM (64GB) was also filling to max when exporting, I won't know until tomorrow how this fix has affected the entire process. We track the server stats with solarwinds so I should have a good idea of how it went.
  • 0 over 6 years ago
    unfortunately didn't work for me
  • 0 over 6 years ago
    had this ticket raised to a P1 by our account manager last week, still waiting for anything more than that
  • 0 over 6 years ago
    Hey FredBloggs,

    So after changing the cache policy in the registry, I was asked to upgrade our RAID controller firmware.

    We have a DL4300 appliance which I had upgraded all firmware on (using the IDRAC FTP update utility), however a manual check showed our RAID controller (Perc H730P Mini) was a still version out; Was running 25.3.0.0016, now running 25.5.0.0018. Dell pulled the 25.40.x.xxx driver due to an unresolvable issue, leading to

    After the update I now have 15 virtual standby's all running at reasonable speeds of 30-80MB/s, sometimes I do see one go a little slower getting down below 20MB/s however because in the latest version of RR transfers can still take place this isn't having a large of an impact as before were our backup window would be missed.

    I would suggest updating your firmware to the latest for all hardware, and ensuring all windows updates are applied.

    I have found overall that with exception to "Deleting RPFS File" tasks which can run at 1GB/s+, all other tasks appear to be limited to 100-150MB/s which I suspect is down to the de-dup and compression performance of the repository file system. It's a shame that the data couldn't be cached on a separate RAID array and then de-duped and compressed once the backup has been finished.

    Hope this helps, if you want any more info on our setup please feel free to PM me pal. Also a thanks to all at Quest and otherwise which have provided support, I think we are finally getting somewhere with this issue.

    James