• Replies 1 reply
  • Subscribers 8 subscribers
  • Views 3997 views
  • Users 0 members are here
Options
Related

Why is my Spotlight Diagnostic Server using excessive CPU or memory?

Gita Sharifi
over 3 years ago

The Spotlight Diagnostic Server is the data collection engine of Spotlight. A Windows Service that calls the SpotlightOOPCollector.exe program to collect data from monitored hosts and displays data on the Spotlight Client application. A sluggish server or application can be due to a CPU or memory hog such as the Spotlight Diagnostic Server. Why the sudden change of behavior? Let us look at a few possibilities:

Deployment Changes:

  1. Spotlight was upgraded
  • Evaluate and compare the changes between the previous and current versions.
  • See Release Notes and KB 325056 article for the changes and new features of the latest and previous releases.

   2. Spotlight was migrated to a new server

  • Evaluate and compare the available resources and the running processes on the new server to the previous one.
  • See System Requirements of the latest release.

Configuration Changes:

Changes to the configuration of Spotlight settings:

  1. Run the Audit Log report and check for the following configuration changes of enablement or increased usage of the following settings:
  • Collection schedule intervals
  • Extended Events
  • Spotlight Cloud

   2. Revert back the changes and re-evaluate the resource consumption

Load Balancing:

  1. The number of monitored connections per Diagnostic Server is within the guardrails of best practices.
  2. The Spotlight Playback and Statistics Repository databases are on the instance sharing the same host as the Diagnostic Server. Consider separating the instance holding the database to an instance outside of this host.
  3. Persistent accumulation of Spotlight Playback and Statistics Repository data queuing on disk. The .bat Files can be found in installation path, by default: C:\Program Files\Quest Software\Diagnostic Server\Agent\Data\repository and \playback folders. This is an indication that your Spotlight databases cannot keep up with the Diagnostic Server transactions. Monitor the instance in Spotlight to look for areas of poor performance and bottlenecks.
  4. Allocate enough memory to Diagnostic Server by adjusting the configuration file – Check KB 314649 to increase the memory cap allocated to the Spotlight Diagnostic Server
  5. Identify what other processes are running on the same host. SQL Server and Reporting Services amongst other applications.
  6. Identify issues with the Spotlight Cloud feature. From Configure | Spotlight Cloud icon first confirm feature is enabled. Next, locate possible errors with uploading data to Cloud by checking the content of the Cloud.log file. In case of excessive number of errors, consider temporarily disabling the Spotlight Cloud feature.
    File location: C:\Program Files\Quest Software\Diagnostic Server\Agent\log\cloud.lo
    Example of error:
        07 Jul 2020 16:21:49,184 ERROR [LucyServiceHttpClientCleanup(Job#5024886)] cloud - Request to https://mds-alarmingest-v4-spot-test.spotlightcloud.io/ingest/api/v4/XXX failed : 500 - Connection timed out: no further information

I hope you find the above checklist helpful.

Contact the Spotlight Support Team for additional help.

  • over 1 year ago

    This is a sign that the Diagnostic Server operations are outpacing your Spotlight databases. Look for performance issues and bottlenecks by keeping an eye on the instance in Spotlight. Find out what more processes are active on the same host. The data gathering tool used by Spotlight is called the Spotlight Diagnostic Server. a Windows service that uses SpotlightOOPCollector.exe to gather information from monitored hosts and present it in the Spotlight Client software. The Spotlight Diagnostic Server, for example, is a CPU or memory hog that may be the cause of a slow server or application. Why the abrupt alteration in behaviour? Let's consider a few options: