Deployment Manager: Last Event more than 3 weeks old

InTrust: version 11.2

Collection comprised of only Domain controllers.

 

Collecting status is green, however, the Last Event collected is more than 3 weeks old for each Domain Controller. I opened repository Viewer and ran a query. Most current events showing are also more than 3 weeks old.

I deleted my collection and created a new collection. Same results, the Last Event is never updated.

Any suggestions?

 

David

  • Hi David,

    What OS is the InTrust server?

    Please have a look at the following KB and check if the InTrust server is possibly impacted:

    support.quest.com/.../real-time-collections-not-working-after-installing-kb4056890-windows-server-2016-kb4056898-windows-server-2012-r2-or-kb4056899-windows-2012-

    Also, if 11.2 ensure you have the latest roll-up hot-fix applied (Update_20170707) or consider upgrade to 11.3.1 at your convenience.

    Regards,
    Chris
  • In reply to Chris.Hood:

    Hi Chris,
    Bamm!
    Hotfix KB4056898 was installed on the InTrust Server the date the collection stopped!
    InTrust is running on Windows Server 2012 R2.

    Question: Real-time colection has stopped working because of KB4056898 on the InTrust server, correct?

    david
  • In reply to david.werner:

    Hi David,

    Correct.

    Microsoft fixed the issue in later KB4057401 for Windows 2012 R2. It should be listed under 'Optional' updates currently.

    Regards,
    Chris
  • In reply to Chris.Hood:

    Thank you, Chris!

    I would never have noticed that events were not being collected if I hadn't logged on to the InTrust server. Is there a way to monitor this? That events from our DCs were not being collected for 3 weeks is very annoying.

    david
  • In reply to david.werner:

    Hi David,

    In this case the real-time event flow issue is side-effect of agent communication sub-system itself being impacted in an unusual way by that KB.

    Normally if agent is not well connected, collection objects will update their state to the following within 5 minutes or so of being disconnected from IT server:

    "Failed" Computer Status and "One or more datasources failed." error

    This would draw attention quicker that there is a problem.

    That being said, there should be alternative way to draw attention to abnormal event flow (or lack thereof). We have logged product enhancement to investigate adding alert (or similar mechanism) to notify InTrust Admins if there is a problem in RTC, like agent stopped collecting events.

    The enhancement ID is IN-975.

    Regards,
    Chris
  • In reply to Chris.Hood:

    Hi Chris,

    Just to let you know, the hotfix solved my issue.

    What is strange is that the Last Event date seems to switch to being updated and current and say 10 minutes later, it switches back to the old date. I checked the repository and the events are being collected. Very strange! Also, each computer in my collection is showing Computer Status = Green (Collecting) with Status showing nothing.

    david
  • In reply to david.werner:

    Hi David,

    That is one of the known issues addressed in InTrust update mentioned earlier actually.

    Issue: After a restart of the SQL server that hosts the configuration database, information about the last event time for datasources in InTrust Deployment Manager collections isn't updated any more.
    Defect ID: 667933

    I would suggest to:

    1. Download Update 20170707:

    support.quest.com/.../6080202

    2. Stop InTust services on all IT servers in the InTrust Organization
    3. Clear the stale records from [dbo].[ITRTAgentDataSource] table in the InTrust_Cfg_DB:

    Delete From [dbo].[ITRTAgentDataSource]

    4. Install the hot-fix on all InTrust servers in the organization.
    5. Restart InTrust services on collector servers
    6. Allow InTrust to run for a bit. Will take some time for collections to be updated. As new event logs arrive from collection objects 'last event' status will be updated.

    Regards,
    Chris

  • In reply to Chris.Hood:

    Hi Chris,

    As previously mentioned, the Update 20170707 and kb4057401 (Windows 2012 R2) definitely helped. Events are being collected! I confirm this by running the Repository viewer.

    I still have the problem that the Last event date is wrong. It is sometimes showing the current date, other times several months old.

    Can you please provide detailed steps I need to carry out to get the Last event date corrected.

    Thank you,
    David
  • In reply to david.werner:

    Hi David,

    Please see my previous reply just before this one above where the steps are included for last event time-stamp issue. Let me know how it goes.

    Thank you,
    Chris
  • In reply to Chris.Hood:

    Hi Chris,

    the steps I don't quite understand are the following:


    3. Clear the stale records from [dbo].[ITRTAgentDataSource] table in the InTrust_Cfg_DB:
    How do i identify stale records?
    Delete From [dbo].[ITRTAgentDataSource]
    delete the Stale records?

    david
  • In reply to david.werner:

    Hi David,

    That's right. Run the following query against the InTrust_Cfg_DB via SQL Studio:

    Delete From [dbo].[ITRTAgentDataSource]

    Thank you,
    Chris
  • In reply to Chris.Hood:

    Hi Chris,

    The computer Status for my domain controllers in my collection is showing 'Configuring' and from time to time when I refresh, some of the servers are Green and collecting and the Last event date is current.

    I can refresh after 10 minutes and the Collecting servers turn back to configuring.

    I did not install the update and hotfix AFTER removing the stale records, I did it before. could this have an impact on what I am seeing?

    David
  • In reply to david.werner:

    Hi David,

    Please try stopping the InTrust services once again and run the following 2 queries against the InTrust_cfg_db:

    Truncate Table [dbo].[ITRTAgentDataSource]
    Truncate Table [dbo].[ITRTAgent]

    Once complete re-start the services and allow some time for the collections to update.

    Based on the results, it sounds like there was some duplicates created previously in ITRTAgent table as well. Clearing both should help resolve the issue. Now that you have the patch applied will prevent the problem from returning.

    Let us know the results.

    Thank you,
    Chris
  • In reply to Chris.Hood:

    Hi Chris,

    That did the trick! Al servers are 'Collecting' and the Last event date is current.

    Thank you for your support!

    david
  • In reply to david.werner:

    Hi Chris,

    As mentioned, all servers are 'Collecting' and the Last event date is current. This is working fine in my DEV environment.

    In my PRODUCTION environment, all servers are 'Collecting', however, the Last event date for some servers is old, other servers it is up-to-date. This can also change. On an up-to-date server, if I reopen the Deployment Manager, it will no longer show an up-to-date value for the Last event date, instead it switches back to the older date. This is random for the servers that are collecting.

    I have performed the SQL Server actions on the InTrust_Cfg_DB database and have installed the hotfixes.

    Any other thoughts or options?

    david