InTrust 11.4 Custom Data source

Trying to create custom data source to collect WSUS logs but when I setup custom data source for error.log it does not align in InTrust 11.4 but I am able to align it in Excel. How to setup custom data source for each type of logs below?  Also can this be collected in Real-time need to use Traditional (scheduled job) collection.

 

  1. WSUS

Windows Internal  Database

C:\Windows\WID\Log

      Erro*.log

log_*.trc

system_health_*.xel

 

                   C:\windows\system32\logfiles\httperr

Httperr*.log

 

                   C:\Program Files\Update Services\Logfiles

           

Change.log

SoftwareDistribution.log

Also for Windows Firewall log can it be collected in real-time.

  • Hi Payank Shah,

    I beg your pardon, but the set of logs you mentioned is so manifold that I cannot give you a certain answer.

    Generally, the text logs can be configured only in "InTrust Manager" and collected only on schedule, not in real-time.

    Erro*.log, log_*.trc, system_health_*.xel: these three are from SQL folder, aren't they? Error log can be collected by existing InTrust data source named "Microsoft SQL Server Error Log", other two are binary, not text ones.

    Under Windows Firewall log do you mean "Microsoft-Windows-Windows Firewall With Advanced Security/Firewall" log which can be found in event viewer? (%SystemRoot%\System32\Winevt\Logs\Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx). If yes, then it can be collected both on schedule and in real-time way if you add it in "InTrust Manager" or "InTrust Deployment Manager".

    All other log names and folders I did not find on my desktop, sorry.

    I propose the following procedure: Please focus on one log, create a separate topic here on this forum and provide the log example, I mean a dozen of lines from the log . I will grab it and help to create the InTrust Data Source.

  • Thx for the Answer Igor.

    Using SQL server data source i was able to try collect err*.log.   But since This and pfirewall.log are text and as you said they have to be scheduled how do you forward this events?  Since forwarding capability are only in InTrust Deployment Manager console. 

  • Unfortunately we cannot forward events collected on schedule, sorry. The second thing is that we forward only named strings of events, and even if we create a custom solution for you to forward arbitrary text logs, they will come to a destination host as empty events. I will pass your request to InTrust Product Manager.

  • Hi Payank,

    I have created a data source for the the first and the simplest file WSUS change.log. 

    1. Please unpack the zip

    2. Use the InTrustPdoImport utility (you have it on DVD) to import the objects to InTrust configuration:

    C:\Temp>InTrustPDOImport.exe -import "WSUS change.log Basic.xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    C:\Temp>InTrustPDOImport.exe -import "WSUS Basic Gathering Policy.xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    3. Go to the data source named "WSUS change.log Basic" and in "Settings->Edit" change the path to the file to the one you use on real system. Finish the wizard.

    4. Create a task using the policy named "WSUS Basic" and the site containing the computer you want to collect from.

    5. Run the task

    6. Go to Repository Viewer and create a custom search with the filter "Log=WSUS" and columns "When" and "Description"

    Other two logs have multi-line comments, and it will take more time to create data sources for them. I will continue to work on them little bit later.

    Thank you, waiting for your feedback.

    WSUS_ChangeLog.zip

  • Hi Payank,

    Here comes the second and the most complex data source for the the Software Distribution log.
    The difficulty with this log resides in multiline events, sometimes an exception stack or a configuration xml are attached to the event.
    The longest line I found in your example (shared privately) was 7167 symbols, but not sure this length will not be exceeded in another file.
    The default buffer for InTrust CTL data source is 1024 symbols, that's why some actions might be required on your side.
    I will provide data sources with 8192 and 16384 buffer which I hope should be enough for all cases.

    1. Please unpack the zip

    2. Use the InTrustPdoImport utility (you have it on DVD) to import the objects to InTrust configuration:

    C:\Temp>InTrustPDOImport.exe -import "SoftwareDistribution Advanced Multiline (1024 buffer).xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    C:\Temp>InTrustPDOImport.exe -import "SoftwareDistribution Advanced Multiline Raw (16384 buffer).xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    C:\Temp>InTrustPDOImport.exe -import "SoftwareDistribution Advanced Multiline Raw (8192 buffer).xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    C:\Temp>InTrustPDOImport.exe -import "SoftwareDistribution Advanced Gathering Policy.xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    3. Go to the data source named "SoftwareDistribution Advanced Multiline Raw (16384 buffer)" and on the "Parameters" tab change the path to the one you use on the real system. Close the properties.

    4. Create a task using the policy named "SoftwareDistribution Advanced" and the site containing the computer you want to collect from.

    5. Run the task

    6. Go to Repository Viewer and create a custom search with the filter "Log=SoftwareDistribution" and columns "When", "Source", "Category" and "Description". Also you may add to the grid Insertion Strings 2 to 5. Review the events.

    PS:
    Once you might experience an issue if the line exceeds 16K (though unlikely), the issue will come as a warning in the gathering session:
    Object Name: InTrustServer
    Data Source: SoftwareDistribution Advanced Multiline Raw (16384 buffer)
    Description: Error occurred while processing file C:\...areDistribution.log. The log file contains an invalid record: "The assembly or..." starting at symbol 11586394.

    Workaround:
    Copy-Paste the data source and in the properties of the new object change 16384 to 32768, and name this object SoftwareDistribution Advanced Multiline Raw (32768 buffer)
    BTW, all these "raw" data sources are just copies of the original data source named "SoftwareDistribution Advanced Multiline (1024 buffer)". This has to be done because the buffer size can be changed only in raw format.

    Thank you, waiting for your feedback.

    SoftwareDistribution.zip

  • Here comes the third and the last data source for the HTTP errors log.
    It is a W3C-like log.

    1. Please unpack the zip

    2. Use the InTrustPdoImport utility (you have it on DVD) to import the objects to InTrust configuration:

    C:\Temp>InTrustPDOImport.exe -import "HTTP Errors Advanced.xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    C:\Temp>InTrustPDOImport.exe -import "HTTP Errors Advanced Gathering Policy.xml"
    Quest InTrust PDO Import Utility version 11.4.0.3868
    Copyright 2018 Quest Software Inc. ALL RIGHTS RESERVED.

    Importing ...
    Import finished.

    3. Go to the data source named "HTTP Errors Advanced" and in "Settings->Edit" change the path to the one you use on the real system. Finish the wizard.

    4. Create a task using the policy named "HTTP Errors Advanced" and the site containing the computer you want to collect from.

    5. Run the task

    6. Go to Repository Viewer and create a custom search with the filter "Log=HTTPErrors" and columns "When", and Insertion Strings 2 to 13. Review the events.

    Thank you, waiting for your feedback.

    HTTPErrors.zip

  • Thx Igor for all your help.  I finally got to implementing this and two other you provided and they work.