This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

TPAM Password Reset Issues

Hi Team,

we have configured to reset computer administrator accounts monthly once. 

In our Monthly report we see that there are 3 types of failures happening.

Scheduled change failed

Password Mismatch - Mis-Match reset pending since May 19th 10:00AM

Initial Forced reset - Initial forced reset pending since May 14th 2PM.

Can someone Please advise the differences of this issues and tips to minimize these failures.

All Our systems are windows based configured for Automatic password reset.

Thank you for your help in this request. 

Parents
  • Hello,

    Please see the information below on how TPAM attempts to Test System, Check or Change passwords:

    (Please note: Different platforms may have slightly different approach to performing the following operations but this is just in general for the purpose of my explanation)

    Test System:
    - TPAM opens a connection to the target server using the username/password of the functional account.
    -If the connection can be established, the test is successful; otherwise, it is considered a failure.

    Check Password:
    -TPAM opens a connection to the target system using the username/password of the account being checked.
    - If the connection can be established, the Check password is successful. If not, TPAM then connects to the target server using the functional account and queries for the existence of the account. If the account exists, it is reported that there is a password mismatch, if it does not, the error indicates that the account does not exist, and if this connection cannot be established, then an “unable to connect” message is returned.

    Change Password:
    – TPAM connects to the target system using the username/password of the functional account and executes the password change command.

    Scheduled Change failed:
    - This means that the password change was scheduled using a Password Change Profile that is associated with the Account and so on the date when the password change schedule was executed, it failed (Could be many reasons, such as target server was not online or failure to login with functional account due to password mismatch of the actual functional account or managed account does not exist)

    Password Mismatch reset pending:
    - This means that the managed account has a password mismatch (Password in TPAM does not match the password on target system) and therefore it requires TPAM to perform a password reset but there is an issue causing the reset to be pending (Could be that TPAM is failing to connect or login to the target System in order to carry out the reset command)

    Initial Forced reset pending:
    - This means that when the managed account was added to TPAM initially, TPAM would initiate a forced reset to change the password for the account because it is likely set to be managed automatically. However, in this case, there is an issue causing the initial password reset to be pending (Similarly could be related to TPAM failing to connect to the target machine if its offline or failing to login to it due to bad credentials of the functional account and therefore cannot carry out the forced reset command)

Reply
  • Hello,

    Please see the information below on how TPAM attempts to Test System, Check or Change passwords:

    (Please note: Different platforms may have slightly different approach to performing the following operations but this is just in general for the purpose of my explanation)

    Test System:
    - TPAM opens a connection to the target server using the username/password of the functional account.
    -If the connection can be established, the test is successful; otherwise, it is considered a failure.

    Check Password:
    -TPAM opens a connection to the target system using the username/password of the account being checked.
    - If the connection can be established, the Check password is successful. If not, TPAM then connects to the target server using the functional account and queries for the existence of the account. If the account exists, it is reported that there is a password mismatch, if it does not, the error indicates that the account does not exist, and if this connection cannot be established, then an “unable to connect” message is returned.

    Change Password:
    – TPAM connects to the target system using the username/password of the functional account and executes the password change command.

    Scheduled Change failed:
    - This means that the password change was scheduled using a Password Change Profile that is associated with the Account and so on the date when the password change schedule was executed, it failed (Could be many reasons, such as target server was not online or failure to login with functional account due to password mismatch of the actual functional account or managed account does not exist)

    Password Mismatch reset pending:
    - This means that the managed account has a password mismatch (Password in TPAM does not match the password on target system) and therefore it requires TPAM to perform a password reset but there is an issue causing the reset to be pending (Could be that TPAM is failing to connect or login to the target System in order to carry out the reset command)

    Initial Forced reset pending:
    - This means that when the managed account was added to TPAM initially, TPAM would initiate a forced reset to change the password for the account because it is likely set to be managed automatically. However, in this case, there is an issue causing the initial password reset to be pending (Similarly could be related to TPAM failing to connect to the target machine if its offline or failing to login to it due to bad credentials of the functional account and therefore cannot carry out the forced reset command)

Children
  • Thank you Very much Tawfiq for detailed explanation.

    Could you please advise any recommendations to minimize failures from TPAM side as we are planning to configure 5000 systems in monthly password reset. 

  • To minimize failures, both TPAM and the target systems must be configured correctly, here is a good article on what is required:

    https://support.oneidentity.com/tpam/kb/75126/how-to-troubleshooting-password-management-and-system-connection-issues-in-tpam

  • The steps outlined in the article are great when you are working through a small number of errors.

    If you  have a large number of issues to remediate then and you want to carry out a bulk check then a CLI script can be used to test connectivity to the various systems.

    You will find CLI or API commands to allow you to automate the test system and check buttons on the GUI as well as changing passwords as well.

    You can have the script sort TPAM responses to give you a bunch of txt files sorted by the response.

    From experience the biggest issues are caused when systems are de-commissioned or taken down for maintenance so I would add to the fault finding in the article to make sure that the TPAM management team are included in the notification loop for such events.

    Systems can then have scheduled change and change after release auto-password functions disabled to prevent issues.

  • Hi Tim,

    Could you please share the cli scripts if you have any for testing bulk systems. Thank you.

  • The following script runs the test system command but the principle is the same for what ever command you wish to run.

    It was written as a batch file partly because the customer did not allow PS to run on local PC's and because being an old bugger my batch file scripting is better than my PS scripting.

    You can call the plink lines verbatim from within a PS script if you prefer.

    You will need a CLI user with admin privileges to run the script and the scrip must be able to access a local copy of the key file. You also need to supply the IP address of the TPAM appliance.

    As plink is used you need to convert the key file downloaded from TPAM with puttygen.exe to convert it into a format plink understands.

    Uses errorlevel to sort good and bad tests. Not quick as each time it executes a CLI command it has to establish a connection to TPAM, run the command, return results and drop connection. This is by design to stop persistent connections so test with a few systems and then run overnight.

    Supplied as is without any guarantee and not supported by Quest. Something I wrote to help out a customer.

    @echo off


    :start
    REM Remove System.txt (if it exists) to ensure no bad platform errors.
     
      del system.txt

    REM This section selects the TPAM Platform

    echo.        
    echo.
    echo.
    echo.        
    echo.
    echo.
    echo.
    echo          ***************************************************************
    echo          *                                                             *
    echo          * Please Enter the name of the TPAM Platform you wish to test *
    echo          *                                                             *
    echo          *        Enter All to test all Platform types                 *
    echo          *                                                             *
    echo          ***************************************************************
    echo.
    set /p Plat=.        :
    echo.        
    echo.
    echo.


    @echo off
    cls
    echo.
    echo.
    echo          ************************************************
    echo.                      
    echo             Extract a list of %Plat% systems from TPAM
    echo.        
    echo             and place them in       
    echo.
    echo             systems.txt                               
    echo.                                                                      
    echo             This could take some time!!!
    echo.
    echo          ************************************************
    echo.
    echo.

     setlocal enabledelayedexpansion

    REM -------------------------------------------------------------------------------------------------------------------------

    REM Retrieving All system configuration information from TPAM and copy to a tab delimited file called system.txt


     plink -2 -i id_dsa.ppk CLI_Admin@172.16.110.16  listsystems --platform %plat% --MaxRows 20000 >>system.txt 2>&1


    REM -------------------------------------------------------------------------------------------------------------------------

    REM This section checks for valid platform selection.

     

     findstr /C:"PlatformName must be one of the supported platforms" system.txt

     IF errorlevel 1 (
     echo.
     echo.
     echo. 
     
     echo           Retrieved a list of %Plat% systems from TPAM instance.
     goto run 
     )
     ( 
     cls
     echo. 
     echo. 
     echo.
     echo         ***********************************************
     echo         *                                             *
     echo         * Sorry that platform type is not recpgnised. *
     echo         *                                             *
     echo         * Please Try again                            *
     echo         *                                             *
     echo         ***********************************************
     ping localhost -n 4 >nul
     cls
     goto Start
     )
     
    :run

    REM -------------------------------------------------------------------------------------------------------------------------

    REM Pad unused fields with Nul due to limitations of token in for command to deal with empty fields

    REM Loop round twice to fill empty fields with "#NUL#", just in case consecutive fields are empty
    REM NB twice is enough as only intested in data at start of data string. If dealing with all data (1,2,X) needs to be
    REM changed to reflect number of blank fields.
    REM NB setlocal enabledelayedexpansion required to be set for this section to work.
     


     for /f "tokens=*" %%X in (system.txt) do (
     set "work=%%X"
     for /l %%i in (1,1,2) do set "work=!work:  = #NUL# !"
     for /F "tokens=* delims= " %%i in ("!work!") do (
     echo  %%i >>systempad.txt
     )
     )


    rem -------------------------------------------------------------------------------------------------------------------------

    REM Remove the first line (column headings) from systempad.txt

     for /f "skip=1 tokens=*" %%A in (systempad.txt) do echo %%A >> temp1.txt
     del systempad.txt
     rename temp1.txt systempad.txt

    REM -------------------------------------------------------------------------------------------------------------------------

    REM extract the 1st colums from systempad.txt using tab delimiter and save as a csv in a file
    REM called systemname.csv  
     
     
     echo           extracting system name to Systemname.csv
     echo.
       
     for /f "tokens=1" %%a in ('type systempad.txt') do (echo %%a >> systemname.csv )

    REM -------------------------------------------------------------------------------------------------------------------------

    REM Remove the Local Appliance Accounts from the list of systems

    REM NB setlocal enabledelayedexpansion required to be set for this section to work.


     echo           Removing Local Parmaster from list of systems
     echo. 
     for /f "tokens=1 delims=," %%l in ('type systemname.csv') do   (echo %%l |findstr /C:"Local_Appliance_parmaster" >nul
     if errorlevel 1 (
     echo %%l >>temp2.txt
     )
     )


     echo           Removing Local Appliance paradmin from list of systems
     echo.

     for /f "tokens=1 delims=," %%p in ('type temp2.txt') do   (echo %%p |findstr /C:"Local_Appliance_paradmin" >nul
     if errorlevel 1 (
     echo %%p>>systemNoLocal.csv
     )
     )

    REM --------------------------------------------------------------------------------------------------------------------------
    cls

    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo.
    echo.
    echo           Running TPAM system test against list of systems contained
    echo           in systemNoLocal.csv and place results rep.txt
    echo.
    echo           Filtered results can be found in:
    echo.
    echo             Pass.txt - Systems that pass TPAM test.
    echo             Pass.csv - Systems that pass TPAM test in .csv format.
    echo.
    echo             Fail.txt - Systems that fail the TPAM test.
    echo             Fail.csv - Systems that failed the TPAM test in a .csv format.
    echo.
    echo.
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo.

     
    rem --------------------------------------------------------------------------------------------------------------------------------------------------------

     echo System,Failure Reason >> fail.csv

     for /f "tokens=1 delims=," %%x in ('type systemNoLocal.csv') do (
     echo Testing System  %%x
    rem echo in loop
     del test.txt > nul
     set system=%%x
     
     plink -2 -i id_dsa.ppk CLI_Admin@172.16.110.16 testsystem --SystemName %%x >> test.txt
     type test.txt >> rep.txt
     call :check

     )
     goto clean
     
    REM ----------------------------------------------------------------------------------------------------------------------

    REM Start of check subroutine 
     
    :check

    rem Debug lines.
    rem echo start sub
    rem echo %system%

    rem the following list are the errors which are filtered on for the reports.
    rem "was successful."
    rem "The RPC server is unavailable."
    rem "An error occurred checking the system"
    rem "Access is denied."
    rem "timed out after" 
     
     findstr /C:"was successful." test.txt >> nul
     if errorlevel 1  (
     echo Successfull completed test of System %system%. Result was %errorlevel% Check failed >> fail.txt
     )
     
     findstr /C:"The RPC server is unavailable." test.txt >> nul
     if not errorlevel 1 (
     echo Reason = %system% "Win32::OLE(0.1709) error 0x800706ba: The RPC server is unavailable". >> fail.txt
     echo %system%,RPC server is unavailable >> fail.csv
     echo. >> fail.txt
     goto next
     )

     findstr /C:"An error occurred checking the system" test.txt >> nul
     if not errorlevel 1 (
     echo "Reason = An error occurred checking the system %system%" >> fail.txt
     echo %system%,An error occurred checking the system >> fail.csv
     echo. >> fail.txt
     goto next
     )
     
     findstr /C:"Access is denied." test.txt >> nul
     if not errorlevel 1 (
     echo Reason = %system% "Win32::OLE(0.1709) error 0x80070005: Access is denied" >> fail.txt
     echo %system%,Access is denied >> fail.csv
     echo. >> fail.txt
     goto next
     ) 

     findstr /C:"timed out after" test.txt >> nul
     if not errorlevel 1 (
     echo Reason = "Test of system %system% time out" >> fail.txt
     echo %system%,Timed Out >> fail.csv
     echo. >> fail.txt
     goto next
     ) 
     
    REM ----------------------------------------------------------------------------------------------------------------------

    REM this section reterns results for sucessful system test.
     
     findstr /C:"was successful." test.txt >> nul
     if errorlevel 0  (
     echo Successfull completed test of System %system%. Result was %errorlevel% Check Passed >>Pass.txt
     echo Results = TPAM Test of %System% passed. >>Pass.txt
     echo %system%,Passed >> pass.csv
     echo. >> pass.txt
     )

    REM ------------------------------------------------------------------------------------------------------------------------- 

    REM this section echos the time the specific system test finished and returns from the subroutine.

    :next
     
     
     echo %time% Test of System  %system%  complete.
     echo.
     exit /b
     
    REM ------------------------------------------------------------------------------------------------------------------------

    REM This section pauses the batch process so that the tempory files produced while it is running can be viewed before they
    REM are deleted.

    :clean


    echo.
    echo. 
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= 
    echo.
    echo.
    echo           Press any key to delete temp and scratch files. 
    echo           and finish testing.
    echo.
    echo.
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo  =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    echo.
    echo.
    pause

    REM ------------------------------------------------------------------------------------------------------------------------

    REM Clean up temp files used in this batch

     del system.txt 
     del systempad.txt
     del temp2.txt
     del systemname.csv
     del systemNoLocal.csv
     del test.txt

  • Hi Tim,

    we see some strange  issues in TPAM system configuration.

    While adding systems we either give FQDN of system like wfm-prod-app-a1.vmware.com or ip address 10.113.61.19 in Network Address section.

    for few systems test system is successful if we give FQDN, for few systems it is successful for ip.

    for few systems test sytem is successful but in password change log it says host unreachable.

    Please see following error message and advise the logic behind this. Thank you.

    [05/21/2019 10:35:53] PartitionName=,SName=wdc-esrs-pm,PartitionedSName=wdc-esrs-pm
    [05/21/2019 10:35:53] Gathering the information necessary to perform a check of wdc-esrs-pm...
    [05/21/2019 10:35:53] Checking the Windows System wdc-esrs-pm(wdc-esrs-pm.vmware.com) using win...
    [05/21/2019 10:35:53] Functional account vmware.com\svc.tpam
    [05/21/2019 10:35:54] Error: (0X80004005) Unspecified error
    [05/21/2019 10:35:54] The test of system [wdc-esrs-pm] failed!
    [05/21/2019 10:35:54] ComputerName: wdc-esrs-pm
    [05/21/2019 10:35:55] Processed the system check on wdc-esrs-pm in 1.8631863 seconds

  • As per the fault finding KB you need to break down each stage of the process to see where it fails.

    You can use the TPAM tools (ping, dns lookup telnet) to check to see if can you reach resolve and that ports are not blocked to the target system.

    The change agent  and proc logs on the /Admin I/F may give you any more details of the issue.

    If the test is passing then this would suggest that the functional account is able to authenticate against the domain BUT:

    Does the functional account have permissions to change the password on the target system?

    Does the account you are trying to change the password for actually exist on the target system?

    Does a Check work?

    Where you are using a FQDN does the system actually exist in the domain the TPAM functional account is a member of or are there trust relationships to allow the functional account to contact and change the password on the target system?

    Are there any firewall rules that are getting in the way of the password change?  TPAM uses 445 to change a password in a windows environment.

    These should give you a good point to start your fault finding process.

    All the best

    Tim

  • Thank you Tim and Tawfiq for your assistance.