How to reestablish connection to Sharepoint after a failed migration

While trying to migrate Notes databases with a large number of records, sometimes one or more will fail with an error. If the task is not resumed quickly enough, the authenticated connection expires and all further writes fail with a "(401) Unauthorized" or "(403) Forbidden" error message.

How can I reestablish connection with authorized credentials without terminating the current migration job?

Because if the source is large and >5000 records have already been written, I cannot start another migration to the list without first recreating it and starting from scratch.

Parents Reply Children
  • i) Yes, those are the settings that are selected for authentication, and the account being used is a Sharepoint/AD account.

    i-a) I'm not sure what settings are ideal, but these are active/checked

      Retry 5 times before reporting throttle error

      Refresh cookies 30 mins before they are due to expire

      Cache authentication cookies in configuration file

    i-b) I don't see an option for SSO; am I just blind?

    ii) I'm trying another migration with this setting unchecked, so I'll see how it fares.

    iii-a) I'm honestly not sure if that's the case. There aren't any folders specifically created in the list(s) ahead of time, so I'm not setting any limits that I know of. There's a field called FolderPath from the source database that's mapped to Folder in Sharepoint, so that's what is being used to create the folder structure in the list.

    The source db has a structure of 


        -- 2016

            - Invoice1

            - Invoice2

        -- 2017

            - Invoice1

            - Invoice2

        -- 2018

            - Invoice1

            - Invoice2

    and that's how the folders are being created during the migration, using any / in the FolderPath value to designate a sub-directory

    iii-b) To coincide with the new migration attempt, I setup indexes on a handful of columns in the list ahead of time, so I'll see if that helps as well

  • i-b)
    What version of MNSP are you using? This was added in 7.0.0 (but please use at least 7.0.1, preferably 7.1.0):

    i-a) I am honestly not up-to-date what the best settings are currently - I know we haven't changed the defaults for years, and since Microsoft has changed SharePoint Online behavior often, these are likely not ideal.

    iiia) If your default view in SharePoint starts showing folders (and no items within folders), you should be good.
    You don't need to pre-create folders - mapping a Notes fields to the "Folders" field in SharePoint is exactly the recommended way.

  • Hi Peter,

    i-b) We have version This software was installed previously by a Quest consultant - do we need to have a proper license to upgrade to newer versions? Is there any reason I'd have trouble upgrading the software myself, or is it a pretty standard install/upgrade procedure?

    When I got to work this morning, the migration was still running. Unchecking the "prompt when recoverable errors" option allowed it to run all night and continue past the 2 errors that were encountered during that time, so it was a success beyond what I'd been experiencing.

    However, right before the migration hit the 24 hour mark, every record it tried to write gave either a 401 unauthorized or 403 forbidden error (I started at 11:58 AM 8/24, this started happening at 11:47 8/25). It ended with 6258 errors by the end.

    I guess because the "prompt for errors" option isn't checked there's no option at the end of migration to migrate failed items? So, I tried to load failed items from the log but the limit is 1500, so after doing some surgery on the log, I tried migrating 460 of the failed items. At that point, I get the "exceeded list view threshold" error.

    The list now contains 30,000+ records, but because of the folders, the default view only has 528 items. Any ideas?

  • 7.1 will run with a version 6 license - but you need to be under a valid support contract to obtain the new binaries/installer.

    The installer upgrades from at least two versions back, so 6.15.0 -> 7.0.1 should work. I would expect 6.15 -> 7.1 will also work, but I am not sure.

    I am ooo today - will try to do some testing with >5000 items into folders on Monday.

  • I am testing right now with 8600+ items stored in folders in a SharePoint Online list, and I can start new migration jobs without problems (all using O365 authentication, MNSP 7).

    Microsoft names another limit at 20,000 items, but I can only find that it is relevant for index creation: "Automatic index creation when sorting in the modern experience is limited to lists and libraries with less than 20,000 items."