Product: ImportOmatic
Description: This solution discusses what to do when importing a large file and the import appears to be frozen
Environment: Hosted
Version: All


Do you have Supervisor rights in Raiser's Edge (RE)?

Has your import been running for more than 15 minutes?

If you answered yes to those questions, then it is very likely that your Hosting session has timed out and that you have actually been disconnected from RE. By default, all RE users are logged out after 15 minutes of inactivity. This setting can be changed for all users EXCEPT those with Supervisor rights. Users with Supervisor rights are always logged out after 15 minutes of inactivity, to help protect your data.

The place to change this setting for non-Supervisor users is:

RE:Config--->Business Rules--->System Access Options--->Disable inactivity timeout for all non-Supervisor users

Unfortunately, the RE database does not always see importing as activity.

To try to avoid being timed out, you can set up an RE user specifically for importing that does not have Supervisor rights and change the aforementioned setting in RE:Config. Note: We have had clients report they have still been timed out in Hosting even when importing as a user without Supervisor rights, with the setting changed.

If your session has timed out, you will have to re-start your connection to Hosting, and re-start RE. While you should not lose the data for rows that have been successfully imported up to the point of the timeout, you will have to do some investigating to determine exactly where the import stopped. Since the entire session timed out, there was no way for ImportOmatic (IOM) to create an entry in the Error.txt to tell you where it stopped. You can think of it almost as though you rebooted mid-import - there was no time for the applications that were running to tidy up.

The possibility of your Hosting session timing out is one of the reasons we suggest you run smaller files (a few hundred versus several thousand), but there are other reasons that are just as important. If, for instance, you discovered you had the wrong value in a column, or had a field mapped incorrectly and the data ended up in the wrong place, you'd probably rather find out with a smaller group of records when you went back and spot checked your data. If it was 100 rows, or even 500 rows, that imported with a mistake, you could recover from it manually in RE. If 5,000 rows imported with a mistake, then you're probably going to consider getting a backup restored. An import cannot be reversed once the import has been committed.

Blackbaud backs up Hosted clients every 15 minutes, and we do not believe they charge to restore your backup as of the writing of this article. That said, you still have to call Hosting Support and wait to have the backup restored, and you will lose anything done in RE after the restore point.

We have developed a free utility, called SplitOmatic, to help you easily break up large import files into more manageable chunks. Simply point it at a source folder that contains your .CSV files, specify an output folder and the number of data rows per file that you would like to have, then hit the start button. When tested on a file with 60k+ rows, breaking the file into 1000 record chunks, it took under a minute to run. You can download SplitOmatic here.