Improved file importing

From June 5th 2016 all new projects started benefiting from a completely rewritten file importing algorithm.

Old projects were not upgraded automatically, so if your project was created before June 5th 2016 see below for how to update your settings.

As of 2019, projects using the old system can no longer use the file importer. Your project data won't be affected in any other way, but importing new files will require you switch to the new system.

What changed?

The core functionality didn't change much, but previously the file importer had two major flaws:

1. When importing a file indexed by ID, any string that didn't look like an ID would be "fixed". This caused problems when the use of certain characters was intentional. For example: "foo bar"="Baz" would have its key converted to "foo-bar" even if you wanted your IDs to have spaces.

2. When importing a file indexed by source text, entries were matched to existing assets by first deriving a unique ID from the text and then matching against that. This caused the problem that assets with very similar text would be matched incorrectly when imported in a different order. It also meant that we couldn't improve our algorithms without breaking backward compatibility.

Both of these issues have been fixed:

1. ID strings are never altered and there are no restrictions to the format of your IDs.
   See the developer guide to Asset IDs and importing by ID.

2. Source text is matched exactly and importing strings in any order will produce the same result.
   See the developer guide to working without IDs and importing by text

How do I upgrade?

If you have an old project please disable the old algorithm in your project settings.

Open your project settings from its :cog icon: in your dashboard, and select the "Developer" tab. Under "Legacy options" switch the "Use deprecated import algorithm" option OFF. You can switch back if you need to, but newly created projects won't have this option.

Project settings showing the old import behaviour enabled. Switch this OFF to upgrade to the new import algorithm.

What could go wrong?

Although the legacy system was broken and had to go, there's a small chance you were relying on its quirks without realizing. We can't account for what peculiarities may have been working in your favour. All we know is the new system is many times better, so we can only recommend that you use it.

Please check your files import as you expect in a fresh project before changing your settings. If in doubt, please get in touch.