The Long and Winding Path of the Cook

I just wanted to drop in a quick blog post to suggest a small change to how UE4 deals with long paths to packages when cooking. It’s one that I know many developers, including some AAA studios that I’ve spoken to about this in the past, will welcome. I know that many of these have looked at workarounds for the problem that I’m about to discuss… all related to the following error message:-

Couldn’t save package, filename is too long :[FILENAME]

There’s a limitation, defined by MAX_FILEPATH_LENGTH, that limits the complete path to assets. For most platforms, this will be defined to be MAX_PATH (260). Certainly, for Windows, that’s the case.

Right now, if an asset’s unrolled path (including drive, folders, filename and extension) is too long, the cooker will simply refuse to cook it, will output an error message and then move straight onto the next.

I’ve seen several workarounds to this employed by various studios/developers… for example:-

Reducing the “path-to-game” to a bare minimum … eg. rather than “E:\Perforce\Source\CoconutLizard\OurAwesomeGame\”, reducing to “E:\P4\Awesome”;
Substituting a network drive path, eg. mapping N: to be E:\Perforce\Source\CoconutLizard\OurAwesomeGame” and then running everything from N:

My suggestion is much simpler: change the code to cook the file regardless, don’t output an error – and don’t even bother outputting a warning. Three reasons:-

The final installation path may be shorter than the cook path – so it may no longer be a problem (false negative);
The final installation path may be longer than the cook path – meaning we may have passed over an eventually problematic file (false positive);
Assuming that the shipped product is likely to be packaged (where we mostly just have a single PAK file rather than lots of individual cooked assets), the path to each asset is irrelevant to an end user anyway..!

So.. here’s how UCookCommandlet::SaveCookedPackage() looks in 4.15 (CookCommandlet.cpp):-

const FString FullFilename = FPaths::ConvertRelativePathToFull( PlatFilename );
if( FullFilename.Len() >= PLATFORM_MAX_FILEPATH_LENGTH )
{
   UE_LOG( LogCookCommandlet, Error, TEXT( "Couldn't save package, filename is too long :%s" ), *FullFilename );
   bSavedCorrectly = false;
}
else
{
   ESavePackageResult Result = GEditor->Save(Package, World, Flags, *PlatFilename, GError, NULL, bSwap, false, SaveFlags, Target, FDateTime::MinValue());
   .. other stuff

I suggest that this should all be changed to just:-

ESavePackageResult Result = GEditor->Save(Package, World, Flags, *PlatFilename, GError, NULL, bSwap, false, SaveFlags, Target, FDateTime::MinValue());
... other stuff

Similarly, here’s UCookOnTheFlyServer::SaveCookedPackage() (CookOnTheFlyServer.cpp):-

const FString FullFilename = FPaths::ConvertRelativePathToFull(PlatFilename);
if (FullFilename.Len() >= (PLATFORM_MAX_FILEPATH_LENGTH - CompressedPackageFileLengthRequirement))
{
   LogCookerMessage(FString::Printf(TEXT("Couldn't save package, filename is too long: %s"), *PlatFilename), EMessageSeverity::Error);
   UE_LOG(LogCook, Error, TEXT("Couldn't save package, filename is too long :%s"), *PlatFilename);
   Result = ESavePackageResult::Error;
}
else
{
   SCOPE_TIMER(GEditorSavePackage);
   GIsCookerLoadingPackage = true;
   Result = GEditor->Save(Package, World, Flags, *PlatFilename, GError, NULL, bSwap, false, SaveFlags, Target, FDateTime::MinValue(), false);
   ... other stuff

Which I’d like to change to:-

SCOPE_TIMER(GEditorSavePackage);
Result = GEditor->Save(Package, World, Flags, *PlatFilename, GError, NULL, bSwap, false, SaveFlags, Target, FDateTime::MinValue(), false);
... other stuff

The cook is now much more likely to succeed without errors if there are long paths.

Further Investigation

Delving a little deeper into the whole MAX_PATH thing, for Windows 10 users, it is possible to fix this problem completely (assuming that nobody wants to go totally crazy and use 40,000-character paths etc). To do this, all existing code that’s using MAX_PATH to store filepaths would need to be removed and the manifest files would just need to have the following added:-

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

The package save functions are currently “mostly” setup to save to a temporary folder, rather than the final destination – note this code from UPackage::Save():-

// Make temp file. CreateTempFilename guarantees unique, non-existing filename.
// The temp file will be saved in the game save folder to not have to deal with potentially too long paths.
FString TempFilename;
TempFilename = FPaths::CreateTempFilename(*FPaths::GameSavedDir(), *BaseFilename);

At the end of that (incredibly long) function the temporary file is then moved with MoveFileW(). Without the manifest change above, MoveFileW() will always fail with a long destination file path – even on Windows 10.

So, yeah, with a little bit of careful work, UE4 could add support for long paths.

Credit(s): Robert Troughton (Coconut Lizard)
Status: Currently unimplemented in 4.15