Malloc Profiler 2

 In Tools

Latest Downloads

You can currently grab Malloc Profiler 2 through our Github Pull Request: github.com/EpicGames/UnrealEngine/pull/3677

Summary

Malloc Profiler 2 (Lizard Edition) is a replacement of Malloc Profiler that adds several very important and, we feel, incredibly useful features. With this tool you can, amongst other things:-

  • Easily find memory leaks in your application or game
  • Find memory usage inefficiencies, giving great insights into where you could optimize

Malloc Profiler and it’s associated tool, Memory Profiler, allow you to analyse all memory allocations passing through Unreal Engine, detailing the callstack that led to each allocation and allowing you to drill down into the numbers and size of allocations… helping track down memory leaks, find excessive memory churn (often creating a performance bottleneck) and to find bugs or “mishaps” in code. We’ve used it extensively at Coconut Lizard, both for research for our various blog posts and for work that we’ve done with clients.


Shortcomings of Malloc Profiler (Epic Edition)

Despite it’s potential, there are some shortcomings with Malloc Profiler – some of which make it almost useless for some of the things that it would otherwise be incredibly useful for. For example:-

  • the files output by Malloc Profiler, that are later to be read by the tool, very quickly become too large to handle;
  • there’s a significant impact on performance of the application when profiling is enabled;
  • a full rebuild is required to enable/disable profiling;
  • functionality is lacking for starting/stopping profiling over durations of time, within memory limits, etc;
  • Minor niggle: due to the profiler saving .mprof files within subfolders of the Profiling folder, one folder per file, when you “clean” old files from your harddrive, Memory Profiler usage is slowed somewhat by having to re-browse back to the profiling folder (as the “last folder opened” no longer exists).

All of these, of course, we’ve addressed and improved/fixed for Malloc Profiler 2.


Malloc Profiler 2 (Lizard Edition)

Here’s a simple table to show you the benefits of our replacement for Malloc Profiler:-

Comparison MatrixMalloc Profiler
(Original)
Malloc Profiler 2
(Lizard Edition)
Impact on performanceSubstantial
(mostly due to HDD access)
Minimal
(~2-3% on average)
How to enable/disableFull sourcecode rebuild
(#define USE_MALLOC_PROFILER in Build.h)
Commandline options or simple c++ function calls
Ability to profile "forever"NoYes
(if your application can cope that long in itself)
Time-based profilingYesOptional
(through commandline or code)
Sample profile filesize
(from profiling editor startup)
450mb25mb
Profile full cook of KiteDemo?
(or other complex map)
No, the output file will grow too large
(eg. ~1gb in 3mins is common)
Yes
(with only ~2-3% perf impact)
Ability to profile over given time rangesNoYes
(time ranges provided on the commandline)
Can profile up to a given memory allocation limitNoYes
(limit, if any, can be specified on the commandline)

Installing Malloc Profiler 2

Malloc Profiler 2 is available through our Github page here: ADD GITHUB LINK


Using Malloc Profiler 2

There are multiple ways that you can start and control Malloc Profiler 2 – through the command-line, console commands and by calling into the library through code. Each have their own benefits under different circumstances. For example you could:-

  • track memory allocations over the entire run of a commandlet by simply adding the commandline option “-mprof2”. This will start profiling from the very launch of the app and stop on exit;
  • track allocations over a certain portion of a game or app by initialising the profiler in a paused state (with commandline options “-mprof2” and “-mprof_startpaused”) and by having it start/stop tracking using the console commands “MPROF_START” and “MPROF_STOP”;
  • try to find out why a commandlet is eating up 6gb of memory at a certain point with commandline options “-mprof2” and “-mprof_maxallocationsize=5500” (~5.5gb);
  • track allocations over the first 3 minutes of a commandlet with commandline options “-mprof2” and “-mprof_duration=180”;
  • etc…

Once allocation tracking is stopped, a .mprof file is written to disc in a folder along the lines of “\KiteDemo\Saved\Profiling” or (if running a cooked build) “\KiteDemo\Saved\Cooked\WindowsNoEditor\KiteDemo\Saved\Profiling”. This file should be opened with Memory Profiler.


Memory Profiler

Memory Profiler is the tool that you need to read back and display Malloc Profiler .mprof files.

There’s not currently much documentation for Memory Profiler… so, for now, here’s a quick summary of the toolbar options:-

OptionDescription
Diff Start / Diff EndUsed to focus our attention on certain events that are snapshotted in code - eg. as well as Start/End. NB, for now at least, these are only make sense in "timeline" mode (off by default)
Sort bySort the tree by either Size or Count
Active Allocations / Lifetime AllocationsWhether we care about what's currently ("active") or what was ever ("lifetime") allocated
Filtering OptionsYou can filter the view using these options
"Go"Takes the options you've specified and repopulates the tree below

Malloc Profiler 2 Commandline Options

OptionDescription
-mprofEnables MallocProfiler (Epic Edition)
-mprof2Enables MallocProfiler2 (Lizard Edition)
-mprof_startpausedInitially, MallocProfiler will be in a "paused" state
-mprof_duration=[float, Seconds]Allows you to specify how many seconds you'd like profiling to run
-mprof_maxallocationsize=[float, Megs]Sets a limit on how much memory should be allocated before we force allocation tracking to stop

Malloc Profiler 2 Console Commands

These can be called at any time from the Unreal Console. Note: you’ll need to enable Malloc Profiler on the commandline (“-mprof”) and probably to begin it in a paused state (“-mprof_startpaused”).

CommandDescription
MPROF STARTStarts tracking
MPROF STOPStops tracking
MPROF STOPALLOCSStops tracking of new allocations (but continues to monitor deallocations until MPROF STOPTRACKING)

Triggering Malloc Profiler 2 Through Code

To use Malloc Profiler 2 in C++ code, you just need to call into the functions below to start/stop allocation tracking. Note: you’ll need to enable Malloc Profiler on the commandline (“-mprof2”) and start it in a paused state (“-mprof_startpaused”).

FunctionDescription
MALLOC_PROFILER( GMalloc->Exec(nullptr, TEXT("MPROF START"), *GLog); );Starts tracking allocations
MALLOC_PROFILER( GMalloc->Exec(nullptr, TEXT("MPROF STOP"), *GLog); );Stops tracking
MALLOC_PROFILER( GMalloc->Exec(nullptr, TEXT("MPROF STOPALLOCS"), *GLog); );Stops tracking allocations but continues tracking deallocations

Further Reading

You can read more about Malloc Profiler 2 on the following blog page: Introducing Malloc Profiler 2

 


Credits:
Programming/Research: Robert Troughton, Josef Gluyas (Coconut Lizard)
Feedback: Lee Clark, Joe Tidmarsh, Gareth Martin (Coconut Lizard)


Recent Posts

Leave a Reply

Be the First to Comment!

Notify of
avatar
wpDiscuz

Start typing and press Enter to search