28. SB Examples, Malloc Logging

Written by Derek Selander

For the final chapter in this section, you’ll go through the same steps I myself took to understand how the MallocStackLogging environment variable is used to get the stack trace when an object is created.

From there, you’ll create a custom LLDB command which gives you the stack trace of when an object was allocated or deallocated in memory — even after the stack trace is long gone from the debugger.

Knowing the stack trace of where an object was created in your program is not only useful for reverse engineering, but also has great use cases in your typical day-to-day debugging. When a process crashes, it’s incredibly helpful to know the history of that memory and any allocation or deallocation events that occurred before your process went off the deep end.

This is another example of a script using stack-related logic, but this chapter will focus on the complete cycle of how to explore, learn, then implement a rather powerful custom command.

Setting up the scripts

You have a couple of scripts to use (and implement!) for this chapter. Let’s go through each one of them and how you’ll use them:

• msl.py: This is the command (which is an abbreviation for MallocStackLogging) is the script you’ll be working on in this chapter. This has a basic skeleton of the logic.

• lookup.py: Wait — you already made this command, right? Yes, but I’ll give you my own version of the lookup command that adds a couple of additional options at the price of uglier code. You’ll use one of the options to filter your searches to specific modules within a process.

• sbt.py: This command will take a backtrace with unsymbolicated symbols, and symbolicate it. You made this in the previous chapter, and you’ll need it at the very end of this chapter. And in case you didn’t work through the previous chapter, it’s included in this chapter’s resources for you to install.

• search.py: This command will enumerate all objects in the heap and search for a particular subclass. This is a very convenient command for quickly grabbing references to instances of a particular class.

Note: These scripts come from https://github.com/DerekSelander/lldb. If I need a tool that I don’t have, I’ll build it, and stick it in the above repo. Check it out for some other novel ideas for LLDB scripts. It’s important to note that a lot of scripts in the above repo have dependencies on other files included in the repo, so if you only download one script, it might not compile until the full set of files is included.

Now for the usual setup. Take all the Python files found in the starter directory for this chapter and copy them into your ~/lldb directory. I am assuming you have the lldbinit.py file already set up, found in Chapter 26, “SB Examples, Improved Lookup.”

Launch an LLDB session in Terminal and go through all the help commands to make sure each script has loaded successfully:

(lldb) help msl
(lldb) help lookup
(lldb) help sbt
(lldb) help search

MallocStackLogging explained

In case you’re unfamiliar with the MallocStackLogging environment variable, I’ll describe it and show how it’s typically used.

When the MallocStackLogging environment variable is passed into a process, and is set to true, it’ll monitor allocations and deallocations of memory on the heap. Pretty neat!

Included within the starter directory is the 50 Shades of Ray Xcode project with some additional logic for this chapter. Open the project.

Before you run it, you’ll need to modify the scheme for your purposes. Select the 50 Shades of Ray scheme (make sure there’s no “Stripped” in the name), then press ⌘ + Shift + < to edit the scheme.

Select Run, then Diagnostics, then select Malloc Stack, then All Allocation and Free History.

Once you’ve enabled this environment variable, build the 50 Shades of Ray program and run it on the iPhone 8 Simulator.

If the MallocStackLogging environment variable is enabled, you’ll see some output from the LLDB console similar to the following:

ShadesOfRay(12911,0x104e663c0) malloc: stack logs being written into /tmp/stack-logs.12911.10d42a000.ShadesOfRay.gjehFY.index

ShadesOfRay(12911,0x104e663c0) malloc: recording malloc and VM allocation stacks to disk using standard recorder

ShadesOfRay(12911,0x104e663c0) malloc: process 12673 no longer exists, stack logs deleted from /tmp/stack-logs.12673.11b51d000.ShadesOfRay.GVo3li.index

Don’t worry about the details of the output; simply look for the presence of output like this as it indicates the MallocStackLogging is working properly.

While the app is running, click the Generate a Ray button at the bottom.

Once a new Ray is created (that is, you see an instance of Ray Wenderlich’s amazingly innovative & handsome face pop up in the Simulator), perform the following steps:

1. Select the Debug Memory Graph located at the top of the LLDB console in Xcode.
2. Select the Show the Debug navigator in the left panel.
3. At the bottom of the left panel select the Show only content from workspace.
4. Select the reference to the RayView.
5. In the right panel of Xcode, make sure the Show the Memory Inspector is selected.

Once you’ve jumped through all those hoops, you’ll have the exact stack trace of where this RayView instance was created through the Backtrace section in Xcode. How cool is that?! The authors of Xcode (and its many modules) have made our lives a bit easier with these memory debugging features!

Plan of attack

You know it’s possible to grab a stack trace for an instantiated object, but you’re going to do one better than Apple.

Your command will be able to turn on the MallocStackLogging functionality at will through LLDB, which means you won’t have to rely on an environment variable. This has the additional benefit that you won’t need to restart your process in case you forget to turn it on during a debug session.

So how are you going to figure out how this MallocStackLogging feature works?

When I am absolutely clueless as to where to begin when exploring built-in code, I follow the rather loose process below and alter queries, depending on the scenario or the output:

• I look for chokepoints where I can safely assume some logic of interest will be executed in a process I am attached to. If I know I can replicate something of interest, I’ll force that action to occur while monitoring it.

• When monitoring the code of interest, I’ll use various tools like LLDB or DTrace (which you’ll learn about in the next chapter) to find the module which holds the code of interest. Again, the module is a dynamic library, framework, NSBundle, or something of that sort.

• Once I find the module of interest, I’ll dump all the code within the module, then filter for what I need using various custom scripts like lookup.py.

• If I find a particular function that looks relevant to my interests, I’ll first try Googling it. I’ll often find some incredibly useful hints on https://opensource.apple.com/ that reveals how I can use what I’ve found.

• Searching through Apple’s opensource URLs, I’ll grab as much context as I can about the code of interest. Sometimes there’s code in the C/C++ source file that will give me an idea of how to formulate the parameters into the function, or perhaps I’ll get a description of the code or its purpose in the header file.

• If there’s no documentation to be gained from Googling, I’ll set breakpoints on the code of interest and see if I can trigger that function naturally. Once hit, I’ll explore both the stack frames and registers to see what kind of parameters are being passed in, as well as the context it’s used in.

You’re going to follow the exact same steps to see where the code for MallocStackLogging resides, explore the module responsible for handling stack tracing logic, then explore any interesting code of interest within that module.

Let’s get cracking!

Hunting in getenv

MallocStackLogging is an environment variable passed into the process. This means the C getenv function is likely used to check if this argument is supplied, and perform additional logic if it is.

You need to dump all the items queried with getenv when the process starts up. You’ll perform the same action you did in Chapter 16, “Hooking & Executing Code with dlopen & dlsym” by creating a symbolic breakpoint to dump the char* parameter when getenv is being called.

In Xcode, create a symbolic breakpoint with the following logic:

• Symbol: getenv
• Action: po (char *)$arg1
• Automatically continue after evaluating actions: yep!

Build and run the program with the MallocStackLogging variable still checked.

From the output, you can see that somewhere in the startup process, there’s code that checks for the presence of MallocStackLogging.

Modify your symbolic breakpoint to only dump the stack trace when the program is checking for the MallocStackLogging environment variable:

• Symbol: getenv
• Condition: ((int)strcmp("MallocStackLogging", $arg1) == 0)
• Action: bt
• Automatically continue after evaluating actions: ¡Sí!

Once your augmented symbolic breakpoint is set up, rerun the app.

You’ll get a couple of stack traces in the console. Check out the very first one:

  * frame #0: 0x0000000112b4da26 libsystem_c.dylib`getenv
    frame #1: 0x0000000112c7dd53 libsystem_malloc.dylib`_malloc_initialize + 466
    frame #2: 0x0000000112ddcac1 libsystem_platform.dylib`_os_once + 36
    frame #3: 0x0000000112c7d849 libsystem_malloc.dylib`default_zone_malloc + 77
    frame #4: 0x0000000112c7d259 libsystem_malloc.dylib`malloc_zone_malloc + 103
    frame #5: 0x0000000112c7f44a libsystem_malloc.dylib`malloc + 24
    frame #6: 0x0000000112aa2947 libdyld.dylib`tlv_load_notification + 286
    frame #7: 0x000000010e0f68a9 dyld_sim`dyld::registerAddCallback(void (*)(mach_header const*, long)) + 134
    frame #8: 0x0000000112aa1a0d libdyld.dylib`_dyld_register_func_for_add_image + 61
    frame #9: 0x0000000112aa1be7 libdyld.dylib`_dyld_initializer + 47

Interesting… Check out stack frame 1:

frame #1: 0x0000000112c7dd53 libsystem_malloc.dylib`_malloc_initialize + 466

If I were an Apple author, I would likely be checking for an environment variable to conditionally see if my code should run right when it’s initialized. This looks like it’s doing the same, plus the module name, libsystem_malloc.dylib fits the bill for something implementing malloc stack logging related logic. Is this it? Maybe. Worth checking out? Totally!

Take a deeper dive into this module and see what it has to offer you.

Using the fancy, new & improved lookup command, explore all the methods implemented by the libsystem_malloc.dylib module that you can execute within your process.

Pause the app in the debugger, and then type the following in your LLDB console:

(lldb) lookup . -m libsystem_malloc.dylib

In iOS 12.0, I get 420 hits. I could gloss through all these methods, but I am getting increasingly lazy as a debugger person. Let’s just hunt for everything that pertains to the word “log” (for logging) and see what we get. Type the following in LLDB:

(lldb) lookup (?i)log -m libsystem_malloc.dylib

I get 54 hits from using a case insensitive search for the word log inside the libsystem_malloc.dylib module.

This hit count is bearable enough to weed through.

Do any of those functions look interesting? Hell yeah! Here are some of the following functions that look interesting to me:

create_log_file

open_log_file_from_directory

__mach_stack_logging_get_frames

turn_off_stack_logging

turn_on_stack_logging

Of my top 5, the turn_on_stack_logging and the __mach_stack_logging_get_frames look like they’re worth checking out.

You’ve found the module of interest, as well as some functions worth further exploration. Time to jump over to Google and see what’s out there.

Googling JIT function candidates

Google for any code pertaining to turn_on_stack_logging. Take a look at this search query:

At the time I wrote this, I got three hits from Google (well, it was actually eight hits with “exclude similar searches” off, but that’s not the point).

These functions are not well-known and are not typically discussed in any circle outside of Apple. In fact, I am rather confident the majority of iOS application developers in Apple don’t know about them either, because when would they use them for writing apps?

This stuff belongs to the low-level C developers of Apple, whom we totally take for granted.

From the Google search, check out the following code from the header file found at https://opensource.apple.com/source/libmalloc/libmalloc-116/private/stack_logging.h.auto.html:

typedef enum {
  stack_logging_mode_none = 0,
  stack_logging_mode_all,
  stack_logging_mode_malloc,
  stack_logging_mode_vm,
  stack_logging_mode_lite
} stack_logging_mode_type;

extern boolean_t turn_on_stack_logging(stack_logging_mode_type mode);

This is some really good information to work with. The turn_on_stack_logging function expects one parameter of type int (C enum). The enum stack_logging_mode_type tells you if you want the stack_logging_mode_all option, it will be at value 1.

You’ll run an experiment by turning off the stack logging environment variable, execute the above function via LLDB, and see if Xcode is recording stack traces for any malloc’d object after you’ve called turn_on_stack_logging.

Before you do that, you’ll first explore the other function, __mach_stack_logging_get_frames.

Exploring __mach_stack_logging_get_frames

Fortunately, for your exploration efforts, __mach_stack_logging_get_frames can also be found in the same header file. This function signature looks like the following:

extern kern_return_t __mach_stack_logging_get_frames(
                                        task_t task,   
                          mach_vm_address_t address, 
             mach_vm_address_t *stack_frames_buffer,
                          uint32_t max_stack_frames,
                                   uint32_t *count);
    /* Gets the last allocation record (malloc, realloc, or free) about address */

This is a good starting point, but what if there are parameters you’re not 100% sure how to obtain? For example, what’s task_t task all about? This is basically a parameter which specifies the process you want this function to act on. But what if you didn’t know that?

Using Google and searching for any implementation files that contain __mach_stack_logging_get_frames can be a big help when you’re uncertain about things like this.

After a casual Googling, the https://llvm.org/svn/llvm-project/lldb/trunk/examples/darwin/heap_find/heap/heap_find.cpp URL provides insight to the first parameter that’s expected within this function.

This file contains the following code:

task_t task = mach_task_self();
/* Omitted code.... */
    stack_entry->address = addr;
    stack_entry->type_flags = stack_logging_type_alloc;
    stack_entry->argument = 0;
    stack_entry->num_frames = 0;
    stack_entry->frames[0] = 0;

    err = __mach_stack_logging_get_frames(task, 
                       (mach_vm_address_t)addr,
                           stack_entry->frames, 
                                    MAX_FRAMES,
                      &stack_entry->num_frames);

    if (err == 0 && stack_entry->num_frames > 0) {
      // Terminate the frames with zero if there is room
      if (stack_entry->num_frames < MAX_FRAMES)
        stack_entry->frames[stack_entry->num_frames] = 0;
    } else {
      g_malloc_stack_history.clear();
    }
  }
}

The task_t parameter has a easy way to get the task representing the current process through the mach_task_self function located in libsystem_kernel.dylib. You can confirm this yourself with the lookup LLDB command.

Testing the functions

To prevent you from getting bored to tears, I’ve already implemented the logic for the __mach_stack_logging_get_frames inside the app.

Hopefully, you still have the application running. If not, get the app running with MallocStackLogging still enabled.

It’s always a good idea to build your proof-of-concept JIT code in Xcode first, and once it’s working, then (and only then!) transfer it to your LLDB script. You’re gonna hate your life if you try to write your POC JIT script code straight in LLDB first. Trust me.

In Xcode, navigate to the stack_logger.cpp file. __mach_stack_logging_get_frames was written in C++, so you’ll need to use C++ code to execute it.

The only function in this file is trace_address:

void trace_address(mach_vm_address_t addr) {

  typedef struct LLDBStackAddress {
    mach_vm_address_t *addresses;
    uint32_t count = 0;
  } LLDBStackAddress;   // 1 
  
  LLDBStackAddress stackaddress; // 2
  __unused mach_vm_address_t address = (mach_vm_address_t)addr;
  __unused task_t task = mach_task_self_;  // 3

  stackaddress.addresses = (mach_vm_address_t *)calloc(100,
                                sizeof(mach_vm_address_t)); // 4

  __mach_stack_logging_get_frames(task, 
                               address, 
                stackaddress.addresses, 
                                   100, 
                  &stackaddress.count); // 5
  
  // 6
  for (int i = 0; i < stackaddress.count; i++) {
    
    printf("[%d] %llu\n", i, stackaddress.addresses[i]);
  }
  
  free(stackaddress.addresses); // 7
}

Breakdown time!

1. As you know, LLDB only lets you return one object to be evaluated. But, as a creative string-theory version of yourself, can create C structs that contain any types you want to be returned.
2. Declare an instance of said struct for use within the function.
3. Remember mach_task_self that was referenced earlier? The global variable mach_task_self_ is the value returned when calling mach_task_self.
4. Since you’re in a lower level, you don’t have ARC to help you allocate items on the heap. You’re allocating 100 mach_vm_address_t’s, which is more than enough to handle any stack trace.
5. The __mach_stack_logging_get_frames then executes. The addresses array of the LLDBStackAddress struct will be populated with the addresses if there’s any stack trace information available.
6. Print out all the addresses that were found
7. Finally, the mach_vm_address_t objects you created are freed.

Time to give it a whirl!

LLDB testing

Make sure the app is running, then tap the Generate a Ray! button. Pause execution and enter the following into LLDB:

(lldb) search RayView -b

The search script will enumerate all objects of a certain type in the heap. This command will hunt for all RayView instances that are currently alive.

The -b option will give you the --brief functionality, free of the class’s description or debugDescription method. Depending on the amount of Ray Wenderlich faces on your Simulator, you’ll get a variable amount of hits.

I have three wondrously magical Ray Wenderlich faces on my simulator, so I get the following output:

(lldb) search RayView -b
RayView * [0x00007fa838414330]
RayView * [0x00007fa8384125f0]
RayView * [0x00007fa83860c000]

Grab any one of those addresses and execute the logic in the trace_address function:

(lldb) po trace_address(0x00007fa838414330)

You’ll get output that looks like the following truncated snippet:

[0] 4533269637
[1] 4460190625
[2] 4460232164
[3] 4454012240
[4] 4478307618
[5] 4482741703
[6] 4478307618
[7] 4479898204
[8] 4479898999
[9] 4479899371
...

These are the actual addresses of the code where this object is created. Verify the first address is code in memory using image lookup:

(lldb) image lookup -a 4533269637

You’ll get the details about that function:

Address: libsystem_malloc.dylib[0x000000000000f485] (libsystem_malloc.dylib.__TEXT.__text + 56217)
Summary: libsystem_malloc.dylib`calloc + 30

There’s more than one way to skin a memory address. Copy the address at frame three and use SBAddress to get the information out of this address:

(lldb) script print lldb.SBAddress(4454012240, lldb.target)

You’ll get stack frame 3, like so:

ShadesOfRay`-[ViewController generateRayViewTapped:] + 64 at ViewController.m:38

Navigating a C array with lldb.value

You’ll again use the lldb.value class to parse the return value of this C struct which was generated inline while executing this function.

Set a GUI breakpoint at the end of the trace_address function.

Use LLDB to execute the same function, but honor breakpoints, and remember to replace the address with one of your RayView instances:

(lldb) e -lobjc++ -O -i0 -- trace_address(0x00007fa838414330)

Execution will stop on the final line of trace_address. You know the drill. Grab the reference to the C struct LLDBStackAddress, stackaddress.

(lldb) script print lldb.frame.FindVariable('stackaddress')

If successful, you’ll get the synthetic format of the stackaddress variable:

(LLDBStackAddress) stackaddress = {
  addresses = 0x00007fa838515cd0
  count = 25
}

Cast this struct into a lldb.value and call the reference a:

(lldb) script a = lldb.value(lldb.frame.FindVariable('stackaddress'))

Ensure a is valid:

(lldb) script print a

You can now easily reference the variables you declared in the LLDBStackAddress struct inside the lldb.value. Type the following into LLDB:

(lldb) script print a.count

You’ll get the stack frame count:

(uint32_t) count = 25

What about the addresses array inside the LLDBStackAddress struct?

(lldb) script print a.addresses[0]

That’s the memory address of the first frame. What about that generateRayViewTapped: method found in frame 3?

(lldb) script print a.addresses[3]

You’ll get something similar to:

(mach_vm_address_t) [3] = 4454012240

Do you see how this tool is coming together? From finding chokepoints of items of interest, to exploring code in modules, to researching tidbits of useful information in https://opensource.apple.com/, to implementing proof of concepts in Xcode before jumping to LLDB Python code, there’s a lot of power under the hood.

Don’t slow down — it’s command implementin’ time!

Turning numbers into stack frames

Included within the starter directory for this chapter is the msl.py script for malloc script logging. You’ve already installed this msl.py script earlier in the “Setting up the scripts” section.

Unfortunately, this script doesn’t do much at the moment, as it doesn’t produce any output. Time to change that.

Open up ~/lldb/msl.py in your favorite editor. Find handle_command and add the following code to it:

command_args = shlex.split(command)
parser = generateOptionParser()
try:
    (options, args) = parser.parse_args(command_args)
except:
    result.SetError(parser.usage)
    return

cleanCommand = args[0]
process = debugger.GetSelectedTarget().GetProcess()
frame = process.GetSelectedThread().GetSelectedFrame()
target = debugger.GetSelectedTarget()

All this logic shouldn’t be new to you, as it’s the “preamble” required to start up the command. The only thing of interest is you opted to omit the posix=False argument that’s sometimes used in the shlex.split(command). There’s no need to provide this parameter, since this command won’t be handling any weird backslash or dash characters. This means the parsing of the output from the options and args variables is much cleaner as well.

Now that you have the basic script going, implement the meat of this script right below the code you just wrote:

# 1
script = generateScript(cleanCommand, options)

# 2
sbval = frame.EvaluateExpression(script, generateOptions())

# 3
if sbval.error.fail: 
    result.AppendMessage(str(sbval.error))
    return

val = lldb.value(sbval)
addresses = []

# 4
for i in range(val.count.sbvalue.unsigned):
    address = val.addresses[i].sbvalue.unsigned
    sbaddr = target.ResolveLoadAddress(address)
    loadAddr = sbaddr.GetLoadAddress(target)
    addresses.append(loadAddr)

# 5
retString = processStackTraceStringFromAddresses(
                                        addresses, 
                                           target)

# 6
freeExpr = 'free('+str(val.addresses.sbvalue.unsigned)+')'
frame.EvaluateExpression(freeExpr, generateOptions())
result.AppendMessage(retString)

Here are the items of interest:

1. Use the generateScript function I supplied, which returns a string containing roughly the same code as in the trace_address function.
2. Execute the code. You know this will return an SBValue.
3. Do a sanity check to see if the EvaluateExpression fails. If it does, dump out the error and exit early.
4. This for-loop will enumerate the memory addresses in the val object, which are the output of the script code, and pull them out into the addresses list.
5. Now that the addresses are pulled out into a list, you pass that list to a predefined function for processing. This will return the stack trace string you’ll spit out.
6. Finally, you manually allocate memory, as you’re a good memory citizen and always clean up after yourself. Most of these scripts you’ve written leak memory, but now that you’re getting more advanced with this stuff, it’s time to do the right thing and free any allocated memory.

Jump back to the Xcode LLDB console and reload your stuff:

(lldb) reload_script

Provided you have no errors, grab a reference to a RayView using the search LLDB command:

(lldb) search RayView -b

Just for kicks, here’s another way to search for all UIViews whose class is implemented in the ShadesOfRay module:

(lldb) search UIView -m ShadesOfRay -b

Once you have a reference to a RayView, run your newly created msl command on it, like so:

(lldb) msl 0x00007fa838414330

You’ll get your expected output just like in Xcode!

frame #0 : 0x11197d485 libsystem_malloc.dylib`calloc + 30
frame #1 : 0x10d3cbba1 libobjc.A.dylib`class_createInstance + 85
frame #2 : 0x10d3d5de4 libobjc.A.dylib`_objc_rootAlloc + 42
frame #3 : 0x10cde7550 ShadesOfRay`-[ViewController generateRayViewTapped:] + 64
frame #4 : 0x10e512d22 UIKit`-[UIApplication sendAction:to:from:forEvent:] + 83

Congratulations! You’ve created a script that will give you the stack trace for an object. Now it’s time to level up and give this script some cool options!

Stack trace from a Swift object

OK — I know you want me to talk about Swift code. You’ll cover a Swift example as well.

Included in the 50 Shades of Ray app is a Swift module, ingeniously named SomeSwiftModule. Within this module is a class named SomeSwiftCode with a static variable to get your singleton quota going.

The code in SomeSwiftCode.swift is about as simple as you can get:

public final class SomeSwiftCode {
  private init() {}
  static let shared = SomeSwiftCode()
}

You’ll use LLDB to call this singleton and examine the stack trace where this function was created.

First off, you have to import your Swift modules! Enter the following into LLDB:

(lldb) e -lswift -O -- import SomeSwiftModule

You’ll get no result if the above was successful.

In LLDB, access the singleton, like so:

(lldb) e -lswift -O -- SomeSwiftCode.shared

You’ll get the address to this object:

<SomeSwiftCode: 0x600000033640>

Now you’ll pass this address in to the msl command. But simply copying and pasting from the current output is waaaaaaaaaaaaaaaay too easy. Use the search command instead and search for all subclasses of SwiftObject:

(lldb) search SwiftObject

You’ll get something like the following:

<__NSArrayM 0x6000004578b0>(
SomeSwiftModule.SomeSwiftCode
)

Again, Swift tries to hide the pointer from you in description. That’s part of its magic!

Use the --brief (-b) option one final time in the search command to grab the instance and ignore the object’s description method.

(lldb) search SwiftObject -b

This will grab the mangled name, but it’s the same reference in memory!

_TtC15SomeSwiftModule13SomeSwiftCode * [0x0000600000033640]

Use the msl command on this address:

(lldb) msl 0x0000600000033640

You’ll get your expected stack trace.

The highlighted frame here is clearly the frame where you call the singleton accessor from LLDB. Yours might be different.

Let’s jump to one final topic I want to discuss briefly: how to build these scripts so you “Don’t Repeat Yourself” when creating functionality in your LLDB scripts.

DRY Python code

Stop the app! In the schemes, select the Stripped 50 Shades of Ray Xcode scheme.

Ensure the MallocStackLogging environment variable is unchecked in the Stripped 50 Shades of Ray scheme.

Good. Ray approves.

Time to try out the turn_on_stack_logging function. Build and run the application.

As you found out in the previous chapter, the “Stripped 50 Shades of Ray” scheme strips the main executable’s contents so there’s no debugging information available. Remember that factoid when you use the msl command.

Once the application is up and running, tap the Generate a Ray! button to create a new instance of the RayView. Since the MallocStackLogging isn’t enabled, let’s see what happens…

Pause execution and search for all RayViews by typing the following into LLDB:

(lldb) search RayView -b

You’ll get something like:

RayView * [0x00007fc23eb00620]

See if the msl command works on this address:

(lldb) msl 0x00007fc23eb00620

Nothing. That makes sense though, because the environment variable was not supplied to the process. Time to circle back and call turn_on_stack_logging to see what it does. Type the following in LLDB:

(lldb) po turn_on_stack_logging(1)

You’ll get some output similar to the kind you get when you supply your process with the MallocStackLogging environment variable:

Resume execution and create another instance of RayView by tapping the bottom button.

Once you’ve done that, pause execution and search for all instances of RayView again.

You’ll get a new address this time. Hopefully with the stack logging enabled, you’ll get a backtrace for this.

Copy this new address and apply the msl command to it.

(lldb) msl 0x00007f8250f0a170

This will give you the stack trace!

This is awesome! You can enable malloc logging at will to monitor any allocation or deallocation events without having to restart your process.

Wait wait wait. Hold on a second… there’s a symbol that’s stripped.

Ray don’t like no stripped functions.

If you recall in the previous chapter, you created the sbt command which symbolicated a stack trace. In the sbt.py script, you created the processStackTraceStringFromAddresses function which took a list of numbers (representing memory addresses for code) and the SBTarget. This function then returned a potentially symbolicated string for the stack trace.

You’ve already done the hard work to write this function, so why not include this work in the msl.py script to optionally execute it?

Jump to the very top of the msl.py function and add the following import statement:

import sbt

In the handle_command function in msl.py, hunt for the following code:

retString = sbt.processStackTraceStringFromAddresses(
                                            addresses, 
                                               target)

Replace that code with the following:

if options.resymbolicate:
    retString = sbt.processStackTraceStringFromAddresses(
                                                addresses, 
                                                   target)
else:
    retString = processStackTraceStringFromAddresses(
                                        addresses, 
                                           target)

You’re conditionally checking for the options.resymbolicate option (which I’ve already set up for you). If True, then call the logic in the sbt module to see if it can generate a string of resymbolicated functions.

Since you wrote that function to be generic and handle a list of Python numbers, you can easily pass this information from your msl script.

Before you test this out, there’s one final component to implement. You need to make a convenience command to enable the turn_on_stack_logging.

Jump up to the __lldb_init_module function (still in msl.py) and add the following line of code:

debugger.HandleCommand('command alias enable_logging expression -lobjc -O -- extern void turn_on_stack_logging(int); turn_on_stack_logging(1);')

This declares a convenience command to turn on malloc stack logging.

Woot! Done! Jump back to Xcode and reload your script:

(lldb) reload_script

Use the --resymbolicate option on the previous RayView to see the stack in its fully symbolicated form.

(lldb) msl 0x00007f8250f0a170 -r

I am literally crying with happiness in the face of this wholly beautiful stack trace. Snif.

Where to go from here?

Hopefully, this full circle of idea, research & implementation has proven useful and even inspired you to create your own scripts. There’s a lot of power hidden quietly away in the many frameworks that already exist on your [i|mac|tv|watch]OS device.

All you need to do is find these hidden gems and exploit them for some crazy commercial debugging tools, or even to use in reverse engineering to better understand what’s happening.

Here’s a list of directories you should explore on your actual iOS device:

• /Developer/
• /usr/lib/
• /System/Library/PrivateFrameworks/

Go forth my little debuggers, and build something that completely blows my mind!