22. Debugging Script Bridging

Written by Derek Selander

You’ve learned the basics of LLDB’s Python script bridging. Now you’re about to embark on the frustrating yet exhilarating world of making full LLDB Python scripts.

As you learn about the classes and methods in the Python lldb module, you’re bound to make false assumptions or simply type incorrect code. In short, you’re going to screw up. Depending on the error, sometimes these scripts fail silently, or they may blow up with an angry stderr.

You need a methodical way to figure out what went wrong in your LLDB script so you don’t pull your hair out. In this chapter, you’ll explore how to inspect your LLDB Python scripts using the Python pdb module, which is used for debugging Python scripts. In addition, you can execute your own “normal” Objective-C, Objective-C++, C or Swift code (or even other languages) within SBDebugger’s (or SBCommandReturnObject’s) HandleCommand method.

In fact, there’s alternative ways to execute non-Python code that you’ll learn about in an upcoming chapter, but for now, you’ll stick to HandleCommand and see how to manage a build time error, or fix a script that produces an incorrect result.

Although it might not seem like it at first, this is the most important chapter in the LLDB Python section, since it will teach you how to explore and debug methods while you’re learning this new Python module. I would have (figuratively?) killed for a chapter like this when I was first learning the Script Bridging module.

Debugging your debugging scripts with pdb

Included in the Python distribution on your system is a Python module named pdb you can use to set breakpoints in a Python script, just like you do with LLDB itself! In addition, pdb has other debugging essential features that let you step into, out of, and over code to inspect potential areas of interest.

You’re going to continue using the helloworld.py script in ~/lldb from the previous chapter. If you haven’t read that chapter yet, copy the helloworld.py from the starter directory into a directory named lldb inside your home directory.

Either way, you should now have a file at ~/lldb/helloworld.py.

Open up helloworld.py and navigate to the your_first_command function, replacing it with the following:

def your_first_command(debugger, command, result, internal_dict):
    import pdb; pdb.set_trace()
    print ("hello world")

Note: It’s worth pointing out pdb will not work when you’re debugging Python scripts in Xcode. The Xcode console window will hang once pdb is tracing a script, so you’ll need to do all pdb Python script debugging in a Terminal window.

Save your changes and open a Terminal window to create a new LLDB session. In Terminal, type:

lldb

Next, execute the yay command (which is defined in helloworld.py, remember?) like so:

(lldb) yay woot

Execution will stop and you’ll get output similar to the following:

> /Users/derekselander/lldb/helloworld.py(3)your_first_command()
-> print ("hello world")
(Pdb) 

The LLDB script gave way to pdb. The Python debugger has stopped execution on the print line of code within helloworld.py inside the function your_first_command.

When creating a LLDB command using Python, there are specific parameters expected in the defining Python function. You’ll now explore these parameters, namely debugger, command, and result.

Explore the command argument first, by typing the following into your pdb session:

(Pdb) command

This will dump out the commands you supplied to your yay custom LLDB command. This will always come in the form of a str, even if you have multiple arguments or integers as input. Since there’s no logic to handle any commands, the yay command will silently ignore all input. If you typed in yay woot as indicated earlier, only woot would be spat out as the command.

Next up on the parameter exploration list is the result parameter. Type the following into pdb:

(Pdb) result

This will dump out something similar to the following:

<lldb.SBCommandReturnObject; proxy of <Swig Object of type 'lldb::SBCommandReturnObject *' at 0x110323060> >

This is an instance of SBCommandReturnObject, which is a class the lldb module uses to let you indicate if the execution of an LLDB command was successful. In addition, you can append messages that will be displayed when your command finishes.

Type the following into pdb:

(Pdb) result.AppendMessage("2nd hello world!")

This appends a message which will be shown by LLDB when this command finishes. In this case, once your command finishes executing, 2nd hello world! will be displayed. However, your script is still frozen in time thanks to pdb.

Once your LLDB scripts get more complicated, the SBCommandReturnObject will come into play, but for simple LLDB scripts, it’s not really needed. You’ll explore the SBCommandReturnObject command more later in this chapter.

Finally, onto the debugger parameter. Type the following into pdb:

(Pdb) debugger

This will dump out another object of class SBDebugger, similar to the following:

<lldb.SBDebugger; proxy of <Swig Object of type 'lldb::SBDebugger *' at 0x110067180> >

You explored this class briefly in the previous chapter to help create the LLDB yay command. You’ve already learned one of the most useful commands in SBDebugger: HandleCommand.

Resume execution in pdb. Like LLDB, it has logic to handle a c or continue to resume execution.

Type the following into pdb:

(Pdb) c

You’ll get the following output:

hello world!
2nd hello world!

pdb is great when you need to pause execution in a certain spot to figure out what’s gone wrong. For example, you could have some complicated setup code, and pause in an area where the logic doesn’t seem to be correct.

This is a much more attractive solution than constantly typing script in LLDB to execute one line of Python code at a time.

pdb’s post mortem debugging

Now that you’ve a basic understanding of the process of debugging your scripts, it’s time to throw you into the deep end with an actual LLDB script and see if you can fix it using pdb’s post-mortem debugging features.

Depending on the type of error, pdb has an attractive option that lets you explore the problematic stack trace in the event the code you’re running threw an exception. This type of debugging methodology will only work if Python threw an exception; this method will not work if you receive unexpected output but your code executed without errors.

However, if your code has error handling (and as your scripts get more complex, they really should), you can easily hunt down potential errors while building your scripts.

Find the starter folder of the resources for this chapter. Next, copy the findclass.py file over to your default ~/lldb directory. Remember, if you’re stubborn and decided to go with a different directory location, you’ll need to adjust accordingly.

Don’t even look at what this code does yet. It’s not going to finish executing as-is, and you’ll use pdb to inspect it after you view the error.

Once the script has been copied to the correct directory, open a Terminal window and launch and attach LLDB to any program which contains Objective-C. You could choose a macOS application or something on the iOS Simulator, or maybe even a watchOS application.

For this example, I’ll attach to the macOS Photos application, but you’re strongly encouraged to attach to a different application. Hey, that’s part of being an explorer!

Note: You will need to disable SIP to attach to any process on your mac. Also in LLDB version 1000.11.37.1, there’s a pretty serious bug which incorrectly imports the macOS headers — even if you are attached to an iOS Simulator application. You can see if this bug affects you by executing a po @import Foundation in LLDB and observing the output. If you’re affected by this bug, you will need to use a different version of LLDB or check out the Appendix to get around this bug.

Make sure the application is alive and running and attach LLDB to it:

lldb -n Photos

Once the process has attached, import the new script into LLDB:

(lldb) command script import ~/lldb/findclass.py

Provided you placed the script in the correct directory, you should get no output. The script will install quietly.

Figure out what this command does by looking at the documentation, since you haven’t even looked at the source code for it yet. Type the following into LLDB:

(lldb) help findclass

You’ll get output similar to the following:

Syntax: findclass

The `findclass` command will dump all the Objective-C runtime classes it knows about. Alternatively, if you supply an argument for it, it will do a case-sensitive search looking only for the classes that contain the input. 

Usage: findclass  # All Classes
Usage: findclass UIViewController # Only classes that contain UIViewController in name

Cool! Let’s try this command. Try dumping out all classes the Objective-C runtime knows about.

(lldb) findclass

You’ll get a rather annoying error assertion similar to the following:

Traceback (most recent call last):
  File "/Users/derekselander/lldb/findclass.py", line 40, in findclass
    raise AssertionError("Uhoh... something went wrong, can you figure it out? :]")
AssertionError: Uhoh... something went wrong, can you figure it out? :]

It’s clear the author of this script is horrible at providing decent information into what happened in the AssertionError. Fortunately, it raised an error! You can use pdb to inspect the stack trace at the time the error was thrown.

In LLDB, type the following:

(lldb) script import pdb
(lldb) findclass
(lldb) script pdb.pm()

This imports pdb into LLDB’s Python context, runs findclass again, then asks pdb to perform a “post mortem”.

LLDB will change to the pdb interface and jump to the line that threw the error.

> /Users/derekselander/lldb/findclass.py(40)findclass()
-> raise AssertionError("Uhoh... something went wrong, can you figure it out? :]")
(Pdb) 

From here, you can use pdb as your new BFF to help explore what’s happening.

Speaking of what’s happening, you haven’t even looked at the source code yet! Lets change that.

Type the following into pdb:

(Pdb) l 1, 50

This will list lines 1, 50 of the findclass.py script.

You have the typical function signature which handles the majority of the logic in these commands:

def findclass(debugger, command, result, internal_dict):

Next up in interesting tidbits is a big long string named codeString, which starts its definition on line 18. It’s a Python multi-line string, which starts with three quotes and finishes with three quotes on line 35. This string is where the meat of this command’s logic lives.

In your pdb session, type the following:

(Pdb) codeString

You’ll get some not-so-pretty output, since dumping a Python string includes all newlines.

'\n    @import Foundation;\n    int numClasses;\n    Class * classes = NULL;\n    classes = NULL;\n    numClasses = objc_getClassList(NULL, 0);\n    NSMutableString *returnString = [NSMutableString string];\n    classes = (__unsafe_unretained Class *)malloc(sizeof(Class) * numClasses);\n    numClasses = objc_getClassList(classes, numClasses);\n\n    for (int i = 0; i < numClasses; i++) {\n      Class c = classes[i];\n      [returnString appendFormat:@"%s,", class_getName(c)];\n    }\n    free(classes);\n    \n    returnString;\n    '

Let’s try that again. Use pdb to print out a pretty version of the codeString variable.

(Pdb) print codeString

Much better!

@import Foundation;
int numClasses;
Class * classes = NULL;
classes = NULL;
numClasses = objc_getClassList(NULL, 0);
NSMutableString *returnString = [NSMutableString string];
classes = (__unsafe_unretained Class *)malloc(sizeof(Class) * numClasses);
numClasses = objc_getClassList(classes, numClasses);

for (int i = 0; i < numClasses; i++) {
  Class c = classes[i];
  [returnString appendFormat:@"%s,", class_getName(c)];
}
free(classes);

returnString;    

This codeString contains Objective-C code which uses the Objective-C runtime to get all the classes it knows about. The final line of this code, returnString, essentially lets you return the value of returnString back to the Python script. More on that shortly.

Scan for the next interesting part. On line 40, the debugger is currently at a raise call. This is also the line that provided the annoyingly vague message you received from LLDB.

37    res = lldb.SBCommandReturnObject()
38    debugger.GetCommandInterpreter().HandleCommand("po " ...
39    if res.GetError(): 
40 ->     raise AssertionError("Uhoh... something went wron...
41    elif not res.HasResult():
42        raise AssertionError("There's no result. Womp wom...

Note the -> on line 40. This indicates where pdb is currently paused.

But wait, res.GetError() looks interesting. Since everything is fair game to explore while pdb has the stack trace, why don’t you explore this error to see if you can actually get some useful info out of this?

(Pdb) print res.GetError()

There you go! Depending whether you decided to break on a macOS, iOS, watchOS, or tvOS app, you might get a slightly different count of error messages, but the idea is the same.

error: warning: got name from symbols: classes
error: 'objc_getClassList' has unknown return type; cast the call to its declared return type
error: 'objc_getClassList' has unknown return type; cast the call to its declared return type
error: 'class_getName' has unknown return type; cast the call to its declared return type

The problem here is the code within codeString is causing LLDB some confusion. This sort of error is very common in LLDB. You often need to tell LLDB the return type of a function, because it doesn’t know what it is. In this case, both objc_getClassList and class_getName have unknown return types.

A quick consultation with Google tells us the two problematic methods in question have the following signatures:

int objc_getClassList(Class *buffer, int bufferCount);
const char * class_getName(Class cls);

All you need to do is cast the return type to the correct value in the codeString code.

Open up ~/lldb/findclass.py and replace the definition of codeString with the following:

codeString = r'''
@import Foundation;
int numClasses;
Class * classes = NULL;
classes = NULL;
numClasses = (int)objc_getClassList(NULL, 0);
NSMutableString *returnString = [NSMutableString string];
classes = (__unsafe_unretained Class *)malloc(sizeof(Class) * numClasses);
numClasses = (int)objc_getClassList(classes, numClasses);

for (int i = 0; i < numClasses; i++) {
  Class c = classes[i];
  [returnString appendFormat:@"%s,", (char *)class_getName(c)];
}
free(classes);

returnString;
'''

Save your work and jump back to your LLDB Terminal window. You’ll still be inside pdb, so type Ctrl + D to exit. Next, type the following:

(lldb) command script import ~/lldb/findclass.py

This will reload the script into LLDB with the new changes in the source code. This is required if you make any changes to the source code and you want to test out the command again without having to restart LLDB.

Try your luck again and dump all of the Objective-C classes available in your process.

(lldb) findclass

Boom! You’ll get a slew of output containing all the Objective-C classes in your program. From your app, from Foundation, from CoreFoundation, and so on. Heh… there’s more than you thought there would be, right?

Try limiting your query to something slightly more manageable. Search for all classes containing the word ViewController:

(lldb) findclass ViewController

Depending on the process you’ve attached to, you’ll get a different amount of classes containing the name ViewController.

When developing commands using the Python script bridging, pdb is a superb tool to keep in your toolbox to help you understand what is happening. It works well for inspecting complicated sections and breaking on problematic areas in your Python script.

expression’s Debug Option

As you saw in Chapter 5, “Expression,” LLDB’s expression command has a slew of options available for when LLDB is evaluating code provided to this command. One of these options, overlooked until now, is the --debug option, or more simply -g. If you supply this option to expression, LLDB will evaluate the expression, but the expression will be written to a file and control will stop as soon as execution hits your command.

Confused? Maybe it would be better to see this option in action. Jump back to your findclass.py file and jump to line 38, which contains the following line of code:

debugger.GetCommandInterpreter().HandleCommand("expression -lobjc -O -- " + codeString, res)

In the options section of the expression command, add the -g option so it now looks like the following:

debugger.GetCommandInterpreter().HandleCommand("expression -lobjc -g -O -- " + codeString, res)

Save your work in findclass.py and reload your script through LLDB:

(lldb) command script import ~/lldb/findclass.py

Once reloaded, give findclass a spin:

(lldb) findclass

Execution will now stop in a method created by the JIT (just in time) compiler and let you debug the code yourself in LLDB!

Note: This script will raise an error because the --debug option was turned on. If you were to use pdb to inspect the res.GetError(), you’ll find that it contains the following message: Execution was halted at the first instruction of the expression function because “debug” was requested… This is OK and not part of an error you should worry about since you’re debugging your own expression. It’s worth noting that you’ll not get a return value from this script since it errored out.

Now you can inspect, step, and even augment parameters just like you would any LLDB expression. Since you’re in the Terminal window, you’ll need to inspect the source code using the source list, or more conveniently, the list or l LLDB command.

In LLDB, type the following:

(lldb) l

This will list the current line and slowly move down through the source file. Repeat to view the next set of lines.

(lldb) l

If you were to keep executing the same command, it would eventually cover all the source lines available and produce no more output. Another solution to viewing and stepping through source code while in a LLDB Terminal window is to use the gui LLDB command. This recently-added command in LLDB will transform your Terminal window into a curses-style GUI.

Type the following to jump into the LLDB GUI window:

(lldb) gui

From here, you can step through code using the N key, or step into code using S. Once you’re at a location of interest, you can exit out of the LLDB GUI by typing Fn + F1 (or just F1 if you don’t have the standard function keys enabled) to bring up the LLDB menu.

From there, press X to Exit out of the LLDB GUI and back into your console to print out/modify or alter control.

Using the --debug option is a great way to hunt for logic that returns unexpected results in your script that is running “actual” code — that is, JIT code — inside the process.

For example, if your script gave you unexpected results, I would get rid of all pdb instances, add the -g option to a expression command executed by HandleCommand and then execute the custom command I was working on. From there, I would use the LLDB console (through Terminal or through Xcode… which is a far better way to view the source code) and then hunt for the reason why my JIT code isn’t returning the expected results.

Note: It’s worth noting I have occasionally experienced errors when using po LLDB command while exploring contents inside a paused JIT function created with the -g option. If that’s the case, I’ll fall back to using the frame variable command to explore the parameters of interest. Check out Chapter 6, “Thread, Frame & Stepping Around” to learn more about the frame LLDB command.

Once you’re satisfied with exploring the --debug option, remove the -g option for your Python script.

How to handle problems

As I alluded to in the introduction to this chapter, you’re going to run into problems when building these scripts. Let’s recap what options you have, depending on the type of problem you encounter when building out these scripts.

Typically, you should perform iterative development on a Python script, save, then reload your script while LLDB is attached to a process and the process is still running.

Python build errors

When reloading your script, you might encounter something like this:

This is an example of a build error that occurred when I was creating my script. This command will not successfully load since there are Python syntax errors in it.

This is the most straightforward type of problem, because reloading the script will show me the error. I can tell that on line 37, I have unmatched indentation in the findclass Python script.

Python runtime errors or unexpected values

What if your Python script loads just fine, and you don’t get any build errors to the console when reloading — but you receive unexpected output, or your script crashes and you need to further inspect what’s happening?

Now, you can use the Python pdb module. Go to your Python script (in this case, findclass.py) and add the following line of code right before you expect the problem to occur:

import pdb; pdb.set_trace()

Jump over to Terminal (again, pdb will freeze Xcode, so Terminal is your only option for pdb) and attach to a process with LLDB, then try your command again.

From there, execution will eventually freeze and hit your pdb-triggered breakpoint, where you can inspect parameters and step through the flow of execution.

JIT code build errors

Often, you’re executing actual code inside the process and then return the value back to your Python script. Again, this will be referred to as JIT code throughout the remainder of the book.

Imagine the following: you’re executing a long batch of JIT code, and when running the JIT code in a HandleCommand method from the LLDB Python module you get an error saying something is not working.

This is one of the more annoying aspects with working with these scripts, since the debugger won’t give you line information along with the error. If you can’t uniquely identify where the error could have originated, you’ll need to systematically comment out areas of your code until HandleCommand produces no errors for the JIT code.

From there, you can hone in on any locations giving you problems, and fix them.

JIT code with unexpected results

The final types of errors you could encounter are unexpected results from your JIT code. For example, in the findclass.py script, what if you didn’t get an expected class? What if you get more hits than you would have expected, searching for a particular query?

This is when that --debug option from the LLDB expression command comes in handy. Hunt down the method for SBDebugger’s or SBCommandReturnObject’s HandleCommand and add the -g option when the expression command is being used.

debugger.GetCommandInterpreter().HandleCommand("expression -lobjc -O -g -- " + codeString, res)

Reload your script, then execute the command.

Control will stop on the JIT code and let you inspect it to determine what went wrong. If you do this in Xcode, you have all the conveniences of your hotkeys while viewing the source code to let you inspect and step over execution to hunt down the problem.

Where to go from here?

You’re now equipped to tackle the toughest debugging problems while making your own custom scripts!

There’s a lot more you can do with pdb than what I described here. Check out https://docs.python.org/2.7/library/pdb.html and read up on the other cool features of pdb. Be sure to remember that the version of pdb must match the version of Python that LLDB is using.

While you’re at it, now’s the time to start exploring other Python modules to see what other cool features they have. Not only do you have the lldb Python module, but you also have the full power of Python to use when creating advanced debugging scripts.