26. SB Examples, Improved Lookup

Written by Derek Selander

For the rest of the chapters in this section, you’ll focus on Python scripts.

As alluded to in the previous chapter, the image lookup -rn command is on its way out. Time to make a prettier script to display content.

Here’s what you get right now with the image lookup -rn command:

When you finish this chapter, you’ll have a new script named lookup which queries in a much cleaner way.

In addition, you’ll add a couple of parameters to the lookup command to add some bells and whistles for your new searches.

Automating script creation

Included with the starter directory of this project are two Python scripts that will make your life easier when creating LLDB script content. They are as follows:

• generate_new_script.py: This will create a new skeleton script with whatever name you provide it and stick it into the same directory generate_new_script resides in.
• lldbinit.py: This script will enumerate all scripts (files that end with .py) located within the same directory as itself and try to load them into LLDB. In addition, if there are any files with a txt extension, LLDB will try to load those files’ contents through command import.

Take both of these files found in the starter folder of this chapter and stick them into your ~/lldb/ directory.

Once the files are in their correct locations, jump over to your ~/.lldbinit file and add following line of code:

command script import ~/lldb/lldbinit.py

This will load the lldbinit.py file which will enumerate all .py files and .txt files found in the same directory and load them into LLDB. This means that from here on out, simply adding a script file into the ~/lldb directory will load it automatically once LLDB starts.

Creating the lookup command

With your new tools properly set up, open up a Terminal window. Launch a new instance of LLDB:

lldb

As expected, you’ll be greeted by the LLDB prompt.

Make sure there are no build errors in any of your existing LLDB scripts:

(lldb) reload_script

If your output is free of errors, it’s time to try out your new command __generate_script (implemented from the generate_new_script.py file).

In LLDB, type:

(lldb) __generate_script lookup

If everything went as expected, you’ll get output similar to the following:

Opening "/Users/derekselander/lldb/lookup.py"...

In addition, a Finder window will pop up showing you the location of the file. It’s pretty crazy what you can do with these Python scripts, right?

Hold onto the Finder window for a second — don’t close it. Head back to the LLDB Terminal window and apply the reload_script command.

Since the lookup.py script was created in the same directory as the lldbinit.py file and you have just reloaded the contents of ~/.lldbinit, you’ll now have a working skeleton of the lookup.py file. Give the command a go.

(lldb) lookup

You’ll get the following output:

Hello! the lookup command is working!

Now you can create and use custom commands in as little as two LLDB commands. Yeah, you could do all the setup in one command, but I like having control over when my scripts reload.

lldbinit directory structure suggestions

The way I’ve structured my own lldbinit files might be insightful to some. This is not a required section, but more of a suggestion on how to organize all of your custom scripts and content for LLDB.

I tend to keep my ~/.lldbinit as light as possible and use a script like lldbinit.py to load all my contents from a particular directory. Facebook’s Chisel does the same thing with the fblldb.py file. Check it out if you’re interested.

I keep that directory under source control in case I need to transfer logic to a different computer, or in case I completely screw something up. For example, my actual ~/.lldbinit file (when not working on this book) only contains the following items:

command script import /Users/derekselander/lldb_repo/lldb_commands/lldbinit.py
command script import /Users/derekselander/chisel/chisel/fblldb.py

The lldb_repo is a public git repository at https://github.com/DerekSelander/lldb which contains some LLDB scripts designed for reverse engineering.

I also have Facebook’s Chisel on source control, so whenever those developers push a new, interesting release, I’ll just pull the latest from my Chisel source control directory at https://github.com/facebook/chisel and I’ll have everything I need the next time I run LLDB, or reload my scripts through reload_script.

Inside my lldb_commands directory, I have all my Python scripts as well as two text files. One text file is named cmds.txt and holds all my command regex’s and command alias’s. I also have another file named settings.txt, which I use to augment any LLDB settings.

For example, the only content I have in my settings.txt file at the moment is:

settings set target.skip-prologue false
settings set target.x86-disassembly-flavor intel

You’ve already added these settings to your ~/.lldbinit file earlier in this book, but I prefer this implementation to separate out my custom LLDB commands to my LLDB settings so I don’t get lost when grep’ing my ~/.lldbinit file.

However, for this book, I chose to keep each chapter content independent for each script installation. This means you’ve manually added content to your ~/.lldbinit file so you know what’s happening. You should revisit this new structure implementation when (if?) you finish this book, as there are several benefits to this suggested layout. The benefits are as follows:

1. Calling reload_script only displays the commands ~/.lldbinit is loading; it will not display the sub-scripts being loaded. For example this will echo back the lldbinit.py being loaded, but not echo out the content lldbinit.py itself loads.

   This makes it easier to create scripts because I often use reload_script as a way to check for any error messages on the latest script I am working on. The less output there is from executing reload_script, the less output there is to review when checking for errors in the console.

2. As noted, having as little content as possible in ~/.lldbinit will let you easily transfer content between computers, especially if that content is under source control.

3. Finally, it’s much easier to add new scripts with this implementation. Just stick them in the same directory as the lldbinit.py file and it will be loaded next time. The alternative is to manually add the path to your script to the ~/.lldbinit file, which can get annoying if you do this frequently.

That’s my two cents on the subject. You’ll use this implementation strategy for the remaining scripts in this section as you only have to add scripts to your ~/lldb directory for them to get loaded into LLDB… which is rather nice, right?

Back to the lookup command!

Implementing the lookup command

As you saw briefly in the previous chapter, the foundation behind this lookup command is rather simple. The main “secret” is using SBTarget’s FindGlobalFunctions API. After that, all you need to do is format the output as you like.

You’ll continue working with the Allocator Xcode project, found in the starter folder for this chapter.

Open the project, and build and run on a iPhone XS Simulator. You’ll use this project to test out your new lookup command queries as the script progresses throughout the chapter.

Once running, pause the application and bring up LLDB.

My memory is a little fuzzy. Which parameters does this FindGlobalFunctions specify? Type the following into LLDB:

(lldb) script help(lldb.SBTarget.FindGlobalFunctions)

You’ll get the following output showing the method signature:

FindGlobalFunctions(self, *args) unbound lldb.SBTarget method
    FindGlobalFunctions(self, str name, uint32_t max_matches, MatchType matchtype) -> SBSymbolContextList

Since it’s a Python class, you can ignore that first self parameter. The str parameter named name will be your lookup query. max_matches will dictate the maximum number of hits you want. If you specify the number 0, it will return all available matches. The matchType parameter is a lldb Python enum on which you can perform different types of searches, such as regex or non-regex.

Since regex searching really is the only way to go, you’ll use the LLDB enum value lldb.eMatchTypeRegex.

The other enum values can be found here: https://lldb.llvm.org/python_reference/_lldb%27-module.html#eMatchTypeRegex

Time to implement this in the lookup.py script. Open up ~/lldb/lookup.py in your favorite text editor. Find the following code at the end of handle_command:

# Uncomment if you are expecting at least one argument
# clean_command = shlex.split(args[0])[0]
result.AppendMessage('Hello! the lookup command is working!')

Delete the above code, and replace it with the following, making sure you preserve the indentation:

# 1
clean_command = shlex.split(args[0])[0]
# 2
target = debugger.GetSelectedTarget()

# 3
contextlist = target.FindGlobalFunctions(clean_command, 0, lldb.eMatchTypeRegex)
# 4
result.AppendMessage(str(contextlist))

Here’s what this does:

1. Obtains a cleaned version of the command that was passed to the script, using the same magic as you saw in Chapter 24.
2. Grabs the instance of SBTarget through SBDebugger.
3. Uses the FindGlobalFunctions API with clean_command. You’re supplying 0, for no upper limit on number of results and giving it the eMatchTypeRegex match type to use a regular expression search.
4. You’re turning the contextlist into a Python str and then appending it to the SBCommandReturnObject.

Back in Xcode, reload the contents through the LLDB console:

(lldb) reload_script

Give the lookup command a go. Remember that DSObjectiveCObject class you spelunked in the previous chapter? Dump everything pertaining to that through LLDB:

lookup DSObjectiveCObject

You’ll get output that actually looks worse than image lookup -rn DSObjectiveCObject:

Use LLDB’s script command to figure out which APIs to explore further:

(lldb) script k = lldb.target.FindGlobalFunctions('DSObjectiveCObject', 0, lldb.eMatchTypeRegex)

This will replicate what you’ve done in the lookup.py script and assign the instance of SBSymbolContextList to the value k. I am a fan of short variables names when exploring API names — if you haven’t noticed.

Explore the documentation of SBSymbolContextList:

(lldb) gdocumentation SBSymbolContextList

While you’re at it, dump all the all the methods implemented by SBSymbolContextList. In LLDB:

(lldb) script dir(lldb.SBSymbolContextList)

This will dump out all the methods SBSymbolContextList implements or overrides. There’s a lot there. But focus on the __iter__ and the __getitem__.

This is good news for your script, since this means SBSymbolContextList is iterable as well as indexable. A second ago, you just assigned an instance of SBSymbolContextList to a variable named k through LLDB.

In the LLDB console, use indexing to grab an item in the k object.

(lldb) script k[0]

This is equivalent to (though much more ugly) typing script k.__getitem__(0). You’ll get something like:

<lldb.SBSymbolContext; proxy of <Swig Object of type 'lldb::SBSymbolContext *' at 0x113a83780> >

Good to know! The SBSymbolContextList holds an “array” of SBSymbolContext.

Use the print command to get the context of this SBSymbolContext:

(lldb) script print k[0]

Your output could differ, but I got the SBSymbolContext which represents -[DSObjectiveCObject setLastName:], like so:

     Module: file = "/Users/derekselander/Library/Developer/Xcode/DerivedData/Allocator-czsgsdzfgtmanrdjnydkbzdmhifw/Build/Products/Debug-iphonesimulator/Allocator.app/Allocator", arch = "x86_64"
CompileUnit: id = {0x00000000}, file = "/Users/derekselander/iOS/dbg/s4-custom-lldb-commands/22.  Ex 1, Improved Lookup/projects/final/Allocator/Allocator/DSObjectiveCObject.m", language = "objective-c"
   Function: id = {0x100000268}, name = "-[DSObjectiveCObject setLastName:]", range = [0x0000000100001c00-0x0000000100001c37)
   FuncType: id = {0x100000268}, decl = DSObjectiveCObject.h:33, compiler_type = "void (NSString *)"
     Symbol: id = {0x0000001e}, range = [0x0000000100001c00-0x0000000100001c40), name="-[DSObjectiveCObject setLastName:]"

You’ll use properties and/or getter methods from the SBSymbolContext to grab the name of this function.

The easiest way to do this is to grab the SBSymbol from the SBSymbolContext through the symbol property. From there the SBSymbol contains a name property, which will return your happy Python string.

Make sure this works in your LLDB console:

(lldb) script print k[0].symbol.name

In my case, I received the following:

-[DSObjectiveCObject setLastName:]

This is enough information to work with in building out your script. You’ll take the SBSymbolContextList, iterate through the items and print out the name of the function it finds.

Head back over to your lookup.py script and modify the contents in the handle_command function. Find the following lines:

# 3
contextlist = target.FindGlobalFunctions(clean_command, 0, lldb.eMatchTypeRegex)
# 4
result.AppendMessage(str(contextlist))

Replace them with the following (indenting correctly!):

contextlist = target.FindGlobalFunctions(clean_command, 0, lldb.eMatchTypeRegex)

output = ''
for context in contextlist:
    output += context.symbol.name + '\n\n'

result.AppendMessage(output)

You’re now iterating all SBSymbolContext’s within the returned SBSymbolContextList, hunting down the name of the function and separating it by two newlines.

Jump back to Xcode, and reload your script:

(lldb) reload_script

Then give your updated lookup command a test in LLDB:

(lldb) lookup DSObjectiveCObject

You’ll get much prettier output than before:

-[DSObjectiveCObject setLastName:]

-[DSObjectiveCObject .cxx_destruct]

-[DSObjectiveCObject setFirstName:]

-[DSObjectiveCObject eyeColor]

-[DSObjectiveCObject init]

-[DSObjectiveCObject lastName]

-[DSObjectiveCObject setEyeColor:]

-[DSObjectiveCObject firstName]

This is nice and all, but I want to see where these functions reside in my process. I want to group all functions to a particular module (an SBModule) when they’re being printed out separated by a header with the module name and number of hits for the module.

Head on back to the lookup.py file. You’ll now create two new functions.

The first function will be named generateFunctionDictionary, which will take your SBBreakpointContextList and generate a Python Dictionary of lists. This dict will contain keys for each module. For the value in the dict, you’ll have a Python list for each SBSymbolContext that gets hit.

The second function will be named generateOutput, which will parse this dictionary you’ve created along with the options you’ve received from the OptionParser instance. This method will return a String to be printed back to the console.

Start by implementing the generateModuleDictionary function right below the handle_command function in your lookup.py script:

def generateModuleDictionary(contextlist):
    mdict = {}
    for context in contextlist:
        # 1
        key = context.module.file.fullpath
        # 2
        if not key in mdict:
            mdict[key] = []

        # 3
        mdict[key].append(context)
    return mdict

Here’s what’s going on:

1. From within the SBSymbolContext, you’re grabbing the SBModule (module), then the SBFileSpec (file), then the Python string of the fullPath and assigning it to a variable named key. It’s important to grab the fullPath (instead of, say, SBFileSpec’s basename property, since there could be multiple modules with the same basename).

2. This mdict variable is going to hold a list of all symbols found, split by module. The key in this dictionary will be the module name, and the value will be an array of symbols found in that module. On this line, you’re checking if the dictionary already contains a list for this module. If not, a blank list is added for this module key.

3. You’re adding the SBSymbolContext instance to the appropriate list for this module. You can safely assume that for every key in the mdict variable, there will be at least one or more SBSymbolContext instances.

Note: A much easier way of getting a unique key would be to just use the __str__() method SBModule has (and pretty much every class in the LLDB Python module). This is the function that gets called when you call Python’s print on one of these objects. However, you wouldn’t be learning about all these classes, properties and methods in the process if you just relied on the __str__() method.

Right below the generateModuleDictionary function, implement the generateOutput function:

def generateOutput(mdict, options, target):
    # 1
    output = ''
    separator = '*' * 60 + '\n'
    # 2
    for key in mdict:
        # 3
        count = len(mdict[key])
        firstItem = mdict[key][0]
        # 4
        moduleName = firstItem.module.file.basename
        output += '{0}{1} hits in {2}\n{0}'.format(separator, 
                                                   count, 
                                                   moduleName)
        # 5
        for context in mdict[key]:
            query = ''
            query += context.symbol.name
            query += '\n\n'
            output += query
  return output

Here’s what this does:

1. The output variable will be the return string that contains all the content eventually passed to your SBCommandReturnObject.
2. Enumerate all the keys found in the mdict dictionary.
3. This will grab the count for the array and the very first item in the list. You’ll use this information to query the module name later.
4. You’re grabbing the module name to use in the header output for each section.
5. This will iterate all the SBSymbolContext items in the Python list and add the names to the output variable.

One final tweak before you can test this out.

Augment the code in the handle_command function so it utilizes the two new methods you’ve just created. Find the following code:

output = ''
for context in contextlist:
    output += context.symbol.name + '\n\n'

And replace it with the following:

mdict = generateModuleDictionary(contextlist)
output = generateOutput(mdict, options, target)

You know what to do. Go to Xcode; reload contents in LLDB.

(lldb) reload_script

Check out your new and improved lookup command:

(lldb) lookup DSObjectiveCObject

You’ll get something like this:

************************************************************
8 hits in Allocator
************************************************************
-[DSObjectiveCObject setLastName:]

-[DSObjectiveCObject .cxx_destruct]

-[DSObjectiveCObject setFirstName:]

-[DSObjectiveCObject eyeColor]

-[DSObjectiveCObject init]

-[DSObjectiveCObject lastName]

-[DSObjectiveCObject setEyeColor:]

-[DSObjectiveCObject firstName]

Cool. Go after all Objective-C methods that begin with initWith, and only contain two parameters.

(lldb) lookup initWith(\w+\:){2,2}\]

You’ll get hits from both public and private modules, all loaded into the Allocator process.

Adding options to lookup

You’ll keep the options nice and simple and implement only two options that don’t require any extra parameters.

You’ll implement the following:

• Add load addresses to each query. This is ideal if you want to know where the actual function is in memory.
• Provide a module summary only. Don’t produce function names, only list the count of hits per module.

The __generate_script command added some placeholders for the generateOptionParser method found at the bottom of the lookup.py file. In the generateOptionParser function, change the function so it contains the following code:

def generateOptionParser():
    usage = "usage: %prog [options] code_to_query"
    parser = optparse.OptionParser(usage=usage, prog="lookup")

    parser.add_option("-l", "--load_address",
          action="store_true",
          default=False,
          dest="load_address",
          help="Show the load addresses for a particular hit")

    parser.add_option("-s", "--module_summary",
          action="store_true",
          default=False,
          dest="module_summary",
          help="Only show the amount of queries in the module")
    return parser

There’s no need to take a deep dive in this code since you learned about this in a previous chapter. You’re creating two supported options, -s, or --module_summary and -l, or --load_address.

You’ll implement the load address option first. In the generateOutput function, navigate to the for-loop iterating over the SBSymbolContext, which starts with the for context in mdict[key]: line of code.

Make that for-loop look like this:

for context in mdict[key]:
    query = ''

    # 1
    if options.load_address:
        # 2
        start = context.symbol.addr.GetLoadAddress(target)
        end = context.symbol.end_addr.GetLoadAddress(target)
        # 3
        startHex = '0x' + format(start, '012x')
        endHex = '0x' + format(end, '012x')
        query += '[{}-{}]\n'.format(startHex, endHex)

    query += context.symbol.name
    query += '\n\n'
    output += query

Here’s what that does:

1. You’re adding the conditional to see if the load_address option is set. If so, this will add content to the output.
2. This traverses the SBSymbolContext to the SBSymbol (symbol property) to the SBAddress (addr or end\_addr) and gets a Python long through the GetLoadAddress method.

There’s actually a load_addr available to SBAddress, but I’ve found it to be a bit buggy at times, so I’ve defaulted to using the GetLoadAddress API instead. This method expects the SBTarget as an input parameter. 3. After you have the start and end addresses expressed in Python long’s, you are formatting them to look pretty and consistent using the Python format function.

This pads the number with zeros if needed, notes it should be 12 digits long, and formats it in hexadecimal.

Save your work and revisit Xcode and the LLDB console. Reload.

(lldb) reload_script

Give your new option a go:

(lldb) lookup -l DSObjectiveCObject

You’ll get output similar to the truncated output:

************************************************************
8 hits in Allocator
************************************************************
[0x0001099d2c00-0x0001099d2c40]
-[DSObjectiveCObject setLastName:]

[0x0001099d2c40-0x0001099d2cae]
-[DSObjectiveCObject .cxx_destruct]

Put a breakpoint at an address from this list to see if it matches with the function. Do it like so, replacing the address with one from your list:

(lldb) b 0x0001099d2c00
Breakpoint 3: where = Allocator`-[DSObjectiveCObject setLastName:] at DSObjectiveCObject.h:33, address = 0x00000001099d2c00

Great job! One more option to implement and then you’re done!

Revisit the generateOutput function for the final time. Find the following line:

moduleName = firstItem.module.file.basename

Add the following code right after that line:

if options.module_summary:
    output += '{} hits in {}\n'.format(count, moduleName)
    continue

This simply adds the number of hits in each module and skips adding the actual symbols.

That’s it. No more code. Save, then head back to Xcode to reload your script:

(lldb) reload_script

Give your module_summary option a go:

(lldb) lookup -s viewWillAppear

You’ll get something similar to this:

1 hits in: GLKit
18 hits in: ContactsUI
3 hits in: DocumentManager
8 hits in: MapKit
49 hits in: UIKitCore
4 hits in: Allocator

That’s it! You’re done! You’ve made a pretty powerful script from scratch. You’ll use this script to search for code in future chapters. The summary option is a great tool to have when you’re casting a wide search and then want to narrow it down further.

Where to go from here?

There are many more options you could add to this lookup command. You could make a -S or -Swift_only query by going after SBSymbolContext’s SBFunction (through the function property) to access the GetLanguage() API. While you’re at it, you should also add a -m or --module option to filter content to a certain module.

If you want to see what else is possible, check out my implementation of lookup here: https://github.com/DerekSelander/LLDB/blob/master/lldb_commands/lookup.py.

Enjoy adding those options!