2. Overview & Getting Help

Written by Walter Tyree

LLDB is a powerful debugger included in a suite of compiler tools and technologies known as the LLVM project. LLDB is an open source project so you can compile it and run your own version if you are so inclined. LLDB supports a number of platforms (like Linux, Windows, NetBSD), but the primary consumers of LLDB are engineers using Apple platforms such as iOS and macOS. As such, this book is primarily geared towards Apple development.

Developers building for Apple platforms will likely have little need to build LLDB from source, opting to use the precompiled LLDB version packaged in Xcode, located at Xcode.app/Contents/Developer/usr/bin/lldb. The LLDB version shipping with Xcode is in constant flux and is not associated with the public LLVM versioning. Typing lldb --version will give you a version internal to Apple. However, https://en.wikipedia.org/wiki/Xcode, has a table mapping the correlation of Xcode versions to the associated LLVM toolchain release. At the time of writing, Apple is shipping LLVM 15 in Xcode 14.3. Xcode’s LLVM toolchain version is usually a little behind what comes from the LLVM project directly. This can be useful for tracking down bugs or expected features.

Note: When referring to the program executable, lldb is lowercase. While referring to LLDB as a system of LLVM, uppercase is used.

Building LLDB via Xcode

You are not required to build LLDB for any of the chapters in this book. However, I can’t give you a book about LLDB without providing instruction to the source and building it for those who are interested. If this does not interest you, feel free to skip this section.

If compiling LLDB is of interest, you’ll need several tools as well as about 50GB of free space on your computer. The first tool you’ll need to build LLDB is CMake. CMake is a program used to generate build instructions that other tools can use to actually build a project. That is, if you want to build with the command line and a GNU Makefile, you can tell CMake to do that. Or, if you’re like me and want to build using Xcode, you can tell CMake to generate an xcodeproj file. The “preferred” way to build LLVM components is to use CMake and Ninja. Ninja is a lightweight build system that was designed for speed. But approaching a complex codebase such as LLVM, it’s best to use the tools you’re most experienced with, so you’ll use Xcode at the price of a slightly slower build time.

Install CMake either via https://brew.sh or via https://cmake.org. If you have brew already setup, it’s a simple matter of the following command:

brew install cmake

In addition to CMake, you’ll need the clang compiler, which I expect you to have via Xcode 13 or higher.

After that, you’re good to go. Change directories to where you want to install LLVM:

cd code

Clone the LLVM repo:

git clone --depth 1 --branch "release/13.x" https://github.com/llvm/llvm-project.git
cd llvm-project

You are specifying a --depth of 1 as LLVM is a huge repository. This --depth flag limits the downloaded commit history and speeds up cloning (if your internet doesn’t suck, you can skip the --depth 1 option). You are checking out the release/13.x branch, which will likely be the current LLVM release by the time you’re reading this.

Once cloned, create a build directory, cd into that build directory and run CMake:

mkdir build-lldb-xcode
cd build-lldb-xcode
cmake -G Xcode -DLLVM_ENABLE_PROJECTS="clang;lldb;libcxx" -DLLVM_BUILD_RUNTIME=Off -DLLVM_INCLUDE_TESTS=Off  -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_INCLUDE_EXAMPLES=Off ../llvm

The CMake command is instructed to build for an Xcode project via the -G Xcode argument. Any argument beginning with a -D is an argument that is passed into the build system. CMake uses the arguments and a file called CMakeLists.txt to decide what to do. With the arguments above you are telling the build system you want to build clang, LLDB, and the C++ standard library (libcxx). LLDB relies on clang, which in turn, relies on the libcxx library.

On my computer, the CMake command took 4 minutes to run. Once complete, an LLVM.xcodeproj is generated in the build-lldb directory.

If you were to list all of Xcode build targets, the output might set off alarm bells.

xcodebuild -list | wc -l
1281

The LLVM.xcodeproj is a HUUUUUGE project that will incorporate all the executables for lldb, clang, and C++ (determined by the LLVM_ENABLE_PROJECTS argument passed to CMake earlier).

Now that you’ve got the infastructure setup, compile lldb:

xcodebuild -scheme lldb

Running the above command on my 2020 M1 Macbook Air took about 25 minutes; your mileage may vary depending on CPU count and RAM. The compiled artifacts created by building LLDB took ~24GB of space. Again, your disk usage will vary depending on the build type, what commit you’re building from and your elected CMake options.

If everything went ok, then a new lldb executable will be located in ./lldb-build/Debug/bin/lldb.

Building LLDB via Ninja

If you’ve come from a more C++ background, or if you despise Xcode, you can also build the LLDB project using Ninja. This is ideal if you prefer using Visual Studio Code or CLion as an IDE.

The process is largely the same as above. Ensure that CMake installed and clone the relevant version/commit of LLVM. Install Ninja if needed:

brew install ninja

Create a build directory specific for the Ninja build and use CMake to generate the Ninja project.

mkdir build-lldb-ninja
cd build-lldb-ninja
cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang;lldb;libcxx" -DLLVM_BUILD_RUNTIME=Off -DLLVM_INCLUDE_TESTS=Off  -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_INCLUDE_EXAMPLES=Off ../llvm

You can use ninja -t targets to display all the items that can be built.

Build the lldb executable with the following command:

ninja lldb

Note: A strong knowledge of CMake and its CMakeLists.txt structure is needed when making major changes to LLDB’s source code. CMake is outside the scope of this book, but interested readers are advised to Google and practice some toy examples before tackling the complex intricacies of LLVM’s CMake structure.

LLDB’s Design

LLDB is broken down into several components in order to create a functional debugger. First, there’s the lldb executable, which is responsible for coordinating everything.

Accompanying the lldb executable is a “special” framework. If you’ve ever tried executing a command in lldb via a (lldb) po something_here, there’s hidden complexities that might need to occur. A compiled language like Swift needs to be compiled. As a result, LLDB needs a compiler to dynamically parse your expression, compile it on the fly, then return the results to lldb’s output. To achieve this, LLDB uses a “watered down” clang compiler inside of a framework. If you have Xcode installed, this framework can be found at Xcode.app/Contents/SharedFrameworks/LLDB.framework.

This framework provides 3rd party consumer APIs discussed in Section IV of this book. These APIs are known as the Script Bridging code, a series of powerful C++ APIs that have remained relatively stable over the years. Since not everyone is a fan of C++, LLDB uses SWIG, a language translator, to produce Python wrappers for the C++ Script Bridging APIs. Much like a Swift Playground, using the Python Script Bridging wrappers is the preferred way to quickly play with live programs to learn and produce advanced commands in LLDB. This book primarily focuses on these APIs through the Python interface, but you can always drop down to C++ if execution speed is critical.

Finally, a “controlling process” called debugserver (or sometimes referred to as lldb-server) is responsible for manipulating a process being debugged. debugserver acts as a mindless, but powerful minion, and lldb acts as the brains, communicating its wishes to debugserver over Unix ports.

LLDB’s Design + Remote Apple Devices

The above description is fine when you’re debugging a process on your own computer, but what about on a remote host? What about debugging an iOS application running on an iPhone or iPad?

To achieve this, a DeveloperDiskImage.dmg mount (located at Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/) is loaded onto the iOS device. This DMG includes a debugserver as well as multiple debugging executables and frameworks. But even with a lldb on your machine, and debugserver on the remote iOS device, there still needs to be a way to start the communication bridge between the 2 devices. This is where /usr/libexec/lockdown (found on the iOS device) comes into play. This daemon is the gatekeeper for communicating with software requests over an iOS device’s lightning/USB cable connection. Tools like Xcode initiate a request to a device for a particular “service”. If lockdownd accepts the request, it will perform the required setup. For the case of debugging, lockdownd will forward the port to debugserver and set up the bridge between lldb on the computer and debugserver on the remote host.

Xcode uses a private framework called MobileDevice to facilitate the communication between Xcode and lockdownd. This framework has been thoroughly reversed by many and interested readers should check out the following URLs:

• https://github.com/ios-control/ios-deploy, Install and debug iOS apps from the command line.
• https://libimobiledevice.org, complete reimplementation of the MobileDevice framework.
• https://github.com/search?q=AMDeviceSecureStartService&type=code, search Github for the usage of a common MobileDevice “connection” symbol reference.

Now that you’ve got an overview of LLDB the project, and how lldb interacts with other components, it’s time to see how you can get help when you need it.

If you’ve come from a more C++ background, or if you despise Xcode, you can also build the LLDB project using Ninja. This is ideal if you prefer using Visual Studio Code or CLion as your IDE.

Just like any respectable developer tool, lldb ships with a healthy amount of documentation. Knowing how to navigate through this documentation — including some of the more obscure command flags — is essential to mastering LLDB.

Open a Terminal window and type lldb. The lldb prompt will appear. From there, type the help command:

(lldb) help

This will dump out all available commands, including the custom commands loaded from your ~/.lldbinit — but more on that later.

Debugger commands:
  apropos           -- List debugger commands related to a word or subject.
  breakpoint        -- Commands for operating on breakpoints (see 'help b' for
                       shorthand.)
  command           -- Commands for managing custom LLDB commands.
  disassemble       -- Disassemble specified instructions in the current
                       target.  Defaults to the current function for the
                       current thread and stack frame.
  expression        -- Evaluate an expression on the current thread.  Displays
                       any returned value with LLDB's default formatting.
  frame             -- Commands for selecting and examining the current thread's
                       stack frames.
  gdb-remote        -- Connect to a process via remote GDB server.
                       If no host is specified, localhost is assumed.
                       gdb-remote is an abbreviation for 'process connect
                       --plugin gdb-remote connect://<hostname>:<port>'
  gui               -- Switch into the curses based GUI mode.
  help              -- Show a list of all debugger commands, or give details
                       about a specific command.
  kdp-remote        -- Connect to a process via remote KDP server.
                       If no UDP port is specified, port 41139 is
                       assumed.
                       kdp-remote is an abbreviation for 'process connect
                       --plugin kdp-remote udp://<hostname>:<port>'
  language          -- Commands specific to a source language.
...

There’s quite a few commands one can use with LLDB. Unfortunately, help doesn’t have a pagination switch, so you have to use the scroll bars in your terminal window to read long entries.

Commands can also contain subcommands, which in turn can have sub-subcommands containing their own documentation. I told you it was a healthy amount of documentation!

Take the breakpoint command for instance. Run the documentation for breakpoint by typing the following:

(lldb) help breakpoint

You’ll see the following output:

Commands for operating on breakpoints (see 'help b' for shorthand.)

Syntax: breakpoint <subcommand> [<command-options>]

The following subcommands are supported:

      clear   -- Delete or disable breakpoints matching the specified source file and line.
      command -- Commands for adding, removing and listing LLDB commands executed when a breakpoint is hit.
      delete  -- Delete the specified breakpoint(s).  If no breakpoints are specified, delete them all.
      disable -- Disable the specified breakpoint(s) without deleting them.  If none are specified, disable all breakpoints.
      enable  -- Enable the specified disabled breakpoint(s). If no breakpoints are specified, enable all of them.
      list    -- List some or all breakpoints at configurable levels of detail.
      modify  -- Modify the options on a breakpoint or set of breakpoints in the executable.  If no breakpoint is specified, acts on the last created breakpoint.  With the
                 exception of -e, -d and -i, passing an empty argument clears the modification.
      name    -- Commands to manage name tags for breakpoints
      read    -- Read and set the breakpoints previously saved to a file with "breakpoint write".
      set     -- Sets a breakpoint or set of breakpoints in the executable.
      write   -- Write the breakpoints listed to a file that can be read in with "breakpoint read".  If given no arguments, writes all breakpoints.

For more help on any particular subcommand, type 'help <command> <subcommand>'.

From there, you can see several supported subcommands. Look up the documentation for breakpoint name by typing the following:

(lldb) help breakpoint name

You’ll see the following output:

Commands to manage name tags for breakpoints

Syntax: breakpoint name <subcommand> [<command-options>]

The following subcommands are supported:

      add       -- Add a name to the breakpoints provided.
      configure -- Configure the options for the breakpoint name provided.  If you provide a breakpoint id, the options will be copied from the breakpoint, otherwise only the
                   options specified will be set on the name.
      delete    -- Delete a name from the breakpoints provided.
      list      -- List either the names for a breakpoint or info about a given name.  With no arguments, lists all names

For more help on any particular subcommand, type 'help <command> <subcommand>'.

If you don’t understand breakpoint name at the moment, don’t worry — you’ll become familiar with breakpoints and all of the subsequent commands soon. For now, the help command is the most important command you can remember.

The apropos Command

Sometimes you don’t know the name of the command you’re searching for, but you know a certain word or phrase that might point you in the right direction. The apropos command can do this for you; it’s a bit like using a search engine to find something on the web.

apropos will do a case-insensitive search for any word or string against the LLDB documentation and return any matching results. For example, try searching for anything pertaining to Swift:

(lldb) apropos swift

You’ll see the following output:

The following commands may relate to 'swift':
  swift    -- A set of commands for operating on the Swift Language Runtime.
  demangle -- Demangle a Swift mangled name
  refcount -- Inspect the reference count data for a Swift object

The following settings variables may relate to 'swift':


  target.swift-extra-clang-flags -- Additional -Xcc flags to be passed to the
                                    Swift ClangImporter.
  target.swift-framework-search-paths -- List of directories to be searched
                                         when locating frameworks for Swift.
  target.swift-module-search-paths -- List of directories to be searched when
                                      locating modules for Swift.
  target.use-all-compiler-flags -- Try to use compiler flags for all modules
                                   when setting up the Swift expression parser,
                                   not just the main executable.
  target.experimental.swift-create-module-contexts-in-parallel -- Create the per-module Swift AST contexts in parallel.
  symbols.swift-module-loading-mode -- The module loading mode to use when
                                       loading modules for Swift.
  symbols.use-swift-clangimporter -- Reconstruct Clang module dependencies from
                                     headers when debugging Swift code
  symbols.use-swift-dwarfimporter -- Reconstruct Clang module dependencies from
                                     DWARF when debugging Swift code
  symbols.use-swift-typeref-typesystem -- Prefer Swift Remote Mirrors over
                                          Remote AST

This dumped everything that might pertain to the word Swift: first the commands, and then the settings which can be used to control how lldb operates.

You can also use apropos to search for a particular sentence. For example, if you were searching for something that can help with reference counting, you might try the following:

(lldb) apropos "reference count"
The following commands may relate to 'reference count':
  refcount -- Inspect the reference count data for a Swift object

Notice the quotes surrounding the words "reference count". apropos will only accept one argument to search for, so the quotes are necessary to treat the input as a single argument.

Isn’t that neat? apropos is a handy tool for querying. It’s not quite as sophisticated as modern internet search engines, but with some playing around, you can usually find what you’re looking for.

Other Helpful Resources

In addition to the lldb tool itself, there’s a plethora of helpful resources sprinkled around the web:

• Stack Overflow’s LLDB tag, including snooping responses from core LLDB developers, like Jim Ingham
• https://www.reddit.com/r/llvm
• LLDB’s source docs
• LLDB’s archives

It’s easy to forget the onslaught of LLDB commands that will soon come, but try to commit these two commands, help and apropos, to heart. They’re the foundation for querying information on commands and you’ll be using them all the time as you master debugging.