12. Assembly & Memory

Written by Derek Selander

You’ve begun the journey and learned the dark arts of the x64 calling convention in the previous chapter. When a function is called, you now know how parameters are passed to functions, and how function return values come back. What you haven’t learned yet is how code is executed when it’s loaded into memory.

In this chapter, you’ll explore how a program executes. You’ll look at a special register used to tell the processor where it should read the next instruction from, as well as how different sizes and groupings of memory can produce very different results.

Setting up the Intel-Flavored Assembly Experience™

As mentioned in the previous chapter, there are two main ways to display assembly. One type, AT&T assembly, is the default assembly set for LLDB. This flavor has the following format:

opcode  source  destination

Take a look at a concrete example:

movq  $0x78, %rax

This will move the hexadecimal value 0x78 into the RAX register. Although this assembly flavor is nice for some, you’ll use the Intel flavor instead from here on out.

Why opt for Intel over AT&T? The answer can be best explained by this simple tweet…

Note: In all seriousness, the choice of assembly flavor is somewhat of a flame war — check out this discussion in StackOverflow: https://stackoverflow.com/questions/972602/att-vs-intel-syntax-and-limitations.

Using Intel was based on the admittedly loose consensus that Intel is better for reading, but at times, worse for writing. Since you’re learning about debugging, the majority of time you’ll be reading assembly as opposed to writing it.

Add the following lines to the bottom of your ~/.lldbinit file:

settings set target.x86-disassembly-flavor intel
settings set target.skip-prologue false

The first line tells LLDB to display x86 assembly (both 32-bit and 64-bit) in the Intel flavor.

The second line tells LLDB to not skip the function prologue. You came across this earlier in this book, and from now on it’s prudent to not skip the prologue since you’ll be inspecting assembly right from the first instruction in a function.

Note: When editing your ~/.lldbinit file, make sure you don’t use a program like TextEdit for this, as it will add unnecessary characters into the file that could result in LLDB not correctly parsing the file. An easy (although dangerous) way to add this is through a Terminal command like so: echo "settings set target.x86-disassembly-flavor intel" >> ~/.lldbinit.

Make sure you have two ‘>>’ in there or else you’ll overwrite all your previous content in your ~/.lldbinit file. If you’re not comfortable with the Terminal, editors like nano (which you’ve used earlier) are your best bet.

The Intel flavor will swap the source and destination values, remove the ‘%’ and ‘$’ characters as well as do many, many other changes. Since you’re not using the AT&T syntax, it’s better to not explain the full differences between the two assembly flavors, and instead just learn the Intel format.

Take a look at the previous example, now shown in the Intel flavor and see how much cleaner it looks:

mov  rax, 0x78

Again, this will move the hexadecimal value 0x78 into the RAX register.

Compared to the AT&T flavor shown earlier, the Intel flavor swaps the source and destination operands. The destination operand now precedes the source operand. When working with assembly, it’s important that you always identify the correct flavor, since a different action could occur if you’re not clear which flavor you’re working with.

From here on out, the Intel flavor will be the path forward. If you ever see a numeric hexadecimal constant that begins with a $ character, or a register that begins with %, know that you’re in the wrong assembly flavor and should change it using the process described above.

Creating the cpx command

First of all, you’re going to create your own LLDB command to help later on.

Open ~/.lldbinit again in your favorite text editor (vim, right?). Then add the following to the bottom of the file:

command alias -H "Print value in ObjC context in hexadecimal" -h "Print in hex" -- cpx expression -f x -l objc -- 

This command, cpx, is a convenience command you can use to print out something in hexadecimal format, using the Objective-C context. This will be useful when printing out register contents.

Remember, registers aren’t available in the Swift context, so you need to use the Objective-C context instead.

Now you have the tools needed to explore memory in this chapter through an assembly point of view!

Bits, bytes, and other terminology

Before you begin exploring memory, you need to be aware of some vocabulary about how memory is grouped. A value that can contain either a 1 or a 0 is known as a bit. You can say there are 64 bits per address in a 64-bit architecture. Simple enough.

When there are 8 bits grouped together, they’re known as a byte. How many unique values can a byte hold? You can determine that by calculating 2^8 which will be 256 values, starting from 0 and going to 255.

Lots of information is expressed in bytes. For example, the C sizeof() function returns the size of the object in bytes.

If you are familiar with ASCII character encoding, you’ll recall all ASCII characters can be held in a single byte.

It’s time to take a look at this terminology in action and learn some tricks along the way.

Open up the Registers macOS application, which you’ll find in the resources folder for this chapter. Next, build and run the app. Once it’s running, pause the program and bring up the LLDB console. As mentioned previously, this will result in the non-Swift debugging context being used.

(lldb) p sizeof('A')

This will print out the number of bytes required to make up the A character:

(unsigned long) $0 = 1

Next, type the following:

(lldb) p/t 'A'

You’ll get the following output:

(char) $1 = 0b01000001

This is the binary representation for the character A in ASCII.

Another more common way to display a byte of information is using hexadecimal values. Two hexadecimal digits are required to represent a byte of information in hexadecimal.

Print out the hexadecimal representation of A:

(lldb) p/x 'A'

You’ll get the following output:

(char) $2 = 0x41

Hexadecimal is great for viewing memory because a single hexadecimal digit represents exactly 4 bits. So if you have 2 hexadecimal digits, you have 1 byte. If you have 8 hexadecimal digits, you have 4 bytes. And so on.

Here are a few more terms for you that you’ll find useful in the chapters to come:

• Nybble: 4 bits, a single value in hexadecimal
• Half word: 16 bits, or 2 bytes
• Word: 32 bits, or 4 bytes
• Double word or Giant word: 64 bits or 8 bytes.

With this terminology, you’re all set to explore the different memory chunks.

The RIP register

Ah, the exact register to put on your gravestone.

When a program executes, code to be executed is loaded into memory. The location of which code to execute next in the program is determined by one magically important register: the RIP or instruction pointer register.

You’ll now take a look at this register in action. Open the Registers application again and navigate to the AppDelegate.swift file. Modify the file so it contains the following code:

@NSApplicationMain
class AppDelegate: NSObject, NSApplicationDelegate {

  func applicationWillBecomeActive(
    _ notification: Notification) {
      print("\(#function)")
      self.aBadMethod()
  }

  func aBadMethod() {
    print("\(#function)")
  }
  
  func aGoodMethod() {
    print("\(#function)")
  }
}

Build and run the application. Unsuprisingly, the method name will get spat out in applicationWillBecomeActive(_:) to the debug console, followed by the aBadMethod output. There will be no execution of aGoodMethod.

Create a breakpoint at the very begining of the aBadMethod using the Xcode GUI:

Build and run again. Once the breakpoint is hit at the beginning of the aBadMethod, navigate to Debug ▸ Debug Workflow ▸ Always Show Disassembly in Xcode. You’ll now see the actual assembly of the program!

Next, type the following into the LLDB console:

(lldb) cpx $rip

This prints out the instruction pointer register using the cpx command you created earlier.

You’ll notice the output LLDB spits out will match the address highlighted by the green line in Xcode:

(unsigned long) $1 = 0x0000000100007c20

It’s worth noting your address could be different than the above output, but the address of the green line and the RIP console output will match. Now, enter the following command in LLDB:

(lldb) image lookup -vrn ^Registers.*aGoodMethod

This is the tried-and-true image lookup command with the typical regular expression arguments plus an added argument, -v, which dumps the verbose output.

You’ll get a fair bit of content. Search for the content immediately following range = [; Command + F will prove useful here. It’s the first value in the range brackets that you’re looking for.

This address is known as the load address. This is the actual physical address of this function in memory.

This differs from the usual output you’ve seen in the image lookup command, in that it only displays the offset of the function relative to the executable, also known as the implementation offset. When hunting for a function’s address, it’s important to differentiate the load address from the implementation offset in an executable, as it will differ.

Copy this new address at the beginning of the range brackets. For this particular example, the load address of aGoodMethod is located at 0x0000000100003a10. Now, write this address which points the beginning of the aGoodMethod method to the RIP register.

(lldb) register write rip 0x0000000100003a10

Click continue using the Xcode debug button. It’s important you do this instead of typing continue in LLDB, as there is a bug that will trip you up when modifying the RIP register and continuing in the console.

After pressing the Xcode continue button, you’ll see that aBadMethod() is not executed and aGoodMethod() is executed instead. Verify this by viewing the output in the console log.

Note: Modifying the RIP register is actually a bit dangerous. You need to make sure the registers holding data for a previous value in the RIP register do not get applied to a new function which would make an incorrect assumption with the registers. Since aGoodMethod and aBadMethod are very similar in functionality, you’ve stopped at the beginning, and as no optimizations were applied to the Registers application, this is not a worry.

Registers and breaking up the bits

As mentioned in the previous chapter, x64 has 16 general purpose registers: RDI, RSI, RAX, RDX, RBP, RSP, RCX, RDX, R8, R9, R10, R11, R12, R13, R14 and R15.

In order to maintain compatibility with previous architectures, such as i386’s 32-bit architecture, registers can be broken up into their 32, 16, or 8-bit values.

For registers that have had a history across different architectures, the frontmost character in the name given to the register determines the size of the register. For example, the RIP register starts with R, which signifies 64 bits. If you wanted the 32 bit equivalent of the RIP register, you’d swap out the R character with an E, to get the EIP register.

Why is this useful? When working with registers, sometimes the value passed into a register does not need to use all 64 bits. For example, consider the Boolean data type.

All you really need is a 1 or a 0 to indicate true or false, right? Based upon the languages features and constraints, the compiler knows this and will sometimes only write information to certain parts of a register.

Let’s see this in action.

Remove all breakpoints in the Registers project. Build and run the project. Now, pause the program out of the blue.

Once stopped, type the following:

(lldb) register write rdx 0x0123456789ABCDEF

This writes a value to the RDX register.

Let’s halt for a minute. A word of warning: You should be aware that writing to registers could cause your program to tank, especially if the register you write to is expected to have a certain type of data. But you’re doing this in the name of science, so don’t worry if your program does crash!

Confirm that this value has been successfully written to the RDX register:

(lldb) p/x $rdx 

Since this is a 64-bit program, you’ll get a double word, i.e. 64 bits, or 8 bytes, or 16 hexadecimal digits.

Now, try printing out the EDX register:

(lldb) p/x $edx 

The EDX register is the least-significant half of the RDX register. So you’ll only see the least-significant half of the double word, i.e., a word. You should see the following:

0x89abcdef

Next, type the following:

(lldb) p/x $dx

This will print out the DX register, which is the least-significant half of the EDX register. It is therefore a half word. You should see the following:

0xcdef

Next, type the following:

(lldb) p/x $dl

This prints out the DL register, which is the least-significant half of the DX register — a byte this time. You should see the following:

0xef

Finally, type the following:

(lldb) p/x $dh  

This gives you the most significant half of the DX register, i.e. the other half to that given by DL. It should come as no surprise that the L in DL stands for “low” and the H in DH stands for “high”.

Keep an eye out for registers with different sizes when exploring assembly. The size of the registers can give clues about the values contained within. For example, you can easily hunt down functions that return Booleans by looking for registers having the L suffix, since a Boolean needs only a single bit to be used.

Registers R8 to R15

Since the R8 to R15 family of registers were created only for 64-bit architectures, they use a completely different format for signifying their smaller counterparts.

Now you’ll explore R9’s different sizing options. Build and run the Registers application, and pause the debugger. Like before, write the same hex value to the R9 register:

(lldb) register write $r9 0x0123456789abcdef

Confirm that you’ve set the R9 register by typing the following:

(lldb) p/x $r9

Next type the following:

(lldb) p/x $r9d

This will print the lower 32 bits of the R9 register. Note how it’s different than how you specified the lower 32 bits for RDX (that is, EDX, if you’ve forgotten already).

Next, type the following:

(lldb) p/x $r9w

This time you get the lower 16 bits of R9. Again, this is different than how you did this for RDX.

Finally, type the following:

(lldb) p/x $r9l

This prints out the lower 8 bits of R9.

Although this seems a bit tedious, you’re building up the skills to read an onslaught of assembly.

Breaking down the memory

Now that you’ve taken a look at the instruction pointer, it’s time to further explore the memory behind it.

As its name suggests, the instruction pointer is actually a pointer. It’s not executing the instructions stored in the RIP register — it’s executing the instructions pointed to in the RIP register.

Seeing this in LLDB will perhaps describe it better. Back in the Registers application, open AppDelegate.swift and once again set a breakpoint on aBadMethod. Build and run the app.

Once the breakpoint is hit and the program is stopped, navigate back to the assembly view. If you forgot, and haven’t created a keyboard shortcut for it, it’s found under Debug ▸ Debug Workflow ▸ Always Show Disassembly.

You’ll be greeted by the onslaught of opcodes and registers. Take a look at the location of the RIP register, which should be pointing to the very beginning of the function.

For this particular build, the beginning address of aBadMethod begins as 0x100007c20. As usual, your address will likely be different.

In the LLDB console, type the following:

(lldb) cpx $rip

As you know by now, this prints out the contents of the instruction pointer register.

As expected, you’ll get the address of the start of aBadMethod. But again, the RIP register points to a value in memory. What is it pointing to?

Well… you could dust off your mad C coding skillz (you remember those, right?) and dereference the pointer, but there’s a much more elegant way to go about it using LLDB.

Type the following, replacing the address with the address of your aBadMethod function:

(lldb) memory read -fi -c1 0x100007c20

Wow, what the heck does that command do?!

memory read takes a value and reads the contents pointed at by the memory address you supply. The -f command is a formatting argument; in this case, it’s the assembly instruction format. Finally you’re saying you only want one assembly instruction to be printed out with the count, or -c argument.

You’ll get output that looks similar to this:

->  0x100007c20:  55  push   rbp

This here is some gooooooooood output. It’s telling you the assembly instruction, as well as the opcode, provided in hexadecimal (0x55) that is responsible for the pushq rbp operation.

Look at that “55” there in the output some more. This is an encoding of the entire instruction, i.e. the whole pushq rbp. Don’t believe me? You can verify it. Type the following into LLDB:

(lldb) expression -f i -l objc -- 0x55

This effectively asks LLDB to decode 0x55. You’ll get the following output:

(int) $0 = 55  push   rbp

That command is a little long, but it’s because you need the required switch to Objective-C context if you are in the Swift debugging context. However, if you move to the Objective-C debugging context, you can use a convenience expression that is a lot shorter.

Try clicking on a different frame in the left panel of Xcode to get into an Objective-C context which doesn’t contain Swift or Objective-C/Swift bridging code.

Click on any frame which is in an Objective-C function.

Next, type the following into the LLDB console:

(lldb) p/i 0x55

Much better, right?

Now, back to the application in hand. Type the following into LLDB, replacing the address once again with your aBadMethod function address:

(lldb) memory read -fi -c4 0x100007c20

You’ll get 10x the output! That’s something worthy to put on that LinkedIn résumé…

By the way, there’s a shorthand convenience way to execute the above command. You can simply type the following to achieve the same result.

(lldb) x/4i 0x100007c20

With either command you choose, you’ll get something similar to following output:

0x100007c20: 55                      push rbp
0x100007c21: 48 89 e5                mov  rbp, rsp
0x100007c24: 41 55                   push r13
0x100007c26: 48 81 ec a8 00 00 00    sub  rsp, 0xa8

There’s something interesting to note here: assembly instructions can have variable lengths. Take a look at the first instruction, versus the rest of the instructions in the output. The first instruction is 1 byte long, represented by 0x55. The following instruction is 3 bytes long.

Make sure you are still in an Objective-C context, and try to print out the opcode responsible for this instruction. It’s just 3 bytes, so all you have to do is join them together, right?

(lldb) p/i 0x4889e5

You’ll get a different instruction completely unrelated to the mov %rsp, %rbp instruction! You’ll see this:

e5 89  inl    $0x89, %eax

What gives? Perhaps now would be a good time to talk about endianness.

Endianness… this stuff is reversed?

The x64 as well as the ARM family architecture devices all use little-endian, which means that data is stored in memory with the least significant byte first. If you were to store the number 0xabcd in memory, the 0xcd byte would be stored first, followed by the 0xab byte.

Back to the instruction example, this means that the instruction 0x4889e5 will be stored in memory as 0xe5, followed by 0x89, followed by 0x48.

Jumping back to that mov instruction you encountered earlier, try reversing the bytes that used to make up the assembly instruction. Type the following into LLDB:

(lldb) p/i 0xe58948

You’ll now get your expected assembly instruction:

(Int) $R1 = 48 89 e5  mov    rbp, rsp

Let’s see some more examples of little-endian in action. Type the following into LLDB:

(lldb) memory read -s1 -c20 -fx 0x100003840

This command reads the memory at address 0x100003840. It reads in size chunks of 1 byte thanks to the -s1 option, and a count of 20 thanks to the -c20 option.

You’ll see something like this:

0x100003840: 0x55 0x48 0x89 0xe5 0x48 0x83 0xec 0x60
0x100003848: 0xb8 0x01 0x00 0x00 0x00 0x89 0xc1 0x48
0x100003850: 0x89 0x7d 0xf8 0x48

Now, double the size and half the count like so:

(lldb) memory read -s2 -c10 -fx 0x100003840

You will see something like this:

0x100003840: 0x4855 0xe589 0x8348 0x60ec 0x01b8 0x0000 0x8900 0x48c1
0x100003850: 0x7d89 0x48f8

Notice how when the memory values are grouped together, they are reversed thanks to being in little-endian.

Now double the size and half the count again:

(lldb) memory read -s4 -c5 -fx 0x100003840

And now you’ll get something like this:

0x100003840: 0xe5894855 0x60ec8348 0x000001b8 0x48c18900
0x100003850: 0x48f87d89

Once again the values are reversed compared to the previous output.

This is very important to remember and also a source of confusion when exploring memory. Not only will the size of memory give you a potentially incorrect answer, but also the order. Remember this when you start yelling at your computer when you’re trying to figure out how something should work!

Where to go from here?

Good job getting through this one. Memory layout can be a confusing topic. Try exploring memory on other devices to make sure you have a solid understanding of the little-endian architecture and how assembly is grouped together.

In the next chapter, you’ll explore the stack frame and how a function gets called.