Linking Lab
Due: Wednesday, October 19, 11:59pm
This assignment is about understanding how the ELF format supports the run-time linking of shared libraries, including the way that references to global variables are implemented and represented. The assignment also requires you to implement a small amount of disassembly of machine code.
Please note that the output required from your program is specified in Output Format. We provide tests and a test harness as described in Test Cases and Test Harness for three levels of completion. Your work is not a solution is if doesn’t match the given output specification, whether that’s because it prints extra spaces or extra blank lines or extra names or extra debugging output.
Functions and Variables a Shared Library
A shared library can import and export both functions and global variables. The functions can read and write to global variables, whether or not the shared library defines the variables.
For example, this .c program could be compiled to a shared library:
int a; |
extern int b; |
|
int do_something(int v) { |
b = a; |
a = v; |
return b; |
} |
The shared library provides a do_something function and a variable named a, and it uses a variable b to be supplied by another shared library or the main executable.
If a programmer has the above .c source, then it’s obvious that do_something uses the global variables a and b. If a programmer is given only the code as a compiled shared library, then that information is not nearly as apparent. It can be extracted only by a careful reading of the machine code and an understanding of how shared libraries are linked for a running program. The objdump program decodes the relevant information, but mixed among much other information.
We’d like to have an inspect tool that takes a shared library and reports which global variables are used by each provided function. Your job is to create an initial prototype of that tool. In principle, determining the variables that a function actually uses could require solving the halting problem. Many approximations are useful, however, and your task will involve a particularly simple approximation.
Inspecting a Shared Library
Suppose that the C code with do_something above is in "demo.c" and compiled with
$ gcc -O2 -fPIC -c demo.c |
$ gcc -shared -o demo.so demo.o |
Then, running your inspect prototype as
$ ./inspect demo.so |
should print out
do_something |
a |
b |
This output shows that a function call do_something is provided by the library, and it refers to global variables a and b.
If "demo.c" is instead
int a; |
extern int b; |
|
int do_something(int v) { |
b = a; |
a = v; |
return b; |
} |
|
int do_something_else(int v) { |
return v; |
} |
then, running your inspect prototype should print out
do_something |
a |
b |
do_something_else |
because do_something_else doesn’t use global variables. Finally, because a function might not itself use variables but might call another function that does, in the case of
int a; |
extern int b; |
|
int do_something(int v) { |
b = a; |
a = v; |
return b; |
} |
|
int do_something_else(int v) { |
return v; |
} |
|
int do_the_third_thing(int v) { |
return do_something(v); |
} |
then your inspect program will print
do_something |
a |
b |
do_something_else |
do_the_third_thing |
(do_something) |
to expose the fact that do_the_third_thing calls do_something (which, in turn, uses a and b).
To determine function and variable information, your program will read
an ELF shared-object file directly. As usual, your program should be
written in ANSI standard C using only standard libraries and
headers—
In fact, you should start with inspect.c, which maps a given shared-object file into memory, so you can traverse ELF information by following pointers in memory. The starting code demonstrates using the in-memory image to check fields of an ELF file that identify the file type.
Levels of Completion
For a basic check grade (80%), complete the assignment to Level 1: simply print all functions that are exported from a shared library, with no information variables that are used or functions that are called.
For a check+ grade (100%), complete the assignment to Level 2: print only information about variables that are used in an exported function before that function performs any jumps (i.e., treat jumps as returns).
For a check++ grade (110%), complete the assignment fully: print all information as described above, which involves following jumps to determine what kind of function is called and (if it’s not an exported function) the variables that function uses.
When grading, we will infer an intended level of completion based on your program’s output as compared to expected output for the three levels.
Prototype Assumptions
To simplify the problem for this prototype, you can make several assumptions:
Your program should only report information about functions that are registered in the dynamic-symbol table (i.e., the .dynsym section) as a function type with a non-zero size.
You must handle a function only to the degree that its implementation uses a constrained set of x86-64 instructions: mov, movs, ret, and jmp. The specific instruction constraints are described in x86-64 Instruction Constraints.
You can assume the usual strategies for implementing relocatable variables and functions in Linux. In particular, accessing a variable will go through a memory location that is listed in the .rela.dyn section, and jumping to an imported or exported function will go through the global offset table as described by a rela.plt section.
For example, if do_something is implemented as
int do_something(int v) { |
b = a + v; |
a = v; |
return b; |
} |
then your program does not need to report a use of b or a, because the implementation uses an addition instruction.
Output Format
Your program must print the name of each function provided by a shared library, and each must be on its own line with no preceding or following spaces.
The function names can be in any order, but each function must be reported exactly once.
For Level 2 and full completion: The name of each variable used by the function (within the constraints of Prototype Assumptions) must be printed on its own line, preceded by exactly two spaces and followed by no additional spaces. The variable names must appear after the name of the function that uses them and before the next function name.
The variables used by a function can be reported in any order, and multiple lines for the same variable name are allowed.
For full completion: The name of an imported or exported function called by an exported function (within the constraints of Prototype Assumptions) must be printed on its own line, preceded by exactly two spaces, surrounded by parentheses, and followed by no additional spaces. Each parenthesized function names must appear after the name of the function that calls it and before the next function name.
Since a function call is recognized only as a jump, calling a function is always the last action of the enclosing function, so at most one called function will be reported for each exported function. The called function can be reported before, after, or among the names of used variables for the enclosing function.
No additional lines, blank or not, can appear in the output.
x86-64 Instruction Constraints
Detecting variable uses will require not only finding functions that are listed in the .dynsym section, but disassembling the function implementation. Disassembling x86-64 is no fun, so your prototype need only handle the following instruction patterns (expressed as byte sequences):
0xc3 —
The 0xc3 opcode represents a ret instruction. A ret instruction terminates a function, so stop checking a function when the 0xc3 opcode is found. 0xe9 followed by four bytes —
The 0xe9 opcode represents a jmp instruction. It is followed by four bytes that represent an int. The int is not the target of the jump; rather, it is a relative displacement for the jump. That is, the target of the jump is the address of the next instruction plus the signed int. This instruction pattern might indicate a call (as the last action of an enclosing function) to another function that is imported or exported for the shared object. If so, the jump will be to the .plt section, which will then start with another jump instruction using a 0xff 0x25 pattern described below; you’ll have to match up the computed address there with one listed in .rela.plt to be sure.
If the call is not to an address within the same section, then the jump effectively terminates the function.
0xeb followed by one byte —
The 0xeb opcode also represents a jmp instruction, but with a signed char displacement. This instruction pattern could correspond to a call to an imported or exported function, but it more likely represents a jump within a function or a call to a file-local function (still in the same section) that you should follow to continue checking for variable uses and other functions calls.
If the call is not to an address within the same section, then the jump effectively terminates the function.
The sequence 0x48 0x8b reg followed by four bytes, where the top two bits of reg are 0 and the bottom three bits of reg equal 5 —
The 0x8b opcode represents a mov instruction, and the 0x48 prefix makes it specifically a movq instruction. The combination of 0 in the top two bits of reg and 5 in the bottom three bits make it a PC-relative computation where the following four bytes provide a int value to add to the PC (i.e., to the address of the following instruction). The remaining three bits of reg specify the destination register. This instruction pattern typically indicates a variable access through the indirection that is described by .rela.dyn, but you’ll have to match up a computed address with one listed in .rela.dyn to be sure.
The sequence mov reg, where mov is 0x63, 0x89, or 0x8b, and the top two bits of reg are set —
The opcode 0x63 is a movslq instruction, while 0x89 and 0x8B are mov instructions. When the top two bits of reg are set, then the move is from one register to another. The movslq opcode doesn’t really work without a 0x48 prefix, so you’re won’t see it without. We include 0x63 with 0x89 and 0x8b in non-0x48 cases, anyway, because it’s simpler to consistently group the three mov opcodes.
The sequence 0x48 mov reg, where mov is 0x63, 0x89 or 0x8b, and the top two bits of reg are set —
Like the preceding case, but the prefix 0x48 makes the mov instruction a movq instruction. The sequence mov reg followed by four bytes, where mov is 0x63, 0x89, or 0x8b, the top two bits of reg are 0, and the bottom three bits of reg equal 5 —
Similar to the preceding mov cases, but the combination of 0 in the top two bits of reg and 5 in the bottom three bits make it a PC-relative computation where the following four bytes provide a int value to add to the PC. The sequence 0x48 mov reg followed by four bytes, where mov is 0x63, 0x89, or 0x8b, the top two bits of reg are 0, and the bottom three bits of reg equal 5 —
Like the preceding case, but the prefix 0x48 makes the mov instruction a movq instruction. This is a generalization of the sequence described further above: 0x48 0x8b reg followed by four bytes. If the opcode is 0x89 instead of 0x8b, then the mov is from a register to a memory location, so that will not correspond to a use of an exported or imported variable. A 0x63 opcode for movslq similarly will not access a variable’s address.
The sequence mov reg followed by one additional byte, where mov is 0x63, 0x89, or 0x8b, the top two bits of reg are 0, and the bottom three bits of reg equal 4 —
The combination of 0 in the top two bits of reg and 4 in the bottom three bits indicate that an additional byte is needed to encode the source or destination. The sequence 0x48 mov reg followed by one additional byte, where mov is 0x63, 0x89, or 0x8b, the top two bits of reg are 0, and the bottom three bits equal 4 —
Like the preceding case, but the prefix 0x48 makes the mov instruction a movq instruction. The sequence mov reg, where mov is 0x63, 0x89, or 0x8b, end either the top two bits of reg are not 0 or the bottom three bits of reg do not equal 4 or 5 —
A non-0 value for the top two bits of reg or a value other than 4 or 5 in the bottom three bits means that source and destination registers are fully specified in reg. The sequence 0x48 mov reg, where mov is 0x63, 0x89, or 0x8b, end either the top two bits of reg are not 0 or the bottom three bits of reg do not equal 4 or 5 —
Like the previous case, but the prefix 0x48 makes the mov instruction a movq instruction.
If your program encounters any other opcode sequence, it should probably abandon disassembling the function. We will apply your program only to functions that are compiled to fit the constraints above.
Similar to the above patterns, you’ll need to recognize exactly one machine-code pattern in the .plt section:
The sequence 0xff 0x25 followed by four bytes —
The four bytes represent a PC-relative address that holds the target of an indirect jump. (The address is the four bytes as an int plus the address of the following instruction.) That address will match up with an entry in the .rela.plt section to identify which imported or exported function is called. This one pattern appears at the beginning of the code that is the target of a jump into the .plt section. You do not need to recognize it in the same places as the other patterns.
Tips
Use readelf to get a human-readable form of the content of an ELF file to get an idea of what your program should find. Use objdump -d to disassemble functions to get an idea of what your program should recognize. Note that objdump -d prints opcodes alongside the assembly code that it prints.
All of the information that you need from the ELF file can be found via section headers and section content, so you will not need to use program headers. In particular, you’ll need to consult the .dynsym, .dynstr, .rela.dyn, .plt, and .rela.plt sections.
When working with ELF content, you have to keep track of which things are in terms of file offsets and which are in terms of (tentative) memory addresses where the library will eventually run. When working with ELF content that is mmapped into memory (as in the starting inspect.c), then you have one more way of referencing things, which is an address in memory at present. Be careful to keep in mind which kind of reference you have at any time.
Don’t confuse “symbol” with “string,” and keep in mind that they are referenced in different ways. Symbols are referenced by an index that corresponds to the symbol’s position in an array of symbols. Strings are referenced by an offset in bytes within the relevant string section. Every symbol has a string for its name.
Take small steps. Start out by printing the symbol index for every function that is provided by the shared library. Then, print the name instead of the symbol index. Then, print the address where each function’s implementation is found, and so on. Make reporting for variables work before attempting to implement reporting for called functions.
The information about ELF that you need to complete this assignment is mostly covered by Videos: ELF. You might use "/usr/include/elf.h" as a reference to find relevant structures, fields, and macros. You might also consult any number of other ELF references on the web.
As a lower bound, a complete solution can fit in about 200 lines of C code, including the 70 lines in the provided inspect.c. Depending on whitespace (fine), comments (good), error checking (commendable), and duplicated code instead abstracting into a function (bad), many solutions will be the range of 300-400 lines.
Test Cases and Test Harness
The archive linklab-handout.zip provides a "Makefile":
The default inspect target will build your program from "inspect.c".
The test target runs your inspect on a number of shared libraries and compares your program’s output to expected output for full completion.
The test-2 target runs your inspect on a number of shared libraries and compares your program’s output to expected output for Level 2 completion. Use test-2 only if you intend to complete the assignment to Level 2; it won’t work if your inspect is intended to implement the complete assignment or Level 1.
The test-1 target runs your inspect on a number of shared libraries and compares your program’s output to expected output for Level 1 completion. Use test-1 only if you intend to complete the assignment to Level 1; it won’t work if your inspect is intended to implement the complete assignment or Level 2.
For example, the test file "f_uses_a.c" gets compiled to "f_uses_a.so", and the result of inspect f_uses_a.so is written to "f_uses_a.so.out". For a complete solution, that file is compared against the provided "f_uses_a.so.expect", and differences are reported as a failure. For Level 2 completion, our program’s output is checked against "f_uses_a.so.expect-2", instead. For Level 1 completion, you program’s output is checked against "f_uses_a.so.expect-1".
By default, comparison uses diff, which checks whether your output matches the provided output exactly. The output specification from Output Format, however, allows some flexibility in the output. The "diff.rkt" script supports all of the allowed flexibility, so, if necessary, adjust the DIFF definition in "Makefile" to use "diff.rkt" instead of plain diff.
You are not required to use the test files, but for grading purposes,
we expect your program’s output to match the specification here—