Elf linking

From ProjectWiki
Jump to: navigation, search

Adventures in dynamic linking


What is this

One of the most common concerns when developing on embedded systems is memory limitations. In particular on the NDS there is 4mb of ram which is almost impossible to fit games with rich content. The solution so far is NitroFS where you can store game and application data so that ram only contains the code necessary to run the game and most of the data is loaded as needed from external files. This works great for data.

Loading animated clips, maps, or music for a new level or cut scene is just like loading and playing movies or mp3s. But what about code?

To dynamically load new (or unload old) code into running program is a bit more complicated because code segments interact with each other, Take the functions below:

void someFunc(void) {
  printf("hai2u! \\^_^\n");

In this example there is a function named 'someFunc', some data "hai2u! \^_^\n", and a reference to an external function 'printf'. When that bit of code is loaded the program and data segments may be loaded into some arbitrary unpredictable location, so how does someFunc know the address of printf?? This process is called linking because each section of code needs to be linked to the others.


First it needs some way to identify locations in memory. This is accomplished with symbols. A symbol entry minimally contains the location of the symbol (within memory or withing a file), and a textual name for the symbol. It may also contain information about the type of symbol (program, data, global data, etc), who (or which process) owns the symbol, which section of the object file the symbol came from, group it belongs to, etc.

//Typical symbol table
typedef struct _SYMBOL {
	char *name;
	char *value;
	SYM_TYPE  type;
	UINT16    group;

Many schemes exist for symbol table implementation, some use a hash table for quicker lookups, some use linked lists while others have a finite array. They all do the same thing, which is, to provide a database relating addresses to names. My implementation is based of VxWork's symLib. But anyways, what about linking?

Extended Linker Format (elf)

When the compiler runs it does not know where these sections will be in RAM, so it instead provides references to locations within the object file of data or other functions from this file, and indicates when a symbol's value is unknown and must be provided externally. The format of these outputted files is typically ELF or Extended Linker Format.

The ELF file is divided into a number of parts, first there is the elf header (Elf32_Ehdr) which defines the number and location of the section and program segments. next there are the program segments which contain prelinked absolute data, and section segments. These are both described using program (Elf32_Shdr) and section (Elf32_Phdr) headers respectively. Definitions of these structures is in elf.h which varies from build to build of binutils/gcc toolchain. Because the elf format works equally well with all CPUs, and most operating systems there are many many variants such as 32 and 64 bit, arm, mips, x86 cpu etc.

The specifics of which system a particular elf object file are for as well as which version of the toolchain was used to compile it is are found in the elf header. These should be checked for compatibility before linking.

Sections within the object file contain data of various types. Some is actual data or code to be linked, others contain information about how to link the code or data, even others describe the symbol table, etc etc.

Sections from a typical ELF file (found using objdump -h):

Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         0000003a  00000000  00000000  00000034  2**2
  1 .data         00000000  00000000  00000000  00000070  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  00000000  00000000  00000070  2**2
  3 .debug_abbrev 0000007d  00000000  00000000  00000070  2**0
  4 .debug_info   0000019f  00000000  00000000  000000ed  2**0
  5 .debug_line   0000003b  00000000  00000000  0000028c  2**0
  6 .rodata       00000018  00000000  00000000  000002c7  2**0
  7 .debug_frame  00000054  00000000  00000000  000002e0  2**2
  8 .debug_loc    00000058  00000000  00000000  00000334  2**0
  9 .debug_pubnames 0000002d  00000000  00000000  0000038c  2**0
 10 .debug_aranges 00000020  00000000  00000000  000003b9  2**0
 11 .comment      0000002a  00000000  00000000  000003d9  2**0
                  CONTENTS, READONLY
 12 .note.GNU-stack 00000000  00000000  00000000  00000403  2**0
                  CONTENTS, READONLY

For most applications you can ignore the .debug* sections, these are not normally used for linking but instead for gdb. Sections within the file which are to be linked are marked as "LOAD" (SHF_ALLOC). Sections with an accompanying relocate table are marked as "RELOC" (SHF_ALLOC?), the allocation table seems to always follow in the section immediately following it.

The definitions for headers are system and version specific, however mostly interchangeable. For instance for testing I used my /usr/include/elf.h from standard Ubuntu Linux distro and just added some ARM specific defines from another file and it worked fine. Another file of interest is elf32-arm.h (or elf32-x86.h) but more on that later... ^^


So first step in loading would be to identify all the loadable sections. The next step is to add all symbols of which the addresses are known from the file to the symbol table, such as any named addresses within the previously loaded sections.

After loading all the loadable sections and adding known symbols can do final relocation. This involves modifying the relocatable sections marked with RELOC.

Taking a look at the disassembly of the 'otherFunc' example can see (objdump -d):

  • otherFunc starts at 0x00000026 in it's section within the file
  • section .rodata + 0xd contains the string to print
  • puts has no value set (just -4)
00000026 <otherFunc>:
  26:	55                   	push   %ebp
  27:	89 e5                	mov    %esp,%ebp
  29:	83 ec 08             	sub    $0x8,%esp
  2c:	c7 04 24 0d 00 00 00 	movl   $0xd,(%esp)
			2f: R_386_32	.rodata
  33:	e8 fc ff ff ff       	call   34 <otherFunc+0xe>
			34: R_386_PC32	puts
  38:	c9                   	leave  
  39:	c3                   	ret

What needs to now happen is the ram addresses of .rodata and puts functions need to be found and the values within otherFunc replaced with the proper ones. To assist in this is the relocation table section, found immediately following the section marked RELOC (not shown by objdump command?! but its there, trust me.. :P) Entries in this table define what address within the relocatable segment need external references, which symbol should be referenced and how the data should be modified.

Relocation records for .text section (objdump -r)

OFFSET   TYPE              VALUE 
00000010 R_386_32          .rodata
00000015 R_386_PC32        puts
0000002f R_386_32          .rodata
00000034 R_386_PC32        puts

For example lets say: .rodata section was loaded into address 0x8000 .text (program code) was loaded into 0x8100

The relocation entry for .text address 0x2f specifies that at that adress there is a 32 bit reference to what was in the .rodata section.(the text "Hai2u! \^_^\n"). Since we have loaded them previously .text is now at 0x8100 so 0x2f becomes 0x8100+0x2f, and .rodata is now at 0x8000. These name/value pairs are also added to the system serial table.

The linker needs to modify 0x812f to contain the proper address for .rodata (found by looking in the running system's symbol table since it was already added during a previous phase) which is 0x8000. But if you look the original value at that locaion is 0xd! This is because this value is intended to be added with whatever symbol value is requested, so finally... the proper address to write to 0x812f would be 0x8000+0xd or 0x800d :D

For the value of 'puts', first it looks up where puts is in the system's symbol table. Since the relocation entry specifies it is R_386_PC32 this means the value is to be program counter relative (PC + value == call address). So to figure out the proper value it takes the address value of the symbol puts (for sake of demonstration lets say its at 0x70f4) minus the address value of the instruction (0x8134) and finally add on whatever original value was in the location... so: 0x70f4 - 0x8134 + 0xfffffffc (0xfffffffc == -4), or symaddr - relocaddr + original value;

Now that function is loaded in ram and linked into all the other parts of the current system it needs. The otherFunc now can be called by looking up the symbol value address 'otherFunc' and casting it as a function:

typedef void(*_someFunc)(void);

Makes perfect sense right! u_u;;;;

ARM Thumb insanity

Simple linking test. :D

Immediately one problem became clear, while it worked fine on x86 code, even after adding the required relocation handling code for R_ARM_CALL and R_ARM_ABS32 (this was same as x86 abs32 btw :D) the arm9 linking test wasn't working properly. After digging a bit realized that libnds operates mostly in 16 bit Thumb mode. For this mode R_ARM_THM_PC22 relocation handling was needed.

To get an idea how arm-eabi handles R_ARM_THM_PC22 i examined elf32-arm.h which contains the processor specific handling code and macros relating to relocation among other things. From this i immediately realized thumb being 16 bit uses two instructions for 'bl' each containing 11 bits representing the branch address and 5 bits for the branch high or branch low instrunctions. (0xf000 and 0xf800) Perfect sense right @_@)?

Below is the source code that was linked into thlink (see image):

#include <stdio.h>
//Just filler so that otherFunc is offset for better testing
int otherFunc(int argc, char **argv) {
int someFunc(void) {
        printf("Zomg! It Works! ^_^\n");


Great!! Now you've got it all loaded and ran it and now dunt need it no more, so now what?! If only there was some scheme to keep track of which object files are loaded and what needs to be done to unload them. Well, this is what modules are for, I will probably base mine on the structure of moduleLib. More on this later..

Personal tools
irssi scripts
eggdrop scripts