Section re-ordering confuses `strip`

[staticfloat]

Running patchelf to enlarge my RUNPATH entry causes my section headers to change from this:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .interp           PROGBITS         0000000000400238  00000238
       000000000000001c  0000000000000000   A       0     0     1
  [ 2] .note.ABI-tag     NOTE             0000000000400254  00000254
       0000000000000020  0000000000000000   A       0     0     4
  [ 3] .note.gnu.build-i NOTE             0000000000400274  00000274
       0000000000000024  0000000000000000   A       0     0     4
  [ 4] .gnu.hash         GNU_HASH         0000000000400298  00000298
       0000000000000064  0000000000000000   A       5     0     8
  [ 5] .dynsym           DYNSYM           0000000000400300  00000300
       00000000000011b8  0000000000000018   A       6     1     8
  [ 6] .dynstr           STRTAB           00000000004014b8  000014b8
       0000000000000868  0000000000000000   A       0     0     1
...

To this:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .dynsym           DYNSYM           00000000003ffb00  001ffb00
       00000000000011b8  0000000000000018   A       2     1     8
  [ 2] .dynstr           STRTAB           00000000003ff270  00000270
       0000000000000890  0000000000000000   A       0     0     8
  [ 3] .gnu.hash         GNU_HASH         0000000000400cb8  00001cb8
       0000000000000064  0000000000000000   A       1     0     8
  [ 4] .interp           PROGBITS         0000000000400d20  00001d20
       000000000000001c  0000000000000000   A       0     0     8
  [ 5] .note.ABI-tag     NOTE             0000000000400d40  00001d40
       0000000000000020  0000000000000000   A       0     0     8
  [ 6] .note.gnu.build-i NOTE             0000000000400d60  00001d60
       0000000000000024  0000000000000000   A       0     0     8
...

This confuses tools such as strip because they use a header offset of 0 as an error code; here's the relevant function assign_file_positions_for_non_load_sections in elf.c from binutils. Running e.g. gdb --args strip -g my_binary, and setting a breakpoint at that position shows that we get a valid section, but since its filepos is equal to zero, binutils ignores it and continues on, getting confused. Note that even readelf gets a little confused by this, as it doesn't even list the .dynsym segment:

$ readelf -l my_binary
...
 Section to Segment mapping:
  Segment Sections...
   00     
   01     .interp 
   02     .interp .note.ABI-tag .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 
   03     .init_array .fini_array .jcr .data.rel.ro .dynamic .got .got.plt .data .bss 
   04     .dynamic 
   05     .note.ABI-tag .note.gnu.build-id 
   06     .eh_frame_hdr 
   07     
   08     .init_array .fini_array .jcr .data.rel.ro .dynamic .got 
$ readelf my_patched_binary
...
 Section to Segment mapping:
  Segment Sections...
   00     
   01     
   02     .dynstr 
   03     .gnu.hash .interp .note.ABI-tag .note.gnu.build-id .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 
   04     
   05     .interp 
   06     .eh_frame_hdr 
   07     .init_array .fini_array .jcr .data.rel.ro .dynamic .got .got.plt .data .bss 
   08     .init_array .fini_array .jcr .data.rel.ro .dynamic .got 
   09     .dynamic 

I theorize that if we keep the .interp section at the beginning of the file, we might be able to still use tools such as strip, but I don't actually know. Is there an easy way to permute the order of headers that patchelf outputs? If you can just point me in the direction of how to do this, I can submit a PR.

[chenz]

ldconfig gets confused as well, but only in combination with strip:

lib.so -> strip -> patchelf -> ldconfig

/sbin/ldconfig.real: file ./libstdc++.so.6.0.19 is truncated

lib.so -> patchelf -> strip

BFD: stas0up1: Not enough room for program headers, try linking with -N
strip:stas0up1[.hash]: Bad value

lib.so -> patchelf -> ldconfig

OK

[sjackman]

I'm seeing this error as well, which makes it impossible to run strip after patchelf.

[davidedelvento]

FWIW this problem affects me too, via https://github.com/schmir/bbfreeze

[mitchblank]

Yeah, I'm seeing something similar. On CentOS 7 strip (2.23.52.0.1-30.el7_1.2) I don't get that exact error, but patchelf 0.9 followed by strip does produce a broken binary.

In the post-patchelf'ed binary, DT_STRTAB seems to end up in the first page of the executable, which includes references to things like needed libraries.

$ dd bs=4k count=1 if=after-patchelf 2>/dev/null | strings | head
__libc_csu_fini
_start
__tls_get_addr
GLIBC_2.3
ld-linux-x86-64.so.2
[...]

but after running strip on it , they're no longer in that position of the file:

$ dd bs=4k count=1 if=after-patchelf-and-strip 2>/dev/null | strings | wc -l
0

However, when I've been walking through the dynamic linker code in gdb, it seems that in _dl_check_map_versions the strtab points into this page and is all \0 bytes, breaking its ability to find the libcs:

(gdb) x/40c 0x3ff580
0x3ff580:   0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'
0x3ff588:   0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'
0x3ff590:   0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'
0x3ff598:   0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'
0x3ff5a0:   0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'    0 '\000'

I'm still digging a bit, hoping to find at least a workaround. Unfortunately I need to produce unstripped binaries to hand to our release engineering process where do the final strip before shipping, so right now trying to use patchelf to fix the rpath inside is resulting in binaries that can't be stripped.

[mitchblank]

OK, I dug into this more. The problem doesn't seem specific to the re-ordering, but that the PT_LOAD program sections that patchelf set up are...dubious. Here is some of the information from readelf -Wl on my original executable (with a little manual reformatting for clarity. I'm only including the LOAD segments since that's what controls what the kernel will actually map into our image.

  Type  Offset    VirtAddr           PhysAddr           FileSiz   MemSiz    Flg Align
  LOAD  0x0000000 0x0000000000400000 0x0000000000400000 0x1c3faa0 0x1c3faa0 R E 0x1000
  LOAD  0x1c3faa0 0x0000000002040aa0 0x0000000002040aa0 0x000bfc8 0x005b170 RW  0x1000

   02     .interp .note.ABI-tag .note.gnu.build-id .dynsym .dynstr .gnu.hash .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .gcc_except_table .stapsdt.base .eh_frame .eh_frame_hdr
   03     .dynamic .got .got.plt .data .jcr .tm_clone_table .fini_array .init_array .data.rel.ro.local .data.rel.ro .bss

The .dynstr is one of the many segments in the first LOAD section... so far so unremarkable.

Now after running patchelf 0.9 things get odd:

  Type  Offset    VirtAddr           PhysAddr           FileSiz   MemSiz    Flg Align
  LOAD  0x0000000 0x00000000003ff000 0x00000000003ff000 0x0001000 0x0001000 RW  0x1000
  LOAD  0x0001000 0x0000000000400000 0x0000000000400000 0x1c3faa0 0x1c3faa0 R E 0x1000
  LOAD  0x1c40aa0 0x0000000002040aa0 0x0000000002040aa0 0x000bfc8 0x005b170 RW  0x1000

   02     .dynamic
   04     .dynsym .interp .note.ABI-tag .note.gnu.build-id .gnu.hash .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .gcc_except_table .stapsdt.base .eh_frame .eh_frame_hdr
   08     .got .got.plt .data .jcr .tm_clone_table .fini_array .init_array .data.rel.ro.local .data.rel.ro .bss

so now there are three LOAD segments. The first rw- one covers just one page (just under address 0x400000) and then is immediately followed by a larger r-x one. But if you look at the dynstr section...

  [Nr] Name                   Type       Address            Off      Size   ES Flg Lk Inf Al
  [ 2] .dynstr                STRTAB     00000000003ff580   000580   2fc53d 00   A  0   0  8

...it starts in the first PT_LOAD section and continues onto the second one! That definitely looks wrong: having a segment cross section boundaries can't really be safe can it?

Anyway, then when we run strip on this binary it unsurprisingly doesn't survive. You still get the three LOAD sections:

  Type  Offset    VirtAddr           PhysAddr           FileSiz   MemSiz    Flg Align
  LOAD  0x0000000 0x00000000003ff000 0x00000000003ff000 0x0000580 0x0000580 RW  0x1000
  LOAD  0x0001000 0x0000000000400000 0x0000000000400000 0x1c3faa0 0x1c3faa0 R E 0x1000
  LOAD  0x1c40aa0 0x0000000002040aa0 0x0000000002040aa0 0x000bfc8 0x005b170 RW  0x1000

...notice that the FileSiz/MemSiz has now shrunk and now doesn't cover the whole page. strip has moved the dynstr and now it's no longer covered in the LOAD sections.

  [Nr] Name                   Type       Address          Off     Size    ES Flg Lk Inf Al
  [ 3] .dynstr                STRTAB     00000000003ff580 1dff580  2fc53d 00   A  0   0  8

...and thus ld-linux.so can no longer find our dependencies. The will appear to be there if you look at the file with objdump since they're present in the file, but they're no longer in a section of the file that gets memory mapped in at execve() time.

[Shonallein]

I'm having a hard time with this issue as well. @mitchblank Did you find a work around the issue since you narrowed it down ? Or do you just don't strip your executable after patchelf ?

[mitchblank]

No, I didn't find a workaround. For now I'm pursuing other options.

[ogrisel]

Any idea how to fix this issue? The problem is also affecting binary wheel packages for Python using the manylinux1 specification (patchelf is internally used by the auditwheel tool to graft third party dynamic libs into those Python packages when necessary).