ELF/x86_64: map object file sections separately into the low 2GB
ClosedPublic

Authored by simonmar on Jun 11 2015, 9:18 AM.

Details

Summary

On 64-bit ELF we need to link object files into the low 2GB due to the
small memory model. Previously we would map the entire object file
using MAP_32BIT, but the object file can consist of 75% or more
symbols, which only need to be present during linking, so this is
wasteful. In our particular application, we're already running out of
space here.

This patch changes the way we load object files on ELF platforms so
that the object is first mapped above the 2GB boundary, parsed, and
then the important sections are re-mapped into the low 2GB area.

Test Plan

validate
(also needs testing on OS X & Windows, preferably 32 & 64 bit)

Diff Detail

Repository
rGHC Glasgow Haskell Compiler
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
simonmar updated this revision to Diff 3192.Jun 11 2015, 9:18 AM
simonmar retitled this revision from to ELF/x86_64: map object file sections separately into the low 2GB.
simonmar updated this object.
simonmar edited the test plan for this revision. (Show Details)
simonmar added a reviewer: austin.
Phyx edited edge metadata.Jun 12 2015, 8:07 AM

Hi Simon,

The changes break the windows build, I've highlighted a few of them but here's the full error log:

cc1.exe: warnings being treated as errors
rts\Linker.c: In function 'freePreloadObjectFile':

rts\Linker.c:2364:9: error:
     error: implicit declaration of function 'munmap'

rts\Linker.c:2364:9: error:
     error: nested extern declaration of 'munmap'
rts\Linker.c: In function 'preloadObjectFile':

rts\Linker.c:3039:4: error:
     error: 'f' undeclared (first use in this function)

rts\Linker.c:3039:4: error:
     note: each undeclared identifier is reported only once for each function it appears in
rts\Linker.c: In function 'ocGetNames_PEi386':

rts\Linker.c:4453:11: error:
     error: passing argument 1 of 'addSection' from incompatible pointer type

rts\Linker.c:3384:1: error:
     note: expected 'struct Section *' but argument is of type 'struct ObjectCode *'

rts\Linker.c:4453:11: error:
     error: incompatible type for argument 3 of 'addSection'

rts\Linker.c:3384:1: error:
     note: expected 'SectionAlloc' but argument is of type 'UChar *'

rts\Linker.c:4453:11: error:
     error: too few arguments to function 'addSection'

rts\Linker.c:3384:1: error:  note: declared here

rts\Linker.c:4502:21: error:
     error: passing argument 1 of 'addSection' from incompatible pointer type

rts\Linker.c:3384:1: error:
     note: expected 'struct Section *' but argument is of type 'struct ObjectCode *'

rts\Linker.c:4502:21: error:
     error: incompatible type for argument 3 of 'addSection'

rts\Linker.c:3384:1: error:
     note: expected 'SectionAlloc' but argument is of type 'void *'

rts\Linker.c:4502:21: error:
     error: too few arguments to function 'addSection'

rts\Linker.c:3384:1: error:  note: declared here
rts/Linker.c
2588–2598

The function is defined outside of an USE_MMAP block, so this breaks the windows build.

2629

and by association this call

2641–2644

and this branch

2972

This declaration is now missing and breaks the compile.

Phyx requested changes to this revision.Jun 12 2015, 8:09 AM
Phyx edited edge metadata.
This revision now requires changes to proceed.Jun 12 2015, 8:09 AM
simonmar updated this revision to Diff 3217.Jun 12 2015, 1:22 PM
simonmar edited edge metadata.

Fix a bug. @Phyx - thanks, I'll look at the Windows breakage.

bgamari added a comment.EditedJun 15 2015, 6:16 AM

@simonmar, could you include a reference to what you mean by "the small memory model?" Is this an x86_64 ABI restriction?

bgamari accepted this revision.Jun 22 2015, 12:57 PM
bgamari added a reviewer: bgamari.

Looks good to me.

We should hold off on merging until we have a positive validation on OS X. Could someone with hardware give this a shot?

Phyx added a comment.Jun 22 2015, 2:08 PM

Just a reminder, the Windows build hasn't been fixed yet. So we need to hold off merging until then too.

The errors aren't terribly complicated so if it's easier I'm more than happy to fix it and submit it for review.

bgamari requested changes to this revision.Jun 22 2015, 2:29 PM
bgamari edited edge metadata.

Ahh, I misunderstood. Thanks for pointing this out, @Phyx. Requesting changes to clear the review queue.

This revision now requires changes to proceed.Jun 22 2015, 2:29 PM
trommler edited edge metadata.Jun 22 2015, 5:05 PM
In D975#26920, @bgamari wrote:

We should hold off on merging until we have a positive validation on OS X. Could someone with hardware give this a shot?

I will do a validate on osx tomorrow.

trommler requested changes to this revision.Jun 23 2015, 1:43 AM
trommler edited edge metadata.

Validate fails on OS X 10.10.3 with the following:

rts/Linker.c:2844:13: error:
     error: use of undeclared identifier 'misalignment'
                misalignment = machoGetMisalignment(f);
                ^

rts/Linker.c:2844:28: error:
     warning: implicit declaration of function 'machoGetMisalignment' is invalid in C99 [-Wimplicit-function-declaration]
                misalignment = machoGetMisalignment(f);
                               ^

rts/Linker.c:2845:49: error:
     error: use of undeclared identifier 'misalignment'
                image = stgMallocBytes(memberSize + misalignment, "loadArchive(image)");
                                                    ^

rts/Linker.c:2846:22: error:
     error: use of undeclared identifier 'misalignment'
                image += misalignment;
                         ^

rts/Linker.c:6445:1: error:
     error: redefinition of 'ocAllocateSymbolExtras_MachO'
ocAllocateSymbolExtras_MachO(ObjectCode* oc)
^

rts/Linker.c:6388:1: error:  note: previous definition is here
ocAllocateSymbolExtras_MachO(ObjectCode* oc)
^

rts/Linker.c:7213:84: error:
     error: too few arguments to function call, expected 4, have 3
                char * zeroFillArea = mmapForLinker(sections[i].size, MAP_ANONYMOUS, -1);
                                      ~~~~~~~~~~~~~                                    ^

rts/Linker.c:2231:1: error:  note: 'mmapForLinker' declared here
static void * mmapForLinker (size_t bytes, nat flags, int fd, int offset)
^

rts/Linker.c:7239:72: error:
     error: too few arguments to function call, expected 8, have 4
                    (void*) (image + sections[i].offset + sections[i].size));
                                                                           ^

rts/Linker.c:3383:1: error:  note: 'addSection' declared here
static void
^
1 warning and 6 errors generated.
make[1]: *** [rts/dist/build/Linker.o] Error 1
simonmar updated this revision to Diff 3337.Jun 26 2015, 10:58 AM
simonmar edited edge metadata.

Fix on Windows, and rebase

There will still be problems on OS X, I'll need to get access to an OS
X box to test on somehow.

simonmar updated this revision to Diff 3355.Jun 29 2015, 3:11 AM
simonmar edited edge metadata.

Fix ELF breakage

bgamari requested changes to this revision.Jul 3 2015, 5:13 AM
bgamari edited edge metadata.

Bumping out of review queue due to OS X issues. @simonmar, let me know if you think these are actually resolved.

This revision now requires changes to proceed.Jul 3 2015, 5:13 AM
simonmar updated this revision to Diff 3489.Jul 9 2015, 3:47 AM
simonmar edited edge metadata.
  • Make the USE_CONTIGUOUS_MMAP case work again (essentially disables the optimisation in this patch)
hsyl20 added a subscriber: hsyl20.Jul 10 2015, 5:27 PM

As far as I understand it, it looks ok to me. Thanks for doing the merge.

We should check memory consumption (cf Trac #9314). I'll do it if I find some time.

rts/Linker.c
5727

Typo *sections*

trommler requested changes to this revision.Jul 13 2015, 3:44 AM
trommler edited edge metadata.

The build on OS X still fails:
(First line folded for readability.)

"inplace/bin/ghc-stage1" -optc-m64 -optc-fno-stack-protector -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes 
-optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith 
-optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist 
-optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build 
-optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/autogen 
-optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-DRtsWay=\"rts_v\" -static  -H32m -O 
-Wall -H64m -O0  -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header 
-Irts -Irts/dist/build -DCOMPILING_RTS -this-package-key rts -dcmm-lint  -DDTRACE     -i -irts 
-irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen           
-O2    -c rts/Linker.c -o rts/dist/build/Linker.o

rts/Linker.c:3087:13: error:
     error: use of undeclared identifier 'misalignment'
                misalignment = machoGetMisalignment(f);
                ^

rts/Linker.c:3087:28: error:
     warning: implicit declaration of function 'machoGetMisalignment' is invalid in C99 [-Wimplicit-function-declaration]
                misalignment = machoGetMisalignment(f);
                               ^

rts/Linker.c:3088:49: error:
     error: use of undeclared identifier 'misalignment'
                image = stgMallocBytes(memberSize + misalignment, "loadArchive(image)");
                                                    ^

rts/Linker.c:3089:22: error:
     error: use of undeclared identifier 'misalignment'
                image += misalignment;
                         ^

rts/Linker.c:6708:1: error:
     error: redefinition of 'ocAllocateSymbolExtras_MachO'
ocAllocateSymbolExtras_MachO(ObjectCode* oc)
^

rts/Linker.c:6651:1: error:  note: previous definition is here
ocAllocateSymbolExtras_MachO(ObjectCode* oc)
^

rts/Linker.c:7476:84: error:
     error: too few arguments to function call, expected 4, have 3
                char * zeroFillArea = mmapForLinker(sections[i].size, MAP_ANONYMOUS, -1);
                                      ~~~~~~~~~~~~~                                    ^

rts/Linker.c:2265:1: error:  note: 'mmapForLinker' declared here
static void * mmapForLinker (size_t bytes, nat flags, int fd, int offset)
^

rts/Linker.c:7502:72: error:
     error: too few arguments to function call, expected 8, have 4
                    (void*) (image + sections[i].offset + sections[i].size));
                                                                           ^

rts/Linker.c:3631:1: error:  note: 'addSection' declared here
static void
^
1 warning and 6 errors generated.
This revision now requires changes to proceed.Jul 13 2015, 3:44 AM

@simonmar, it looks like this could use another iteration.

simonmar added a comment.EditedAug 29 2015, 5:45 AM

Yes, I'd really love to get this in. This is one of very few patches in my local GHC that I haven't managed to get into master yet. It's blocked on fixing it up for OS X, which is blocked on me borrowing a machine and working on it. I'll aim to do it for 7.12.

simonmar updated this revision to Diff 4501.Oct 15 2015, 8:17 AM
simonmar edited edge metadata.

Fixup for OS X, and rebase

This revision was automatically updated to reflect the committed changes.