The Runtime Linker is currently eagerly loading all object files on all
platforms which do not use the system linker for GHCi.
The problem with this approach is that it requires all symbols to be found.
Even those of functions never used/called. This makes the number of
libraries required to link things like mingwex quite high.
To work around this the rts was relying on a trick. It itself was compiled
with MingW64-w's GCC. So it was already linked against mingwex.
As such, it re-exported the symbols from itself.
While this worked it made it impossible to link against mingwex in user
libraries. And with this means no C99 code could ever run in GHCi on
Windows without having the required symbols re-exported from the rts.
Consequently this rules out a large number of packages on Windows.
SDL2, HMatrix etc.
After talking with @rwbarton I have taken the approach of loading entire
object files when a symbol is needed instead of doing the dependency
tracking on a per symbol basis. This is a lot less fragile and a lot
less complicated to implement.
The changes come down to the following steps:
- modify the linker to and introduce a new state for ObjectCode: Needed. A Needed object is one that is required for the linking to succeed. The initial set consists of all Object files passed as arguments to the link.
- Change ObjectCode's to be indexed but not initialized or resolved. This means we know where we would load the symbols, but haven't actually done so.
- Mark any ObjectCode belonging to .o passed as argument as required: ObjectState NEEDED.
- During Resolve object calls, mark all ObjectCode containing the required symbols as NEEDED
- During lookupSymbol lookups, (which is called from linkExpr and linkDecl in GHCI.hs) is the symbol is in a not-yet-loaded ObjectCode then load the ObjectCode on demand and return the address of the symbol. Otherwise produce an unresolved symbols error as expected.
- On unloadObj we then change the state of the object and remove it's symbols from the reqSymHash table so it can be reloaded.
This change affects all platforms and OSes which use the runtime linker.
It seems there are no real perf tests for GHCi, but performance shouldn't
be impacted much. We gain a lot of time not loading all obj files, and
we lose some time in lookupSymbol when we're finding sections that have
to be loaded. The actual finding itself is O(1) (Assuming the hashtnl is perfect)
It also consumes slighly more memory as instead of storing just the Address of
a symbol I also store some other information, like if the symbol is weak or not.
This change will break any packages relying on renamed POSIX functions
that were re-named and re-exported by the rts. Any packages following
the proper naming for functions as found on MSDN will work fine.