angerman (Moritz Angermann)
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

Badges

User Since
Sep 11 2014, 7:17 AM (132 w, 13 h)
Availability
Available

Recent Activity

Tue, Mar 21

angerman updated the diff for D3255: Adds aarch64 linker for mach-o files..
  • Always populate the info struct.
Tue, Mar 21, 3:39 AM
angerman updated the diff for D3252: Refactor MachO.c.
  • rebase
Tue, Mar 21, 3:36 AM
angerman updated the diff for D3251: Add ocInit_MachO.
  • don't init unverified
Tue, Mar 21, 3:33 AM

Fri, Mar 17

angerman added a comment to D3348: Check TargetPlatform instead of HostPlatform for leading underscore.

The following test can be found in https://opensource.apple.com/source/gcc/gcc-1495/libjava/libltdl/aclocal.m4.auto.html.

AC_DEFUN(AC_LTDL_SYMBOL_USCORE,
[dnl does the compiler prefix global symbols with an underscore?
AC_REQUIRE([AC_LTDL_GLOBAL_SYMBOL_PIPE])dnl
AC_MSG_CHECKING([for _ prefix in compiled symbols])
AC_CACHE_VAL(ac_cv_sys_symbol_underscore,
[ac_cv_sys_symbol_underscore=no
cat > conftest.$ac_ext <<EOF
void nm_test_func(){}
int main(){nm_test_func;return 0;}
EOF
if AC_TRY_EVAL(ac_compile); then
  # Now try to grab the symbols.
  ac_nlist=conftest.nm
  if AC_TRY_EVAL(NM conftest.$ac_objext \| $ac_cv_sys_global_symbol_pipe \> $ac_nlist) && test -s "$ac_nlist"; then
    # See whether the symbols have a leading underscore.
    if egrep '^. _nm_test_func' "$ac_nlist" >/dev/null; then
      ac_cv_sys_symbol_underscore=yes
    else
      if egrep '^. nm_test_func ' "$ac_nlist" >/dev/null; then
	:
      else
	echo "configure: cannot find nm_test_func in $ac_nlist" >&AC_FD_CC
      fi
    fi
  else
    echo "configure: cannot run $ac_cv_sys_global_symbol_pipe" >&AC_FD_CC
  fi
else
  echo "configure: failed program was:" >&AC_FD_CC
  cat conftest.c >&AC_FD_CC
fi
rm -rf conftest*
])
AC_MSG_RESULT($ac_cv_sys_symbol_underscore)
AC_LTDL_DLSYM_USCORE
])
Fri, Mar 17, 10:20 PM
angerman added a comment to D3348: Check TargetPlatform instead of HostPlatform for leading underscore.

iverilog seems to do this test as well, however more in line with what @rwbarton suggested:

Fri, Mar 17, 10:04 PM
angerman added inline comments to D3348: Check TargetPlatform instead of HostPlatform for leading underscore.
Fri, Mar 17, 9:55 PM
angerman updated the diff for D3348: Check TargetPlatform instead of HostPlatform for leading underscore.
  • Drop confusing comment.
Fri, Mar 17, 9:47 PM

Thu, Mar 16

angerman created D3356: Various patches to support android cross compilation.
Thu, Mar 16, 10:55 PM
angerman updated the diff for D3255: Adds aarch64 linker for mach-o files..
Thu, Mar 16, 10:51 PM
angerman updated the diff for D3252: Refactor MachO.c.
Thu, Mar 16, 10:48 PM
angerman updated the diff for D3251: Add ocInit_MachO.
Thu, Mar 16, 10:46 PM
angerman updated the diff for D3239: rts linker: Introduce MachOTypes.
  • Improve LinkerInternals
Thu, Mar 16, 10:45 PM
angerman updated the diff for D3350: Drop dead code in rts/{Prelude.h,package.conf.in}.
  • fix superfluous a
Thu, Mar 16, 10:33 PM
angerman added a comment to D3352: Replace opt and llc with clang.
In D3352#96309, @erikd wrote:

However, @angerman does have some orthogonal changes in the pipeline which will require a patched LLVM (at least until the next release). We will need to figure out how we want to manage this.

Just to add what this change is. We currently use the Mangler to basically prevent any ghc generated code from being -dead_striped (by the mach-o linker) by removing the .subsections_via_symbols directive
from the assembly, which llvm ingeniously unconditionally injects into any assembly it produces for mach-o. The basic issue here is that the linker does not understand the prefix data (our info tables) belongs
to the function, and just strippes them away. After some digging we found the .alt_entry directive for mach-o. An upstream patch in llvm (which has been landed by now, and will therefore be available in llvm5
i believe) emits a temporary symbol at the start of the info table, and marks the function entry as an .alt_entry. This is sufficient for the linker to understand that the prefix data and the function belong together
and must not be separated nor stripped. This should allow proper dead_stripping with ghc generated code on mach-o!

Thu, Mar 16, 8:16 PM
angerman added a comment to D3349: [libffi] use master..

Indeed I remember from my own cross-compilation adventures that this was necessary. However, I'm really not excited about this approach. For one, it introduces a git dependency into the build system. Secondly, it pulls down untrusted code from the Internet (I would at very least freeze us at a particular commit so we know what we are getting).

Could we add libffi as a git submodule until we have a release?

Let's instead just document our way out of this issue for the time being: add a note to the Wiki's cross compiling page noting that the user must use a pre-release libffi and --with-system-libffi on ARM. We could even fail during configure if we really wanted.

I'd rather have have the configure script fail, and be explicit about this? (e.g. say you are building for armv7 with clang, be told that to successfully configure this, you need a prerelease version.)
I'm really not a fan of the wiki, and I find it hard to locate information there.

Moreover, we should push the libffi upstream to do a release.

Absolutely, I'm trying to. Maybe you can chime in as well? https://github.com/libffi/libffi/issues/296

Thu, Mar 16, 8:06 PM
angerman added a comment to D3348: Check TargetPlatform instead of HostPlatform for leading underscore.

Thanks for the review @bgamari, I tried to explain my reasoning a bit.

Thu, Mar 16, 8:01 PM
angerman added a comment to D3352: Replace opt and llc with clang.
In D3352#96309, @erikd wrote:

You get clang as part of the llvm suite (http://releases.llvm.org/download.html) it is often broken into clang and tools, as you do not necessarily need the tools, unless you want to do something very specific.

Up until now, I have just installed what LLVM (and also Clang) version is needed from the Debian repositories. Debian provides packages for multiple versions of LLVM and the can install in parallel.

clang should come hand in hand with the rest of the llvm tools. So again in principle it will be the same as before.

So where are the configure script changes to validate the Clang version? I repeat, building a local clang is not IMO a reasonable approach.

Thu, Mar 16, 5:07 AM
angerman updated subscribers of D3351: Don’t force gold on me..
In D3351#96301, @erikd wrote:

And yes, these lines prevent cross compilation to android.

How do you know they don't break compiles (cross and/or native) for Linux?

Thu, Mar 16, 3:48 AM
angerman added a comment to D3352: Replace opt and llc with clang.
In D3352#96302, @erikd wrote:

What are the reasons to prefer opt and llc over clang for you?

At the moment I know that opt and llc work and I don't know enough about the implications and mechanics of this change.

This patch raises so many questions. Some of those questions:

  • On arm/linux and aarch64/linux does GHC still produce LLVM text IR and run that through Clang?

In principle yes; in this draft state however, not yet, as these lines are currently missing (in runPhase (RealPhase LlvmAs) input_fn dflags)

(ArchARM ARMv7 _ _, OSLinux) -> return "armv6-unknown-linux-gnueabihf"
(ArchARM64,         OSLinux) -> return "aarch64-none-linux-gnu"
  • Are we going to be locked to a single version of Clang like we are currently locked to a single version of LLVM?

clang should come hand in hand with the rest of the llvm tools. So again in principle it will be the same as before.
You get clang as part of the llvm suite (http://releases.llvm.org/download.html) it is often broken into clang and tools, as you do not
necessarily need the tools, unless you want to do something very specific.
Think of clang as the unified interface to llvm, and opt, llc, ... as the development tools for the llvm suite.

Thu, Mar 16, 3:44 AM
angerman added a comment to D3351: Don’t force gold on me..

@erikd I've just figured that my comment may raise more questions than answers. Killing code that only compiles for arm on linux, and then saying I did not test on arm linux might sound quite bonkers.

Thu, Mar 16, 3:20 AM
angerman added a comment to D3351: Don’t force gold on me..
In D3351#96296, @erikd wrote:

Have you tested that on Arm64 Linux? For a long time BFD ld was broken on both 32 and 63 bit Arm. If BFD ld has been fixed we probably need a version check on it.

Thu, Mar 16, 3:12 AM
angerman added a comment to D3351: Don’t force gold on me..

@dfeuer I'm happy to discuss this. And if we truly want to force gold here, we need to at least guard it with check that the compiler is gcc.

Thu, Mar 16, 3:10 AM
angerman added a comment to D3352: Replace opt and llc with clang.
In D3352#96290, @erikd wrote:

Currently, I can build GHC on say Arm/Linux with ghc, gcc, opt and llc. That is specifically, *without* clang.

Is this patch forcing me to install clang?

Thu, Mar 16, 3:08 AM

Wed, Mar 15

angerman added a comment to D3352: Replace opt and llc with clang.

This is my first stab at simplifying the llvm pipeline, any suggestions are welcome. I'm uploading this rough diff to solicit some early feedback before sinking more time into this.

Wed, Mar 15, 9:05 PM
angerman created D3352: Replace opt and llc with clang.
Wed, Mar 15, 9:01 PM
angerman created D3351: Don’t force gold on me..
Wed, Mar 15, 8:56 PM
angerman created D3350: Drop dead code in rts/{Prelude.h,package.conf.in}.
Wed, Mar 15, 8:53 PM
angerman created D3349: [libffi] use master..
Wed, Mar 15, 8:51 PM
angerman created D3348: Check TargetPlatform instead of HostPlatform for leading underscore.
Wed, Mar 15, 8:47 PM

Fri, Mar 10

angerman requested review of D3311: Disable Mangler by default.

Upon closer inspection, the issue demonstrated in https://github.com/bgamari/symbol-type-repro, can be rectified by supplying -fPIC or -relocation-model=pic to all relevant commands.
What leaves to be seen is why the function/object rewrite worked in the first place, as objects would end up in non-executable memory regions.

Fri, Mar 10, 10:43 PM
angerman planned changes to D3311: Disable Mangler by default.

@bgamari has come up with an explicit example for the PLT mangling requirement in https://github.com/bgamari/symbol-type-repro. This however only affects ELF, the mangler also only fixes the issue for ELF.
Mach-O has the same issue, at least on aarch64. However there is no symbol type that would allow us to prevent this. Preliminary testing though has shown that the .alt_entry change, in https://reviews.llvm.org/D30770,
will resolve this issue for Mach-O. Where as https://reviews.llvm.org/D30812 is the llvm upstream fix for the function to object mangling.

Fri, Mar 10, 3:27 AM
angerman added inline comments to D3311: Disable Mangler by default.
Fri, Mar 10, 12:07 AM
angerman updated the diff for D3311: Disable Mangler by default.
  • fix comment
Fri, Mar 10, 12:07 AM

Thu, Mar 9

angerman created D3311: Disable Mangler by default.
Thu, Mar 9, 9:11 PM
angerman updated the diff for D3290: Make LLVM output robust to -dead_strip on mach-o platforms.
  • rebase
Thu, Mar 9, 7:56 PM

Wed, Mar 8

angerman added a comment to D3297: Drop AVX mangling..

The proper test would be a full run of DPH with an AVX-enabled vector library. Unfortunately, the DPH code has bit-rotted, so I can't provide you with an easily-run test.

Uhh. O.k.

Wed, Mar 8, 9:18 PM
angerman added a comment to D3297: Drop AVX mangling..

Yes, we pass -stack-alignment=32 to llc, but the point is that this doesn't guarantee alignment of the stack—as the comment in the code notes.

For this small example, it looks like LLVM does not assume the stack is aligned. Has LLVM's behavior changed, so that it never assumes the stack is aligned? If so, then this mangling code can be removed. Otherwise, we need to test more thoroughly.

Wed, Mar 8, 6:47 PM
angerman updated the diff for D3290: Make LLVM output robust to -dead_strip on mach-o platforms.
  • drop alt_entry
Wed, Mar 8, 9:14 AM
angerman added a comment to D3290: Make LLVM output robust to -dead_strip on mach-o platforms.

I might have had a wrong interpretation of .alt_entry. I'll give this a build, and update the diff, unless something falls apart along the way.

Wed, Mar 8, 8:26 AM
angerman added a comment to D3290: Make LLVM output robust to -dead_strip on mach-o platforms.

You don't need .alt_entry any more if you are putting .no_dead_strip on the info tables (it could only be harmful, by causing even more things to stay live).

Wed, Mar 8, 8:11 AM
angerman added a comment to D3290: Make LLVM output robust to -dead_strip on mach-o platforms.

This does work, and has the same effect as the mangling of the .subsection_via_symbols directive. This is all rather unfortunate. However I'd rather work towards dropping the Mangler for good,
and have every symbol explicitly marked .no_dead_strip, rather than having to mangle the .subsections_via_symbols in the Mangler, in some hot patch post process way.

Wed, Mar 8, 6:50 AM
angerman updated the diff for D3290: Make LLVM output robust to -dead_strip on mach-o platforms.
  • don’t dead_strip dsps.
Wed, Mar 8, 6:47 AM
angerman added a comment to D3297: Drop AVX mangling..

The pre and post mangled files do differ. In that the post mangled file has the .subsections_via_symbols removed :)

$ diff SimdTest.s SimdTest.lm_s
376c376
< ## no .subsection_via_symbols for ghc. We need our info tables!
---
> .subsections_via_symbols
Wed, Mar 8, 4:09 AM
angerman created D3297: Drop AVX mangling..
Wed, Mar 8, 4:06 AM
angerman raised a concern with rGHCecb880caea44: base: kevent.filter is signed.

This breaks macOS (quick-llvm, even though I doubt that matters)

Wed, Mar 8, 3:13 AM

Tue, Mar 7

angerman planned changes to D3290: Make LLVM output robust to -dead_strip on mach-o platforms.

This is currently not working as expected. Additional changes that make the visibility of symbols clearer to the linker are necessary, such that the linker does not strip symbols that are essential (e.g. _stg_STACK_info$def).

Tue, Mar 7, 10:05 PM
angerman created D3290: Make LLVM output robust to -dead_strip on mach-o platforms.
Tue, Mar 7, 3:43 AM

Mon, Mar 6

angerman updated the diff for D3287: Mangle .subsections_via_symbols away..
  • Drop redundancy.
Mon, Mar 6, 9:21 AM
angerman added a comment to D3287: Mangle .subsections_via_symbols away..

This should fix the breakage introduced in D2911.

Mon, Mar 6, 9:13 AM
angerman created D3287: Mangle .subsections_via_symbols away..
Mon, Mar 6, 9:11 AM
angerman updated subscribers of rGHC0a6c257de5c2: -dead_strip is now the default on Darwin.
Mon, Mar 6, 2:50 AM
angerman added 3 auditor(s) for rGHC0a6c257de5c2: -dead_strip is now the default on Darwin: bgamari, rwbarton, erikd.

Friends, I'd really like to bring this to your attention. Please let me know where my debugging is faulty. I'd be glad if this can be resolved some other way.

Mon, Mar 6, 2:50 AM
angerman added a comment to rGHC0a6c257de5c2: -dead_strip is now the default on Darwin.

Alright, I believe this needs quite a bit more work.

Mon, Mar 6, 2:48 AM
angerman raised a concern with rGHC0a6c257de5c2: -dead_strip is now the default on Darwin.

After a few hours of bisecting ghc, this is the final commit that broken
building on macOS with llvm. (flavor quick-llvm on macOS).

Mon, Mar 6, 12:06 AM

Sat, Mar 4

angerman added a comment to D3278: Enable new warning for fragile/incorrect CPP #if usage.

Looks good to me. I'm however a bit irritated about the inconsistent use of

  • #ifdef
  • #if defined X
  • #if defined(X).
Sat, Mar 4, 7:23 PM

Fri, Mar 3

angerman updated the diff for D3255: Adds aarch64 linker for mach-o files..
  • update to latest consistant state (with object code info structs)
Fri, Mar 3, 1:43 AM
angerman updated the diff for D3252: Refactor MachO.c.
  • update to latest consistant state (with object code info structs)
Fri, Mar 3, 1:37 AM
angerman updated the diff for D3251: Add ocInit_MachO.
  • update to latest consistant state (with object code info structs)
Fri, Mar 3, 1:35 AM
angerman updated the diff for D3239: rts linker: Introduce MachOTypes.
  • update to latest consistant state (with object code info structs)
Fri, Mar 3, 1:30 AM
angerman updated the diff for D3238: Make mmap r+w only during preload for iOS..
  • update to latest consistant state.
Fri, Mar 3, 1:29 AM

Thu, Mar 2

angerman updated the diff for D3238: Make mmap r+w only during preload for iOS..
  • improve comment.
Thu, Mar 2, 7:52 PM
angerman updated the diff for D3240: Allow iOS to load archives through the linker.
  • rebase onto master
Thu, Mar 2, 7:27 PM
angerman updated the diff for D3233: Enter iserv-proxy.
  • rebase onto master
Thu, Mar 2, 7:25 PM
angerman added a comment to D3239: rts linker: Introduce MachOTypes.

Thanks @bgamari for taking a look at this.

Thu, Mar 2, 10:29 AM
angerman added a comment to D3238: Make mmap r+w only during preload for iOS..

You say "While we do not yet enable mmap for ios builds. If we later do, we must not try to mmap r+w+x, on iOS as that clearly fails." Does "not yet" mean even after this entire set of MachO linker-related patches, so this will still be effectively dead code?

No, this is not dead code. The updated autoconfig file is in the final diff D3255.

Thu, Mar 2, 10:13 AM
angerman added inline comments to D3239: rts linker: Introduce MachOTypes.
Thu, Mar 2, 9:55 AM
angerman added a comment to D3239: rts linker: Introduce MachOTypes.

@simonmar thanks for taking the time to review this!

Thu, Mar 2, 9:53 AM
angerman updated the diff for D3255: Adds aarch64 linker for mach-o files..
  • Range Check, Proddable sections.
Thu, Mar 2, 9:46 AM
angerman accepted D3259: Properly acquire locks on not yet existing package databases.

Rectifies the issue with no preexisting package database in ~/ghc, and for also the ghci problem, which complained about -B<top> missing.
I do however not understand why the second issue is related; it is however fixed for cbe569a5 with this patch applied on top.

Thu, Mar 2, 12:14 AM
angerman added a comment to rGHC0d86aa5904e5: Add support for concurrent package db access and updates.
In null, @arybczak wrote:

[...]
The first issue is fixed by D3259 - I'm not sure about the second one, but it might be a byproduct of the first problem. Can you check D3259?

Thu, Mar 2, 12:12 AM

Wed, Mar 1

angerman raised a concern with rGHC0d86aa5904e5: Add support for concurrent package db access and updates.

This also breaks ghc-pkg and ghci on macOS as far as I can tell.

Wed, Mar 1, 10:52 PM
angerman updated the summary of D3252: Refactor MachO.c.
Wed, Mar 1, 7:36 PM
angerman updated the summary of D3251: Add ocInit_MachO.
Wed, Mar 1, 7:35 PM
angerman updated the summary of D3239: rts linker: Introduce MachOTypes.
Wed, Mar 1, 7:33 PM
angerman updated the summary of D3238: Make mmap r+w only during preload for iOS..
Wed, Mar 1, 7:33 PM
angerman updated the summary of D3240: Allow iOS to load archives through the linker.
Wed, Mar 1, 7:31 PM
angerman updated the diff for D3252: Refactor MachO.c.
  • Add lost comment back in.
Wed, Mar 1, 7:26 PM
angerman updated the test plan for D3255: Adds aarch64 linker for mach-o files..
Wed, Mar 1, 8:01 AM
angerman updated the test plan for D3255: Adds aarch64 linker for mach-o files..
Wed, Mar 1, 7:59 AM
angerman updated the test plan for D3255: Adds aarch64 linker for mach-o files..
Wed, Mar 1, 7:57 AM
angerman created D3255: Adds aarch64 linker for mach-o files..
Wed, Mar 1, 7:54 AM
angerman updated the diff for D3252: Refactor MachO.c.
  • proper rebase
Wed, Mar 1, 2:53 AM
angerman updated the diff for D3251: Add ocInit_MachO.
  • add ocDeinit
Wed, Mar 1, 2:51 AM
angerman updated the diff for D3239: rts linker: Introduce MachOTypes.
  • Adds missing got_size field.
Wed, Mar 1, 2:47 AM
angerman updated the diff for D3252: Refactor MachO.c.
  • fix the dsymLC to symCmd mistake. And have it be properly refactored with dsymCmd!
Wed, Mar 1, 2:03 AM
angerman created D3252: Refactor MachO.c.
Wed, Mar 1, 1:08 AM
angerman added a dependent revision for D3239: rts linker: Introduce MachOTypes: D3252: Refactor MachO.c.
Wed, Mar 1, 1:08 AM
angerman added a dependent revision for D3251: Add ocInit_MachO: D3252: Refactor MachO.c.
Wed, Mar 1, 1:08 AM
angerman updated the diff for D3251: Add ocInit_MachO.
  • Diff against proper base.
Wed, Mar 1, 12:19 AM
angerman updated the diff for D3239: rts linker: Introduce MachOTypes.
  • Adds scattered_relocation_info struct that somehow went unnoticed.
  • Renames symbol to nlist to improve readability.
  • Uses SymbolName and SymbolAddr.
Wed, Mar 1, 12:18 AM
angerman updated the diff for D3251: Add ocInit_MachO.
  • Rename symbol to nlist, to reduce confusion down the line.
Wed, Mar 1, 12:16 AM

Tue, Feb 28

angerman created D3251: Add ocInit_MachO.
Tue, Feb 28, 10:45 PM
angerman added a dependent revision for D3239: rts linker: Introduce MachOTypes: D3251: Add ocInit_MachO.
Tue, Feb 28, 10:45 PM
angerman added a dependent revision for D3240: Allow iOS to load archives through the linker: D3251: Add ocInit_MachO.
Tue, Feb 28, 10:45 PM
angerman added inline comments to D3239: rts linker: Introduce MachOTypes.
Tue, Feb 28, 9:56 PM
angerman updated the diff for D3239: rts linker: Introduce MachOTypes.
  • Adds comment.
Tue, Feb 28, 9:53 PM
angerman updated the diff for D3240: Allow iOS to load archives through the linker.

rebase onto master

Tue, Feb 28, 9:18 PM
angerman updated the diff for D3239: rts linker: Introduce MachOTypes.

Rebase onto master.

Tue, Feb 28, 9:17 PM
angerman added a comment to D3238: Make mmap r+w only during preload for iOS..

So where does the mprotect currently happen?

I think we probably want to be doing this for all platforms if we can. @simonmar, do you know of any reason why this shouldn't be possible?

Tue, Feb 28, 8:53 PM
angerman added a comment to D3233: Enter iserv-proxy.

Why does the proxy need to understand the content of messages, rather than just forwarding data in both directions whenever it arrives? In fact, it makes me wonder whether the proxy can't be a script wrapping netcat or something like that. Perhaps the answer to this is the extra SlaveMessage protocol for sending files - but I wouldn't object to making this part of the main protocol, unused in the case of a local server. It's quite confusing that we have these extra messages exchanged between the slave and proxy in certain circumstances.

Tue, Feb 28, 8:48 PM