Fix unwinding of C -> Haskell FFI calls with -threaded
ClosedPublic

Authored by niteria on May 3 2018, 4:15 PM.

Details

Summary

See the new note.

Test Plan

manual testing with patched gdb
maybe I can write something automated with libdw

Diff Detail

Repository
rGHC Glasgow Haskell Compiler
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
niteria created this revision.May 3 2018, 4:15 PM

Why is it that we apparently do not need to unwind rbp in the non-threaded runtime?

Why is it that we apparently do not need to unwind rbp in the non-threaded runtime?

non-threaded

Here's scheduleWaitThread from Schedule.o (non-threaded runtime) disassembled (objdump -xD), apparently schedule gets inlined into scheduleWaitThread:
Full text at P178.

00000000000008f0 <scheduleWaitThread>:
     8f0:	41 57                	push   %r15
     8f2:	41 56                	push   %r14
     8f4:	49 89 d6             	mov    %rdx,%r14
     8f7:	41 55                	push   %r13
     8f9:	41 54                	push   %r12
     8fb:	49 89 fc             	mov    %rdi,%r12
     8fe:	55                   	push   %rbp
     8ff:	53                   	push   %rbx
     900:	48 83 ec 18          	sub    $0x18,%rsp

...

     b43:	66 41 83 ff 02       	cmp    $0x2,%r15w
     b48:	0f 84 f8 00 00 00    	je     c46 <scheduleWaitThread+0x356>
     b4e:	77 68                	ja     bb8 <scheduleWaitThread+0x2c8>
     b50:	66 41 83 ff 01       	cmp    $0x1,%r15w
     b55:	0f 85 c5 01 00 00    	jne    d20 <scheduleWaitThread+0x430>
     b5b:	48 8d 73 18          	lea    0x18(%rbx),%rsi
     b5f:	bf 00 00 00 00       	mov    $0x0,%edi
			b60: R_X86_64_32	stg_returnToStackTop
     b64:	e8 00 00 00 00       	callq  b69 <scheduleWaitThread+0x279>
			b65: R_X86_64_PC32	StgRun-0x4
     b69:	44 8b b8 90 03 00 00 	mov    0x390(%rax),%r15d
     b70:	48 8d 58 e8          	lea    -0x18(%rax),%rbx
     b74:	c6 83 c0 03 00 00 00 	movb   $0x0,0x3c0(%rbx)
     b7b:	4c 8b ab 80 03 00 00 	mov    0x380(%rbx),%r13
     b82:	41 83 ff 04          	cmp    $0x4,%r15d

...

The call to StgRun is at b64. Relevant unwinding info (readelf --debug-dump=frames-interp):

0000022c 0000000000000048 00000230 FDE cie=00000000 pc=00000000000008f0..000000000000126a
   LOC           CFA      rbx   rbp   r12   r13   r14   r15   ra
00000000000008f0 rsp+8    u     u     u     u     u     u     c-8
00000000000008f2 rsp+16   u     u     u     u     u     c-16  c-8
00000000000008f4 rsp+24   u     u     u     u     c-24  c-16  c-8
00000000000008f9 rsp+32   u     u     u     c-32  c-24  c-16  c-8
00000000000008fb rsp+40   u     u     c-40  c-32  c-24  c-16  c-8
00000000000008ff rsp+48   u     c-48  c-40  c-32  c-24  c-16  c-8
0000000000000900 rsp+56   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000904 rsp+80   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d67 rsp+56   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d68 rsp+48   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d69 rsp+40   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d6b rsp+32   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d6d rsp+24   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d6f rsp+16   c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d71 rsp+8    c-56  c-48  c-40  c-32  c-24  c-16  c-8
0000000000000d78 rsp+80   c-56  c-48  c-40  c-32  c-24  c-16  c-8

Our unwind line is: 0000000000000904 rsp+80 c-56 c-48 c-40 c-32 c-24 c-16 c-8 here.

-threaded

With -threaded, schedule is its own function. Full text at P179.

0000000000000a10 <schedule>:
     a10:	55                   	push   %rbp
     a11:	48 89 e5             	mov    %rsp,%rbp
     a14:	41 57                	push   %r15
     a16:	41 56                	push   %r14
     a18:	41 55                	push   %r13
     a1a:	41 54                	push   %r12
     a1c:	49 89 f4             	mov    %rsi,%r12
     a1f:	53                   	push   %rbx
     a20:	48 83 ec 68          	sub    $0x68,%rsp
     a24:	48 89 b5 78 ff ff ff 	mov    %rsi,-0x88(%rbp)
     a2b:	48 89 7d c0          	mov    %rdi,-0x40(%rbp)
     a2f:	80 bf c0 03 00 00 00 	cmpb   $0x0,0x3c0(%rdi)

...

    12da:	0f 85 9f 01 00 00    	jne    147f <schedule+0xa6f>
    12e0:	48 8b 45 c0          	mov    -0x40(%rbp),%rax
    12e4:	bf 00 00 00 00       	mov    $0x0,%edi
			12e5: R_X86_64_32	stg_returnToStackTop
    12e9:	48 8d 70 18          	lea    0x18(%rax),%rsi
    12ed:	e8 00 00 00 00       	callq  12f2 <schedule+0x8e2>
			12ee: R_X86_64_PC32	StgRun-0x4
    12f2:	48 8d 50 e8          	lea    -0x18(%rax),%rdx
    12f6:	8b 98 90 03 00 00    	mov    0x390(%rax),%ebx
    12fc:	48 89 55 c0          	mov    %rdx,-0x40(%rbp)
    1300:	48 89 d0             	mov    %rdx,%rax

...

The call to StgRun is at 12ed.

Relevant unwind info:

00000198 000000000000002c 0000019c FDE cie=00000000 pc=0000000000000a10..00000000000019bc
   LOC           CFA      rbx   rbp   r12   r13   r14   r15   ra
0000000000000a10 rsp+8    u     u     u     u     u     u     c-8
0000000000000a11 rsp+16   u     c-16  u     u     u     u     c-8
0000000000000a14 rbp+16   u     c-16  u     u     u     u     c-8
0000000000000a1c rbp+16   u     c-16  c-48  c-40  c-32  c-24  c-8
0000000000000a24 rbp+16   c-56  c-16  c-48  c-40  c-32  c-24  c-8
0000000000001389 rsp+8    c-56  c-16  c-48  c-40  c-32  c-24  c-8
000000000000138a rbp+16   c-56  c-16  c-48  c-40  c-32  c-24  c-8

Our unwind line: 0000000000000a24 rbp+16 c-56 c-16 c-48 c-40 c-32 c-24 c-8.

bring it back together

For not-threaded unwind line was: 0000000000000904 rsp+80 c-56 c-48 c-40 c-32 c-24 c-16 c-8 here.
For -threaded it was: 0000000000000a24 rbp+16 c-56 c-16 c-48 c-40 c-32 c-24 c-8.

One uses rbp to define the CFA, the other uses rsp.

simonmar requested changes to this revision.May 8 2018, 4:25 AM

Just a few nits

includes/Rts.h
305–306 ↗(On Diff #16295)

Dangerous to define something called str() in a header that might be #included outside of RTS code. We don't have an obvious place to put these private things that we want to share amongst different files in the RTS. Maybe put them in rts/RtsUtils.h?

rts/StgCRun.c
255

I think this note is about unwinding from Haskell code into C, not the other way around, right? Could you also say something about how we unwind from C to Haskell, or is that in a different note?

257

A diagram showing the stacks here would really help me visualise what's going on. FYI I've been using https://textik.com/ for this.

421–428

magic

This revision now requires changes to proceed.May 8 2018, 4:25 AM
niteria updated this revision to Diff 16358.May 10 2018, 9:42 AM
  • move str/xstr to RtsUtils.h
  • add a diagram
  • (hopefully) make it unambiguous what calls what and what is unwound
niteria updated this revision to Diff 16359.May 10 2018, 9:47 AM
niteria marked an inline comment as done.

eat extra newlines

niteria marked 2 inline comments as done.May 10 2018, 9:48 AM
niteria added inline comments.
rts/StgCRun.c
255

Ah, I see the ambiguity of this sentence. What I had in mind when writing this is that the call was from C into Haskell and we unwind it.
But unwinding goes the opposite way, so depending how you parenthesise you get different meaning.

I don't think the other case is documented.

257

textik is really cool, thanks!

niteria marked an inline comment as done.May 10 2018, 9:49 AM
This revision is now accepted and ready to land.May 10 2018, 12:21 PM
This revision was automatically updated to reflect the committed changes.