[CRIU] [PATCH 1/4] pie/x86_64: syscall clobbers rcx and r11

[Cyrill Gorcunov]

On Tue, Jun 16, 2015 at 07:34:55AM -0600, Tycho Andersen wrote:
> On Tue, Jun 16, 2015 at 04:31:50PM +0300, Cyrill Gorcunov wrote:
> > On Tue, Jun 16, 2015 at 07:20:56AM -0600, Tycho Andersen wrote:
> > > 
> > > Ok, I just added rcx because the above link said it could be clobbered
> > > too, but I can send a patch to remove it if you want.
> > 
> > no need to, i'm fine with as is.
> 
> Ok. I think the reason we've never seen a seg fault here is that the
> code path hasn't changed in a while (and neither has gcc's register
> allocator apparently, yay reaload.c). When I changed it for this
> patchset, gcc decided to use r11 for something (viz. the dissasembly
> in the commit message), which is what caused the fault.

I still wonder why it's happened ;) Look, the kernel saves all the data
it going to modify inside syscall

ENTRY(system_call)
	...
	SAVE_ARGS 8,0

the SAVE_ARGS macro expands to (see thunk_64.S)

	...
     3e6:       48 83 ec 50             sub    $0x50,%rsp
     3ea:       48 89 7c 24 40          mov    %rdi,0x40(%rsp)
     3ef:       48 89 74 24 38          mov    %rsi,0x38(%rsp)
     3f4:       48 89 54 24 30          mov    %rdx,0x30(%rsp)
     3f9:       48 89 44 24 20          mov    %rax,0x20(%rsp)
     3fe:       4c 89 44 24 18          mov    %r8,0x18(%rsp)
     403:       4c 89 4c 24 10          mov    %r9,0x10(%rsp)
     408:       4c 89 54 24 08          mov    %r10,0x8(%rsp)
     40d:       4c 89 1c 24             mov    %r11,(%rsp)
     411:       48 89 44 24 48          mov    %rax,0x48(%rsp)
     416:       48 89 4c 24 50          mov    %rcx,0x50(%rsp)
     41b:       f7 84 24 88 e0 ff ff    testl  $0x100801d1,-0x1f78(%rsp)
     422:       d1 01 08 10 
     426:       0f 85 d6 00 00 00       jne    502 <tracesys>
...

then when it's time to return from syscall we move them back

	RESTORE_ARGS 1,-ARG_SKIP,0

which expands to

     467:       48 8b 4c 24 50          mov    0x50(%rsp),%rcx
     46c:       4c 8b 1c 24             mov    (%rsp),%r11
     470:       4c 8b 54 24 08          mov    0x8(%rsp),%r10
     475:       4c 8b 4c 24 10          mov    0x10(%rsp),%r9
     47a:       4c 8b 44 24 18          mov    0x18(%rsp),%r8
     47f:       48 8b 44 24 20          mov    0x20(%rsp),%rax
     484:       48 8b 54 24 30          mov    0x30(%rsp),%rdx
     489:       48 8b 74 24 38          mov    0x38(%rsp),%rsi
     48e:       48 8b 7c 24 40          mov    0x40(%rsp),%rdi
     493:       65 48 8b 24 25 00 00    mov    %gs:0x0,%rsp
     49a:       00 00 
     49c:       0f 01 f8                swapgs 
     49f:       48 0f 07                sysretq 

Until I miss something obvious (which is very possible), we should not have r11 corruption here.
Could you please send me pie/restorer.built-in.o which was failing?
Actually better both one which failed and one with your patch applied.