[CRIU] [PATCH] aarch64/pie/util-vdso: workaround for vdso_symbols relocation

[Christopher Covington]

Thanks Dmitry!

On 05/10/2016 01:45 PM, Dmitry Safonov wrote:
> This seems to be known problem in util-vdso.c on aarch64 [1].
> 
> Now restorer segfaults with the following log:
> [ 8107.384817] criu[5135]: unhandled level 3 translation fault (11) at 0x00005b98, esr 0x90000007
> [ 8107.385538] pgd = ffffffc038dbc000
> [ 8107.386046] [00005b98] *pgd=0000000078d6c003, *pud=0000000078d6c003, *pmd=0000000073c31003, *pte=0000000000000000
> [ 8107.391920]
> [ 8107.392521] CPU: 0 PID: 5135 Comm: criu Not tainted 4.5.0 #29
> [ 8107.392805] Hardware name: linux,dummy-virt (DT)
> [ 8107.393140] task: ffffffc039a2a400 ti: ffffffc033c34000 task.ti: ffffffc033c34000
> [ 8107.393782] PC is at 0x13514
> [ 8107.406332] LR is at 0x1342c
> [ 8107.406550] pc : [<0000000000013514>] lr : [<000000000001342c>] pstate: a0000000
> 
> This is because gcc for aarch64 adds vdso_symbols array to symbols
> table and by that reason, it needs run-time relocations in place.
> 
> How it goes (with cut not interesting assembly):
>   0x14104:	adrp	x7, 0x14000	; adrp+add loading of
>   0x14114:	add	x0, x7, #0x928	; symbol table's address,
>   0x14134:	ldp	x2, x3, [x0]	; loading address of symbol from
> 					; symbol table
>   0x1414c:	stp	x2, x3, [x29,#112] ; saving it on stack on
> 					; function's begin
>   0x14188:	ldr	x2, [x29,#112]	; using symbol's address in code
> 
> The symbol for this in symbol table is:
>   [root at alarm cr]# readelf -s criu/pie/restorer.built-in.bin.o | grep 5b98
>     248: 0000000000005b98     0 NOTYPE  LOCAL  DEFAULT    1 $d
> And also may be visible this way:
>   objdump -dS criu/pie/restorer.built-in.bin.o | less
>   ...
>   0000000000004924 <cur_loglevel>:
>       4924:       00000002 00005b98 00000000 00005ed0     .....[.......^..
>       4934:       00000000 00005ee8 00000000 00005f00     .....^......._..
> 
> So, in a symbol table lies not relocated address of symbol.
> The real address may be visible with added printing of vdso_symbols[0]:
> pie: vdso: vdso_symbols[0] 0x15b98
> (this way gcc by some reason does access symbol through
> local adrp+add calculations, than through stack-saved pointer
> of a symbol from symbol table).
> 
> While we don't handling properly relocs here, I suggest this
> ugly workaround.
>
> Temporary fix for: #150

I wonder if the best permanent fix would be to GCC.

> [1]: https://lists.openvz.org/pipermail/criu/2015-October/022453.html
> 
> Cc: Wang Long <long.wanglong at huawei.com>
> Cc: Christopher Covington <cov at codeaurora.org>
> Cc: Cyrill Gorcunov <gorcunov at openvz.org>
> Signed-off-by: Dmitry Safonov <dsafonov at virtuozzo.com>
> ---
> NOTE: I test it on qemu-system-aarch64 v2.5.0 with gcc version 5.3.0 (GCC),
>       but before applying, I want to have tested-by from Wang Long - that
>       this workaround fixed his problem.
> 
>  criu/arch/aarch64/include/asm/vdso.h | 18 ++++++++++++++----
>  1 file changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/criu/arch/aarch64/include/asm/vdso.h b/criu/arch/aarch64/include/asm/vdso.h
> index d015c63877e8..26dcf8b8885c 100644
> --- a/criu/arch/aarch64/include/asm/vdso.h
> +++ b/criu/arch/aarch64/include/asm/vdso.h
> @@ -10,11 +10,21 @@
>   */
>  #define VDSO_SYMBOL_MAX		4
>  
> +/*
> + * Workaround for VDSO array symbol table's relocation.
> + * XXX: remove when compel/piegen will support aarch64.
> + */
> +static const char* __maybe_unused aarch_vdso_symbol1 = "__kernel_clock_getres";
> +static const char* __maybe_unused aarch_vdso_symbol2 = "__kernel_clock_gettime";
> +static const char* __maybe_unused aarch_vdso_symbol3 = "__kernel_gettimeofday";
> +static const char* __maybe_unused aarch_vdso_symbol4 = "__kernel_rt_sigreturn";
> +
>  #define ARCH_VDSO_SYMBOLS			\
> -	"__kernel_clock_getres",		\
> -	"__kernel_clock_gettime",		\
> -	"__kernel_gettimeofday",		\
> -	"__kernel_rt_sigreturn"
> +	aarch_vdso_symbol1,			\
> +	aarch_vdso_symbol2,			\
> +	aarch_vdso_symbol3,			\
> +	aarch_vdso_symbol4
> +
>  
>  struct vdso_symtable;
>  extern int vdso_redirect_calls(unsigned long base_to,
> 

Reviewed-by: Christopher Covington <cov at codeaurora.org>

-- 
Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project