[CRIU] Dealing with VDSO remap

[Laurent Dufour]

On 09/03/2015 14:39, Cyrill Gorcunov wrote:
> On Mon, Mar 09, 2015 at 02:32:09PM +0100, Laurent Dufour wrote:
>>>
>>> Hi Laurent. As far as I remember this indeed a problem on arm64, and
>>> there should be some CONFIG_ option for that (but I have a little in
>>> my memory what exactly workaround was). Thus the real fix is somehow
>>> tune the kernel itself but looks like nothing was implemented in this
>>> area yet :/ So any help would be appreciated.
>>
>> Hi Cyrill,
>>
>> I didn't find any CONFIG_ option relative to that. I should have missed it.
> 
> I could be messing it with something else, so don't pay attention on this moment.
> 
>> Anyway, I agree that some stuff is needed in the kernel to deal with
>> that. The big step will now to find the best way to achieve that. I'll
>> try to be helpful there... even if that mm part of the kernel looks
>> tricky to me ;)
> 
> Look, I'm not relly knowning person in internal vdso structure for ARM/ARM64,
> but if the problem is only that on start the kernel provides own _unmovable_
> (ie not pie compiled code, unlike x86 where the vdso is in pie format) then
> indeed some kernel tuning is need. On x86 we do the following trick
> 
>  - remember current vdso the criu has (we name it rt-vdso)
>  - read dumped vdso from image file
>  - if vdso in image is different in compare to rt-vdso
>    - we simply move rt-vdso into some new place (remapping it)
>    - map original vdso from image into original position and
>      patch it so that every function in this image vdso would
>      jump execution to rt-vdso. we call this process vdso proxy,
>      maybe something similar needed on arm

Hi Cyrill,

Thanks for the details.
VDSO is quite similar on PPC, it's a real pie piece of code which is
mapped on each process at a randomized address. As I guess it is on x86,
this not a real map but a page sharing.
Moving the VDSO is not breaking it and if the VDSO is the same between
restarted and checkpointed process I guess we can safely move it.

I did a *ugly* hack in the kernel's remap code to update the
reference the kernel keep to the process's VDSO base address when it is
moved and restart of a simple process tree is fine. So that seems to be
the way to do that.

Now the next step will be to get such a feature accepted upstream in the
kernel. I'll send a RFC to the dedicated kernel list soon. Hoping this
will not be a too long story.

Regarding the VDSO trampoline used when the VDSO is different at restart
compared to the checkpointed one, I had to study it further.
If you have any tips there, you're welcome ;-)

Laurent.