[CRIU] dlopen() and inodes

[Nicolas Viennot]

Hello,

When loading a shared library via a call to dlopen(libpath), glibc and musl compare
the pair device+inode of the file opened at libpath with the one of previously opened libraries.
If the newly opened library is a duplicate of another already loaded library from this comparison,
the old library is instead used. This ensures that a library accessible via multiple paths does
not get loaded twice. Typically the paths are soft links, and rarely hard links.

We checkpoint and restore using CRIU on Kubernetes. Even though we use the same
container image when restoring applications, we find different device+inodes for the
same files compared to the checkpoint environment, leading to issues with dlopen().

We thought of patching glibc to either follow library soft links to canonicalize
paths, or use the DT_SONAME field in the ELF header to uniquely identify a library.
We also thought of interposing fstat() to virtualize devices and inodes, but this
inevitably adds some execution overhead.

Has anyone encountered this issue? Any suggestions? 

Thanks,
Nico