[CRIU] [PATCH][RFC] zdtm: Move towards the new generation of criu testing

[Pavel Emelyanov]

On 10/01/2015 03:59 PM, Christopher Covington wrote:
> Hi Pavel,
> 
> On 10/01/2015 08:11 AM, Pavel Emelyanov wrote:
>> Hi.
>>
>> After recent set with fault-injection it seemed to me that the
>> existing 1.2KLOC shell script would be quite hard to extend and
>> maintain in the future. I propose to bring our test system onto
>> the next level and write it on a more suitable language :)
> 
> Great idea.

BTW, won't you have issues on your ARM boxes? IIRC you had some
strange ones even with shell-based zdtm, will python be OK for 
your setups?

>> So, things to get fixed in the first place
>>
>> 1. Introduce per-test extendable description. Right now this
>>    description is scatered over test itself, .opts file, scripts
>>    and the zdtm.sh itself (list forming and random hacks).
>>
>>    In new approach all the tests' metadata is in the zdtm.list
>>    file in yaml format.
> 
> So you've reduced the scattering from test, .opts, scripts, and zdtm.sh (4) to
> test and zdtm.list (2). Do you think it's useful to have the information in
> two places (the redundancy could help catch errors for example)? Why not have
> all the information in the test?

Yes, yes, suggestions are welcome :) I'm not 100% happy with zdtm.list I 
currently have.

> For example:
> 
> A) Build the list by recursively finding executables. 

Tried that :) First of all, we don't run everything that executes :) E.g.
the zdtm/ dir contains some tests that we know don't work, but they're
still compile. Second, some tests out there can be scripts.

> This can take care of
> the arch flag as there is no point building x86-specific tests for ARM and
> Power (and some of them won't build even if you wanted to).
> 
> B) Determine the flags, flavor, opts, etc. by parsing the output of `$test
> --help`.

And this is what I was thinking. Executing every binary twice heavily slows 
things down :(

-- Pavel