[CRIU] [PATCH] zdtm: print a summary of failed tests if using --keep-going

[Adrian Reber]

On Mon, Nov 28, 2016 at 10:50:36PM -0800, Andrei Vagin wrote:
> On Tue, Nov 29, 2016 at 07:20:18AM +0100, Adrian Reber wrote:
> > On Mon, Nov 28, 2016 at 11:40:40AM -0800, Andrei Vagin wrote:
> > > On Mon, Nov 28, 2016 at 05:26:08PM +0100, Adrian Reber wrote:
> > > > From: Adrian Reber <areber at redhat.com>
> > > > 
> > > > When using the option --keep-going to get a quick overview of the test
> > > > results there is now a summary which tests have failed like:
> > > > 
> > > >  ############# 7 TEST(S) FAILED (TOTAL 268/SKIPPED 83) ##############
> > > >   * zdtm/static/sched_policy00
> > > >   * zdtm/static/pthread00
> > > >   * zdtm/static/pthread01
> > > >   * zdtm/static/maps01
> > > >   * zdtm/static/maps00
> > > >   * zdtm/transition/maps008
> > > >   * zdtm/transition/shmem
> > > >  ############################### FAIL ###############################
> > > >
> > > 
> > > 
> > > 
> > > > Signed-off-by: Adrian Reber <areber at redhat.com>
> > > > ---
> > > >  test/zdtm.py | 9 +++++++++
> > > >  1 file changed, 9 insertions(+)
> > > > 
> > > > diff --git a/test/zdtm.py b/test/zdtm.py
> > > > index 9d682dd..8ccb9a8 100755
> > > > --- a/test/zdtm.py
> > > > +++ b/test/zdtm.py
> > > > @@ -1325,6 +1325,8 @@ class launcher:
> > > >  		self.__subs = {}
> > > >  		self.__fail = False
> > > >  		self.__file_report = None
> > > > +		self.__failed = []
> > > > +		self.__nr_skip = 0
> > > >  		if self.__max > 1 and self.__total > 1:
> > > >  			self.__use_log = True
> > > >  		elif opts['report']:
> > > > @@ -1355,6 +1357,7 @@ class launcher:
> > > >  		print "Skipping %s (%s)" % (name, reason)
> > > >  		self.__nr += 1
> > > >  		self.__runtest += 1
> > > > +		self.__nr_skip += 1
> > > >  		if self.__file_report:
> > > >  			testline = "ok %d - %s # SKIP %s" % (self.__runtest, name, reason)
> > > >  			print >> self.__file_report, testline
> > > > @@ -1398,6 +1401,7 @@ class launcher:
> > > >  			if status != 0:
> > > >  				self.__fail = True
> > > >  				failed_flavor = decode_flav(os.WEXITSTATUS(status))
> > > > +				self.__failed.append({sub['name']: failed_flavor})
> > > 
> > > I would like to print results for tests in the same order how they
> > > were executed. I saw many time when the first fail was a reason for
> > > other ones.
> > 
> > As far as I have seen it, the results all over always printed in the
> > order they were executed. I am appending to a list so the order should
> > never change.
> 
> You are right, I saw a dict and thought that the order will be not
> saved. In such cases usually a tuple is used.

Right, the dict was from an earlier version. New patch without a dict
on its way.

		Adrian