[CRIU] [PATCH] zdtm: print a summary of failed tests if using --keep-going

[Andrei Vagin]

On Mon, Nov 28, 2016 at 05:26:08PM +0100, Adrian Reber wrote:
> From: Adrian Reber <areber at redhat.com>
> 
> When using the option --keep-going to get a quick overview of the test
> results there is now a summary which tests have failed like:
> 
>  ############# 7 TEST(S) FAILED (TOTAL 268/SKIPPED 83) ##############
>   * zdtm/static/sched_policy00
>   * zdtm/static/pthread00
>   * zdtm/static/pthread01
>   * zdtm/static/maps01
>   * zdtm/static/maps00
>   * zdtm/transition/maps008
>   * zdtm/transition/shmem
>  ############################### FAIL ###############################
>



> Signed-off-by: Adrian Reber <areber at redhat.com>
> ---
>  test/zdtm.py | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/test/zdtm.py b/test/zdtm.py
> index 9d682dd..8ccb9a8 100755
> --- a/test/zdtm.py
> +++ b/test/zdtm.py
> @@ -1325,6 +1325,8 @@ class launcher:
>  		self.__subs = {}
>  		self.__fail = False
>  		self.__file_report = None
> +		self.__failed = []
> +		self.__nr_skip = 0
>  		if self.__max > 1 and self.__total > 1:
>  			self.__use_log = True
>  		elif opts['report']:
> @@ -1355,6 +1357,7 @@ class launcher:
>  		print "Skipping %s (%s)" % (name, reason)
>  		self.__nr += 1
>  		self.__runtest += 1
> +		self.__nr_skip += 1
>  		if self.__file_report:
>  			testline = "ok %d - %s # SKIP %s" % (self.__runtest, name, reason)
>  			print >> self.__file_report, testline
> @@ -1398,6 +1401,7 @@ class launcher:
>  			if status != 0:
>  				self.__fail = True
>  				failed_flavor = decode_flav(os.WEXITSTATUS(status))
> +				self.__failed.append({sub['name']: failed_flavor})

I would like to print results for tests in the same order how they
were executed. I saw many time when the first fail was a reason for
other ones.

>  				if self.__file_report:
>  					testline = "not ok %d - %s # flavor %s" % (self.__runtest, sub['name'], failed_flavor)
>  					details = {'output': open(sub['log']).read()}
> @@ -1442,6 +1446,11 @@ class launcher:
>  		if self.__file_report:
>  			self.__file_report.close()
>  		if self.__fail:
> +			if opts['keep_going']:
> +				print_sep("%d TEST(S) FAILED (TOTAL %d/SKIPPED %d)"
> +						% (len(self.__failed), self.__total, self.__nr_skip), "#")
> +				for failed in self.__failed:
> +					print " * %s" % failed.popitem()[0]
>  			print_sep("FAIL", "#")
>  			sys.exit(1)
>  
> -- 
> 2.9.3
> 
> _______________________________________________
> CRIU mailing list
> CRIU at openvz.org
> https://lists.openvz.org/mailman/listinfo/criu