[Devel] [PATCH 6/8] mm: Fix int overflow in callers of do_shrink_slab()

[Andrey Zhadchenko]

From: Kirill Tkhai <ktkhai at virtuozzo.com>

do_shrink_slab() returns unsigned long value, and
the placing into int variable cuts high bytes off.
Then we compare ret and 0xfffffffe (since SHRINK_EMPTY
is converted to ret type).

Thus, big number of objects returned by do_shrink_slab()
may be interpreted as SHRINK_EMPTY, if low bytes of
their value are equal to 0xfffffffe. Fix that
by declaration ret as unsigned long in these functions.

Reported-by: Cyrill Gorcunov <gorcunov at openvz.org>
Signed-off-by: Kirill Tkhai <ktkhai at virtuozzo.com>
Acked-by: Cyrill Gorcunov <gorcunov at gmail.com>

(cherry-picked from b55ead794a0f2bbca90b0bf138131731ce354699)
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko at virtuozzo.com>
---
 mm/vmscan.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 7dbc3de..863d1c2 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -539,8 +539,8 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
 			struct mem_cgroup *memcg, int priority)
 {
 	struct memcg_shrinker_map *map;
-	unsigned long freed = 0;
-	int ret, i;
+	unsigned long ret, freed = 0;
+	int i;
 
 	if (!memcg_kmem_enabled() || !mem_cgroup_online(memcg))
 		return 0;
@@ -636,9 +636,8 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid,
 				 struct mem_cgroup *memcg,
 				 int priority, bool for_drop_caches)
 {
+	unsigned long ret, freed = 0;
 	struct shrinker *shrinker;
-	unsigned long freed = 0;
-	int ret;
 
 	/*
 	 * The root memcg might be allocated even though memcg is disabled
-- 
1.8.3.1