Andrew Bresticker c1e2ee2dc4
memcg: replace ss->id_lock with a rwlock ...
While back-porting Johannes Weiner's patch "mm: memcg-aware global
reclaim" for an internal effort, we noticed a significant performance
regression during page-reclaim heavy workloads due to high contention of
the ss->id_lock. This lock protects idr map, and serializes calls to
idr_get_next() in css_get_next() (which is used during the memcg hierarchy
walk).
Since idr_get_next() is just doing a look up, we need only serialize it
with respect to idr_remove()/idr_get_new(). By making the ss->id_lock a
rwlock, contention is greatly reduced and performance improves.
Tested: cat a 256m file from a ramdisk in a 128m container 50 times on
each core (one file + container per core) in parallel on a NUMA machine.
Result is the time for the test to complete in 1 of the containers.
Both kernels included Johannes' memcg-aware global reclaim patches.
Before rwlock patch: 1710.778s
After rwlock patch: 152.227s
Signed-off-by: Andrew Bresticker <abrestic@google.com >
Cc: Paul Menage <menage@gmail.com >
Cc: Li Zefan <lizf@cn.fujitsu.com >
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com >
Cc: Ying Han <yinghan@google.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2011-11-02 16:07:03 -07:00
2011-10-26 23:15:28 +02:00
2011-09-19 16:10:11 -04:00
2011-08-26 12:02:50 -04:00
2011-10-17 19:22:46 -04:00
2011-10-28 16:42:18 -07:00
2011-09-03 19:07:21 -03:00
2011-10-11 20:56:41 -07:00
2011-10-18 21:26:55 -07:00
2011-11-01 15:06:20 -07:00
2011-11-01 10:51:38 -07:00
2011-10-27 09:11:34 -04:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-09-13 22:44:10 -04:00
2011-10-20 11:41:49 +02:00
2011-10-31 17:30:55 -07:00
2011-11-01 15:06:20 -07:00
2011-09-11 09:17:53 +08:00
2011-10-31 17:30:54 -07:00
2011-09-13 15:42:30 -04:00
2011-10-25 15:44:06 +02:00
2011-10-31 15:42:54 -07:00
2011-08-03 21:53:27 -10:00
2011-11-02 16:07:03 -07:00
2011-08-10 14:55:29 -07:00
2011-08-26 12:02:50 -04:00
2011-10-26 15:43:26 -04:00
2011-10-26 15:43:25 -04:00
2011-10-31 17:30:54 -07:00
2011-08-26 12:02:50 -04:00
2011-09-14 18:09:38 -07:00
2011-08-03 11:30:42 -04:00
2011-08-23 14:50:29 +02:00
2011-08-24 16:04:34 +02:00
2011-10-31 17:30:54 -07:00
2011-08-09 01:33:04 -07:00
2011-07-31 22:05:09 +02:00
2011-10-17 19:22:46 -04:00
2011-10-20 18:21:36 -04:00
2011-11-02 16:07:03 -07:00
2011-09-27 09:25:02 +01:00
2011-10-13 14:36:58 +01:00
2011-10-03 09:34:16 +02:00
2011-09-08 11:10:56 +02:00
2011-09-28 13:41:50 -04:00
2011-10-31 17:30:49 -07:00
2011-10-31 17:30:44 -07:00
2011-08-25 16:25:33 -07:00
2011-09-23 12:05:29 +05:30
2011-08-03 19:06:37 -04:00
2011-10-30 15:16:41 +01:00
2011-08-08 11:33:23 -07:00
2011-10-21 14:24:01 +02:00
2011-08-06 22:12:37 -07:00
2011-10-21 14:24:07 +02:00
2011-08-06 22:52:40 -07:00
2011-10-02 00:19:34 +02:00
2011-09-25 23:26:21 +01:00
2011-10-31 17:30:54 -07:00
2011-08-02 12:32:06 +01:00
2011-08-02 12:32:04 +01:00
2011-09-21 10:22:03 +02:00
2011-11-02 16:07:02 -07:00
2011-10-31 17:30:44 -07:00
2011-09-15 14:02:57 +02:00
2011-10-04 13:08:18 -07:00
2011-10-31 17:30:54 -07:00
2011-08-01 13:40:51 -10:00
2011-10-30 15:16:43 +01:00
2011-09-16 19:25:10 -04:00
2011-09-14 15:24:51 -04:00
2011-10-31 17:30:54 -07:00
2011-08-03 14:25:20 -10:00
2011-10-19 19:35:51 -04:00
2011-10-28 10:43:32 -07:00
2011-10-31 17:30:54 -07:00
2011-10-31 17:30:54 -07:00
2011-10-05 01:10:12 +00:00
2011-08-08 16:08:08 +02:00
2011-10-31 17:30:54 -07:00
2011-08-03 11:15:57 -04:00
2011-08-29 00:16:19 -07:00
2011-08-03 14:25:21 -10:00
2011-10-24 16:04:06 +02:00
2011-08-17 13:00:20 -07:00
2011-10-25 09:59:04 +02:00
2011-10-31 17:30:48 -07:00
2011-10-11 09:51:22 -06:00
2011-10-29 09:37:09 +01:00
2011-10-30 13:47:25 +01:00
2011-08-03 14:25:20 -10:00
2011-09-30 15:57:07 -04:00
2011-10-11 15:35:42 -04:00
2011-10-16 13:15:38 -07:00
2011-10-21 02:53:07 -04:00
2011-10-03 14:03:48 -04:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-10-30 04:43:30 -04:00
2011-08-17 20:21:27 -07:00
2011-08-26 12:02:50 -04:00
2011-10-12 19:09:18 -04:00
2011-09-13 11:11:55 +02:00
2011-10-28 14:25:01 -07:00
2011-09-14 15:24:52 -04:00
2011-10-26 16:17:32 +02:00
2011-10-17 11:42:49 +02:00
2011-10-28 05:54:23 -07:00
2011-10-21 14:38:55 +02:00
2011-07-31 14:30:59 -10:00
2011-09-20 14:50:00 -04:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-10-04 12:43:49 +02:00
2011-10-03 15:35:26 +02:00
2011-10-03 15:35:26 +02:00
2011-10-31 14:03:23 +01:00
2011-09-27 18:08:04 +02:00
2011-09-21 10:28:51 +02:00
2011-10-31 17:30:54 -07:00
2011-10-31 17:30:54 -07:00
2011-10-31 17:30:57 -07:00
2011-10-31 17:46:07 -07:00
2011-10-31 17:30:54 -07:00
2011-10-31 17:30:54 -07:00
2011-09-13 11:11:45 +02:00
2011-10-31 17:30:54 -07:00
2011-09-25 19:52:59 +03:00
2011-10-30 12:24:05 +02:00
2011-08-26 12:02:50 -04:00
2011-09-16 19:20:20 -04:00
2011-10-31 17:30:54 -07:00
2011-10-31 17:30:52 -07:00
2011-08-26 12:02:50 -04:00
2011-10-31 17:30:57 -07:00
2011-09-28 21:36:37 -07:00
2011-07-31 22:08:04 +02:00
2011-11-02 16:06:59 -07:00
2011-10-31 17:30:47 -07:00
2011-11-02 16:07:00 -07:00
2011-09-15 15:36:34 -04:00
2011-07-31 22:08:04 +02:00
2011-11-02 16:06:57 -07:00
2011-11-02 16:06:58 -07:00
2011-10-31 17:30:54 -07:00
2011-10-31 17:30:47 -07:00
2011-08-25 10:25:37 -07:00
2011-08-10 20:38:14 -04:00
2011-10-26 13:10:39 +10:30
2011-09-27 08:12:33 -07:00
2011-10-19 17:00:35 -04:00
2011-10-31 17:30:54 -07:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-08-26 12:02:50 -04:00
2011-11-01 09:24:41 -07:00
2011-08-26 12:02:50 -04:00
2011-09-20 14:43:49 -04:00
2011-10-19 11:52:12 -04:00
2011-07-31 14:27:04 -10:00
2011-10-18 09:14:34 -07:00
2011-10-19 13:58:38 -07:00
2011-10-19 13:37:34 -07:00
2011-07-31 12:18:16 -04:00
2011-10-14 14:48:23 -04:00
2011-08-04 11:26:24 +01:00
2011-10-31 14:03:22 +01:00
2011-11-01 20:58:25 -07:00
2011-08-29 12:38:51 -03:00
2011-10-31 17:30:45 -07:00
2011-09-30 22:35:12 +02:00
2011-09-13 11:12:05 +02:00
2011-07-30 08:21:48 -10:00
2011-10-14 09:05:27 -07:00
2011-10-14 09:05:35 -07:00
2011-10-14 09:05:35 -07:00
2011-10-28 14:20:44 -07:00
2011-09-13 11:11:47 +02:00
2011-10-06 13:00:28 +02:00
2011-08-25 10:17:28 -07:00
2011-08-26 12:02:50 -04:00
2011-10-24 02:54:50 -04:00
2011-10-29 07:28:36 -07:00
2011-08-25 15:34:19 +02:00
2011-09-26 20:22:02 +02:00
2011-10-04 21:54:26 +02:00
2011-08-25 15:34:19 +02:00
2011-10-22 00:21:52 +02:00
2011-08-03 00:58:42 -04:00
2011-08-10 14:55:26 -07:00
2011-11-02 16:07:02 -07:00
2011-10-31 17:30:54 -07:00
2011-09-13 11:11:50 +02:00
2011-10-12 09:17:24 -07:00
2011-09-29 00:32:03 -04:00
2011-08-25 16:25:34 -07:00
2011-10-31 17:30:54 -07:00
2011-08-03 14:25:24 -10:00
2011-10-28 05:29:07 -07:00
2011-09-13 11:11:54 +02:00
2011-09-28 21:38:35 -07:00
2011-09-28 21:38:20 -07:00
2011-09-28 21:36:42 -07:00
2011-09-30 13:57:47 +01:00
2011-10-03 15:19:19 -04:00
2011-08-30 12:27:45 -04:00
2011-11-02 16:07:01 -07:00
2011-08-25 16:25:34 -07:00
2011-08-26 12:02:50 -04:00
2011-09-13 11:11:59 +02:00
2011-09-13 11:11:59 +02:00
2011-10-26 17:08:43 +02:00
2011-09-09 16:56:33 -07:00
2011-11-02 16:07:01 -07:00
2011-09-13 11:11:57 +02:00
2011-10-31 17:30:54 -07:00
2011-08-24 15:25:49 -07:00
2011-09-22 15:50:38 -07:00
2011-09-22 15:50:38 -07:00
2011-08-24 15:27:59 -07:00
2011-09-28 10:24:14 -07:00
2011-09-28 13:41:51 -04:00
2011-07-30 08:44:19 -10:00
2011-08-03 14:25:24 -10:00
2011-10-31 17:30:49 -07:00
2011-10-24 18:18:09 -04:00
2011-09-27 23:03:30 +03:00
2011-09-15 14:49:43 -04:00
2011-08-07 22:48:07 -07:00
2011-10-31 17:30:47 -07:00
2011-08-16 00:16:49 -07:00
2011-10-16 23:28:51 +02:00
2011-10-31 17:30:44 -07:00
2011-08-03 14:25:22 -10:00
2011-10-31 17:30:44 -07:00
2011-11-02 16:07:02 -07:00
2011-10-19 19:24:14 -04:00
2011-10-03 14:01:21 -04:00
2011-08-02 14:51:57 -04:00
2011-08-22 14:13:32 -07:00
2011-08-26 12:02:50 -04:00
2011-09-19 11:35:58 -04:00
2011-10-31 17:30:54 -07:00
2011-08-10 20:38:14 -04:00
2011-08-23 10:10:38 -07:00
2011-10-26 15:11:09 +02:00
2011-09-28 21:36:38 -07:00
2011-10-12 21:13:11 -07:00
2011-10-18 11:18:57 -07:00
2011-08-26 12:02:50 -04:00
2011-09-26 15:51:10 -07:00
2011-11-02 16:07:02 -07:00
2011-10-08 08:01:06 -03:00
2011-10-24 02:07:21 -04:00
2011-10-31 17:30:47 -07:00
2011-08-19 22:42:07 +08:00
2011-08-26 12:02:50 -04:00
2011-09-14 15:24:51 -04:00
c1e2ee2dc4