Add vendor hooks to reclaim MADV_PAGEOUT memory for asynchrnous
device. It also exports reclaim_pages to reclaim memory.
Bug: 326662423
Change-Id: Ic2516c64a9dbd53173a3bfb19b6cd21636916c27
Signed-off-by: Minchan Kim <minchan@google.com>
(cherry picked from commit 1c0a654f8baeac62ce37e63503a8beb71981dba2)
Signed-off-by: Dmitry Skiba <dskiba@google.com>
Add vendor hook for compaction begin/end. The first use would be
to measure compaction durations.
Bug: 229927848
Test: echo 1 > /proc/sys/vm/compact_memory and observe output change in
/sys/kernel/pixel_stat/mm/compaction/mm_compaction_duration
Signed-off-by: Robin Hsu <robinhsu@google.com>
Change-Id: I3d95434bf49b37199056dc9ddfc36a59a7de17b7
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 13b6bd38bb1f43bfffdb08c8f3a4a20d36ccd670)
Signed-off-by: liangjlee <liangjlee@google.com>
(cherry picked from commit 33b9d9889640fa6825e31e3026f619d66bc7f017)
Signed-off-by: Dmitry Skiba <dskiba@google.com>
To report vendor-specific memory statistics, add restricted
vendor hook since normal vendor hook work with only atomic
context.
Bug: 333482947
Change-Id: I5c32961b30f082a8a4aa78906d2fce1cdf4b0d2b
Signed-off-by: Minchan Kim <minchan@google.com>
(cherry picked from commit 7ea01ee68e6867798309cf0d3044a695b7d56947)
Signed-off-by: Dmitry Skiba <dskiba@google.com>
Add a vendor hook for pagecache hit/miss and other
vendor specific functions.
Bug: 174088128
Bug: 172987241
Signed-off-by: Chiawei Wang <chiaweiwang@google.com>
Change-Id: Ie9f14a69a86b8ed81de766e44e30f2eba1d9bd84
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit db158b4ae0543446d38313c3da942afee9947267)
Signed-off-by: Jack Lee <liangjlee@google.com>
(cherry picked from commit cbac5cbf85540c043d1ddb917f288ebfe62bbca7)
Signed-off-by: Dmitry Skiba <dskiba@google.com>
Add a vendor hook for costly order page counting
and other vendor specific functions.
Bug: 174521902
Bug: 172987241
Signed-off-by: Chiawei Wang <chiaweiwang@google.com>
Change-Id: I89206727a462548cc3500b695d85c83ff003eec7
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 369de3780428a17e9afece2f5747f03619d589b6)
Signed-off-by: liangjlee <liangjlee@google.com>
(cherry picked from commit 86a45bb2dc315fd0b1d0b513b0f2e1be82d25f70)
Signed-off-by: Dmitry Skiba <dskiba@google.com>
Add hooks at process exiting routines so that oems
can control these procedures. One possible benifit is the peak of
system load can be shaved and load can be more smooth when a large
number of threads is killed once upon a time, while a sudden peak
of system load can probably lead to user junk issues.
Bug: 296493318
Change-Id: Ied4cc968e45baa39f1b79e8c7dce75cf6b8db55d
Signed-off-by: Qixia Yang <yangqixia2@oppo.com>
As QTVMs use Gunyah emulated watchdog instead of vcpu_stall detector,
provide trace_hooks to manage the watchdog of the QTVM.
Bug: 399219478
Change-Id: I6ec8a0236835dcc20c66681a747eae09c0c1a32d
Signed-off-by: Prakruthi Deepak Heragu <quic_pheragu@quicinc.com>
The binder.impl parameter currently controls whether C Binder is loaded
or not, even if no Rust Binder driver is present. This patch changes it
so that C Binder always gets loaded even if binder.impl=rust is set. To
actually load a different Binder driver, it must explicitly ask C Binder
to unload itself.
This ensures that:
1. If binder.impl=rust is set on a device without Rust Binder available,
it will boot normally with C Binder instead of failing to boot.
2. By not listing unload_binder in the symbol list, we ensure that
Binder can only be loaded as a GKI module, not as a vendor module.
Bug: 388786466
Change-Id: I791154e8f10abd5c470bb697b5e1035c33af897f
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
since we can't control all kvmalloc_node callsite's gfp_flags, we add
a vendor hook in kvmalloc_node to tune the reclaim behavior for some
really high-order allocation
Bug: 300857012
Bug: 395197442
Change-Id: I5f0c4c2921d204289911704e3a205f6a1dc50d04
Signed-off-by: liwei <liwei1234@oppo.com>
Signed-off-by: wengle <wengle@oppo.com>
Introduce vendor hooks for best fit mechanism of
CMA allocation.
Bug: 379357530
Change-Id: I8149b0ba1c69d3482e726b6ff5227e8344aed6aa
Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com>
For direct memory reclaim, aging can be skipped, and threads such as
kswapd can be allowed to execute, reducing the time spent on direct
memory reclaim.
Why do we need to reseve 6 u64s OEM data?
The two u64s parameters are used to tell kswapd how to age this lrugen.
The other four u64s are used for two list heads to store some special
pages.
The size of 'struct lru_gen_folio' only affects the size of
'struct lruvec', the size of 'struct mem_cgroup_per_node' and
the size of 'struct pglist_data'.
The influence of OEM data:
sizeof(struct lru_gen_folio) is changed from 960 to 1008.
sizeof(struct lruvec) is changed from 1176 to 1224.
sizeof(struct mem_cgroup_per_node) is changed from 1472 to 1536.
sizeof(struct pglist_data) is changed from 7424 to 7488.
The 'struct mem_cgroup_per_node' is alloc for in
alloc_mem_cgroup_per_node_info() using kzalloc_node(). while
the size of 'struct mem_cgroup_per_node' is 1472Byte. Due to
alignment reasons, the memory size allocate is 2KB. Adding 6 u64s
will not increase the alloc size.
Bug: 402316914
Change-Id: Ib9667e83a8227a36e3adc63e84d73a6b529000a8
Signed-off-by: Qianfeng Rong <rongqianfeng@vivo.corp-partner.google.com>
[cmllamas: switch can_swap to swappiness]
Signed-off-by: Carlos Llamas <cmllamas@google.com>
The more memcg groups there are, the longer a single traversal takes,
resulting in poor consistency in memory recycling time.
It is possible to interrupt memcg traversal recycling to speed up
memory application for some important processes.
Bug: 402041892
Change-Id: I8b9a0cce754cb07babe0ce234fbf262b20f9d020
Signed-off-by: Qianfeng Rong <rongqianfeng@vivo.corp-partner.google.com>
Add hooks and additional fields in vm_struct and track structs
to store and report additional information for slab memory leak
debugging.
Bug: 184928480
Bug: 395197442
Change-Id: I6897a6a98d4eaaea492673cefd4111a7ba741940
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Signed-off-by: wengle <wengle@oppo.com>
Transmit the track action type to save_track_hash, otherwise we do not
know it is allocation stack or free stack.
Fixes: 8bc6337823c6 ("ANDROID: vendor_hooks: add hooks for slab memory leak debugging")
Bug: 184928480
Bug: 395197442
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Change-Id: I74c50c02cfb4ebbf3e9fecdf125e76946ff4e7d1
(cherry picked from commit 7d216e2512082853f4cf0c220a1b48119203144f)
Signed-off-by: huzhanyuan <huzhanyuan@oppo.com>
Signed-off-by: wengle <wengle@oppo.com>
In some situations, we want to decrease readaround size for better
performance. So we add this hook.
Bug: 288216516
Bug: 395197442
Change-Id: If2f5f75976c99ff1f82ce29d370f9216926055ab
Signed-off-by: Oven <liyangouwen1@oppo.com>
Signed-off-by: huzhanyuan <huzhanyuan@oppo.com>
Signed-off-by: wengle <wengle@oppo.com>
To monitor the reclaiming ability of kswapd, add vendor hook recording when the kswapd finish the reclaiming job and the reclaim progress.
android_vh_vmscan_kswpad_done(int, unsigned int, unsigned int, unsigned int)
Bug: 301044280
Bug: 330661608
Bug: 395197442
Change-Id: Id6e0a97003f0a156cff4d0996bc38bcd89b1dc69
Signed-off-by: John Hsu <john.hsu@mediatek.com>
Signed-off-by: wengle <wengle@oppo.com>
Provide a vendor hook to allow drain_all_pages to be skipped
during direct reclaim in some cases to avoid delays caused by
it in cases when the benefits of draining pcp lists are known
to be small.
Bug: 220811627
Bug: 234405962
Bug: 395197442
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Change-Id: I0805241f81e0a94afcf62c98e97cff125d4061e2
Signed-off-by: wengle <wengle@oppo.com>
We add these hooks to avoid key threads blocked in memory allocation
path.
-android_vh_free_unref_page_bypass ----We create a memory pool for the
key threads. This hook determines whether a page should be free to the
pool or to buddy freelist. It works with a existing hook
`android_vh_alloc_pages_reclaim_bypass`, which takes pages out of the
pool.
-android_vh_kvmalloc_node_use_vmalloc ----For key threads, we perfer
not to run into direct reclaim. So we clear __GFP_DIRECT_RECLAIM flag.
For threads which are not that important, we perfer use vmalloc.
-android_vh_should_alloc_pages_retry ----Before key threads run into
direct reclaim, we want to retry with a lower watermark.
-android_vh_unreserve_highatomic_bypass ----We want to keep more
highatomic pages when unreserve them to avoid highatomic allocation
failures.
-android_vh_rmqueue_bulk_bypass ----We found sometimes when key threads
run into rmqueue_bulk, it took several milliseconds spinning at
zone->lock or filling per-cpu pages. We use this hook to take pages from
the mempool mentioned above, rather than grab zone->lock and fill a
batch of pages to per-cpu.
Bug: 288216516
Bug: 395197442
Change-Id: I1656032d6819ca627723341987b6094775bc345f
Signed-off-by: Oven <liyangouwen1@oppo.com>
Signed-off-by: huzhanyuan <huzhanyuan@oppo.com>
Signed-off-by: wengle <wengle@oppo.com>
For short-life or high-IO-freq files we marked, the writeback tasks
will act on an additional writeback dispatching list, achieving
hierarchical control of dirty writeback without affecting other original
writeback tasks.
Bug: 400883625
Change-Id: I7973cedeb15896d2cd2224443bf857111b2adac7
Signed-off-by: Bo Lan <lanbo@honor.com>
Add hooks to capture various per-zone memory stats when
a trigger threshold is hit.
In the stability Monkey4SystemLongTime test, the camera startup will
request a large amount of memory in a short period of time,
resulting in a temporary shortage of system memory. In addition,
other processes also need to continuously request memory.In this
case, it is easy for the system to have lmkd blocked and unable to
kill the process in time, resulting in OOM.
This hook can be used to detect when lmkd is in D state and the
memory pressure meets the set conditions, then the kill process
mechanism is triggered to avoid the occurrence of OOM. In the
previous test, 1/20 machines had a problem at 74.49 hours,but after
the modification, 48 machines were tested for 120 hours without
any problems.
Bug: 268290366
Change-Id: Ibe39263ddb05ffc3fa63b5225497a90c6480c8d7
Signed-off-by: Georgi Djakov <quic_c_gdjako@quicinc.com>
Signed-off-by: xiaofeng <xiaofeng5@xiaomi.com>
Add hook for binder_alloc_new_buf_locked, which gives the vendor a
choice whether or not this alloc new buf should go on or not.
Bug: 322915513
Change-Id: Ie7f1567dae6dee0fb22b014cd1eacc3308ccb205
Signed-off-by: chenweitao <chenweitao@oppo.com>
As trace_android_vh_si_meminfo_adjust allows vendors to change for
totalram and freeram, we'd like to change sharedram values. This is
helpful when a driver uses shmem a lot.
Bug: 371132125
Change-Id: Ifd615e1a4d600084f3957267da3189f0edc342e7
Signed-off-by: Jaewon Kim <jaewon31.kim@samsung.com>
Add some vendor hooks for shmem memory stats. This help us track the
number of shmem pages which are belong to the mapping we're interested
in. If shmem pages are swapped out, we can track that number, too.
Bug: 372282099
Change-Id: I0a18a690f37719400faa4a01b1d5d09e979f09c7
Signed-off-by: Jaewon Kim <jaewon31.kim@samsung.com>
Add vendor hooks in shmem folio allocation paths to tune the higher
order folios.
With vendor hook possibly overriding shmem_allowable_huge_orders()
result, `order` inside shmem_get_folio_gfp() becomes variable when
when CONFIG_TRANSPARENT_HUGEPAGE=n and the block that calls
vma_thp_gfp_mask() is not compiled out anymore, leading to the
following linker error:
ld.lld: error: undefined symbol: vma_thp_gfp_mask
>>> referenced by shmem.c
Fix this by making the block dependent on CONFIG_TRANSPARENT_HUGEPAGE.
Bug: 397599006
Change-Id: Ia0e126c674a9f08cb10fe5ee06f5efac7b5cbe7f
Signed-off-by: Charan Teja Kalla <quic_charante@quicinc.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
add two vendor hooks:
android_vh_page_should_be_protected():protect pages from memory
reclaim.
android_vh_page_referenced_check_bypass():bypass rmap in active list
shrink.
The new vendor data field in scan_control are used to track how many
pages are protected in current reclaim and the "protected / scanned"
rate. These parameters are useful for understanding the impact of page
protection operations on LRU and reclaim, helping us make better
decsions.
Bug: 399782297
Change-Id: I49567a4b1f978821a94da0a8339b2b8fdfd52daf
Signed-off-by: Yuxuan Yan <yanyuxuan3@xiaomi.corp-partner.google.com>
When servicemanager process added service proxy from other process
register the service, we want to know the matching relation between
handle in the process and service name. When binder transaction
happened, We want to know what process calls what method on what service.
Patch 3482670 was uploaded in this patchset to avoid conflict.
Bug: 186604985
Bug: 395737475
Signed-off-by: zhengding chen <chenzhengding@oppo.com>
Change-Id: I813d1cde10294d8665f899f7fef0d444ec1f1f5e
Signed-off-by: shenshen mao <maoshenshen@oppo.com>
Signed-off-by: xiaosa liang <liangxiaosa@oppo.com>
binder_proc->fg_todo list
1. The implementation logic of this feature
1) In trace_android_vh_binder_list_add_work, record the sequence of work, work->android_oem_data1 = atomic64_inc_return(&binder_work_seq).
2) In the binder_proc_transaction() function, for special synchronized
binders, if a thread is not found, t->work will be added to proc->fg_todo in trace_android_vh_binder_special_task().
3) binder_thread_read() -> trace_android_vh_binder_select_special_worklist()
will select work to execute from proc->fg_todo according to a specific
algorithm, such as selecting the thread->todo first, then selecting two
proc->fg_todo works, and then one proc->todo work, while taking into account the sequence of work (work->android_oem_data1).
proc->ANDROID_OEM_DATA(1) is used to generate proc->fg_todo list, and it
was committed in the id 8a3740893d9957783493364b118282daa71695cc by other colleagues.
2. trace_android_vh_binder_list_add_work: record the sequence of work.
3. trace_android_vh_binder_has_proc_work_ilocked: Check if there is work
in proc->fg_todo list, and if so, then binder_wakeup_proc_ilocked.
4. trace_android_vh_binder_check_special_work: release the work on
proc->fg_todo list, and print the binder work on fg_list.
Bug: 334746237
Bug: 395737475
Change-Id: I855618ee129fe1bb5925b169d24eb2301ad90664
Signed-off-by: Fuchun Liao <lfc@oppo.com>
1.android_vh_tcp_rcv_rtt_update
To record the rcv rtt and its update time.
We use "srtt_us" in tcp_sock struct to judge the network quality.
But sometimes, an uplink request is sent, and downlink data is received for a long time.
During this period, the "srtt_us" is not updated, but the "rcv_rtt_est" may be updated.
So in this case, we may choose to use rcv rtt instead of rtt to judge the network quality.
Bug: 398691197
Change-Id: I92b26b4302ff7ea5cbabaa9bb3bb71dacfe702db
Signed-off-by: Chenghao Zhao <zhaochenghao@honor.com>
Through this vendor hook, we can get the timing to check
current running task for the validation of its credential
and nanosleep operations. through this we can filter some
app tasks which may fire the nanosleep events storm.
Bug: 396244637
Bug: 341618050
Change-Id: I644ac3d217930aa0a50966996e8001e27ce8a501
Signed-off-by: litao <tao.li@vivo.corp-partner.google.com>
(cherry picked from commit 1d9ed15534d4988da3ac56dd7892d37bf5f96847)
We want to use this hook to record the sleeping time due to Futex
Bug: 210947226
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Change-Id: I637f889dce42937116d10979e0c40fddf96cd1a2
(cherry picked from commit a7ab784f601a93a78c1c22cd0aacc2af64d8e3c8)
Signed-off-by: maoshenshen <maoshenshen@oppo.com>
Add some hooks in the binder module so that we can do task dependency analysis and statistical work in OEM's module for further optimization.
Bug: 235925535
Bug: 281920779
Signed-off-by: Zhuguangqing <zhuguangqing@xiaomi.com>
Change-Id: Id47e59c4e3ccd07b26eef758ada147b98cd1964e
Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Recently we have discovered many lag issues caused by percpu_rwsem
lock-holding tasks not being scheduled for a long time. we need to
identify them and provide appropriate scheduling protection in our
oem scheduler.
To support this, we add one hook below:
trace_android_vh_percpu_rwsem_wq_add
Bug: 301066838
Change-Id: Id770c1a7978842abfc62d3fa9aeb5ac7a1904972
Signed-off-by: xieliujie <xieliujie@oppo.com>
Add hook to show the more information about cpu, when cat /proc/cpuinfo
, some chip info will be appended.
Bug: 341597636
Change-Id: I4dbf7ba305a428f3dcf71e71d700a51dac8f7be0
Signed-off-by: Dezhi Huang <huangdezhi@hihonor.com>
(cherry picked from commit 08c63cde8c2bbdd5ab44c4f644422764f9b9ad14)
add vendor hooks to support link data path feature.
1.android_vh_dc_send_copy
For send copy pkt on the other ways
2.android_vh_dc_receive
For Drop duplicated pkt when dual connection connected
Bug: 343838581
Change-Id: I31acba1827205c366a4f54c4b940d4d8f2c21bff
Signed-off-by: Dezhi Huang <huangdezhi@hihonor.com>
(cherry picked from commit 86dd89554f8308fcd58af6156fd67ac2ac7facbf)
(cherry picked from commit f949397a1b86bdb461a14176db4a7cdc6cc63cbc)
Add vendor hook inside of get_page_from_freelist() to check
and modify the watermark in some special situations.
Additional page flag bit will be set for future identification.
Separately, a vendor hook inside of page_add_new_anon_rmap()
is added to set the referenced bit in some situations, e.g.
if the special bit in the page flag mentioned before is set,
we will give this page one more chance before it gets reclaimed.
Bug: 279793368
Bug: 336900791
Change-Id: I363853a050a87201f6f368ccc580485dddd6c6b6
Signed-off-by: Dezhi Huang <huangdezhi@hihonor.com>
Signed-off-by: huzhanyuan <huzhanyuan@oppo.com>
(cherry picked from commit 49fa9c5a7e11922caa132a0d014b46b7a72f10d3)
Add a vendor hook for start/end of page_cache_readahead and filemap_fault.
Bug: 351175506
Change-Id: Ie00ff34d48db7ef85520576af8986c95d841112e
Signed-off-by: Sooyong Suk <s.suk@samsung.corp-partner.google.com>
Add new vendor hook to get pages with gfp from vendor implemented pool.
Vendor hook trace_android_rvh_try_alloc_pages() was previously added via
https://android-review.googlesource.com/c/kernel/common/+/3184382.
But there were missing gfp_t in the commit, which is needed for
page allocation. So add a new vendor hook with gfp.
Bug: 351175506
Change-Id: I64b382b8d6d528d58146f32ebabe2883234f2653
Signed-off-by: Sooyong Suk <s.suk@samsung.corp-partner.google.com>
Add a vendor hook to let vendor module to take pages on certain
conditions to use later instead of freeing pages directly.
Bug: 351175506
Change-Id: I067ebdd2721dcc1fe95e9ed40d1bbeddf64ac327
Signed-off-by: Sooyong Suk <s.suk@samsung.corp-partner.google.com>
Add a vendor hook to bypass free_pages_prepare for those pages that have
never been used by user after free_pages_prepare. This type of pages can
be created by vendor module implementations.
Bug: 351175506
Change-Id: I5bebc47fe1be410c78924dd5392260ad0dadaa22
Signed-off-by: Sooyong Suk <s.suk@samsung.corp-partner.google.com>
Minchan Kim
Robin Hsu
Chiawei Wang
Qixia Yang
Prakruthi Deepak Heragu
Alice Ryhl
liwei
Bibek Kumar Patro
Qianfeng Rong
Liujie Xie
xiaofeng
Oven
liang zhang
JohnHsu
wudean
Bo Lan
Jiewen Wang
chenweitao
Jaewon Kim
Charan Teja Kalla
yipeng xiang
Yuxuan Yan
zhengding chen
Fuchun Liao
Chenghao Zhao
litao
Zhuguangqing
Dezhi Huang
Sooyong Suk
Sooyong Suk