Commit Graph

3603 Commits

Author SHA1 Message Date
Slobodan Predolac 88b29da00a [EASY][BUGFIX] Spelling and format 2025-08-23 21:47:21 -07:00
lexprfuncall f890bbed4a Define malloc_{write,read}_fd as non-inline global functions
The static inline definition made more sense when these functions just
dispatched to a syscall wrapper.  Since they acquired a retry loop, a
non-inline definition makes more sense.
2025-08-23 11:42:23 -04:00
lexprfuncall 6ced4b5afb Handle interruptions and retries of read(2) and write(2) 2025-08-23 11:42:23 -04:00
lexprfuncall 5f06b14d1c Remove an orphaned comment
This was left behind when definitions of malloc_open and malloc_close
were abstracted from code that had followed.
2025-08-23 11:42:23 -04:00
Shirui Cheng e2da7477f8 Revert PR #2608: Manually revert commits 70c94d..f9c0b5 2025-08-22 21:55:24 -07:00
lexprfuncall 9186700eb3 Fix the compilation check for process madvise
An include of unistd.h is needed to make the declaration of the
syscall function visible to the compiler.  The include of sys/mman.h
is not used at all.
2025-08-21 22:43:02 -07:00
Slobodan Predolac 4f126b47b3 Save and restore errno when calling process_madvise 2025-08-21 16:22:06 -07:00
lexprfuncall acd85e5359 Remove an unused function and global variable
When the dehugify functionality was retired in an previous commit, a
dehugify-related function and global variable in a test was
accidentally left in-place causing builds that add -Werror to CFLAGS
to fail.
2025-08-21 08:59:37 -07:00
Slobodan Predolac f9fae9f1f8 Experimental configuration option for fast path prefetch from cache_bin 2025-08-15 16:06:08 -07:00
lexprfuncall c45b6223e5 Use relaxed atomics to access the process madvise pid fd
Relaxed atomics already provide sequentially consistent access to single
location data structures.
2025-08-13 18:33:27 -07:00
lexprfuncall 16c5abd1cd Do not dehugify when purging
Giving the advice MADV_DONTNEED to a range of virtual memory backed by
a transparent huge page already causes that range of virtual memory to
become backed by regular pages.
2025-08-13 18:31:50 -07:00
lexprfuncall 0d071a086b Fix several spelling errors in comments 2025-08-08 14:12:12 -07:00
Slobodan Predolac 97d25919c3 [process_madvise] Make init lazy so that python tests pass. Reset the pidfd on fork 2025-07-29 15:47:53 -07:00
Slobodan Predolac fb52eac372 Add several USDT probes for hpa 2025-06-26 13:16:04 -07:00
Slobodan Predolac 9dd7b075b6 Add experimental support for usdt systemtap probes 2025-06-26 13:16:04 -07:00
guangli-dai 1c826c924a Ignore the clang-format changes in the git blame. 2025-06-22 09:21:44 -07:00
guangli-dai 34f359e0ca Reformat the codebase with the clang-format 18. 2025-06-20 14:35:15 -07:00
Shirui Cheng ae4e729d15 Update the default value for opt_experimental_tcache_gc and opt_calloc_madvise_threshold 2025-06-17 13:25:20 -07:00
Guangli Dai 95f9f934ca Remove --enable-limit-usize-gap for cirrus CI since the config-time option is removed. 2025-06-16 11:16:40 -07:00
guangli-dai f9e9e0986a Update appveyor settings. 2025-06-16 11:16:40 -07:00
dzhao.ampere d58e311320 test/unit/psset.c: fix SIGSEGV when PAGESIZE is large
When hugepage is enabled and PAGESIZE is large, the test could
ask for a stack size larger than user limit. Allocating the
memory instead can avoid the failure.
2025-06-13 17:08:38 -07:00
Slobodan Predolac 390e70c840 [thread_event] Add support for user events in thread events when stats are enabled 2025-06-11 15:37:03 -07:00
Slobodan Predolac b2a35a905f [thread_event] Remove macros from thread_event and replace with dynamic event objects 2025-06-11 15:37:03 -07:00
Qi Wang 1972241cd2 Remove unused options in the batched madvise unit tests. 2025-06-02 11:25:37 -07:00
Jason Evans 27d7960cf9 Revert "Extend purging algorithm with peak demand tracking"
This reverts commit ad108d50f1.
2025-06-02 10:44:37 -07:00
guangli-dai edaab8b3ad Turn clang-format off for codes with multi-line commands in macros 2025-05-28 19:22:21 -07:00
guangli-dai 4531411abe Modify .clang-format to have declarations aligned 2025-05-28 19:22:21 -07:00
guangli-dai 1818170c8d Fix binshard.sh by specifying bin_shards for all sizes. 2025-05-28 19:21:49 -07:00
guangli-dai fd60645260 Add one more check to double free validation. 2025-05-28 19:21:49 -07:00
Xin Yang 5e460bfea2 Refactor: use the cache_bin_sz_t typedef instead of direct uint16_t
any future changes to the underlying data type for bin sizes
(such as upgrading from `uint16_t` to `uint32_t`) can be achieved
by modifying only the `cache_bin_sz_t` definition.

Signed-off-by: Xin Yang <yangxin.dev@bytedance.com>
2025-05-22 10:43:33 -07:00
Xin Yang 9169e9272a Fix: Adjust CACHE_BIN_NFLUSH_BATCH_MAX size to prevent assert failures
The maximum allowed value for `nflush_batch` is
`CACHE_BIN_NFLUSH_BATCH_MAX`. However, `tcache_bin_flush_impl_small`
could potentially declare an array of `emap_batch_lookup_result_t`
of size `CACHE_BIN_NFLUSH_BATCH_MAX + 1`. leads to a `VARIABLE_ARRAY`
assertion failure, observed when `tcache_nslots_small_max` is
configured to 2048. This patch ensures the array size does not exceed
the allowed maximum.

Signed-off-by: Xin Yang <yangxin.dev@bytedance.com>
2025-05-22 10:27:09 -07:00
guangli-dai f19a569216 Ignore formatting commit in blame. 2025-05-20 14:21:08 -07:00
Slobodan Predolac b6338c4ff6 EASY - be explicit in non-vectorized hpa tests 2025-05-19 16:31:04 -07:00
guangli-dai 554185356b Sample format on tcache_max test 2025-05-19 15:06:13 -07:00
guangli-dai 3cee771cfa Modify .clang-format to make it more aligned with current freebsd style 2025-05-19 15:06:13 -07:00
Jiebin Sun 3c14707b01 To improve reuse efficiency, the maximum coalesced size for large extents
in the dirty ecache has been limited. This patch was tested with real
workloads using ClickHouse (Clickbench Q35) on a system with 2x240 vCPUs.
The results showed a 2X in query per second (QPS) performance and
a reduction in page faults to 29% of the previous rate. Additionally,
microbenchmark testing involved 256 memory reallocations resizing
from 4KB to 16KB in one arena, which demonstrated a 5X performance
improvement.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2025-05-12 15:45:36 -07:00
guangli-dai 37bf846cc3 Fixes to prevent static analysis warnings. 2025-05-06 14:47:35 -07:00
guangli-dai 8347f1045a Renaming limit_usize_gap to disable_large_size_classes 2025-05-06 14:47:35 -07:00
Guangli Dai 01e9ecbeb2 Remove build-time configuration 'config_limit_usize_gap' 2025-05-06 14:47:35 -07:00
Slobodan Predolac 852da1be15 Add experimental option force using SYS_process_madvise 2025-04-28 18:45:30 -07:00
Slobodan Predolac 1956a54a43 [process_madvise] Use process_madvise across multiple huge_pages 2025-04-25 19:19:03 -07:00
Slobodan Predolac 0dfb4a5a1a Add output argument to hpa_purge_begin to count dirty ranges 2025-04-25 19:19:03 -07:00
Slobodan Predolac cfa90dfd80 Refactor hpa purging to prepare for vectorized call across multiple pages 2025-04-25 19:19:03 -07:00
Qi Wang a3910b9802 Avoid forced purging during thread-arena migration when bg thd is on. 2025-04-25 19:18:20 -07:00
guangli-dai c23a6bfdf6 Add opt.limit_usize_gap to stats 2025-04-16 10:38:10 -07:00
guangli-dai c20a63a765 Silence the uninitialized warning from clang. 2025-04-16 10:38:10 -07:00
Qi Wang f81fb92a89 Remove Travis CI macOS configs (not supported anymore). 2025-04-14 15:27:38 -07:00
Slobodan Predolac f19f49ef3e if process_madvise is supported, call it when purging hpa 2025-04-04 13:57:42 -07:00
Kaspar M. Rohrer 80e9001af3 Move `extern "C" specifications for C++ to where they are needed
This should fix errors when compiling C++ code with modules enabled on clang.
2025-03-31 10:41:51 -07:00
Shirui Cheng 3688dfb5c3 fix assertion error in huge_arena_auto_thp_switch() when b0 is deleted in unit test 2025-03-20 12:45:23 -07:00