yuqei/jemalloc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
6841110bd6ed17b32a5fed90c53c64555366a792
jemalloc/include/jemalloc/internal
T
History
Kevin Svetlitski 6841110bd6 Make edata_cmp_summary_comp 30% faster 
`edata_cmp_summary_comp` is one of the very hottest functions, taking up
3% of all time spent inside Jemalloc. I noticed that all existing
callsites rely only on the sign of the value returned by this function,
so I came up with this equivalent branchless implementation which
preserves this property. After empirical measurement, I have found that
this implementation is 30% faster, therefore representing a 1% speed-up
to the allocator as a whole.

At @interwq's suggestion, I've applied the same optimization to
`edata_esnead_comp` in case this function becomes hotter in the future.
2023-05-04 09:59:17 -07:00
..
activity_callback.h
Add experimental.thread.activity_callback.
2020-11-05 12:33:25 -08:00
arena_externs.h
Add arena-level name.
2022-09-16 15:04:59 -07:00
arena_inlines_a.h
Cache an arena's index in the arena.
2021-02-04 14:10:43 -08:00
arena_inlines_b.h
Disallow decay during reentrancy.
2023-04-05 10:16:37 -07:00
arena_stats.h
Add stats for stashed bytes in tcache.
2021-12-29 14:44:43 -08:00
arena_structs.h
Add arena-level name.
2022-09-16 15:04:59 -07:00
arena_types.h
Add arena-level name.
2022-09-16 15:04:59 -07:00
assert.h
…
atomic_c11.h
…
atomic_gcc_atomic.h
…
atomic_gcc_sync.h
…
atomic_msvc.h
…
atomic.h
atomic: add atomic_load_sub_store
2020-07-09 13:41:04 -07:00
background_thread_externs.h
Remove opt_background_thread_hpa_interval_max_ms
2021-09-17 16:56:41 -07:00
background_thread_inlines.h
Allow PAI to calculate time until deferred work
2021-09-17 16:56:41 -07:00
background_thread_structs.h
Wake up background threads on demand
2021-09-17 16:56:41 -07:00
base.h
Fix base_ehooks_get_for_metadata
2021-12-20 15:37:53 -08:00
bin_info.h
…
bin_stats.h
…
bin_types.h
…
bin.h
Bin: Move stats closer to the mutex.
2021-02-04 14:10:43 -08:00
bit_util.h
bit_util: Guarantee popcount's presence.
2020-12-07 06:21:08 -08:00
bitmap.h
bit_util: Change ffs functions indexing.
2020-07-30 15:25:23 -07:00
buf_writer.h
Unify write callback signature
2020-05-11 14:51:24 -07:00
cache_bin.h
Some nits in cache_bin.h
2023-05-01 10:21:17 -07:00
ckh.h
…
counter.h
Migrate counter to use locked int
2020-05-12 08:23:15 -07:00
ctl.h
Optimize stats print using partial name-to-mib
2020-12-18 10:39:58 -08:00
decay.h
Allow PAI to calculate time until deferred work
2021-09-17 16:56:41 -07:00
div.h
…
ecache.h
San: Rename 'guard' to 'san'
2021-12-15 10:39:17 -08:00
edata_cache.h
edata_cache: Allow unbounded fast caching.
2021-07-26 15:14:37 -07:00
edata.h
Make edata_cmp_summary_comp 30% faster
2023-05-04 09:59:17 -07:00
ehooks.h
Implement guard pages.
2021-09-26 16:30:15 -07:00
emap.h
San: Bump alloc frequently reused guarded allocations
2021-12-15 10:39:17 -08:00
emitter.h
Unify write callback signature
2020-05-11 14:51:24 -07:00
eset.h
Eset: Cache summary information for heap edatas.
2021-08-02 15:02:49 -07:00
exp_grow.h
Rename geom_grow -> exp_grow.
2020-11-13 13:42:33 -08:00
extent_dss.h
…
extent_mmap.h
…
extent.h
fix some typos
2022-04-25 11:29:00 -07:00
fb.h
Clean up some minor data structure inconsistencies
2021-05-12 11:14:23 -07:00
fxp.h
fxp: Add FXP_INIT_PERCENT
2021-02-04 20:58:31 -08:00
hash.h
…
hook.h
Implement pvalloc replacement
2022-05-18 17:01:09 -07:00
hpa_hooks.h
Fix the time-since computation in HPA.
2021-12-21 23:37:22 -08:00
hpa_opts.h
HPA: Add min_purge_interval_ms option
2021-09-17 16:56:41 -07:00
hpa.h
Remove unused mutex from hpa_central
2023-03-10 11:25:47 -08:00
hpdata.h
HPA: Correctly calculate retained pages
2021-08-20 18:06:17 -07:00
inspect.h
…
jemalloc_internal_decls.h
OpenBSD build fix. still no cpu affinity.
2022-07-19 13:20:11 -07:00
jemalloc_internal_defs.h.in
Fix the rdtscp detection bug and add prefix for the macro.
2023-03-23 11:16:19 -07:00
jemalloc_internal_externs.h
Inline free and sdallocx into operator delete
2022-11-21 11:14:05 -08:00
jemalloc_internal_includes.h
fix some typos
2022-04-25 11:29:00 -07:00
jemalloc_internal_inlines_a.h
Fix the rdtscp detection bug and add prefix for the macro.
2023-03-23 11:16:19 -07:00
jemalloc_internal_inlines_b.h
Simplify arena_migrate() to take arena_t* instead of indices.
2022-01-11 16:59:22 -08:00
jemalloc_internal_inlines_c.h
Inline free and sdallocx into operator delete
2022-11-21 11:14:05 -08:00
jemalloc_internal_macros.h
…
jemalloc_internal_types.h
jemalloc_internal_types.h: Use alloca if __STDC_NO_VLA__ is defined
2022-10-14 15:48:32 -07:00
jemalloc_preamble.h.in
enabled percpu_arena settings on macOs.
2022-07-19 13:23:08 -07:00
large_externs.h
Record request size in prof info
2021-01-07 20:39:49 -08:00
lockedint.h
Migrate counter to use locked int
2020-05-12 08:23:15 -07:00
log.h
…
malloc_io.h
Unify write callback signature
2020-05-11 14:51:24 -07:00
mpsc_queue.h
mpsc_queue: Add module.
2021-06-24 14:55:49 -07:00
mutex_prof.h
Add prof stats mutex stats
2021-01-07 20:39:49 -08:00
mutex.h
Remove locked flag set in malloc_mutex_trylock
2023-04-06 10:57:04 -07:00
nstime.h
Fix shadowed variable usage.
2021-12-23 10:55:08 -08:00
pa.h
fix some typos
2022-04-25 11:29:00 -07:00
pac.h
San: Bump alloc frequently reused guarded allocations
2021-12-15 10:39:17 -08:00
pages.h
Implement PAGE_FLOOR macro
2022-03-22 17:45:55 -07:00
pai.h
Pass 'frequent_reuse' hint to PAI
2021-12-15 10:39:17 -08:00
peak_event.h
Add thread.peak.[read|reset] mallctls.
2020-06-11 13:54:22 -07:00
peak.h
Add peak_t, for tracking allocator net max.
2020-06-11 13:54:22 -07:00
ph.h
Simplify the logic in ph_remove
2023-03-31 14:35:31 -07:00
private_namespace.sh
…
private_symbols.sh
…
prng.h
PRNG: Remove atomic functionality.
2020-10-05 19:55:57 -07:00
prof_data.h
Inline the storage for thread name in prof_tdata_t.
2023-04-05 10:03:12 -07:00
prof_externs.h
Implement prof sample hooks "experimental.hooks.prof_sample(_free)".
2022-12-07 16:06:49 -08:00
prof_hook.h
Implement prof sample hooks "experimental.hooks.prof_sample(_free)".
2022-12-07 16:06:49 -08:00
prof_inlines.h
Inline the storage for thread name in prof_tdata_t.
2023-04-05 10:03:12 -07:00
prof_log.h
Relocate a few prof utilities to the right modules
2020-06-29 14:27:50 -07:00
prof_recent.h
Correct usize in prof last-N record
2020-09-09 13:31:35 -07:00
prof_stats.h
Track per size class internal fragmentation
2021-01-07 20:39:49 -08:00
prof_structs.h
Rearrange the bools in prof_tdata_t to save some bytes.
2023-04-05 10:03:12 -07:00
prof_sys.h
Allow setting custom backtrace hook
2021-09-22 15:04:01 -07:00
prof_types.h
Inline the storage for thread name in prof_tdata_t.
2023-04-05 10:03:12 -07:00
psset.h
psset: Purge empty slabs first.
2021-07-12 17:59:18 -07:00
public_namespace.sh
…
public_unnamespace.sh
…
ql.h
QL, QR: Add documentation.
2020-04-11 10:32:11 -07:00
qr.h
QL, QR: Add documentation.
2020-04-11 10:32:11 -07:00
quantum.h
jemalloc: fix PowerPC definitions in quantum.h
2022-05-26 10:51:10 -07:00
rb.h
Red-black tree: add summarize/filter.
2021-05-12 11:14:23 -07:00
rtree_tsd.h
Optimize away the tsd_fast() check on free fastpath.
2021-10-28 10:05:59 -07:00
rtree.h
Optimize away a branch on the free fastpath.
2021-10-28 16:55:54 -07:00
safety_check.h
Add double free detection in thread cache for debug build
2022-08-04 16:58:22 -07:00
san_bump.h
San: Bump alloc frequently reused guarded allocations
2021-12-15 10:39:17 -08:00
san.h
Fix unnecessary returns in san_(un)guard_pages_two_sided.
2022-01-04 13:55:06 -08:00
sc.h
fix some typos
2022-04-25 11:29:00 -07:00
sec_opts.h
Do not disable SEC by default for 64k pages platforms
2022-03-24 22:05:35 -07:00
sec.h
SEC: Allow arbitrarily many shards, cached sizes.
2021-05-22 08:17:41 -07:00
seq.h
…
slab_data.h
…
smoothstep.h
…
smoothstep.sh
…
spin.h
…
stats.h
HPA: Add stats for the hpa_shard.
2020-10-23 11:14:34 -07:00
sz.h
Add comments and use meaningful vars in sz_psz2ind.
2022-03-24 16:56:59 -07:00
tcache_externs.h
Implement use-after-free detection using junk and stash.
2021-12-29 14:44:43 -08:00
tcache_inlines.h
Implement use-after-free detection using junk and stash.
2021-12-29 14:44:43 -08:00
tcache_structs.h
Add a hard limit on tcache max size class.
2020-10-16 13:49:51 -07:00
tcache_types.h
Add a hard limit on tcache max size class.
2020-10-16 13:49:51 -07:00
test_hooks.h
Redefine functions with test hooks only for tests
2021-10-15 15:25:36 -07:00
thread_event.h
Optimize away the tsd_fast() check on free fastpath.
2021-10-28 10:05:59 -07:00
ticker.h
Disallow decay during reentrancy.
2023-04-05 10:16:37 -07:00
tsd_generic.h
Fix recursive malloc during bootstrap on QNX
2020-12-18 10:05:59 -08:00
tsd_malloc_thread_cleanup.h
Fix FreeBSD system jemalloc TSD cleanup
2022-03-02 10:10:27 -08:00
tsd_tls.h
…
tsd_types.h
Fix FreeBSD system jemalloc TSD cleanup
2022-03-02 10:10:27 -08:00
tsd_win.h
Fix FreeBSD system jemalloc TSD cleanup
2022-03-02 10:10:27 -08:00
tsd.h
Enable fast thread locals for dealloc-only threads.
2022-10-25 09:54:38 -07:00
typed_list.h
Add empty test and concat for typed list
2020-07-31 09:16:50 -07:00
util.h
Tcache flush: prefetch edata contents.
2021-02-04 14:10:43 -08:00
witness.h
San: Implement bump alloc
2021-12-15 10:39:17 -08:00