When sys_heap_realloc() is expressed in terms of sys_heap_aligned_realloc()
it invokes a longer aligned allocation code path with an extra runtime
overhead even though no alignment is necessary.
Let's reference and invoke the aligned allocation code path only when an
actual aligned allocation is requested. This opens the possibility for
the linker to garbage-collect the aligning code otherwise.
Improve realloc documentation while at it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
When k_malloc() is expressed in terms of k_aligned_alloc() it invokes a
longer aligned allocation code path with an extra runtime overhead even
though no alignment is necessary.
Let's reference and invoke the aligned allocation code path only when an
actual aligned allocation is requested. This opens the possibility for
the linker to garbage-collect the aligning code otherwise.
Also bypass k_heap_malloc() and friends given they're invoked with
K_NO_WAIT. Go directly to sys_heap_*() instead to cut some more unneeded
overhead.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
When k_heap_alloc() is expressed in terms of k_heap_aligned_alloc()
it invokes a longer aligned allocation code path with an extra runtime
overhead even though no alignment is necessary.
Let's reference and invoke the aligned allocation code path only when an
actual aligned allocation is requested. This opens the possibility for
the linker to garbage-collect the aligning code otherwise.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
To request heap statistics, a pointer to a heap structure is required.
This is straightforward for a user-defined heap. However, such a pointer
is not known for heaps created by other components or libraries, like
libc. Therefore, it is not possible to calculate the total heap memory.
The proposed solution is to use an array of pointers, which is filled in
on every sys_heap_init() call. One can then iterate through it to sum up
the total memory allocated for all heaps.
The array size is configurable. The default array size is zero,
which means the feature is disabled. Any other integer greater then zero
defines the array size and enables the feature.
A list of pointers instead of an array could be another approach,
but it requeres a heap, which is not always available.
Signed-off-by: Ivan Pankratov <ivan.pankratov@silabs.com>
Use `do { ... } while (false)' instead of `do { ... } while (0)'.
Use comparisons with zero instead of implicitly testing integers.
Use comparisons with NULL instead of implicitly testing pointers.
Use comparisons with NUL instead of implicitly testing plain chars.
Use `bool' instead of `int' to represent Boolean values.
Use `while (true)' instead of `while (1)' to express infinite loops.
Signed-off-by: frei tycho <tfrei@baumer.com>
Disable tests/kernel/mem_protect/syscalls for qemu_arc_em where
we trigger ARC QEMU bug which cause illegal instruction exception
on perfectly valid ARC code.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
This is a workaround for a compiler bug on (at least) GCC 12.1.0 in
Zephyr SDK 0.15.1. The optimizer generates this function with a last
instruction that is an unconditional branch (a tail call into the
chunk_set() handling). But that means that the NEXT instruction gets
decoded as part of the branch delay slot, but that instruction isn't
part of this function! Some instructions aren't legal in branch delay
slots. One of those is ENTER_S, which is a very common entry
instruction for whatever function the linker places after us. It
seems like the compiler doesn't understand this problem. Stuff a NOP
in to guarantee the code is legal.
Comment above is duplicated in the code. The workaround is
straightforward once the issue is understood, but the path to get here
was hilariously weird.
Fixes#54720
Signed-off-by: Andy Ross <andyross@google.com>
Wire this up the same way ASAN works. Right now it's support only by
recent clang versions (not gcc), and only in 64 bit mode. But it's
capable of detecting uninitialized data reads, which ASAN is not.
This support is wired into the sys_heap (and thus k_heap/k_malloc)
layers, allowing detection of heap misuse like use-after-free. Note
that there is one false negative lurking: due to complexity, in the
case where a sys_heap_realloc() call is able to shrink memory in
place, the now-unused suffix is not marked uninitialized immediately,
making it impossible to detect use-after-free of those particular
bytes. But the system will recover cleanly the next time the memory
gets allocated.
Also no attempt was made to integrate this handling into the newlib or
picolibc allocators, though that should hopefully be possible via
similar means.
Signed-off-by: Andy Ross <andyross@google.com>
In order to bring consistency in-tree, migrate all lib code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Besides the current allocated/free bytes, keep track of
the maximum allocated bytes to help determine the heap
size requirements. Also, provide a function to reset
the statistic.
Signed-off-by: Damian Krolik <damian.krolik@nordicsemi.no>
add functions to get the sys_heap runtime statistics,
include total free bytes, total allocated bytes.
Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
The "small" heap is is way sufficient for most 32-bit systems.
Let's provide the option to have only one type of heap allowing for
smaller and faster heap code due to not having a bunch of runtime
conditionals based on the heap size.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add a simple internal block size predicate to expose the internal
memory region reserved for an allocation. The immediate use case is
cache-incoherent systems wanting to do an invalidate of freed memory,
but it might be useful for apps doing e.g. string processing to better
optimize size changes, etc...
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The K_HEAP_DEFINE macro would allow users to specify heaps that are
too small, leading to potential corruption events (though at least
there were __ASSERTs that would catch this at runtime if enabled).
It would be nice to put the logic to compute this value into the heap
code, but that isn't available in kernel.h (and we don't want to pull
it in as this header is already WAY to thick). So instead we just
hand-compute and document the choice. We can address bitrot problems
with a test.
(Tweaks to heap size asserts and correct size bounds from Nicolas Pitre)
Fixes#33009
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
heap* had several places missing final else statement in the
if else if construct. This commit adds else {} to comply with
coding guideline 15.7.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
The lib/os/ had several places missing final else
statement in the if else if construct. This commit adds
else {} or simple refactor to comply with coding guideline 15.7.
- cbprintf_complete.c
- cbprintf_nano.c
- heap-validate.c
- heap.c
- onoff.c
- p4wq.c
- sem.c
Also resolves the checkpatch issue of comments should align * on
each line.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
This symbol is reserved and usage of reserved symbols violates the
coding guidelines. (MISRA 21.2)
NAME
fgetpos, fseek, fsetpos, ftell, rewind - reposition a stream
SYNOPSIS
#include <stdio.h>
void rewind(FILE *stream);
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The size_t usage, especially in struct z_heap_bucket made the heap
header almost 2x bigger than it needs to be on 64-bit systems.
This prompted me to clean up our type usage to make the code more
efficient and easier to understand. From now on:
- chunkid_t is for absolute chunk position measured in chunk units
- chunksz_t is for chunk sizes measured in chunk units
- size_t is for buffer sizes measured in bytes
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The end marker chunk was represented by the len field of struct z_heap.
It is now renamed to end_chunk to make it more obvious what it is.
And while at it...
Given that it is used in size_too_big() to cap the allocation size
already, we no longer need to test the bucket index against the
biggest index possible derived from end_chunk in alloc_chunk(). The
corresponding bucket_idx() call is relatively expensive on some
architectures so avoiding it (turning it into a CHECK() instead) is
a good thing.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The sys_heap_realloc() code falls back to allocating new memory
and copying the existing data over when it cannot adjust the size
in place. However the size of the data to copy should be the old
size and not the new size if we're extending the allocation.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The definition for realloc() says that it should return a pointer
to the allocated memory which is suitably aligned for any built-in
type.
Turn sys_heap_realloc() into a sys_heap_aligned_realloc() and use it
with __alignof__(z_max_align_t) to implement realloc() with proper
memory alignment for any platform.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The strategy used in z_heap_aligned_alloc() was to allocate an extra
align-sized memory block for storing a pointer to the memory heap.
This is wasteful in terms of memory usage when alignment is larger
than a pointer width. A loop is needed to find the initial memory
start when freeing it which isn't optimal either.
Instead, let's have sys_heap_aligned_alloc() rewind a pointer after
it is aligned to make just enough room for storing our heap reference.
This way the heap reference is always located immediately before the
aligned memory and any unused memory is returned to the heap.
The rewind and alignment values may coincide in which case only
the alignment is necessary anyway.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's do it upfront only once for each entry point and dispense
with overflow checks later to keep the code simple.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
First, the maximum heap size must fit in 31 bits worth of chunks
because the internal 32-bit field holding the size is shared with
the `used` bit.
Then the mention of a 256-byte block in the doc is no longer
relevant. That pertained to the previous allocator implementation.
And ditto for the HEAP_MEM_POOL_MIN_SIZE kconfig option.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Just like commit 0ae04f01b6 ("lib/os/heap: make some checks more
assertive") we shouldn't validate the externally provided align
argument only when CONFIG_SYS_HEAP_VALIDATE is set.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
If the new size amounts to the same number of chunks then:
- If right-chunk is used then we needlessly allocate new memory and
copy data over.
- If right-chunk is free then we attempt to split it with a zero size
which corrupts the prev/next list.
Make sure this case is properly handled and add a test for it.
While at it, let's simplify the code somewhat as well.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add an optimized realloc() implementation that can successfully expand
allocations in place if there exists enough free memory after the
supplied block.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The biggest required padding is equal to `align - chunk_header_bytes`
and not `align - 1` given that the header already contributes to the
padding.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The code that made aligned_alloc work with the 4-byte heap headers was
requesting a block of the correctly padded size, and correctly
aligning the output buffer within that memory, but it was using the
UNALIGNED chunk size for the buffer as the final size of the block
with splitting off the unused suffix. So the final chunk in the
buffer was could be incorrectly returned to the heap and reused,
leading to overlap.
Compute the chunk size of the output buffer based on the
already-aligned output pointer instead.
Initial investigation and fix from Andy Ross <andrew.j.ross@intel.com>.
I reworked his fix, created a test case, and stolen his commit log.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.
Changes are related to converting the integer constants to the
unsigned integer constants
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
MISRA-C Rule 5.3 states that identifiers in inner scope should
not hide identifiers in outer scope.
In the function sys_heap_alloc(), the variable "chunksz"
collide with function named chunksz(). So rename those variable.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
After commit 8a6b02b5bf ("lib/os/heap: some code simplification in
sys_heap_aligned_alloc()") it is no longer required to have a "big"
heap for aligned allocations to work on 32-bit targets. While the
natural alignment for returned memory has an offset of 4 within a chunk
unit due to the smaller header size, returning to a chunkid from a
memory pointer with an offset of 8 will fall back onto the proper chunk
number once the 4 is substracted and then divided by 8.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The code is doing a split in split_alloc(), adding the leftover to the
free list, then splitting the suffix away in sys_heap_aligned_alloc(),
removing the former leftover from the free list, combining it with the
suffix and finally adding the combined chunk back to the free list.
Instead, let's have each allocator do their own splitting only once by
moving the split_alloc() processing upstream rather than downstream.
This also allows for the "used" flag to be set only once at the end
rather than being overwritten along the way.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Instead of limiting the excess split-off to sufficiently large chunks
in split_alloc(), let's allow normal allocations to create "solo free
headers" just like with aligned allocations. There is no point leaving
them in the allocated chunk if the user didn't ask for it. Doing so
makes them eligible for merging at the next opportunity and potentially
reusable sooner.
Also make the validation code aware of them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This makes the code cleaner wrt bucket_idx() usage on chunks for which
solo_free_header() is true. In such case the bucket_idx() computation
is useless, and potentially undefined anyway.
In the same vain, move the clearing of the used flag out of
free_chunks() as only one of its callers actually needs that.
Makes free_chunks singular as there is only one chunk (potentially
spanning multiple chunk units) to free.
Also some cosmetic changes for better code uniformity.
No functional changes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Some checks in sys_heap_init() depend on the externally provided size
parameter. If the check fails, this would be a bug outside of the heap
code and therefore should be flagged despite the value of
CONFIG_SYS_HEAP_VALIDATE.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add support for a C11-style aligned_alloc() in the heap
implementation. This is properly optimized, in the sense that unused
prefix/suffix data around the chosen allocation is returned to the
heap and made available for general allocation.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Miscellaneous refactoring and simplification. No behavioral changes:
Make split_alloc() take and return chunk IDs and not memory pointers,
leaving the conversion between memory/chunks the job of the higher
level sys_heap_alloc() API. This cleans up the internals for code
that wants to do allocation but has its own ideas about what to do
with the resulting chunks.
Add split_chunks() and merge_chunks() utilities to own the linear/size
pointers and have split_alloc() and free_chunks() use them instead of
doing the list management directly.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This struct is taking up most of the heap's constant footprint overhead.
We can easily get rid of the list_size member as it is mostly used to
determine if the list is empty, and that can be determined through
other means.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Nicolas Pitre
Ivan Pankratov
Hess Nathan
frei tycho
Anas Nashif
Evgeniy Paltsev
Andy Ross
Gerard Marull-Paretas
Damian Krolik
Daniel Leung
Chen Peng1
Andy Ross
Jennifer Williams
Aastha Grover