This has not bitten us yet, but it was a ticking timebomb.
This is similar to the issue that was found with irq_lock/irq_unlock
implementations on several architectures. Having a volatile variable is
not the way to force the sched_lock variable to be
incremented/decremented around the accesses to data it protects.
Instead, a compiler barrier must prevent the compiler from reordering
the memory accesses around setting of sched_lock. Needed in the inline
implementations _sched_lock()/_sched_unlock_no_reschedule(), which
resolve to simple decrement/increment of the per-thread sched_lock
variable.
Change-Id: I06f5b3524889f193efe69caa947118404b1be0b5
Signed-off-by: Benjamin Walsh <walsh.benj@gmail.com>
Replace the existing Apache 2.0 boilerplate header with an SPDX tag
throughout the zephyr code tree. This patch was generated via a
script run over the master branch.
Also updated doc/porting/application.rst that had a dependency on
line numbers in a literal include.
Manually updated subsys/logging/sys_log.c that had a malformed
header in the original file. Also cleanup several cases that already
had a SPDX tag and we either got a duplicate or missed updating.
Jira: ZEP-1457
Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The idle priority was not accounted for.
With this change, the philosophers demo runs in coop-only mode.
Change-Id: I23db33687bcf3b2107d5fc07977143730f62e476
Signed-off-by: Benjamin Walsh <walsh.benj@gmail.com>
It's calling a function on every iteration, it's more efficient to just
do the logic inline.
Change-Id: I166e377d4ffb3056749fd625cb789173030904ac
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
This will allow for an enhancement when checking if the thread is
preemptible when exiting an interrupt.
Change-Id: If93ccd1916eacb5e02a4d15b259fb74f9800d6f4
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Not needed, since only the thread itself can modifiy its own
sched_locked count.
Change-Id: I3d3d8be548d2b24ca14f51637cc58bda66f8b9ee
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Some tick frequencies lend themselves to optimized conversions from ms
to ticks and vice-versa.
- 1000Hz which does not need any conversion
- 500Hz, 250Hz, 125Hz where the division/multiplication are a straight
shift since they are power-of-two factors of 1000.
In addition, some more generally used values are made to use optimized
conversion equations rather than the generic one that uses 64-bit math,
and often results in calling compiler intrinsics.
These values are: 100Hz, 50Hz, 25Hz, 20Hz, 10Hz, 1Hz (the last one used
in some testing).
Avoiding the 64-bit math intrisics has the additional benefit, in
addition to increased performance, of using a significant lower amount
of stack space: 52 bytes on ARM Cortex-M and 80 bytes on x86.
Change-Id: I080eb338a2637d6b1c6838c119af1a9fa37fe869
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Also remove mentions of unified kernel in various places in the kernel,
samples and documentation.
Change-Id: Ice43bc73badbe7e14bae40fd6f2a302f6528a77d
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
For very constrained systems, like bootloaders.
Only the main thread is available, so a main() function must be
provided. Kernel objects where pending is in play will not behave as
expected, since the main thread cannot pend, it being the only thread in
the system. Usage of objects should be limited to using K_NO_WAIT as the
timeout parameter, effectively polling on the object.
Change-Id: Iae0261daa98bff388dc482797cde69f94e2e95cc
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Some kernel operations, like scheduler locking can be optmized out,
since coop threads lock the scheduler by their very nature. Also, the
interrupt exit path for all architecture does not have to do any
rescheduling, again by the nature of non-preemptible threads.
Change-Id: I270e926df3ce46e11d77270330f2f4b463971763
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Use _INACTIVE instead of hardcoding -1.
_EXPIRED is defined as -2 and will be used for an improvement so that
interrupts are not locked for a non-deterministic amount of time while
handling expired timeouts.
_abort_timeout/_abort_thread_timeout return _INACTIVE instead of -1 if
the timeout has already been disabled.
Change-Id: If99226ff316a62c27b2a2e4e874388c3c44a8aeb
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The way the ready thread cache was implemented caused it to not always
be "hot", i.e. there could be some misses, which happened when the
cached thread was taken out of the ready queue. When that happened, it
was not replaced immediately, since doing so could mean that the
replacement might not run because the flow could be interrupted and
another thread could take its place. This was the more conservative
approach that insured that moving a thread to the cache would never be
wasted.
However, this caused two problems:
1. The cache could not be refilled until another thread context-switched
in, since there was no thread in the cache to compare priorities
against.
2. Interrupt exit code would always have to call into C to find what
thread to run when the current thread was not coop and did not have the
scheduler locked. Furthermore, it was possible for this code path to
encounter a cold cache and then it had to find out what thread to run
the long way.
To fix this, filling the cache is now more aggressive, i.e. the next
thread to put in the cache is found even in the case the current cached
thread is context-switched out. This ensures the interrupt exit code is
much faster on the slow path. In addition, since finding the next thread
to run is now always "get it from the cache", which is a simple fetch
from memory (_kernel.ready_q.cache), there is no need to call the more
complex C code.
On the ARM FRDM K64F board, this improvement is seen:
Before:
1- Measure time to switch from ISR back to interrupted task
switching time is 215 tcs = 1791 nsec
2- Measure time from ISR to executing a different task (rescheduled)
switch time is 315 tcs = 2625 nsec
After:
1- Measure time to switch from ISR back to interrupted task
switching time is 130 tcs = 1083 nsec
2- Measure time from ISR to executing a different task (rescheduled)
switch time is 225 tcs = 1875 nsec
These are the most dramatic improvements, but most of the numbers
generated by the latency_measure test are improved.
Fixes ZEP-1401.
Change-Id: I2eaac147048b1ec71a93bd0a285e743a39533973
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
To guarantee that the compiler does not reorder the execution of
irq_lock() with preceding operations, a volatile qualifier is
placed before the declaration of the ticks variable, which then
ensures that irq_lock() is executed after the tick calculation but
before accessing the ready and timeout queues.
Without the volatile keyword interrupts will be disabled during the
calculation of the ticks, which increases interrupt latency
significantly.
Change-Id: I2da82a1282e344f3b8d69e9457b36a4cb1d9ec18
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The fact that a thread is timing out was tracked via two flags: the
K_TIMING thread flag bit, and the thread's timeout's
delta_ticks_from_prev being -1 or not. This duplication could
potentially cause discrepancies if the two flags got out-of-sync, and
there was no benfits to having both.
Since timeouts that are not parts of a thread rely on the value of
delta_ticks_from_prev, standardize on it.
Since the K_TIMING bit is removed from the thread's flags, K_READY would
not reflect the reality anymore. It is removed and replaced by
_is_thread_prevented_froM_running(), which looks at the state flags that
are relevant. A thread that is ready now is not prevented from running
and does not have an active timeout.
Change-Id: I902ef9fb7801b00626df491f5108971817750daa
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
In addition to more priorities taking more memory to host them, finding
the next thread to run when it is not cached is slower since each extra
set of 32 priorities maps to a loop iteration. That loop is remove
entirely when the number of priorities is less than 32 (31 + the idle
thread).
Fixes ZEP-1303.
Change-Id: I3205df90d379a0f4456ff1d7f1aaa67ad2cddf15
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
When moving arch-specific thread structure to arch-agnostic, some field
accesses were missed when used in K_DEBUG statements, which are turned
off by default.
Change-Id: Ife0f49b8185a0db468deab73555f7034f20ca3e8
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
There was a lot of duplication between architectures for the definition
of threads and the "nanokernel" guts. These have been consolidated.
Now, a common file kernel/unified/include/kernel_structs.h holds the
common definitions. Architectures provide two files to complement it:
kernel_arch_data.h and kernel_arch_func.h. The first one contains at
least the struct _thread_arch and struct _kernel_arch data structures,
as well as the struct _callee_saved and struct _caller_saved register
layouts. The second file contains anything that needs what is provided
by the common stuff in kernel_structs.h. Those two files are only meant
to be included in kernel_structs.h in very specific locations.
The thread data structure has been separated into three major parts:
common struct _thread_base and struct k_thread, and arch-specific struct
_thread_arch. The first and third ones are included in the second.
The struct s_NANO data structure has been split into two: common struct
_kernel and arch-specific struct _kernel_arch. The latter is included in
the former.
Offsets files have also changed: nano_offsets.h has been renamed
kernel_offsets.h and is still included by the arch-specific offsets.c.
Also, since the thread and kernel data structures are now made of
sub-structures, offsets have to be added to make up the full offset.
Some of these additions have been consolidated in shorter symbols,
available from kernel/unified/include/offsets_short.h, which includes an
arch-specific offsets_arch_short.h. Most of the code include
offsets_short.h now instead of offsets.h.
Change-Id: I084645cb7e6db8db69aeaaf162963fe157045d5a
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Useful for finding out if the current thread is protected against
preemption when using non-preemption to protect data structures.
Change-Id: Ib545a3609af3646ba49eeeb5a2c50dc51af010d4
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Oversight. These functions are used extensively in the kernel guts, but
are also supposed to be an API.
k_sched_lock used to be implemented as a static inline. However, until
the header files are cleaned-up, and everything, including applications
get access to the kernel internal data structures, it must be
implemented as a function. To reduce the cost to the internals of the
kernel, the new internal _sched_lock() contains the same implemetation,
but is inlined.
Change-Id: If2f61d7714f87d81ddbeed69fedd111b8ce01376
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Verify the thread priorities are within the bounds when starting a new
thread and when changing the priority of a thread.
Change-Id: I007b3b249e4b80235b6439cbee44cad2f31973bb
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Ensures that all APIs which accept a timeout value wait for at least
the specified amount of time, and do not time out prematurely.
* The kernel now waits for the next system clock tick to occur before
the timeout interval is considered to have started. (That is, the only
way to ensure a delay of N tick intervals is to wait for N+1 ticks
to occur.)
* Gets rid of ticks -> milliseconds -> ticks conversion in task_sleep()
and fiber_sleep() legacy APIs, since this introduces rounding that
-- coupled with the previous change -- can alter the number of ticks
being requested during the sleep operation.
* Corrects work queue API that was incorrectly shown to use a delay
measured in ticks, rather than milliseconds.
Change-Id: I8b04467237b24fb0364c8f344d872457418c18da
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
Build breaks when enabling CONFIG_NEWLIB_LIBC because it has its own
sched.h file.
This is a bad symptom of a greater issue: the build system passes many
'-I<path>' options to the compiler, and that allows including header
files by simply specifying their names (when located somewhere else than
<zephyr>/include/) and can cause clashes when several files in different
locations have the same name, like in this case.
Fixes ZEP-1062.
Change-Id: I81d1d69ee6669a609cd0c420b1b8f870d17dcb67
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
* Gets rid of k_current_priority_get(). Users can just call
k_thread_priority_get(k_current_get()) instead.
* Declares k_thread_priority_get() in kernel.h, where it
really belongs.
* Removes duplicate declaration of k_thread_priority_set().
Change-Id: I616ae6f2e06c95ecba3b92324186b3fa29162fd1
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
Rename _do_timeout_add to _add_timeout, rename _TIMEOUT_ADD to
_add_thread_timeout to better reflect their functionalities. Have the
latter call the former, remove _do_timeout_add and
_nano_timer_timeout_add.
Change-Id: Ica86bea10d99d72bf78379598a942d277e7002d0
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Rename _do_timeout_abort to _abort_timeout, rename _timeout_abort to
_abort_thread_timeout to better reflect their functionalities. Have the
latter call the former, remove _do_timeout_abort and
_nano_timer_timeout_abort.
Change-Id: I0fea9474b19a2eb47a37489eb06c0d1d56886c9c
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
When adding a thread to the ready queue, it is often known at that time
if the thread added will be the next one to run or not. So, instead of
simply updating the ready queues and the bitmask, also cache what that
thread is, so that when the scheduler is invoked, it can simply fetch it
from there. This is only done if there is a thread in the cache, since
the way the cache is updated is by comparing the priorities of the
thread being added and the cached thread.
When a thread is removed from the ready queue, if it is currently the
cached thread, it is also removed from the cache. The cache is not
updated at this time, since this would be a preemptive fetching that
could be overriden before the newly cached thread would even be
scheduled in.
Finally, when a thread is scheduled in, it now becomes the cached thread
since the fact that it is running means that by definition it was the
next one to run.
Doing this can speed up considerably some context switch times,
especially when a thread is preempted by an interrupt and the same
thread is scheduled when the interrupt exits.
Change-Id: I6dc8391cfca566699bb9b217eafe6bc6a063c8bb
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
When fetching the next thread to run, we know at least one thread is
available.
Change-Id: I568c33a61b6a0a6d6a7f79c337caecffd5ef70b6
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The routine _reschedule_threads() does not need to check that the
scheduler is locked as this is done as part of _must_switch_threads().
Change-Id: I701b811512836eef591b2adcd708991ec9324b3d
Signed-off-by: Peter Mitsis <peter.mitsis@windriver.com>
The routine _is_next_thread_current() checks if the current thread
is still the highest priority thread that is ready to run.
It is useful for determining if a thread must be swapped out when
timeslicing is in effect.
Change-Id: Ide7b89742a64f6082ca4c679a4b2fbd60792e30f
Signed-off-by: Peter Mitsis <peter.mitsis@windriver.com>
Summary of what this includes:
initialization:
Copy from nano_init.c, with the following changes:
- the main thread is the continuation of the init thread, but an idle
thread is created as well
- _main() initializes threads in groups and starts the EXE group
- the ready queues are initialized
- the main thread is marked as non-essential once the system init is
done
- a weak main() symbol is provided if the application does not provide a
main() function
scheduler:
Not an exhaustive list, but basically provide primitives for:
- adding/removing a thread to/from a wait queue
- adding/removing a thread to/from the ready queue
- marking thread as ready
- locking/unlocking the scheduler
- instead of locking interrupts
- getting/setting thread priority
- checking what state (coop/preempt) a thread is currenlty running in
- rescheduling threads
- finding what thread is the next to run
- yielding/sleeping/aborting sleep
- finding the current thread
threads:
- Add operationns on threads, such as creating and starting them.
standardized handling of kernel object return codes:
- Kernel objects now cause _Swap() to return the following values:
0 => operation successful
-EAGAIN => operation timed out
-Exxxxx => operation failed for another reason
- The thread's swap_data field can be used to return any additional
information required to complete the operation, such as the actual
result of a successful operation.
timeouts:
- same as nano timeouts, renamed to simply 'timeouts'
- the kernel is still tick-based, but objects take timeout values in
ms for forward compatibility with a tickless kernel.
semaphores:
- Port of the nanokernel semaphores, which have the same basic behaviour
as the microkernel ones. Semaphore groups are not yet implemented.
- These semaphores are enhanced in that they accept an initial count and a
count limit. This allows configuring them as binary semaphores, and also
provisioning them without having to "give" the semaphore multiple times
before using them.
mutexes:
- Straight port of the microkernel mutexes. An init function is added to
allow defining them at runtime.
pipes:
- straight port
timers:
- amalgamation of nano and micro timers, with all functionalities
intact.
events:
- re-implementation, using semaphores and workqueues.
mailboxes:
- straight port
message queues:
- straight port of microkernel FIFOs
memory maps:
- straight port
workqueues:
- Basically, have all APIs follow the k_ naming rule, and use the _timeout
subsystem from the unified kernel directory, and not the _nano_timeout
one.
stacks:
- Port of the nanokernel stacks. They can now have multiple threads
pending on them and threads can wait with a timeout.
LIFOs:
- Straight port of the nanokernel LIFOs.
FIFOs:
- Straight port of the nanokernel FIFOs.
Work by: Dmitriy Korovkin <dmitriy.korovkin@windriver.com>
Peter Mitsis <peter.mitsis@windriver.com>
Allan Stephens <allan.stephens@windriver.com>
Benjamin Walsh <benjamin.walsh@windriver.com>
Change-Id: Id3cadb3694484ab2ca467889cfb029be3cd3a7d6
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Benjamin Walsh
David B. Kinder
Benjamin Walsh
Anas Nashif
Carles Cufi
Peter Mitsis
Allan Stephens