Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Instead of hardcoding alignment size for pass 2 device handles, use
Z_DECL_ALIGN. This makes sure gen_handles.py is always in sync with the
type defined in device.h. The build assert in device.h can be removed as
a result, since we do not hardcode handles size anywhere else.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Replicate the devicetree dependencies into a sorted list. This ensures
that the structures added to the .__device_handles_pass2 section are
reproducible from build to build.
Tested with: west build -b native_posix tests/drivers/build_all/sensor
Without this change, two consecutive builds do not compare.
Signed-off-by: Keith Short <keithshort@google.com>
Output the final dependency graph as a `.dot` file, which when rendered
by graphviz can be easier to comprehend than the text descriptions.
This output is optional in that it will not be generated if `graphviz`
is not installed.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Use the new elf_parser module to simplify the process of generating the
final device handle arrays.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Rename the placeholder variable generated for PM slots so that the
prefix doesn't colide with the PM structs declared by devices. This
simplifies the process of searching for symbols in `.elf` files.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update the device handle array for injected devices to output device
handles, not device ordinals. This requires that the extra ordinals
provided to `DEVICE_DT_INST` map to an existing device in order to
appear in the final handle array.
This is required for the injected handles to be useful, no meaningful
operations can be done on device ordinals.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Remove an extra `DEVICE_HANDLE_SEP` that was being inserted into the
handle array when injected dependencies were present.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
When updating `sys.path` to allow importing the pickled edtlib instance,
add the path to the front of `sys.path`, not the end. This ensures that
the `devicetree.edtlib` module that is imported is the one relative
to the files being run, not some other version which may exist on the
path.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Update the script so that it handles generates files using the
<zephyr/...> include prefix.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Instead of resizing all devices handles, we just resize devices that are
power domains. This means that a power domain has to be declared as
compatbile with "power-domain" in device tree node.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Add API to add devices to a power domain in runtime. The number of
devices that can be added is defined in build time.
The script gen_handles.py will check the number defined in
`CONFIG_PM_DEVICE_POWER_DOMAIN_DYNAMIC` to resize the handles vector,
adding empty slots in the supported sector to be used later.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Add supported device information to the device `handles` array. This
enables API's to iterate over supported devices for power management
purposes.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
replace with version.parse from packaging module.
prevent this warning message:
DeprecationWarning: The distutils package is deprecated
and slated for removal in Python 3.12. Use setuptools or
check PEP 632 for potential alternatives
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
With `gen_handles.py` now running on the first pre-built image,
`zephyr_pre0.elf` there is no requirement for the device handle arrays
to remain the same size after processing.
Remove the padding generated in `gen_handles.py`, as well as the
temporary option `CONFIG_DEVICE_HANDLE_PADDING` which was added to work
around this issue.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
When CONFIG_USERSPACE is enabled, the ELF file from linker pass 1 is
used to create a hash table that identifies kernel objects by address.
We therefore can't allow the size of any object in the pass 2 ELF to
change in a way that would change those addresses, or we would create
a garbage hash table.
Simultaneously (and regardless of CONFIG_USERSPACE's value),
gen_handles.py must transform arrays of handles from their pass 1
values to their pass 2 values; see the file's docstring for more
details on that transformation.
The way this works is that gen_handles.py just pads out each pass 2
array so its length is the same as its pass 1 value. The padding value
is a repeated run of DEVICE_HANDLE_ENDS values. This value is the
terminator which we look for at runtime in places like
device_required_handles_get(), so there must be at least one, and we
error out in gen_handles.py if there's no room in the pass 2 array for
at least one such value. (If there is extra room, we just keep
inserting extra DEVICE_HANDLE_ENDS values to pad the array to its
original length.)
However, it is possible that a device has more direct dependencies in
the pass 2 handles array than its corresponding devicetree node had in
the pass 1 array. When this happens, users have no recourse, so that's
a potential showstopper.
To work around this possibility for now, add a new config option,
CONFIG_DEVICE_HANDLE_PADDING, whose value defaults to 0.
When nonzero, it is a count of padding handles that are inserted into
each device handles array. When gen_handles.py errors out due to lack
of room, its error message now tells the user how much to increase
CONFIG_DEVICE_HANDLE_PADDING by to work around the problem.
It looks like a real fix for this is to allocate kernel objects whose
addresses are required for hash tables in CONFIG_USERSPACE=y
configurations *before* the handle arrays. The handle arrays could
then be resized as needed in pass 2, which saves ROM by avoiding
unnecessary padding, and would avoid the need for
CONFIG_DEVICE_HANDLE_PADDING altogether.
However, this 'real fix' is not available and we are facing a deadline
to get a temporary solution in for Zephyr v2.7.0, so this is a good
enough workaround for now.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This reverts commit ec331c6fe2.
Although it's a valid simplification under the assumption that we're
going to be padding the array out anyway, it would use extra ROM if we
fix the build system issues that are currently forcing gen_handles.py
to introduce extra padding in the handles arrays for linker pass 2.
On the (perhaps optimistic) assumption that we're going to fix the
build system, let's get rid of a commit that would get in the way. The
extra "complexity" in device_required_handles_get() is trivial.
This gets rid of a comment describing the linker passes, but the
structure of the comment is a bit misleading (and it contains
incorrect information for the results of pass 2: the terminator at the
end is DEVICE_HANDLE_ENDS, not DEVICE_HANDLE_NULL).
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This reverts commit 0c6588ff47.
It's not clear that the supported devices are being properly computed,
so let's revert this for v2.7.0 until we've had more time to think
it through.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The current gen_handles.py script uses linker defined symbols.
As preparation for support of more linkers the gen_tables.py now takes
the device start symbol as argument.
For example, armlink and ld uses different symbols.
With ld those can be named explicitly, but for armlink the linker
decides the names.
For ld, Zephyr defines: __device_start
For armlink, the symbol is defined as: Image$$<section name>$$Base
Therefore knowledge of the linker symbol to be used must be passed to
gen_handles.py so that the correct symbol can be used for locating
devices.
To support this change, the creation of the asm, compiler, compiler-cpp,
linker targets has been moved from target_toolchain_flags.cmake to
target_toolchain.cmake.
All linkers has been updated to support the use of the
device_start_symbol on the linker target.
List of linkers updated:
- ld
- lld
- arcmwdt
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Add supported device information to the device `handles` array. This
enables API's to iterate over supported devices for power management
purposes.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Force the inclusion of a `DEVICE_HANDLE_SEP` at the end of the
devicetree dependency section of the array. This lets us simplify the
implementation of `device_required_handles_get`, as there is only one
symbol the section ends with.
This does not use any extra ROM as the array is padded out to the
original size with `DEVICE_HANDLE_ENDS` anyway.
Also adds a description of the array format where the array is
instantiated.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Optimize the handles array by making the following observations:
* The devicetree ordinal at index 0 in pass1 is discarded by
gen_handles.py, and therefore does not appear in the pass2 array.
* gen_handles.py does not need `DEVICE_HANDLE_ENDS` to determine the
end of the handle array, as that information is present in the .elf.
Therefore, instead of replacing the devicetree ordinal with an
additional `DEVICE_HANDLE_ENDS` at the end (to preserve lengths), we
can simply move the ordinal to the end and have it be the original
`DEVICE_HANDLE_ENDS` symbol. This reduces the size of the array by
one handle per device (2 bytes).
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Initialize `handle.device` to `None` so that the assert after the
matching has a chance to catch errors. Without this, a failed match will
raise an exception on the line above as it attempts to get a property
that doesn't exist.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
We are now in the process of extracting edtlib and dtlib into a
standalone source code library that we intend to share with other
projects.
Links related to the work making this standalone:
https://pypi.org/project/devicetree/https://python-devicetree.readthedocs.io/en/latest/https://github.com/zephyrproject-rtos/python-devicetree
This standalone repo includes the same features as what we have in
Zephyr, but in its own 'devicetree' python package with PyPI
integration, etc.
To avoid making this a hard fork, move the code that's being made
standalone around in Zephyr into a new scripts/dts/python-devicetree
subdirectory, and handle the package and sys.path changes in the
various places in the tree that use it.
From now on, it will be possible to update the standalone repository
by just recursively copying scripts/dts/python-devicetree's contents
into it and committing the results.
This is an interim step; do NOT 'pip install devicetree' yet.
The code in the zephyr repository is still the canonical location.
(In the long term, people will get the devicetree package from PyPI
just like they do the 'yaml' package today, but that won't happen for
the foreseeable future.)
This commit is purely intended to avoid a hard fork for the standalone
code, and no functional changes besides the package structure and
location of the code itself are expected.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The script gen_handles.py was introduce in #32127 but relies on
ZEPHYR_BASE being set in environment.
However, it is not a requirement to set Zephyr base in the users
environment, therefore this is changed to an argument `-z` or
`--zephyr-base` which will be passed from the build system to the
script.
If `-z` or `--zephyr-base` is not provided, the environment will be
checked for a ZEPHYR_BASE setting there.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree. The
device dependency arrays are updated to use these handles.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree. The
device dependency arrays are updated to use these handles.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Jamie McCrae
Gerard Marull-Paretas
Keith Short
Jordan Yates
Anas Nashif
Flavio Ceolin
Julien Massot
Martí Bolívar
Torsten Rasmussen
Peter Bigot