Add two new types: Binding and PropertySpec.
- Binding is a first-class abstraction for a devicetree binding
file as defined by a YAML file in the Zephyr syntax.
- PropertySpec is a helper type which represents a property
definition within a Binding.
Make the Binding constructor a new entry point to the library. This
enables users to deal with bindings as standalone entities, apart from
how they characterize a particular devicetree.
Rework the EDT and Node internals that deal with bindings as dict
values to use the Binding type instead. To make this less ambiguous,
use the variable name 'raw' when we're dealing with a binding as it's
parsed from YAML, and 'binding' when we're dealing with a Python
Binding object.
This commit drops support for the following legacy bindings keys
- '#cells': use '*-cells' instead (e.g. 'gpio-cells', 'pwm-cells')
- "child-bus: foo" and "child: bus: foo": use "bus:" instead
- "parent-bus" and "parent: bus: ": use "on-bus:" instead
Officially, legacy bindings have been gone since
6bf761fc0a ("dts: Remove support for deprecated DTS binding
syntax"), so this is vestigial code, and I couldn't find any in-tree
users.
It also drops the convention in this file that ""-strings are
preferred.
I honestly don't understand why this was ever enforced; the file
itself admits single quotes are common in Python and we use them
elsewhere in Zephyr's Python code.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add the first API functions that directly deal with node dependency
ordinals as determined by edtlib:
- DT_DEP_ORD(node_id): node_id's ordinal
- DT_REQUIRES_DEP_ORDS(node_id): list of dep ordinals for node_id's
direct dependencies
- DT_SUPPORTS_DEP_ORDS(node_id): list of dep ordinals for nodes
depending directly on node_id
- DT_INST_ equivalents
This is not meant to be an exhaustive set of macros related to
dependency ordinals; rather, it's a starting out point meant to enable
initial struct device dependency tracking work. We can add more if
needed.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The legacy macros were first deprecated in Zephyr v2.3. Now that
Zephyr v2.4 has been released, that makes two releases where these
macros have been deprecated, so it's OK to remove them.
This leaves support for legacy binding syntax in place. Removing that
is left to future work.
We need to update various pieces of documentation related to flash
partitions that never got updated when the new API was introduced.
Consolidate this information in the flash_map.h API reference page,
since that's really where users will run into it. This also gives us
the opportunity to improve this documentation.
Adjust a couple of kconfigfunctions.py and sanitycheck bits to use
non-legacy edtlib APIs.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Each controller node in a phandle-array may set the number of cells in
a specifier as any nonnegative integer. Currently, we don't allow
this in edtlib in the case where there are multiple controllers in a
phandle-array property all of which have 0 cells in the relevant
specifier, which is not correct. Fix this, add a regression test, and
improve the error message while we are here.
Fixes: #28709
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a lookup table for finding a node by its dependency ordinal.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Make the scc_order method a property instead. This is in keeping with
the "General biased advice" at the top of file.
The actual order is therefore lazily initialized in this commit and
the order is not computed by the time __init__() returns. The next
commit will invoke scc_order by the time the constructor returns.
This is preparation work for adding a lookup table from dependency
ordinals to nodes. The combination of these two changes will make
intializing that lookup table a bit easier.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We don't want to support cyclic dependency structures, because it
means that Node objects cannot have dep_ordinal attributes as they are
currently documented to possess unconditionally.
Nevertheless, we have some in our tests. Remove them by extracting the
/props/ctrl-X nodes to the same level as the /props nodes. This breaks
a cycle caused by:
- /props/ctrl-X nodes depend on /props because of the parent/child
relationship
- /props depends on /props/ctrl-X because it refers to them by phandle
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Use the pytest test framework in the dtlib.py and edtlib.py test
suites (testdtlib.py and testedtlib.py respectively).
The goal here is not to change what is being tested. The existing test
suite is excellent and very thorough.
However, it is made up of executable scripts where all of the tests
are run using a hand-rolled framework in a single function per file.
This is a bit all-or-nothing and prevents various nice features
available in the de-facto standard pytest test framework from being
used.
In particular, pytest can:
- drop into a debugger (pdb) when there is a problem
- accept a pattern which specifies a subset of tests to run
- print very detailed error messages about the actual and expected
results in various traceback formats from brief to very verbose
- gather coverage data for the python scripts being tested (via plugin)
- run tests in parallel (via plugin)
- It's easy in pytest to run tests with temporary directories
using the tmp_path and other fixtures. This us avoid
temporarily dirtying the working tree as is done now.
Moving to pytest lets us leverage all of these things without any loss
in ease of use (in fact, some things are nicer in pytest):
- Any function that starts with "test_" is automatically picked up and
run. No need for rolling up lists of functions into a test suite.
- Tests are written using ordinary Python 'assert'
statements.
- Pytest magic unpacks the AST of failed asserts to print details on
what went wrong in really nice ways. For example, it will show you
exactly what parts of two strings that are expected to be equal
differ.
For the most part, this is a pretty mechanical conversion:
- extract helpers and test cases into separate functions
- insert temporary paths and adjust tests accordingly to not match
file names exactly
- use 'assert CONDITION' instead of 'if not CONDITION: fail()'
There are a few cases where making this happen required slightly
larger changes than that, but they are limited.
Move the checks from check_compliance.py to a new GitHub workflow,
removing hacks that are no longer needed.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Tell the EDT instance that properties of the /zephyr,user node should
be generated based on the binding types inferred from the property
content.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Clean up of devicetree tooling removed generation of information
present in devicetree in nodes that have no compatible, or for extra
properties not defined by a binding. Discussion proposed that these
properties should be allowed, but only in a defined node /zephyr,user.
For that node infer bindings based on the presence of properties.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Consolidate creation of edtlib.EDT objects from a build directory's
devicetree into one place by loading it from build/zephyr/edt.pickle
everywhere. A previous commit creates edt.pickle from gen_defines.py.
In addition to probably speeding things up slightly by not reparsing
the devicetree, the main benefit of this approach is creating a single
point of truth for the bindings directories and warnings
configuration, meaning we don't have to worry about them getting out
of sync while being passed around between devicetree creation and
usage time.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We only have one use of _binding_compat and it doesn't need self, so
just fold it into _init_compat2binding.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We deprecated a number of aspects of the DTS binding syntax in Zephyr
2.1. Remove the support for the deprecated syntax. Remove from docs
about the deprecated syntax as well.
Removed reference in release-notes-2.1.rst to legacy_binding_syntax
since that anchor doesn't exist anymore.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We need to save and restore the devicetree data to generate optimized
dependency information later on in the build, in particular during the
final application link.
Make this happen by pickling the EDT object in BUILD_DIR/edt.pickle.
The existence of this file is an implementation detail, so do not add
it to the documentation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We have a use case for saving the EDT object to be able to open it up
again later. It would be convenient to be able to do this with the
pickle module from stdlib.
The only thing stopping us from doing that appears to be the open
reference to sys.stderr that's held the edt object even after
EDT.__init__ exits. However, there doesn't seem to be a need to keep
holding on to this object, and in fact it would be a little bit nicer
to drop the reference in case something else (even in the same Python
process that created it originally) wants the EDT object around, but
might want the warn file closed if its refcount zeroes out.
Just drop the reference at the end of __init__ and make EDT._warn()
throw an exception if it's attempted to be used after the constructor
exits.
Make pickle-ability an API guarantee so we can treat any regressions
as bugs going forward.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
I can't see a good reason to be doing this in the Node class's
unit_addr accessor. Move it up to the edtlib initialization so it only
happens once.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add test cases that verify various bits and pieces of the legacy
devicetree macros match the new APIs.
Writing these test cases without giving rise to deprecated macro
warnings which might break people's CI if they build with -Werror
requires turning off the __WARN() generation in
devicetree_legacy_unfixed.h. The entire file is deprecated at this
point and must be explicitly enabled with an opt-in Kconfig option, so
there isn't any harm in doing this.
Nevertheless, take a minimally invasive approach to avoiding __WARN()
generation in gen_legacy_defines.py, to avoid the possibility of
breakage. This code is basically frozen anyway, so hacks like this
won't cause maintainability problems since it isn't being actively
maintained.
Use the new tests as fodder for a migration guide from the old API in
the documentation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Some updates to the reference page for the "core" APIs, and associated
follow-ups in the guides:
- centralize documentation of chosen zephyr nodes in a non-legacy
file, provide a reference to them from the intro page in the guide
- review doxygen docstrings and correct errors for generic APIs
- add introductory text to each section in the API reference
- add missing hardware-specific pages
Documentation for layers built on top of these is mostly left to future
commits, but I do have a smattering of fixes in the guides that I
noticed while I was doing this.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add DT_NODE_BY_FIXED_PARTITION_LABEL that given a "label" in any
fixed-partitions map will return the node_id for that partition node.
Add DT_NODE_HAS_FIXED_PARTITION_LABEL that will test if a given
fixed-partitions "label" is valid.
Add DT_FIXED_PARTITION_ID that will return an unique ordinal value for
the partition give a node_id to the partition.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If we have a fixed-partition on a flash device that is for example on
a spi controller we will not get a binding match currently. This is
because we expect a match between both the compatible and the fact that
fixed-partition node is a decendant of the spi bus.
To address this we treat fixed-partitions as if they are on no bus.
This has the effect of causing a binding match as well as ensuring that
when we process the fixed-partition node we will do anything special to
it because of the bus it happens to be under (for example SPI CS_GPIO
processing).
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Setup node.compats right after we create the Node. This allows access
to the compats information in _bus_node.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Usually, we want to operate only on "available" device
nodes ("available" means "status is okay and a matching binding is
found"), but that's not true in all cases.
Sometimes we want to operate on special nodes without matching
bindings, such as those describing memory.
To handle the distinction, change various additional devicetree APIs
making it clear that they operate only on available device nodes,
adjusting gen_defines and devicetree.h implementation details
accordingly:
- emit macros for all existing nodes in gen_defines.py, regardless
of status or matching binding
- rename DT_NUM_INST to DT_NUM_INST_STATUS_OKAY
- rename DT_NODE_HAS_COMPAT to DT_NODE_HAS_COMPAT_STATUS_OKAY
- rename DT_INST_FOREACH to DT_INST_FOREACH_STATUS_OKAY
- rename DT_ANY_INST_ON_BUS to DT_ANY_INST_ON_BUS_STATUS_OKAY
- rewrite DT_HAS_NODE_STATUS_OKAY in terms of a new DT_NODE_HAS_STATUS
- resurrect DT_HAS_NODE in the form of DT_NODE_EXISTS
- remove DT_COMPAT_ON_BUS as a public API
- use the new default_prop_types edtlib parameter
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Even though it is about to be done for sound technical reasons, a
subsequent patch adding access to all device nodes at the last minute
in the 2.3 release is going to be playing a bit of a fast one on
the Zephyr community, especially users of DT_INST APIs.
In particular, instance numbers are currently allocated only to
enabled nodes, but that will not be true soon: *every* node of a
compatible will be allocated an instance number, even disabled ones.
This is especially unfortunate for drivers and applications that
expect singletons of their compatibles, and use DT_INST(0, ...) to
mean "the one enabled instance of my compatible".
To avoid gratuitous breakage, let's prepare for that by sorting each
edt.compat2nodes sub-list so that enabled instances always come before
disabled ones.
This doesn't break any API guarantees, because there basically *are*
no ordering guarantees, in part precisely to give us the flexibility
to do things like this. And it does help patterns that use instances 0
through N-1, including the important singleton case.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The macro iterates through the list of child nodes and invokes provided
macro for each node.
Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Remove semicolon between instance invocations of DT_FOREACH_IMPL_ and
thus DT_INST_FOREACH. This provides more flexibility to the user. This
requires we fixup in tree users to add semicolon where needed.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
These look up tables generalize the compat2enabled map in a way we
will need to make the API more flexible in Zephyr.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Let's get the actual node status, instead of relying on enabled.
Leave enabled in place for gen_legacy_defines.py's sake.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
If a devicetree node doesn't have a matching binding we will at least
populate a common standard set of properties for that node. The list of
standard properties is:
compatible
status
reg
reg-names
label
interrupt
interrupts-extended
interrupt-names
interrupt-controller
This allows us to handle cases like memory nodes that don't have any
compatible property, we can still generate the reg values.
We limit this to known properties as for any other property we can not
fully determine the property type without a binding and thus we can't
ensure the generation for that property is correct or may not change.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
It's a common operation to want to find a node based on its label. Add
a lookup table to make this easier.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a __WARN("Macro is deprecated") to all DT_<COMPAT>_BUS_<BUS> macros
now that all in tree users should have been converted to the new macros.
This is intended to make sure any PRs don't introduce new usages of
these macros.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add a __WARN("Macro is deprecated") to all DT_INST macros now that all
in tree users should have been converted to the new macros.
This is intended to make sure any PRs don't introduce new usages of
these macros.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Always generate the comment text specifying a node's path identifier.
Add the DT_ prefix so it matches the actual macro usable from C. This
will make a following patch which adds support for accessing a node's
parent result in a generated header file which is easier to read.
Put the node's path right after "Devicetree node:" in the comment.
This makes the section for that node easier to grep for.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The root node's z_path_id value for the duration of this script
doesn't match the value DT_ROOT is defined to in devicetree.h.
I didn't notice this because the root node's compatible doesn't have a
matching binding in practice, so no macros are generated for it, but
we're about to start looking at node parents explicitly and this is an
issue for that. Fix it so the root node's z_path_id is "N", since
DT_ROOT is the token "DT_N".
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add generation of the following macros:
DT_N_<node-id>_REG_IDX_<idx>_EXISTS 1
DT_N_<node-id>_IRQ_IDX_<idx>_EXISTS 1
This will allow us to use IS_ENABLED() in DT_REG_HAS_IDX and
DT_IRQ_HAS_IDX which matches behavior of other DT_*_HAS_* macros as
well as lets use these with COND_CODE_1.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add generation of the following macros:
DT_N_<node-id>_P_<prop-id>_NAME_<NAME>_EXISTS
DT_N_<node-id>_P_<prop-id>_IDX_<idx>_EXISTS
This will be useful to check availability of named or indexed
property like dmas/dma-names.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
The last user of the .conf file format DTS data has been removed. We
can now remove the generation and associated support for the .conf file.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Due to the use of UTIL_EVAL*() macros, the UTIL_LISTIFY() macro used
by DT_INST_FOREACH(foo) can cause long build errors when there is a
build error in the expansion for "foo". More than a thousand lines of
build error output have been observed for an error in a single line of
faulty C.
To improve the situation, re-work the implementation details so the
errors are a bit shorter and easier to read. The use of COND_CODE_1
still makes the error messages quite long, due to GCC generating notes
for various intermediate expansions (__DEBRACKET,
__GET_ARG_2_DEBRACKET, __COND_CODE, Z_COND_CODE_1, COND_CODE1), but
it's better than the long list of UTIL_EVAL notes.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
And implement DT_ANY_INST_ON_BUS() in terms of it.
This makes some error messages quite a bit shorter by avoiding
UTIL_LISTIFY(), which has a nasty temper and tends to explode if not
treated gently.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We get the following error:
ValueError: max() arg is an empty sequence
if the compatiable section of the device tree is empty or doesn't exist.
Fix this by havingin max_len get a default value of 0.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This is joint work with Kumar Gala (see signed-off-by).
This supports a new devicetree macro syntax coming. It's not really
worth mixing up the old and the new generation scripts into one file,
because:
- we aim to remove support for the old macros at some point, so it
will be cleaner to start fresh with a new script based on the old one
that only generates the new syntax
- it will avoid regressions to leave the existing code alone while
we're moving users to the new names
Keep the existing script by moving it to gen_legacy_defines.py and
changing a few comments and strings around. It's responsible for
generating:
- devicetree.conf: only needed by deprecated kconfigfunctions
- devicetree_legacy_unfixed.h: "old" devicetree_unfixed.h macros
Put a new gen_defines.py in its place. It generates:
- zephyr.dts
- devicetree_unfixed.h in the new syntax
Include devicetree_legacy_unfixed.h from devicetree.h so no DT users
are affected by this change.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Kumar Gala
Martí Bolívar
Peter Bigot
Dominik Ermel
Henrik Brix Andersen
Erwan Gouriou