This change enhances the devicetree library by adding support for tracking
the source filename and line number for nodes and properties.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Node properties are not created for all properties defined
by the node's binding (Binding.prop2specs),
only for those that actually have a value.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
PropertySpec.path does NOT tell "the file where the property
was last modified", but instead the binding file specifying
the devicetree node of which this is a property.
See: #65135
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
Add a new "hash" attribute to all Devicetree EDT nodes. The hash is
calculated on the full path of the node; this means that its value
remains stable across rebuilds.
The hash is checked for uniqueness among nodes in the same EDT.
This computed token is then added to `devicetree_generated.h` and made
accessible to Zephyr code via a new DT_NODE_HASH(node_id) macro.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Add a series of unit tests which try to cover somewhat systematically
the possible inputs and what we finally get at the exit
of the Binding constructor.
Running the assumption that any (valid) YAML binding file is
something we can make a Binding instance with:
- check which properties are defined at which level (binding,
child-binding, grandchild-binding, etc) and their specifications
once the binding is initialized
- check how including bindings are permitted to specialize
the specifications of inherited properties
- check the rules applied when overwriting a binding's description
or compatible string (at the binding, child-binding, etc, levels)
Some tests covering known issues are disabled by default:
- this permits to document these issues
- while not causing CI errors (when running the python-devicetree
unit tests)
- enabling these tests without causing errors should allow us
to consider the related issues are fixed
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
This simplifies the code and makes it clearer that both properties are
defined in terms of the Binding object matched to a given DT node.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Moves several node-specific operations inside the Node class to improve
its encapsulation, remove a monkey patch and access to internal methods
and fields.
Signed-off-by: Florian Grandel <fgrandel@code-for-humans.de>
[1] was introduced to get more valuable answers from
the PropertySpec.path API, which is supposed to tell
in which file the property's specification was "last modfied".
Further work on related issues [2] showed that the
approach chosen in [1] is dead end: we need to first rethink
how bindings (and especially child-bindings) are initialized.
[1] edtlib: fix last modified semantic in included property specs
[2] edtlib: Preserve paths of properties from included child bindings
See also: #65221, #78095
This reverts commit b3b5ad8156.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
Use-case "B includes I includes X":
- X is a base binding file, specifying common properties
- I is an intermediary binding file, which includes X
without modification nor filter
- B includes I, filtering the properties it chooses
to inherit with an allowlist or a blocklist
Check that the properties inherited from X via I
are actually filtered as B intends to,
up to the grandchild-binding level.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
This unit test was added to cover the change introduced by [1].
Further work on related issues [2] showed that the chosen approach
is dead end.
We're reverting all changes made in [1].
[1] edtlib: fix last modified semantic in included property specs
[2] edtlib: Preserve paths of properties from included child bindings
See also: #65221, #78095
This reverts commit 70eaa61cb0.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
This unit test was added specifically to cover a regression
reported by the CI while working on [1].
Further work on related issues [2] showed that:
- [1] and [2] are dead end: we need to first rethink
how bindings (and especially child-bindings) are initialized
- the inclusion mechanism supported by Zephyr deserves more systematic
testing in edtlib if we want to work with confidence
The approach we choose is to:
- revert all changes made in [1]
- from there, systematically add unit tests as we address
the issues we identified (or the additional features we need)
one after the other
[1] edtlib: fix last modified semantic in included property specs
[2] edtlib: Preserve paths of properties from included child bindings
See also: #65221, #78095
This reverts commit 33bb3b60d9.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
Adjust existing tests to support the changes and add new tests to test
the newly added feature.
Signed-off-by: Joel Hirsbrunner <jhirsbrunner@baumer.com>
It is currently impossible to use enum with any array like type (i.e.
string-array and array, these are the only ones that make sense) in the
devicetree and dt-bindings.
However, there is no such remark in the dt-bindings section of the docs.
Since this is a feature that comes in very handy and is implemented
fairly easily, I adjusted the scripts for this.
It is now possible to do something like this.
```yaml
compatible = "enums"
properties:
array-enum:
type: string-array
enum:
- bar
- foo
- baz
- zoo
```
```dts
/ {
enums {
compatible = "enums";
array-enum = "foo", "bar";
};
};
```
Signed-off-by: Joel Hirsbrunner <jhirsbrunner@baumer.com>
Previously, dtlib would fail to parse the following:
/delete-node/ &{/};
This is accepted by dtc, so dtlib should be aligned.
The expected behavior is that the contents of the "deleted" root node
are emptied, but the node itself remains in the tree. This means that
it's possible to put that statement at the end of a DTS file and still
get a valid output. A small test case for this scenario is included.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Make sure filters set by property-allowlist and property-blocklist
in an including binding are recursively applied to included bindings.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
Although the PropertySpec.path attribute is documented as
"the file where the property was last modified",
all property specs in Binding.prop2specs will claim
they were last modified by the top-level binding itself.
Consider:
- I1 is a base binding that specifies properties x and y
- I2 is an "intermediate" binding that includes I1,
modifying the specification for property x
- B is a top-level bindings that includes I2,
and specifies an additional property p
When enumerating the properties of B,
we expect the values of PropertySpec.path to tell us:
- y was last modified by I1
- x was last modified by I2
- p was last modified by B
However, the Binding constructor:
- first merges all included bindings into the top-level one
- eventually initializes specifications for all the defined properties
As a consequence, all defined properties claim they were last modified
by the top-level binding file.
We should instead:
- first, take into account their own specifications for the
included properties
- eventually update these specifications with the properties
the top-level binding adds or modifies
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
Make sure the property specs answered by the Binding.prop2specs API
do not all claim (PropertySpec.path) they were last modified
by the top-level binding.
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
The currently used PyYaml version has some vulnerabilies as
described on the pull request description. It updates to
version 6.0, removing these supply chain vulnerabily.
The OSSF Scorecard was the tool used for discovering
these vulnerabilties.
Signed-off-by: Javan lacerda <javanlacerda@google.com>
The current EDT graph logic only use properties directly under a
specific node to add dependencies. For nodes properties in
child-bindings, this means that the child phandles are only linked by
the child node itself, which does have an ordinal but no corresponding
"sturct device" in the code, causing those dependencies to be silently
ignored by gen_handles.py.
Fix that by adding the recursive logic to visit child bindings when
present, which causes all child node property handles to be linked to
the parent node.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When we have an empty Devicetree, ie,
```
/dts-v1/;
/ {
};
```
The node's dep_ordinal is never initialized because the node graph is
empty. This ends up with invalid ordinal tokens (-1) in
devicetree_generated.h which in turn produce some cryptic compiler
errors, see e.g.
```
error: pasting "dts_ord_" and "-" does not give a valid preprocessing
token
95 | #define Z_DEVICE_DT_DEV_ID(node_id) _CONCAT(dts_ord_,
DT_DEP_ORD(node_id))
...
include/zephyr/devicetree.h:2498:41:
note: in expansion of macro 'DT_FOREACH_OKAY_HELPER'
2498 | #define DT_FOREACH_STATUS_OKAY_NODE(fn)
DT_FOREACH_OKAY_HELPER(fn)
|
^~~~~~~~~~~~~~~~~~~~~~
include/zephyr/device.h:1022:1:
note: in expansion of macro 'DT_FOREACH_STATUS_OKAY_NODE'
1022 |
DT_FOREACH_STATUS_OKAY_NODE(Z_MAYBE_DEVICE_DECLARE_INTERNAL)
```
(devicetree_generated.h)
```
...
#define DT_N_ORD -1
#define DT_N_ORD_STR_SORTABLE 000-1
...
```
This patch makes sure root node is always inserted (without any target)
so that it gets initialized later.
Discovered as part of
https://github.com/zephyrproject-rtos/zephyr/pull/63696
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
PCI devices are have some differences to regular nodes:
* node name specifies device/function e.g. "pcie@1,0"
* register address has a different meaning
* zero-sized register is allowed
This improves alignment with Linux DT for PCI devices
Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
In Linux, checkpatch.pl relies on the vendor-prefixes.yaml file
to validate manufacturers in compatible strings.
In addition to the vendors defined in vendor-prefixes.txt,
the YAML file includes expressions for "prefixes which are not vendors":
these expressions do NOT define special manufacturers that may appear
in compatible strings, and are never involved as such in DTS files.
We can rather see them as bulk-definitions of JSON/YAML properties
suitable for the dt-schema tools.
OTHO, in Zephyr, checkpatch.pl relies on the vendor-prefixes.txt file,
which does not include these additional prefixes, but edtlib.EDT adds
them as hard-coded special values.
This is confusing, if not incorrect:
- the fact that edtlib.EDT (and thus its client code in the
zephyr/scripts directory) actually allows these vendors
in compatible strings is buried in the source code
- checkpatch.pl (with vendor-prefixes.txt) in Zephyr behaves neither like
checkpatch.pl (with vendor-prefixes.yaml) in Linux, nor like edtlib.EDT
(with _VENDOR_PREFIX_ALLOWED)
- Zephyr should not treat these "prefixes which are not vendors" as
valid manufacturers in compatible strings to begin with
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
This is a one-line fix for edtlib, which lets gen_defines.py indicate
whether the `ranges` property exists within a given node.
Although address translation through ranges is typically automatic,
users can choose to manually inspect ranges using DT_FOREACH_RANGE(),
DT_NUM_RANGES(), and other DT_RANGES_* macros. These can be used to
implement manual translation at runtime, which is currently done for
PCIe controllers.
The only thing missing is being able to check if a node contains an
empty `ranges;`, which signifies a 1:1 translation to the parent bus.
Checking DT_NUM_RANGES() is insufficient, because it returns zero
whether or not `ranges;` is present.
It should be possible to use DT_NODE_HAS_PROP(), but it was not working,
because edtlib ignores properties which are undeclared in bindings and
don't have a default type. Add a missing PropertySpec for `ranges` with
"compound" type; it can't be "array" because it can be empty-valued.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
This concludes the type annotations for the public API for the module,
along with the relevant internal state. It's not worth type annotating
the internal backwards compatibility shim for !include.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This requires adding a private constructor so that mypy
can tell what all the final instance state is going to be.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Incremental progress towards type annotating the whole module.
Annotate helper procedures used by the class as well.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file. I am
reordering the classes to make it possible to type annotate the module
in a more readable way.
Git might make the diff look bigger than it really is.
To verify this is just moving code, use 'git diff --minimal'.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Benjamin Cabé
Christophe Dufaza
Luca Burelli
James Roy
Grzegorz Swiderski
Pieter De Gendt
Florian Grandel
Joel Hirsbrunner
Benedikt Schmidt
Reto Schneider
Lingao Meng
Javan lacerda
Fabio Baltieri
Gerard Marull-Paretas
Grant Ramsay
Martí Bolívar