Add support for .mot file flash using west flash command
The RX build output .mot as binary file to flash into
board
Signed-off-by: Phi Tran <phi.tran.jg@bp.renesas.com>
Signed-off-by: Duy Nguyen <duy.nguyen.xa@renesas.com>
Add runners entry to the module schema and import the file specified.
Every class that inherits from ZephyrBinaryRunner will be discovered and
added to runners list.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
Adds an optional priority parameter to the flash runner run once
configuration which allows for deciding upon which file should
ultimately be used
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Provide a mechanism to propagate useful arguments from one runner to the
next. The primary use case for this is to propagate a JLink serial
number, so that if it is queried from the terminal the user only needs
to make the choice once.
Implements #76077.
Signed-off-by: Jordan Yates <jordan@embeint.com>
This command runs separately from a debug server, instead of attaching
to a running server. This is both the easiest out of the box experience,
and also should be possible to implement consistently for most runners.
This commit includes an initial implementation for pyocd.
Signed-off-by: Tobias Pisani <mail@topisani.dev>
Adjust error message so that it clearly states runners.yaml is
missing from <build_dir>/zephyr, instead of referencing CMake cache
variable ZEPHYR_RUNNERS_YAML, which is no longer used (since
3124c02987 ).
Also clean up that variable in CMake since it is no longer used
(0 other references in entire tree).
Fixes#70605
Signed-off-by: Louis Feller <louis.feller@st.com>
Fixes an issue whereby the domains file in sysbuild projects
would be loaded and used with outdated information if sysbuild
configuration was changed then west flash was ran directly after
it
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Introduce `simulate `command for running samples on a simulator of
choice. Initial implementation consists of one runner, dedicated for
Renode.
Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
This adds supports for flashing images with sysbuild where there
are multiple images per board to prevent using the same command per
image flash which might cause issues if they are not ran just once
per flash per unique board name. A deferred reset feature is also
introduced that prevents a board (or multiple) from being reset if
multiple images are to be flashed until the final one has been
flashed which prevents issues with e.g. security bits being enabled
that then prevent flashing further images.
These options can be set at a board level (in board.yml) or a SoC
level (in soc.yml), if both are present then the board configuration
will be used instead of the SoC, and regex can be used for matching
of partial names which allows for matching specific SoCs or CPU cores
regardless of the board being used
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Add the exe fle to the runnerconfiguration class,
so we can use it from runners which will need it.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add a new UF2 runner, supporting only the flash capability.
Searches for FAT partitions containing `INFO_UF2.TXT` files,
and can optionally filter on a matching `Board-ID` value in
that file.
Signed-off-by: Peter Johanson <peter@peterjohanson.com>
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
When using west runners with multi domains we want to warn users when
they user runner args on multi domain builds.
Therefore fix the `>1` to the correct `>0` which ensures the warning
is printed in all expected cases.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The introduction of sysbuild and west build / flash / debug / etc.
support raises questions regarding how to handle user specified runner
arguments.
Some arguments may be acceptable for all domains, whereas others are
only valid for a single or few domains in a multi domain build.
Therefore, consider the usages of runner specific options experimental
when applied to multiple domains.
The following warning is printed to the user when using runner specific
arguments on multiple domains:
> WARNING: Specifying runner options for multiple domains is
> experimental.
> If problems are experienced, please specify a single domain using
> '--domain <domain>'
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit extends the west commands build, flash, and debug to support
--domain when having multiple domains (images) defined in a domains.yaml
build file.
The domains.yaml uses the following yaml format to specify the
build directory of each domain in the multi image build:
> default: <domain-n>
> domains:
> <domain-1>:
> build_dir: <build_dir-domain-1>
> <domain-2>:
> build_dir: <build_dir-domain-2>
> ...
`west <build|flash|debug>` has been extended to support
`--domain <domain>`.
`west build` calls CMake to create the build system, and if `--domain`
is given, then the build tool will be invoked afterwards for the
specified domain.
`west flash` will default flash all domains, but `--domain <domain>`
argument can be used to select a specific domain to flash, for example:
> west flash --domain mcuboot
`west debug` only a single domain can be debugged at any given time.
If `--domain` is not specified, then the default domain specified in the
domains.yml file will be used.
Users can still select a different domain, for example with:
> west debug --domain mcuboot
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Running west flash with the --openocd-search arg will result in the
openocd command being run with a -s param added for every character in
the passed DIR.
Fix the argparser to append all openocd-search DIRs into a list, instead
of just passing a string to the subcommand.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
When updating `sys.path` to allow importing the pickled edtlib instance,
add the path to the front of `sys.path`, not the end. This ensures that
the `devicetree.edtlib` module that is imported is the one relative
to the files being run, not some other version which may exist on the
path.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Zephyr thread awareness is available for openocd but boards don't
have debuggers configuration. This configure OpenOCD runner
automatically to complete configuration.
User still require enable CONFIG_DEBUG_THREAD_INFO=y to visualize
thread debug information.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Today, there is a build target is added for each runner: flash, debug,
debugserver, attach.
And those runners will have a dependency to Zephyr logical target that
is built before invoking `west <runner>`.
This design has some flaws, mainly that additional dependencies directly
on the target will not be built when running `west <runner>` directly.
That generator expressions cannot be used for the DEPENDS argument.
Instead, the build target `<runner>` will not have any dependencies, and
will raise a build error if a dependency is added to the target.
Due to how `add_dependencies()` work, this must be done as a build time
check, and not configure time check.
`west <runner>` will invoke a build before executing the runner, and
this way ensure the build target is up-to-date, which again removes the
need for a dedicated `west_<runner>_target`.
It also minimizes the risk of developer errors, as developers no longer
need to consider the need for adding additional dependencies.
If a custom target is part of the default `all` build, then it's ensured
to be up-to-date.
Fixes: Issue reported on slack.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
We are now in the process of extracting edtlib and dtlib into a
standalone source code library that we intend to share with other
projects.
Links related to the work making this standalone:
https://pypi.org/project/devicetree/https://python-devicetree.readthedocs.io/en/latest/https://github.com/zephyrproject-rtos/python-devicetree
This standalone repo includes the same features as what we have in
Zephyr, but in its own 'devicetree' python package with PyPI
integration, etc.
To avoid making this a hard fork, move the code that's being made
standalone around in Zephyr into a new scripts/dts/python-devicetree
subdirectory, and handle the package and sys.path changes in the
various places in the tree that use it.
From now on, it will be possible to update the standalone repository
by just recursively copying scripts/dts/python-devicetree's contents
into it and committing the results.
This is an interim step; do NOT 'pip install devicetree' yet.
The code in the zephyr repository is still the canonical location.
(In the long term, people will get the devicetree package from PyPI
just like they do the 'yaml' package today, but that won't happen for
the foreseeable future.)
This commit is purely intended to avoid a hard fork for the standalone
code, and no functional changes besides the package structure and
location of the code itself are expected.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Runners can (and many should) depend on the devicetree to decide what
to do, especially as it relates to the flash layout. Make it so that
they do not have to manipulate sys.path to get a hold of the edtlib
directory by doing it in one place, before importing any of them.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The RunnerConfig class stores the locations of the Zephyr output files
in various formats (elf, hex, bin). A longstanding issue with the
representation is that these might not exist if the corresponding
Kconfig options are not set. For example, if
CONFIG_BUILD_OUTPUT_BIN=n, there is no .bin file.
Change this so the type system knows these are Optional[str], not str.
Fix the runners that use non-ELF outputs so they check for the
existence of the relevant file before using it, mostly using a new
ZephyrBinaryRunner.ensure_output helper.
I'm not going to bother with checking for the ELF file itself; that's
always there as far as I can tell.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Remove the boards and shields lists from the 'usage' target output.
That might have been readable at some point long ago in Zephyr's
history, when only a few boards were available, but right now it's
obscuring the high level targets we really want 'usage' to print.
Instead, add 'boards' and 'shields' targets which the user can run to
get those lists, and reference them from the 'usage' output. This
makes 'usage' squintable again. We use the new list_boards.py script
from the 'boards' target.
Reference the 'help' target from 'usage' as well, and drop the
recommendation that people run '--target help' from the 'west build
--help' output for the 'west build --target' option. The canonical
place to look is 'usage' now.
Use the new list_boards.py code from 'west boards' as well, which
allows us to add the board's directory as a format string key, in
addition to its name and architecture.
Keep west-completion.bash up to date. While doing that, I noticed that
a bunch of references to this file refer to a stale location, so fix
those too.
Finally, the 'usage' output is what we print for a failed board or
shield lookup, so that needs to be updated also. Handle that by
invoking boards.cmake and a new shields.cmake in CMake script mode to
print the relevant output.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
I keep seeing people miss the --context option in the "west flash
--help" output.
This option is very important for understanding the runner-specific
options and state, and missing it means people get confused about what
west flash, debug, etc. can do and are doing.
Try to avoid this problem by adding a big fat banner about the
omission of runner-specific options in the main help output, and
provide more hints about how to use --context.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit message is a bit of a novel mostly:
- because the issues involved are longstanding
- as evidence this is not a capricious refactoring
The runners.core.RunnerConfig Python class holds common configuration
values used by multiple runners, such as the location of the build
outputs and board directory.
The runners code, first written in 2017-ish, replaced various shell
scripts that got this information from the environment. Avoiding
environment variables was a requirement, however. It's ghastly to set
environment variables for a single command invocation on Windows, and
the whole thing was part of a larger push to make Zephyr development
on Windows better.
I had a hammer (the argparse module). Finding a replacement naturally
looked like a nail, so the information that ends up in RunnerConfig
got shunted from the build system to Python in the form of 'west
flash' / 'west debug' command line options like '--board-dir',
'--elf-file', etc.
I initially stored the options and their values in the CMake cache.
This was chosen in hopes the build system maintainer would like
the strategy (which worked).
I knew the command line arguments approach was a bit hacky (this
wasn't a nail), but I also honestly didn't have a better idea at the
time.
It did indeed cause issues:
- users don't know that just because they specify --bin-file on the
command line doesn't mean that their runner respects the option, and
have gotten confused trying to flash alternate files, usually for
chain-loading by MCUboot (for example, see #15961)
- common options weren't possible to pass via board.cmake files
(#22563, fixed partly via introduction of runners.yaml and the west
flash/debug commands no longer relying on the cache)
- it is confusing that "west flash --help" prints information about
openocd related options even when the user's board has no openocd
support. The same could be said about gdb in potential future use
cases where debugging occurs via some other tool.
Over time, they've caused enough users enough problems that
improvements are a priority.
To work towards this, put these values into runners.yaml using a new
'config: ...' key/value instead of command line options.
For example, instead of this in the generated runners.yaml file:
args:
common:
- --hex-file=.../zephyr.hex
we now have:
config:
hex_file: zephyr.hex
and similarly for other values.
In Python, we still support the command line options, but they are not
generated by the build system for any in-tree boards. Further work is
needed to deprecate the confusing ones (like --hex-file) and move the
runner-specific host tool related options (like --openocd) to the
runners that need them.
Individual board.cmake files should now influence these values by
overriding the relevant target properties of the
runners_yaml_props_target.
For example, instead of:
board_runner_args(foo "--hex-file=bar.hex")
Do this:
set_target_properties(runners_yaml_props_target PROPERTIES
hex_file bar.hex)
This change additionally allows us to stitch cmake/mcuboot.cmake and
the runners together easily by having mcuboot.cmake override the
properties that set the hex or bin file to flash. (The command line
arguments are still supported as-is.)
Combined with 98e0c95d91ae16f14e4997fb64ccdf0956595712 ("build:
auto-generate signed mcuboot binaries"), this will allow users to
build and flash images to be chain loaded by mcuboot in a way that
avoids calling 'west sign' and passing 'west flash' its output files
entirely.
While we are here, rename runner_yml_write to runners_yaml_append().
This function doesn't actually write anything, and we're here
refactoring this file anyway, so we might as well improve the
situation while we're at it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The 'runner_config' variable name is particularly misleading because
there is a class called RunnerConfig, and that variable does not
contain one.
Rename it to 'runners_yaml' since it contains the parsed contents of
the runners.yaml file. Rename the variable that refers to the path
itself to 'runners_yaml_path'. No functional changes expected.
This is prep work for redoing how actual RunnerConfig objects get
made.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit creates a list of a phony targets for each runner, that is:
`west_flash_depends`, `west_debug_depends`, and so on.
Those targets has identical dependencies as CMake runner target.
flash, debug, debugserver, attach targets.
As example `ninja flash` correctly ensures dependencies are taken into
consideration before calling `west flash`.
Unfortunately, calling `west flash` directly would not re-run the flash
dependencies, cause `west flash` would only build the default CMake
target.
Now, `west flash` calls the phony `west_flash_depends` target, ensuring
all deps are up-to-date before flashing (unless --skip-rebuild is given)
The same is true for the other mentioned runners.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Just changes to the west help output; no functional changes expected.
Make option descriptions lowercase to match the argparse module's
conventions. When multiple sentences are required, move them to parser
prolog/epilog or argument group description sections.
Clarify some points that have confused multiple people.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Require all implementations to provide a do_create(), a new
ZephyrBinaryRunner abstract class method, and make create() itself
concrete.
This allows us to enforce common conventions related to individual
runner capabilities as each runner provides to the core via
RunnerCaps.
For now, just enforce that:
- common options related to capabilities are always added, so runners
can't reuse them for different ends
- common options provided for runners which don't support them emit
sensible error messages that should be easy to diagnose and support
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Originally reported in #23539 (though that seems to have been another
problem), west flash and friends are dumping stack when used with an
unconfigured runner.
Let's just promote the warning about this to an error. The idea that
this ever could have worked without explicit support has not worked
out in practice, to my knowledge.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is yet another bug introduced by the move to runners.yaml.
Sigh. I should have tested this better.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This will also make the reason for a following bug fix easier to see.
Update a comment block to include all the work that needs doing.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This makes its true value clearer, and will make a later bug fix patch
easier to understand.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Refactor the code to support the new runners.yaml file created by the
build system.
Compared to fishing around in the CMake cache, this makes it trivial
to put all the command line arguments to a runner-based command on
equal footing, regardless of if they're defined in the runners package
proper or defined in run_common.py.
This allows board.cmake files to do things like this:
board_set_runner_args(foo
--bin-file=${PROJECT_BINARY_DIR}/my-signed.bin)
While at it, make some other cleanups:
- Stop using the obsolete and deprecated west.cmake module while we're
here in favor of the zcmake.py module which was added to Zephyr a long
time ago. Yikes. I had forgotten this was still here.
- Stop using west.util's wrap function in favor of raw use of
textwrap. The west function splits on hyphens, which is breaking
runner names like "em-starterkit".
- Clean up the --context output a bit
Fixes: #22563
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The 'command' variable points at a python command object, not a
string. Take its name so the help text for west flash -h correctly
says 'flash' instead of 'debug'.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Catch RuntimeError when calling runner.run() and print a message
instead of dumping stack unless in verbose mode.
This improves the command line interface when runners raise exceptions.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Should be wrn() instead of warn(). Reported by pylint.
Also remove a {} from the message. It's not being formatted.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
scripts/west_commands/run_common.py:175:12: R1719: The if expression
can be replaced with 'test' (simplifiable-if-expression)
Fixing pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
When using a build folder format with build.dir-fmt that includes any
parameters that need resolving, the west runners cannot find the folder
since the required information (board, source dir or app) is not
available.
Add a very simple heuristic to support the case where a build folder
starts with a hardcoded prefix (for example 'build/') and a single build
is present under that prefix.
The heuristic is gated behind a new configuration option:
build.guess-dir
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Print a friendlier error message on ValueError, but don't throw away
the stack trace.
Move another call to log.die().
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Having common log handlers now lets us improve our logging output so
that info messages are prefixed with the runner they came from, and
doing something similar with the high level steps as we go, like this:
-- west <command>: using runners
-- runners.RUNNER_NAME: doing something
<output from RUNNER_NAME subprocesses go here>
-- runners.RUNNER_NAME: all done, bye
We can also colorize the west output to make it stand out better from
subprocesses, using the same output formatting style that west
commands like west list do.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
I've had some requests to be able to use code in the runners package
without having west installed.
It turns out to be pretty easy to make this happen, as west is
currently only used for west.log and some trivial helper methods:
- To replace west log, use the standard logging module
- Add an appropriate handler for each runner's logger in
run_common.py which delegates to west.log, to keep
output working as expected.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
The runners/jlink.py script has a mechanism for erroring out if a host
tool is not installed. Abstract it into runners/core.py and handle it
from run_common.py. This will let it be used in more places.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Vignesh Pandian
Phi Tran
Jamie McCrae
Camille BAUD
Pieter De Gendt
Jordan Yates
Tobias Pisani
Louis Feller
Michał Szprejda
Alberto Escolar Piedras
Peter Johanson
Gerhard Jörges
Torsten Rasmussen
Jeremy Bettis
Jordan Yates
Gerson Fernando Budke
Martí Bolívar
Ulf Magnusson
Carles Cufi