Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fixes: #40590
This commit updates gen_app_partitions.py to include only files present
in the current build by extracting the information from the CMake
generated `compile_commands.json` file.
This ensures that object files in sub-projects, such as `empty_cpu0`,
will not be considered by the script.
Using the compile_commands.json instead of walking the whole build tree
for finding object files also improves performance:
Time of executing `gen_app_partitions.py` (Old):
__________________________
Executed in 480.06 millis
usr time 425.83 millis
sys time 49.55 millis
Time of executing `gen_app_partitions.py` (New):
________________________________________________________
Executed in 76.22 millis
usr time 49.00 millis
sys time 24.59 millis
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Do not load empty files through the ELF parser and raise exception when
magic number of ELF is not matched.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some architectures (e.g. RISC-V) has .sdata/.sbss section for small
data/bss. Memory partition should also manage the permission of these
sections in library so they should be put into app_smem.
(For example, newlib _impure_ptr is in .sdata section and
__malloc_top_pad is in .sbss section in RISC-V.)
Signed-off-by: Jim Shu <cwshu@andestech.com>
When _app_smem region is empty, alignment is also needed. If there
is no alignment, the _app_smem_start used by arm mpu can be lower
than __rodata_region_end, and this two regions can overlap.
The Armv8-R aarch64 MPU does not allow overlapped regions.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This allows memory partitions to be put into the pinned
section so they are available during boot. For example,
the stack guard (in libc partition) is needed during boot
but before the paging mechanism is initialized. Without
pinning it in physical memory, it would fault in early
boot process.
A new cmake property app_smem,pinned_partitions is
introduced so that additional partitions can be pinned
if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
MWDT toolchain adds additional suffix to sections name in case of
ffunction-sections / fdata-sections are enabled.
As proposed by Andy Ross let's pick a single set of rules
and syntax that work.
Suggested-by: Andy Ross <andy@plausible.org>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Reported by pylint's 'bad-whitespace' warning.
Not gonna enable this warning in the CI check, because it flags stuff
like deliberately aligning assignments and gets too cultish. Just a
cleanup pass.
For whatever reason, the common convention in Python is to skip spaces
around '=' when passing keyword arguments and giving default arguments:
f(x=3, y=4)
def f(x, y=8):
...
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Commit 212ec9a29a / feature #14121 already ordered partitions by
decreasing size, however it was common in samples/userspace/shared_mem/
/sample.kernel.memory_protection.shared_mem for two partitions to have
the same size and be randomly ordered between them. This adds the
partition name as a second sort key.
Unlike previous attempt in commit 725abdf430 this doesn't use the
partition name as the first (and only) key and doesn't break the
decreasing size order. Huge thanks to Sigvart Hovland for spotting this
in a post-merge but prompt code review.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This reverts commit 725abdf430 which did get rid of randomness in the
order of the partition _names_ as claimed but regressed commit
212ec9a29a / feature #14121 and broke the previous size order which I
missed. Huge thanks to Sigvart Hovland for spotting this in a post-merge
but prompt code review. Proper fix in the next commit.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Dictionaries are not ordered in Python 3.5 and before, so building twice
in a row could lead to a different partition order, different
build/zephyr/include/generated/app_smem_*.ld files and different
binaries.
Fix with a minor change to the "for" loop in the output function:
make it iterate on sorted(partitions.items()) instead of the raw and
randomly ordered partitions dictionary.
It is easy to reproduce the issue even without downgrading to an
obsolete Python version; pick a test like samples/userspace/shared_mem/
and simply change the code to this:
--- a/scripts/gen_app_partitions.py
+++ b/scripts/gen_app_partitions.py
@@ -159,10 +159,12 @@ def parse_elf_file(partitions):
partitions[partition_name][SZ] += size
+import random
def generate_final_linker(linker_file, partitions):
string = linker_start_seq
size_string = ''
- for partition, item in sorted(partitions.items()):
+ for partition, item in sorted(partitions.items(),
+ key=lambda x: random.random()):
string += data_template.format(partition)
if LIB in item:
for lib in item[LIB]:
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes pylint warnings like this one:
doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).
It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.
Also note that '\s' and '\\s' turn into the same string.
Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Discovered with pylint3.
Use the placeholder name '_' for unproblematic unused variables. It's
what I'm used to, and pylint knows not to flag it.
Python tip:
for i in range(n):
some_list.append(0)
can be replaced with
some_list += n*[0]
Similarly, 3*'\t' gives '\t\t\t'.
(Relevant here because pylint flagged the loop index as unused.)
To do integer division in Python 3, use // instead of /.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
os.exit() doesn't exist.
Also use the nifty sys.exit(msg) feature, which prints 'msg' to stderr
and exits with status 1.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Not needed in Python. Detected by check C0325 in pylint3.
Also replace an
if len(tag):
with just
if tag:
Empty strings, byte strings, lists, etc., are falsy in Python.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
(OPTIONAL) was a vestiage from the initial import of the Zephyr code
base and we dont utilize it with the GNU linker. Additionally, the way
(OPTIONAL) gets defined to nothing creates a linker script that lld
(from llvm) doesn't like.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT is enabled,
the app shared memory partition may cause waste of memory
due to the need for padding.
For example, tests/subsys/jwt and board mps2_an385:
z_test_mem_partition: addr 0x20000000, size 52
z_libc_partition : addr 0x20000040, size 4
k_mbedtls_partition : addr 0x20008000, size 32736
ending at 0x2000ffff, taking up 65536 bytes
With power-of-two size and alignment requirement,
k_mbedtls_partition takes up 32KB memory and needs to be
aligned on 32KB boundary. If the above partitions are
ordered as shown, there needs to be a lot of padding
after z_libc_partition before k_mbedtls_partition can
start. In order to minimize padding, these partitions
need to be sort by size in descending order.
After the changes here, the partitions are:
k_mbedtls_partition : addr 0x20000000, size 32736
z_test_mem_partition: addr 0x20008000, size 52
z_libc_partition : addr 0x20008040, size 4
ending at 0x2000805f, taking up 32864 bytes
With the above example, sorting results in a saving
of 32672 bytes of saving.
Fixes#14121
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We have several scripts used by the build system related
to generating code for system calls, privileged mode stacks,
kernel object metadata, and application shared memory
partitions. Add some overview documentation for each.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
App shared memory defines a bunch of symbols in the
linker script. Namespace them properly as private
zephyr variables.
The variables which indicate the bounds of the entire
partition now end with "_part_start", "_part_size",
and "_part_end" to make them easy for scripts to
distinguish them from other generated symbols for
data/bss sizes.
Finally, the bss size is not rounded up, this was
causing unnecessary memory to be zeroed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is an integral part of userspace and cannot be used
on its own. Fold into the main userspace configuration.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Zephyr may be linked against third-party libraries which
were not part of the build. These may contain globals
which need to end up in a memory partition.
We can now specify the names of these libraries, as well
as a destination partition for their globals.
Some excessively long variables were renamed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
* K_APP_DMEM_SECTION/K_MEM_BMEM_SECTION macros now exist
to specifically define the name of the sections for data
and bss respectively.
* All boards now use the gen_app_partitions.py script, the
padding hacks for non-power-of-two arches didn't work right
in all cases. Linker scripts have been updated.
* The defined k_mem_partition is now completely initialized
at build time. The region data structures now only exist
to zero BSS.
Based on some work submitted by Adithya Baglody
<adithya.baglody@intel.com>
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Move the definition of _image_ram_start at the beginning
of the RAMMABLE (SRAM) region, so it points to the actual
start of RAM linker sections.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit standardizes and simplifies the way we enforce
linker section alignment, to comply with minimum alignment
requirement for MPU, if we build Zephyr with MPU support:
- it enforces alignment with the minimum MPU granularity at
the beginning and end of linker sections that require to
be protected by MPU,
- it enforces alignment with size if required by the MPU
architecture.
Particularly for the Application Memory section, the commit
simplifies how the proper alignment is enforced, removing
the need of calculating the alignment with a post-linker
python script. It also removes the need for an additional
section for padding.
For the Application Shared Memory section(s), the commit
enforces minimum alignment besides the requirement for
alignment with size (for the respective MPUs) and fixes
a bug where the app_data_align was erronously used in the
scipts for auto-generating the linker scripts.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The size calculation for power of 2 MPUs were incorrect.
The calculation was not taking into account the amount of padding
the linker does when doing the required alignment. Hence the size
being calculated was completely incorrect.
With this patch the code now is optimized and the size of
partitions is now provided by the linker.
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
This feature was failing on a default ARM core MPU. The linker
script that was getting created was not able to align the required
partitions at prebuilt time.
The old implementation relied on the prebuilt to finish then
extract the size information which was then used to align the regions.
This fails because the size of the alignment and the fill in the
linker needs to be available at prebuilt time else it cant manage
the final elf file generation. We cant have 2 different sizes of
prebuilt and final elf file.
This implementation will get the alignment requirements met at
prebuilt time.
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Jamie McCrae
Anas Nashif
Torsten Rasmussen
Jim Shu
Jaxson Han
Daniel Leung
Eugeniy Paltsev
Ulf Magnusson
Marc Herbert
Kumar Gala
Andrew Boie
Ioannis Glaropoulos
Adithya Baglody