Support parsing and serializing of struct fields that are defined as a
char array.
Use the token JSON_TOK_STRING_BUF to parse and serialize a string for a
char array, for example:
struct foo {
const char *str;
char str_buf[30];
};
struct json_obj_descr foo_descr[] = {
JSON_OBJ_DESCR_PRIM(struct foo, str, JSON_TOK_STRING),
JSON_OBJ_DESCR_PRIM(struct foo, str_buf, JSON_TOK_STRING_BUF),
};
The struct 'json_obj_descr' now has an additional union member 'field'
to store the size of the struct field, which is used with the token
'JSON_TOK_STRING_BUF' to determine the element size.
Fixes: #65200
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
Up to now, the handling of type float was offloaded to the users of the
JSON utility, with the token JSON_TOK_FLOAT.
Improve handling of floating point types and support the types 'float'
and 'double' in a built-in way so that they can be directly parsed to
and serialized from variables (of type float or double).
The types are serialized in the shortest representation, either as a
decimal number or in scientific notation:
* float (with JSON_TOK_FLOAT_FP): encoded with maximal 9 digits
* double (with JSON_TOK_DOUBLE_FP): encoded with maximal 16 digits
* NaN, Infinity, -Infinity: encoded and decoded as:
{"nan_val":NaN,"inf_pos":Infinity,"inf_neg":-Infinity}
Enable the floating point functionality with the Kconfig option:
JSON_LIBRARY_FP_SUPPORT=y
It requires a libc implementation with support for floating point
functions: strtof(), strtod(), isnan() and isinf().
Fixes: #59412
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
Add support to decode the special floating point values NaN, Infinity
and -Infinity. For example:
{"nan_val":NaN,"inf_pos":Infinity,"inf_neg":-Infinity}
Note that this commit is a preparation for the built-in support of
floating point values and these are only accepted when compiled with
the flag -DCONFIG_JSON_LIBRARY_FP_SUPPORT.
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
Add support to decode floating point numbers in scientific notation,
e.g. 3.40282347e+38. Only the lower-case specifier 'e' is allowed.
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
The calculation of the object size may be incorrect when the size of
a field is smaller than the struct alignment. When such a struct is
used in an array field, the decoded object contains wrong values.
The alignment influences the object size. For example the following
struct has a calculated object size of 8 bytes, however due to
alignment the real size of the struct is 12 bytes:
struct test_bool {
bool b1; /* offset 0, size 1 */
/* 3-byte padding */
int i1; /* offset 4, size 4 */
bool b2; /* offset 8, size 1 */
/* 3-byte padding */
};
This commit changes the object size calculation and computes the size
with the offset and size of the last field in the struct (rounded up
by the struct alignment).
Fixes: #85121
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
Utilize a code spell-checking tool to scan for and correct spelling errors
in various files within the `lib` directory.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Introduce support for 'uint64_t' type, so that unsigned numbers
can be serialized into JSON payloads.
Signed-off-by: Mykhailo Lohvynenko <Mykhailo_Lohvynenko@epam.com>
Up to now there was only support for parsing/encoding 32-bit integer
numbers, with no support for larger ones.
Introduce support for 'int64_t' type, so that large numbers can be
serialized into JSON payloads.
Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
Currently, json array macros are passing the outer "container" struct
to the Z_JSON_ELEMENT_DESCR macro, causing the alignment to be calculated
from the outer struct which may be incorrect. Fix this by using the
biggest shift from the nested objects to calculate the alignment as the
struct size is calculated based on the biggest alignment of it's members.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Move various utilities out of lib into own folder for better assignement
and management in the maintainer file. lib/os has become another dumping
ground for everything and it the Kconfig and contents in that folder
became difficult to manage, configure and test.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Skip child objects and arrays that are not specified in the given object
descriptor when parsing a JSON input string.
This patch adds support for extra child arrays which previously were not
supported by the parser as opposed to additional child objects.
Fixes#47988
Signed-off-by: Markus Fuchs <markus.fuchs@ch.sauter-bc.com>
This patch fixes support for encoding and decoding multidimensional arrays
as described by the JSON_OBJ_DESCR_ARRAY_ARRAY() macro.
Currently, the JSON array encoding and decoding functions, arr_encode()
and arr_parse(), expect array elements to be of object or primitive type.
However, arrays may be nested and so an array's elements may also be
arrays.
In order to support nested arrays, two special cases must be considered:
1. The array of objects/arrays sub-descriptor is described by two
`json_obj_descr` structs and so two instead of one `json_obj_descr`
structs must be skipped when iterating over the JSON descriptor to get to
an array's elements.
2. The implicit array item count field has to be considered for the
parent itself and all its child array items when calculating an element's
size.
Fixes#50801
Signed-off-by: Markus Fuchs <markus.fuchs@ch.sauter-bc.com>
Analog to json_obj_encode vs. json_calc_encoded_len which
calculates the object len using json_obj_encode, introduce
json_calc_encoded_arr_len which calculates the length using
json_arr_encode. That is needed when the object to be encoded
is array on the root level.
Signed-off-by: Miika Karanki <miika.karanki@vaisala.com>
We get compile warnings of the form:
error: converting the result of
'<<' to a boolean; did you mean
'((__aeabi_ctype_table_ + 1)[(byte)] << 28) != 0'?
[-Werror,-Wint-in-bool-context]
if (!isprint(byte)) {
^
Since isprint (and the other is* functions) return an int, change check
to an explicit test against the return value.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
The limiting factor is the output bitmask that says which elements have
been filled in by the parser. This patch changes the bitmask type from int
to int64_t.
Signed-off-by: Björn Stenberg <bjorn@haxx.se>
New Types:
* Added support for Generic Numeric type (float, 64-bit)
* Added support for Opaque string type.
* Added support parse Array data in object for seprate array parsing
New API for Json Array parsing Object 1 by 1:
* json_arr_separate_object_parse_init() init array parse
* json_arr_separate_parse_object() Parsing 1 Json Object
Rename token and lexer structures and publish those.
Signed-off-by: Juha Heiskanen <juha.heiskanen@nordicsemi.no>
In order to bring consistency in-tree, migrate all lib code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The library supports encoding JSON objects and arrays as well as
parsing JSON objects. Introduce a new function json_arr_parse() adding
support for parsing top-level JSON arrays.
Signed-off-by: Markus Fuchs <markus.fuchs@ch.sauter-bc.com>
Replace all calls to the assert macro that comes from libc by calls to
__ASSERT_NO_MSG(). This is usefull as the former might be different
depending on the libc used and the later can be customized to reduce
flash footprint.
Signed-off-by: Xavier Chapron <xavier.chapron@stimio.fr>
-Wimplicit-fallthrough=2 requires a fallthrough comment or a compiler
to tells gcc that this happens intentionally.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
The library supports the declaration of JSON arrays as both nested and
top-level elements. However, as the provided encoding functions
json_obj_encode() and json_obj_encode_buf() interpret all input
structures as objects, top-level arrays are encoded as
{"<field_name>":[{...},...,{...}]}
instead of
[{...},...,{...}].
Add new functions json_arr_encode() and json_arr_encode_buf() that
enable top-level JSON array encoding.
Signed-off-by: Markus Fuchs <markus.fuchs@de.sauter-bc.com>
The bounds check failed to account for the additional space required
for the terminating NUL after the encoded value was written.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
move misc/util.h to sys/util.h and
create a shim for backward-compatibility.
No functional changes to the headers.
A warning in the shim can be controlled with CONFIG_COMPAT_INCLUDES.
Related to #16539
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
move misc/printk.h to sys/printk.h and
create a shim for backward-compatibility.
No functional changes to the headers.
A warning in the shim can be controlled with CONFIG_COMPAT_INCLUDES.
Related to #16539
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
move json.h to data/json.h and
create a shim for backward-compatibility.
No functional changes to the headers.
A warning in the shim can be controlled with CONFIG_COMPAT_INCLUDES.
Related to #16539
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The struct json_obj_descr definition allocates only 2 bits for type
alignment. Instead of using them literally minus 1 to encode 1, 2, or 4,
let's store the alignment's shift value instead so that 1, 2, 4 or 8 can
be encoded with the same 2 bits to accommodate 64-bit builds.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
lib/ was starting to get messy and inconsitent. Files being either
dumped in the root or in sub-directories without a clear plan.
Move all library components into one single folder and call it 'os'.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If we use newlib the isdigit (and other similar functions) return an
error as char can possibly be viewed as signed:
usr/include/ctype.h:57:54: error: array subscript has type ‘char’ [-Werror=char-subscripts]
#define __ctype_lookup(__c) ((__ctype_ptr__+sizeof(""[__c]))[(int)(__c)])
Explicity cast to unsigned char so we deal with both this warning and
possible warning when -Wpointer-sign is enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This trades a little bit over 40 bytes (on x86) of text for a lot of
savings in rodata. This is accomplished by using bitfields to pack the
field name length, offset, alignment, and the type tag into a single
32-bit unsigned integer instead of scattering this information into
four different integers.
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
This was causing an unaligned pointer read on some architectures,
leading to crashes. This could be alternatively solved by rounding
the size to the nearest power of 2, but this wouldn't work with
packed structs.
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
This appears to be a bug in GCC: when an anonymous union contains
anonymous structs, GCC issues a warning that a field in one of the
anonymous structs has not been initialized. Fix by making the
structs not anonymous.
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
append_bytes_to_buf() already writes a NUL byte; no need to call
append_bytes() again with "" and size 1.
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
This function currently fails when decoding an array with number of
elements exactly equal to the maximum available in the struct.
To fix this, move the check for if the current field is past the end
of the array to just before attempting to decode a value. This allows
the last element to be followed by a JSON_TOK_LIST_END token in the
case that the array is full, and the function to return success.
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
The JSON library doesn't properly encode arrays whose elements are of
object type. Fix that.
This fix avoids allocating a temporary descriptor on the stack, and
keeps the size of struct json_obj_descr unchanged, by preserving an
unintuitive size optimization made by the library. See the comments
in the patch for more details.
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
Move all characters to "char" type: no implicit conversions between
"unsigned char", "u8_t", etc.
Tested with ISSM 2016.2.085.
Jira: ZEP-2159
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
Currently, json_escape() allocates a temporary buffer on the stack
that's the size of the string being escaped.
Stack space is precious and longer JSON strings can run into the
hundreds of bytes, so re-implement this routine to escape in place.
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
Convert code to use u{8,16,32,64}_t and s{8,16,32,64}_t instead of C99
integer types.
Jira: ZEP-2051
Change-Id: I74bc6384c4090f4ae322e3aa151874f583a5fe73
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This is a start to move away from the C99 {u}int{8,16,32,64}_t types to
Zephyr defined u{8,16,32,64}_t and s{8,16,32,64}_t. This allows Zephyr
to define the sized types in a consistent manor across all the
architectures we support and not conflict with what various compilers
and libc might do with regards to the C99 types.
We introduce <zephyr/types.h> as part of this and have it include
<stdint.h> for now until we transition all the code away from the C99
types.
We go with u{8,16,32,64}_t and s{8,16,32,64}_t as there are some
existing variables defined u8 & u16 as well as to be consistent with
Zephyr naming conventions.
Jira: ZEP-2051
Change-Id: I451fed0623b029d65866622e478225dfab2c0ca8
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If we use newlib the isdigit (and other similar functions) return an
error as char can possibly be viewed as signed:
usr/include/ctype.h:57:54: error: array subscript has type ‘char’ [-Werror=char-subscripts]
#define __ctype_lookup(__c) ((__ctype_ptr__+sizeof(""[__c]))[(int)(__c)])
Being explicit about the char being unsigned char deals with this.
Change-Id: If2416218220ef5b29f1a69470cbcc6b4fd49ef86
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Boolean values were being decoded using the descriptor type rather than
the value type.
Jira: ZEP-1607
Change-Id: I0c9324ee705af973ccf738e92785820c3a5fb692
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
The function to ignore spaces was not being called, so some tokens had
whitespace in the beginning. They were correctly lexed, but parsing
could eventually fail.
Jira: ZEP-1607
Change-Id: I796596143895fa0fa652641f56af9a03e7a65b7a
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
Comparing *endptr with '\0' will always be true before replacing
*token->end with prev_end.
Jira: ZEP-1607
Change-Id: I224129586e15380d3919bfba3db4fcf38c28cb07
Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
Christoph Winklhofer
Pisit Sawangvonganan
Mykhailo Lohvynenko
Marcin Niestroj
Mykyta Poturai
Fin Maaß
Anas Nashif
Markus Fuchs
Miika Karanki
Kumar Gala
Björn Stenberg
Juha Heiskanen
Gerard Marull-Paretas
Ramiro Merello
Michael Zimmermann
Xavier Chapron
Flavio Ceolin
Markus Fuchs
Kumar Gala
Peter Bigot
Nicolas Pitre
Leandro Pereira
Marti Bolivar