symbolize.py: add support for TEE core ASLR

With the introduction of ASLR, the abort and panic dumps for the TEE
core have gained a "Load address" line. Update the symbolize.py script
to parse that

symbolize.py: add support for TEE core ASLR

With the introduction of ASLR, the abort and panic dumps for the TEE
core have gained a "Load address" line. Update the symbolize.py script
to parse that line and pass relative addresses to addr2line.

Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Jerome Forissier <jerome@forissier.org>

show more ...

ftrace: Add support for syscall function tracer

This patch adds support for syscall tracing in TEE core. It complements
existing ftrace support for user TAs via adding trace for syscalls that
are in

ftrace: Add support for syscall function tracer

This patch adds support for syscall tracing in TEE core. It complements
existing ftrace support for user TAs via adding trace for syscalls that
are invoked by user TAs into the TEE core.

And after this patch ftrace will cover both TA and TEE core code. So lets
rename config option from CFG_TA_FTRACE_SUPPORT to CFG_FTRACE_SUPPORT.

It is optional to enable syscall trace via CFG_SYSCALL_FTRACE=y config
option in addition to CFG_FTRACE_SUPPORT=y config option.

Signed-off-by: Sumit Garg <sumit.garg@linaro.org>
Reviewed-by: Jerome Forissier <jerome@forissier.org>

show more ...

scripts: add update_changelog.py

Add a simple helper script that makes the tedious and slightly error
prone update process easier and more stable.

Usage example:
$ ./scripts/update_changelog.py -

scripts: add update_changelog.py

Add a simple helper script that makes the tedious and slightly error
prone update process easier and more stable.

Usage example:
$ ./scripts/update_changelog.py --changelog-file CHANGELOG.md \
--release-version 3.7.0 --previous-release-version 3.6.0 \
--release-date 2019-10-18

Signed-off-by: Joakim Bech <joakim.bech@linaro.org>
Acked-by: Jerome Forissier <jerome@forissier.org>

show more ...

scripts/bin_to_c.py: add support for text files

bin_to_c.py may be useful to generate a C file from a text file, not
only from a binary file. This patch adds the --text argument for that
purpose. Wi

scripts/bin_to_c.py: add support for text files

bin_to_c.py may be useful to generate a C file from a text file, not
only from a binary file. This patch adds the --text argument for that
purpose. With --text, the script will:
- define a "const char []" variable,
- error out if a null byte is encountered,
- terminate the input string with a null byte.

Signed-off-by: Jerome Forissier <jerome@forissier.org>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

Experimental Clang support

Allows building with Clang with "make COMPILER=clang [other flags...]".
The clang command has to be in the $PATH, as well as the associated
tools (clang-cpp, ld.lld, llvm-

Experimental Clang support

Allows building with Clang with "make COMPILER=clang [other flags...]".
The clang command has to be in the $PATH, as well as the associated
tools (clang-cpp, ld.lld, llvm-ar, llvm-nm, llvm-objcopy and
llvm-readelf).

Tested with Clang built from the master branch of [1] (development
version for 9.0):

mkdir build; cd build
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=~/llvm-install \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DLLVM_TARGETS_TO_BUILD="AArch64;ARM" \
~/llvm-project/llvm
ninja && ninja install

Limitations:

- CFG_CORE_SANITIZE_KADDRESS=y is not supported.
- CFG_WITH_PAGER is supported, but requires that the TEE core be
linked with the GNU linker. The reason is documented in
mk/clang.mk.

Bug:

- ldelf assertion failure in xtest 1019 when CFG_ULIBS_SHARED=y (QEMU)
E/LD: assertion 'maps[map_idx].sz == sz' failed at ldelf/ta_elf.c:1114 in ta_elf_print_mappings()
Prevents ldelf from displaying the TA mappings on abort or panic, but
does not seem to cause any other problem.

Link: [1] https://github.com/llvm/llvm-project/commits/8351c327647
Signed-off-by: Jerome Forissier <jerome@forissier.org>
Tested-by: Jerome Forissier <jerome@forissier.org> (QEMU pager/no pager)
Tested-by: Jerome Forissier <jerome@forissier.org> (QEMUv8, pager/no pager)
Tested-by: Jerome Forissier <jerome@forissier.org> (HiKey960, 32/64, GP)
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

symbolize.py: fix exception when stdin is not a terminal

Commit 6b4fc6752b3c ("symbolize.py: disable terminal local echo") uses
termios functions on stdin unconditionally. Unfortunately, this will
c

symbolize.py: fix exception when stdin is not a terminal

Commit 6b4fc6752b3c ("symbolize.py: disable terminal local echo") uses
termios functions on stdin unconditionally. Unfortunately, this will
cause an exception when stdin is not a terminal, for instance:

$ echo Hello | ./script/symbolize.py
Traceback (most recent call last):
File "./scripts/symbolize.py", line 497, in <module>
main()
File "./scripts/symbolize.py", line 484, in main
old = termios.tcgetattr(fd)
termios.error: (25, 'Inappropriate ioctl for device')

Fix the issue by making sure stdin is a TTY before using the termios
functions.

Fixes: 6b4fc6752b3c ("symbolize.py: disable terminal local echo")
Signed-off-by: Jerome Forissier <jerome@forissier.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

symbolize.py: disable terminal local echo

When scripts/symbolize.py is used interactively and the input data is
copied and pasted into the terminal, both the input and output text are
displayed on t

symbolize.py: disable terminal local echo

When scripts/symbolize.py is used interactively and the input data is
copied and pasted into the terminal, both the input and output text are
displayed on the same screen. There is nothing preventing both text
streams from mixing up because it depends on buffering and how Python
reads and flushes its buffers.

Things usually go well with Python 2, but I have observed that Python 3
is more problematic. So the issue needs to be properly addressed. This
patch temporarily disables local echo so that only the processed text is
shown.

Signed-off-by: Jerome Forissier <jerome@forissier.org>
Reviewed-by: Rouven Czerwinski <r.czerwinski@pengutronix.de>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts: convert remainging scripts to python3

Convert the remaining scripts to python3. Mostly done with 2to3 and
manual adjustements to the code afterwards.

Signed-off-by: Rouven Czerwinski <r.cz

scripts: convert remainging scripts to python3

Convert the remaining scripts to python3. Mostly done with 2to3 and
manual adjustements to the code afterwards.

Signed-off-by: Rouven Czerwinski <r.czerwinski@pengutronix.de>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome@forissier.org>
Tested-by: Jerome Forissier <jerome@forissier.org> (QEMU, QEMUv8)

show more ...

get_maintainer.py: fix spelling mistake

OSError is spelled incorrectly, causing the following error:

$ ./scripts/get_maintainer.py core/arch/arm/plat-imx/
Traceback (most recent call last):
Fi

get_maintainer.py: fix spelling mistake

OSError is spelled incorrectly, causing the following error:

$ ./scripts/get_maintainer.py core/arch/arm/plat-imx/
Traceback (most recent call last):
File "./scripts/get_maintainer.py", line 102, in split_patchset
f = open(patchset, "r")
IsADirectoryError: [Errno 21] Is a directory: 'core/arch/arm/plat-imx/'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./scripts/get_maintainer.py", line 292, in <module>
main()
File "./scripts/get_maintainer.py", line 242, in main
patches = split_patchset(arg)
File "./scripts/get_maintainer.py", line 103, in split_patchset
except OsError:
NameError: name 'OsError' is not defined

Signed-off-by: Jerome Forissier <jerome@forissier.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts/sign.py: support PyCryptodome

Depending on the Linux distribution, the Crypto package may be
provided by PyCrypto [1] or PyCryptodome [2]. For example, Ubuntu has
the former, while Arch Linu

scripts/sign.py: support PyCryptodome

Depending on the Linux distribution, the Crypto package may be
provided by PyCrypto [1] or PyCryptodome [2]. For example, Ubuntu has
the former, while Arch Linux comes with the latter.

PyCryptodome is a fork of PyCrypto, which is not maintained any more
(the last release dates back to 2013 [3]). It exposes almost the same
API, but there are a few incompatibilities [4].

Newer versions of PyCryptodome throw NotImplementedError when calling
size() method if RSA object. This is because this method is
considered not safe. But this is only one way to get key size in
PyCrypto.

We will use size_in_bytes() in case if size() is not available.

Also, added sanity check to ensure that computed signature size
equals to actual one.

[1] https://pypi.org/project/pycrypto/
[2] https://pycryptodome.readthedocs.io/en/latest/
[3] https://pypi.org/project/pycrypto/#history
[4] https://pycryptodome.readthedocs.io/en/latest/src/vs_pycrypto.html

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

get_maintainer.py: process patches individually

When given the -m/-merge-check option, get_maintainer.py parses the
Acked-by and Reviewed-by tags that may be found in a patchset or
Github PR. In pre

get_maintainer.py: process patches individually

When given the -m/-merge-check option, get_maintainer.py parses the
Acked-by and Reviewed-by tags that may be found in a patchset or
Github PR. In presence of several patches the tags should apply to
each patch individually, not to the whole patchset as is currently
done. As a result, the script may fail to report some unapproved
changes.

Fix this issue by splitting patchsets into individual patches before
processing.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

checkpatch: allow string concatenation without space

Adds --ignore CONCATENATED_STRING to checkpatch to allow constructs
like:
DMESG("value1 %"PRIX32" value2 %"PRIX32, v1, v2);

Without this we're f

checkpatch: allow string concatenation without space

Adds --ignore CONCATENATED_STRING to checkpatch to allow constructs
like:
DMESG("value1 %"PRIX32" value2 %"PRIX32, v1, v2);

Without this we're forced to use:
DMESG("value1 %" PRIX32 " value2 %" PRIX32, v1, v2);

Which is a bit harder to read, especially for more complicated cases.

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

checkpatch: use --strict option

Uses the --strict option with checkpatch to catch more of the common style
issues. Ignoring the PREFER_KERNEL_TYPES since we are normally not using
those types. CAMEL

checkpatch: use --strict option

Uses the --strict option with checkpatch to catch more of the common style
issues. Ignoring the PREFER_KERNEL_TYPES since we are normally not using
those types. CAMELCASE is also ignored for now since it triggers many false
positives due to the TEE_Result type.

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

build: correct sign.py mandated arguments

Script sign.py mandates arguments --key and --inf. This change corrects
argument definition and usage accordingly.

Signed-off-by: Etienne Carriere <etienne

build: correct sign.py mandated arguments

Script sign.py mandates arguments --key and --inf. This change corrects
argument definition and usage accordingly.

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

script/sign.py: introduce --ta-version

Restores argument --version to script sign.py to allow user to
set the trusted application version identifier in the signed
header of the TA binary image. This

script/sign.py: introduce --ta-version

Restores argument --version to script sign.py to allow user to
set the trusted application version identifier in the signed
header of the TA binary image. This argument was removed by [1].
This change also renames argument into --ta-version to prevent
confusion with script or data structure versioning argument.

[1] commit 1cdd95a2a46d ("Support offline signing of TAs.")

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

arm32_sysreg.py: shorten lines to <= 79 characters

pycodestyle v2.4.0 complains:

$ pycodestyle scripts/arm32_sysreg.py
scripts/arm32_sysreg.py:72:80: E501 line too long (80 > 79 characters)
scripts

arm32_sysreg.py: shorten lines to <= 79 characters

pycodestyle v2.4.0 complains:

$ pycodestyle scripts/arm32_sysreg.py
scripts/arm32_sysreg.py:72:80: E501 line too long (80 > 79 characters)
scripts/arm32_sysreg.py:99:80: E501 line too long (80 > 79 characters)

Break those two lines before the 79 character limit.

Fixes: 4486d5866e238 ("libutee: add headers for user-space to access sysregs")
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

libutee: add headers for user-space to access sysregs

User space may require to access system registers like generic timer
registers in case function tracing is enabled etc. So provide headers
for u

libutee: add headers for user-space to access sysregs

User space may require to access system registers like generic timer
registers in case function tracing is enabled etc. So provide headers
for user space to access sysregs.

Signed-off-by: Sumit Garg <sumit.garg@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

Add scripts/get_maintainer.py

Adds a script to help identify all the maintainer/reviewers for some
path in the source tree, or for some patch or Github pull request.
When processing a patch or a pul

Add scripts/get_maintainer.py

Adds a script to help identify all the maintainer/reviewers for some
path in the source tree, or for some patch or Github pull request.
When processing a patch or a pull request the script can take into
account the Acked-by: and Reviewed-by: tags, so that it can easily tell
if a change is ready to be merged or still needs reviewing (and by
whom). Information comes from the MAINTAINERS file.

The script is similar to the Linux kernel's get_maintainer.pl, although
its use cases are a bit different.

Examples:

1. Who maintains the i.MX platform? Do not show the top-level maintainers
(subsystem "THE REST"), unless there is no specific maintainer.

$ ./scripts/get_maintainer.py --strict core/arch/arm/plat-imx
Peng Fan <peng.fan@nxp.com> [@MrVan] (reviewer:NXP (Freescale) i.MX family)
$

2. Who maintains the i.MX configuration file?

$ ./scripts/get_maintainer.py --strict core/arch/arm/plat-imx/conf.mk
Peng Fan <peng.fan@nxp.com> [@MrVan] (reviewer:NXP (Freescale) i.MX family)
Peng Fan <peng.fan@nxp.com> [@MrVan] (reviewer:NXP (Freescale) i.MX7 WaRP7)
Bryan O'Donoghue <bryan.odonoghue@linaro.org> [@bryanodonoghue] (reviewer:NXP (Freescale) i.MX7 WaRP7)
$

3. Is PR 3169 ready for merging?

$ ./scripts/get_maintainer.py --merge-check --github-pr 3169
Downloading https://github.com/OP-TEE/optee_os/pull/3169.patch... Done.
$

4. Is PR 3169 lacking some reviews from subsystem maintainers? In other
words, is it still properly approved for merging if we ignore the tags
given by members of subsystem "THE REST" whenever there is another
subsystem maintainer?

$ ./scripts/get_maintainer.py --merge-check --strict --github-pr 3169
Peng Fan <peng.fan@nxp.com> [@MrVan] (reviewer:NXP (Freescale) i.MX family)
Peng Fan <peng.fan@nxp.com> [@MrVan] (reviewer:NXP (Freescale) i.MX7 WaRP7)
Bryan O'Donoghue <bryan.odonoghue@linaro.org> [@bryanodonoghue] (reviewer:NXP (Freescale) i.MX7 WaRP7)
$

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

build: fix TA signing offline signing script

The script now exits with a return value of 1 instead of 0 if the
key is not usable for signing.
A typo in regenerating the digest during stitching was f

build: fix TA signing offline signing script

The script now exits with a return value of 1 instead of 0 if the
key is not usable for signing.
A typo in regenerating the digest during stitching was fixed.
Command dispatching was simplified.

Fixes: 1cdd95a2a46d ("Support offline signing of TAs.")
Signed-off-by: Markus S. Wamser <markus.wamser@mixed-mode.de>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

gen_ldelf_hex.py: print how to obtain elftools module

There is recurring problem when OP-TEE build fails with cryptic
message from this python script. It occurs when user forgets to
install elftools

gen_ldelf_hex.py: print how to obtain elftools module

There is recurring problem when OP-TEE build fails with cryptic
message from this python script. It occurs when user forgets to
install elftools python module.

Print human-readable error message with instructions how to install
it.

This is mere convenience, with no functional changes to the script
itself.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

Support offline signing of TAs.

Add generate-digest and stitch-ta support to script sign.py. One can
generate a digest for the loadable TA file, sign it offline and later
stitch the generated signat

Support offline signing of TAs.

Add generate-digest and stitch-ta support to script sign.py. One can
generate a digest for the loadable TA file, sign it offline and later
stitch the generated signature to the loadable TA binary file.

For this purpose sign.py gets an extra initial command argument to select
between sign/digest/stitch operations. If no command is provided, `sign`
is assumed, preserving previous usage.

Signed-off-by: Markus S. Wamser <markus.wamser@mixed-mode.de>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

ta_bin_to_c.py: mark as python3 script

This commit marks the script as a python3 script and update
hex string conversion accordingly.

Moving to python3 requires a change in hex string conversion.
I

ta_bin_to_c.py: mark as python3 script

This commit marks the script as a python3 script and update
hex string conversion accordingly.

Moving to python3 requires a change in hex string conversion.
It causes by `zlib.compress()` behaving differently in python2
and python3. It returns a string in python2, and it returns
a bytes object in python3.

With python3 `zlib.compress()` behavior, we can use hex() to
convert bytes[i] of type int into a hex string.

Signed-off-by: Pipat Methavanitpong <pipat.methavanitpong@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Rouven Czerwinski <r.czerwinski@pengutronix.de>

show more ...

ldelf: scripts/gen_ldelf_hex.py: allow multiple RW load segments

Allow multiple read/write load segments as long as they are contiguous.
This is to support Clang > 8.0.0 which generates this:

Prog

ldelf: scripts/gen_ldelf_hex.py: allow multiple RW load segments

Allow multiple read/write load segments as long as they are contiguous.
This is to support Clang > 8.0.0 which generates this:

Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x001000 0x00000000 0x00000000 0x0a50c 0x0a50c R E 0x1000
LOAD 0x00c000 0x0000b000 0x0000b000 0x0006c 0x0006c RW 0x1000
LOAD 0x00c06c 0x0000b06c 0x0000b06c 0x0006c 0x0007c RW 0x1000
DYNAMIC 0x00c000 0x0000b000 0x0000b000 0x00060 0x00060 RW 0x4
GNU_RELRO 0x00c000 0x0000b000 0x0000b000 0x0006c 0x01000 R 0x1
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0
EXIDX 0x00b134 0x0000a134 0x0000a134 0x00258 0x00258 R 0x4

Section to Segment mapping:
Segment Sections...
00 .text .rodata .ARM.exidx .gnu.hash .ARM.extab .dynsym .dynstr .hash .rel.dyn
01 .dynamic .got
02 .data .bss
03 .dynamic
04 .dynamic .got .bss
05
06 .ARM.exidx

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts: port gen_ldelf_hex to python3

Simple port for gen_ldelf_hex to python3.

Signed-off-by: Rouven Czerwinski <r.czerwinski@pengutronix.de>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>

core: dump user TA memory map and call stack with ldelf

Enters special ldelf dump function to dump user TA memory map.

scripts/symbolize.py patch by Jerome Forissier

Reviewed-by: Jerome Forissier

core: dump user TA memory map and call stack with ldelf

Enters special ldelf dump function to dump user TA memory map.

scripts/symbolize.py patch by Jerome Forissier

Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: [symbolize.py] Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...