scripts/symbolize.py: code cleanup

Fix pep8 errors.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>

core: arm32: generate system register access code

Replaces the hand crafted system register code in <arm32.h> and
<arm32_macros.S> with generated code based on arm32_sysreg.txt which is
extracted fr

core: arm32: generate system register access code

Replaces the hand crafted system register code in <arm32.h> and
<arm32_macros.S> with generated code based on arm32_sysreg.txt which is
extracted from the ARM Architecture Reference Manual.

The remaining hand crafted code for cp15 accesses is not covered by the
ARM Architecture Reference Manual.

A script is added to generate both assembly macros and static inline
functions to access the system registers.

Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts/symbolize.py: remove spurious output '(out/arm/core/tee.elf)'

When a TA dump is processed, there is a list of ELF files just before the
call stack. However, when analyzing a TEE core dump, t

scripts/symbolize.py: remove spurious output '(out/arm/core/tee.elf)'

When a TA dump is processed, there is a list of ELF files just before the
call stack. However, when analyzing a TEE core dump, there is no such
list. Make sure this situation is properly handled to avoid displaying a
spurious message. This means fixing incorrect indentation in a
conditional.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts/symbolize.py: ignore undefined symbols

With the introduction of dynamically linked TAs, symbolize.py may
encounter undefined (external) symbols when it parses the output of the nm
command lo

scripts/symbolize.py: ignore undefined symbols

With the introduction of dynamically linked TAs, symbolize.py may
encounter undefined (external) symbols when it parses the output of the nm
command looking for a symbol's address. The current code is not prepared
for that and will raise an exception. Fix the issue by ignoring lines that
have an unexpected format.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts/symbolize.py: print path of ELF files

Show which file corresponds to each ELF binary in a TA. Symbolic links
are followed, so that dynamic libraries are likely shown with their
user-friendly

scripts/symbolize.py: print path of ELF files

Show which file corresponds to each ELF binary in a TA. Symbolic links
are followed, so that dynamic libraries are likely shown with their
user-friendly name (libfoo.so) rather than their UUID (<uuid>.elf) --
see commit 01b8b5ce011d ("TA dev kit: when building a shared library,
create symlink with UUID").

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

scripts/symbolize.py: add support for dynamically linked TAs

Dynamically linked TAs contain sections mapped from several ELF files.
Update symbolize.py accordingly. As a result, the call stack will

scripts/symbolize.py: add support for dynamically linked TAs

Dynamically linked TAs contain sections mapped from several ELF files.
Update symbolize.py accordingly. As a result, the call stack will show
the expected debug information instead of "?? ??:0" for stack frames
that belong to a dynamic library. Also, the memory regions used to
map libraries will be annotated with section names like for the main
executable.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

checkpatch_inc.sh: ignore lib/libmbedtls

Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

checkpatch_inc.sh: remove non-existent directories

Removes directories listed in checkpatch_inc.sh that doesn't
exists any longer.

Reviewed-by: Joakim Bech <joakim.bech@linaro.org>
Acked-by: Etienn

checkpatch_inc.sh: remove non-existent directories

Removes directories listed in checkpatch_inc.sh that doesn't
exists any longer.

Reviewed-by: Joakim Bech <joakim.bech@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

checkpatch_inc.sh: fix case of empty commit due to path filtering

If a commit changes only paths that are filtered out by $_CP_EXCL, thus
making the diff empty, 'git format-patch -1 <sha1>' will sho

checkpatch_inc.sh: fix case of empty commit due to path filtering

If a commit changes only paths that are filtered out by $_CP_EXCL, thus
making the diff empty, 'git format-patch -1 <sha1>' will show the parent
commit instead. As a result, checkpatch.sh will check the wrong commit.

Several things are needed to fix the issue:

1. When calling 'git format-patch', specify the commit ID as a range
(<sha1>^..<sha1>) rather than as a single revision with a maximum count
(-1 <sha1>). This avoids showing the wrong commit.
2. Show the commit message then the diff in two steps, because the
above syntax will not print anything if the diff happens to be empty.
3. Tell checkpatch.pl to ignore the "not a unified diff" error which
is triggered if the commit log is not followed by a diff.

Link: https://travis-ci.org/OP-TEE/optee_os/builds/367058383#L2199-L2204
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

build: introduce asm-defines-y

The Makefile rules that are used to generate a C header file containing
constants for struct offsets etc. are currently in mk/compile.mk. They
are used by core.mk whic

build: introduce asm-defines-y

The Makefile rules that are used to generate a C header file containing
constants for struct offsets etc. are currently in mk/compile.mk. They
are used by core.mk which sets a variable (asm-defines-file) before it
includes compile.mk. This works well for this purpose, but does not
scale to several files.

There is a use case for platform code to be able to use the asm-defines
mechanism, too. Therefore, introduce a variable that can be used in any
sub.mk: asm-defines-y.

In addition, to avoid duplication, the DEFINE and DEFINES macros are
moved to their own header (core/include/gen-asm-defines.h), with the
added benefit that it can be explicitly excluded from the checkpatch
list and thus not generate any warning on the 'DEFINE' macro needing
parentheses.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Tested-by: Peng Fan <peng.fan@nxp.com>
Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

checkpatch_inc.sh: exclude files with inline assembly

checkpatch.pl produces false errors when given files with inline
assembly [1]. It is best to skip those files entirely.

Link: [1] https://travi

checkpatch_inc.sh: exclude files with inline assembly

checkpatch.pl produces false errors when given files with inline
assembly [1]. It is best to skip those files entirely.

Link: [1] https://travis-ci.org/OP-TEE/optee_os/builds/347435939#L2115
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts/symbolize.py: update help text

Commit 546291f4de9b ("trace: make output more compact") has changed the
format of the abort and panic dumps. Update 'symbolize.py --help'
accordingly.

The hel

scripts/symbolize.py: update help text

Commit 546291f4de9b ("trace: make output more compact") has changed the
format of the abort and panic dumps. Update 'symbolize.py --help'
accordingly.

The help text is also reworked slightly to make it clear that the
script may be used with all kinds of crash dumps (TEE core or TA,
aborts and panics).

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

scripts/symbolize.py: strip cwd from file paths by default

It is quite common to invoke symbolize.py from a top-level directory
which contains the source code of OP-TEE as well as of Trusted
Applica

scripts/symbolize.py: strip cwd from file paths by default

It is quite common to invoke symbolize.py from a top-level directory
which contains the source code of OP-TEE as well as of Trusted
Applications. In this case, it is convenient to strip the directory
from all the paths reported by the script.

Therefore, make this behavior the default. In other words, '-s $(pwd)'
is now implied. One can still obtain full paths by passing '-s' with no
argument.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

Remove 'All rights reserved' from Linaro files

The text 'All rights reserved' is useless [1]. The Free Software
Foundation's REUSE Initiative best practices document [2] does not
contain these words

Remove 'All rights reserved' from Linaro files

The text 'All rights reserved' is useless [1]. The Free Software
Foundation's REUSE Initiative best practices document [2] does not
contain these words. Therefore, we can safely remove the text from the
files that are owned by Linaro.

Generated by:
spdxify.py --linaro-only --strip-arr optee_os/

Link: [1] https://en.wikipedia.org/wiki/All_rights_reserved
Link: [2] https://reuse.software/practices/
Link: [3] https://github.com/jforissier/misc/blob/f7b56c8/spdxify.py
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

Remove license notice from Linaro files

Now that we have added SPDX identifiers, we can safely remove the
verbose license text from the files that are owned by Linaro.

Generated by [1]:
spdxify.p

Remove license notice from Linaro files

Now that we have added SPDX identifiers, we can safely remove the
verbose license text from the files that are owned by Linaro.

Generated by [1]:
spdxify.py --linaro-only --strip-license-text optee_os/

Link: [1] https://github.com/jforissier/misc/blob/f7b56c8/spdxify.py
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

Add SPDX license identifiers

Adds one SPDX-License-Identifier line [1] to each source files that
contains license text.

Generated by [2]:
spdxify.py --add-spdx optee_os/

The scancode tool [3] wa

Add SPDX license identifiers

Adds one SPDX-License-Identifier line [1] to each source files that
contains license text.

Generated by [2]:
spdxify.py --add-spdx optee_os/

The scancode tool [3] was used to double check the license matching
code in the Python script. All the licenses detected by scancode are
either detected by spdxify.py, or have no SPDX identifier, or are false
matches.

Link: [1] https://spdx.org/licenses/
Link: [2] https://github.com/jforissier/misc/blob/f7b56c8/spdxify.py
Link: [3] https://github.com/nexB/scancode-toolkit
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

scripts: update regular expression in symbolize.py

Update the regular expression finding the call stack addresses to match
the new trace format.

Signed-off-by: Joakim Bech <joakim.bech@linaro.org>

scripts: update regular expression in symbolize.py

Update the regular expression finding the call stack addresses to match
the new trace format.

Signed-off-by: Joakim Bech <joakim.bech@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

Sign TAs as bootstrap TAs

Changes to TA sign script to sign TAs as Bootstrap TAs
(img_type == SHDR_BOOTSTRAP_TA) instead of the legacy
TA format (img_type == SHDR_TA).

Reviewed-by: Volodymyr Babchu

Sign TAs as bootstrap TAs

Changes to TA sign script to sign TAs as Bootstrap TAs
(img_type == SHDR_BOOTSTRAP_TA) instead of the legacy
TA format (img_type == SHDR_TA).

Reviewed-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

mem_usage.py: fix error with ccache

Fix the following error:

make mem_usage CROSS_COMPILE="ccache arm-linux-gnueabihf-"
...
GEN out/arm-plat-vexpress/core/tee.mem_usage
Traceback (most recent

mem_usage.py: fix error with ccache

Fix the following error:

make mem_usage CROSS_COMPILE="ccache arm-linux-gnueabihf-"
...
GEN out/arm-plat-vexpress/core/tee.mem_usage
Traceback (most recent call last):
File "./scripts/mem_usage.py", line 162, in <module>
main()
File "./scripts/mem_usage.py", line 94, in main
universal_newlines=True)
File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'ccache arm-linux-gnueabihf-readelf'
core/arch/arm/kernel/link.mk:255: recipe for target 'out/arm-plat-vexpress/core/tee.mem_usage' failed
make: *** [out/arm-plat-vexpress/core/tee.mem_usage] Error 1

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

Add pager options to mem_usage.py

Adds options to report the size of the init, paged and unpaged areas
in pager builds (CFG_WITH_PAGER=y). This is helpful to monitor the
evolution of the code size,

Add pager options to mem_usage.py

Adds options to report the size of the init, paged and unpaged areas
in pager builds (CFG_WITH_PAGER=y). This is helpful to monitor the
evolution of the code size, and identify issues such as when useless
functions get pulled into the unpaged area by mistake.
For instance, here is what we can get with 'mem_usage.py -nUr' and a
few lines of script (shows changes in the size of unpaged code and
data for a non-debug qemu_virt build since 2.5.0):

2.5.0 67600
2.5.0-2-g0b8e6e78 67632 +32 core: abort: fix get_fault_type()
2.5.0-11-g87d96185 67728 +96 core: pager: avoid page faults with pager lock held
2.5.0-37-g0073c9a8 67792 +64 core: pager: optimize tee_pager_handle_fault()
2.5.0-40-g935ac3ec 67888 +96 core: arm32: handle aborts in system mode
2.5.0-53-g2e4e94bf 67920 +32 core: default enable program flow prediction on ARMv7 cores
2.5.0-57-ge84e1fec 67704 -216 core: clean and unwind arm32_aeabi_[l]divmod_a32.S
2.5.0-63-g23346f16 67736 +32 Add support for several user TA stores
2.5.0-76-gb2087a20 67608 -128 core: pager: deal with large CFG_TEE_RAM_VA_SIZE
2.5.0-77-g00aa47a4 67688 +80 core: plat-vexpress: increase CFG_TEE_RAM_VA_SIZE
2.5.0-112-g86e50a60 67696 +8 core: arm: psci: add suspend resume common functions
2.5.0-120-g13b3ee90 67768 +72 core: print rwx flags for each MMU region when a user TA aborts
2.5.0-122-ge61644fb 83408 +15640 core: make reset_secondary() unpaged
2.5.0-123-g1506eb6f 83424 +16 core: plat-vexpress: init gic on secondary cores
2.5.0-124-ge2b68c87 91368 +7944 core: plat-vextpress-qemu_virt: update num cores
2.5.0-126-g5402a9fe 92808 +1440 qemu_virt: enable smp boot
2.5.0-132-gda51216b 92840 +32 dts: pass PA of reserved region
2.5.0-173-gb4121bfb 92848 +8 pl011: prevent putc() and flush() function from blocking indefinitely
2.5.0-176-g55d6853c 92912 +64 core: add registered shared memory support
2.5.0-182-g93d3c451 163936 +71024 core: pager: ltc: prng: add entropy to the AE key for paged TAs
2.5.0-202-ga71af55e 163984 +48 core: mobj: add mobj_get_phys_offs()
2.5.0-203-g5c7a19bb 163968 -16 core: mobj: remove double physical offset
2.5.0-205-g430dcbd8 163944 -24 core: reimplement mobj_mapped_shm_alloc()
2.5.0-208-gbbed97b6 163976 +32 core:mmu: fix userland va2pa conversion
2.5.0-209-g42d91b4b 164040 +64 core:mmu: fix userland pa2va conversion
2.6.0-rc1-1-gb6449075 164136 +96 thread.c: free rpc arg mobj during cache disabling
2.6.0-7-g8473540d 77144 -86992 Keep assembly functions in separate sections
2.6.0-8-g64113fca 77112 -32 core: arm32: replace _start with reset() function
2.6.0-9-g486754e8 77176 +64 core: arm32: reset_secondary() set reset vector

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by Etienne Carriere <etienne.carriere@linaro.org>

show more ...

Rewrite mem_usage.awk in Python

Rewrite the memory usage script in Python. No functional change, except
that the script now takes tee.elf as an argument instead of processing
the output of readelf.

Rewrite mem_usage.awk in Python

Rewrite the memory usage script in Python. No functional change, except
that the script now takes tee.elf as an argument instead of processing
the output of readelf. The makefile (make mem_usage) is adjusted
accordingly.

This makes the script shorter and easier to call, and it is now
possible to add command line options.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Volodymyr Babchuk <vlad.babchuk@gmail.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by Etienne Carriere <etienne.carriere@linaro.org>

show more ...

scripts/symbolize.py: reset state when a (new) abort message is found

The symbolize script may not process correctly several dumps in a row.
For instance, if you pipe a TA abort followed by a TEE co

scripts/symbolize.py: reset state when a (new) abort message is found

The symbolize script may not process correctly several dumps in a row.
For instance, if you pipe a TA abort followed by a TEE core abort, the
later is not decoded properly because the script still uses the TA
binary to interpret the core stack. Fix this by resetting the state of
the decoder each time an abort line is encountered.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

TUI: remove unused scripts/render_font.py

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

scripts/gen_ld_sects.py: Explicitly number positional argument specifiers

Before Python 2.7 omitting the positional argument specifier was not
allowed. Some may still be using earlier Python version

scripts/gen_ld_sects.py: Explicitly number positional argument specifiers

Before Python 2.7 omitting the positional argument specifier was not
allowed. Some may still be using earlier Python version where this
will cause build to fail. Fix this here.

Fixes: 5976a0a5b9bf ("core: avoid incremental linking with -gc")
Reported-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Andrew F. Davis <afd@ti.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

scripts/symbolize.py: print ELF sections after MMU region information

When processing a user TA abort dump, list the ELF sections that are
mapped in each MMU region. For example (the lines modified

scripts/symbolize.py: print ELF sections after MMU region information

When processing a user TA abort dump, list the ELF sections that are
mapped in each MMU region. For example (the lines modified by this
patch are prefixed with >>):

User TA undef-abort at address 0x10574e
fsr 0x00000000 ttbr0 0x0e07a06a ttbr1 0x0e07406a cidr 0x1
cpu #0 cpsr 0x60000030
r0 0x20000013 r4 0x0013a6bc r8 0x00000000 r12 0x0e07dd88
r1 0x00000033 r5 0x00121fd3 r9 0x00000000 sp 0x001026cc
r2 0x0010581f r6 0x00102590 r10 0x00000000 lr 0x00105823
r3 0x00000043 r7 0x001026cc r11 0x00000000 pc 0x0010574e
Status of TA 5b9e0e40-2636-11e1-ad9e-0002a5d5c51b (0xe073b70) (active)
arch: arm load address: 0x103000 ctx-idr: 1
stack: 0x100000 10240
region 0: va 0x100000 pa 0xe21e000 size 0x3000 flags rw-
>> region 1: va 0x103000 pa 0xe100000 size 0x2e000 flags r-x .ta_head .text .rodata
>> region 2: va 0x131000 pa 0xe12e000 size 0xa000 flags r-- .rodata .ARM.extab .ARM.exidx .got .dynsym .rel.got .dynamic .dynstr .hash .rel.dyn
>> region 3: va 0x13b000 pa 0xe138000 size 0xe6000 flags rw- .data .bss
region 4: va 0 pa 0 size 0 flags ---
region 5: va 0 pa 0 size 0 flags ---
region 6: va 0 pa 0 size 0 flags ---
region 7: va 0 pa 0 size 0 flags ---
User TA undef-abort at address 0x10574e undef_instr+6 .text+10030
Call stack:
0x0010574e undef_instr at optee_test/ta/os_test/os_test.c:880
0x00105823 ta_entry_bad_mem_access at optee_test/ta/os_test/os_test.c:917
0x00105e75 TA_InvokeCommandEntryPoint at optee_test/ta/os_test/ta_entry.c:101
0x00121fb7 entry_invoke_command at optee_os/lib/libutee/arch/arm/user_ta_entry.c:207
0x00122013 __utee_entry at optee_os/lib/libutee/arch/arm/user_ta_entry.c:235

Suggested-by: Zeng Tao <prime.zeng@hisilicon.com>
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...