core: dynamic allocation of thread_core_local and its stacks

With CFG_DYN_CONFIG enabled, use dynamic allocation of thread_core_local
and the two stacks, tmp_stack and abt_stack, recorded in it.

Si

core: dynamic allocation of thread_core_local and its stacks

With CFG_DYN_CONFIG enabled, use dynamic allocation of thread_core_local
and the two stacks, tmp_stack and abt_stack, recorded in it.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

core: fix crash during syscall ftrace

Syscall ftrace collects data during a syscall. get_fbuf() checks if
thread_get_id_may_fail() != -1 to see if a function is called under
normal thread execution.

core: fix crash during syscall ftrace

Syscall ftrace collects data during a syscall. get_fbuf() checks if
thread_get_id_may_fail() != -1 to see if a function is called under
normal thread execution. This can lead to an inconsistent state if a
native interrupt occur while ftrace_enter() or ftrace_return() is
recording data in the ftrace buffer. So fix this by using
thread_is_in_normal_mode() to exclude ftrace during interrupt
processing.

Reported-by: Jerome Forissier <jerome.forissier@linaro.org>
Closes: https://github.com/OP-TEE/optee_os/issues/7216
Fixes: 099918f6744c ("ftrace: Add support for syscall function tracer")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (vexpress-qemu_armv8a)

show more ...

core: arm32: initialize secure monitor late

Initialize the secure monitor as late as possible before exiting to the
normal world. This is needed in later patches where the stacks aren't
statically a

core: arm32: initialize secure monitor late

Initialize the secure monitor as late as possible before exiting to the
normal world. This is needed in later patches where the stacks aren't
statically allocated.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: arm: Remove duplicated sp assignment for ARM64 in set_ctx_regs()

There are two lines of code to assign value of sp for ARM64. Remove one
of them.

Signed-off-by: Alvin Chang <alvinga@andestech

core: arm: Remove duplicated sp assignment for ARM64 in set_ctx_regs()

There are two lines of code to assign value of sp for ARM64. Remove one
of them.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: open-code thread_init_stack()

The implementations of thread_init_stack() are identical and trivial for
both arm and riscv. So simplify code further and open-code it where it's
called from in c

core: open-code thread_init_stack()

The implementations of thread_init_stack() are identical and trivial for
both arm and riscv. So simplify code further and open-code it where it's
called from in core/kernel/thread.c.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: Enable pointer authentication for SPs

Add support to pauth keys for SPs if pointer authentication is enabled.

Signed-off-by: Gabor Toth <gabor.toth2@arm.com>
Acked-by: Etienne Carriere <etien

core: Enable pointer authentication for SPs

Add support to pauth keys for SPs if pointer authentication is enabled.

Signed-off-by: Gabor Toth <gabor.toth2@arm.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: remove unnecessary includes

Remove unnecessary includes.

Signed-off-by: Clement Faure <clement.faure@nxp.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander

core: remove unnecessary includes

Remove unnecessary includes.

Signed-off-by: Clement Faure <clement.faure@nxp.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: interrupt: registering interrupt providers

Adds interrupt chip framework API functions for an interrupt controller
to register as an interrupt provider in the driver probing sequence
based on

core: interrupt: registering interrupt providers

Adds interrupt chip framework API functions for an interrupt controller
to register as an interrupt provider in the driver probing sequence
based on device tree. This allows interrupt consumer to be deferred
when a dependent interrupt controller is not yet initialized.

Interrupt controllers register a driver in DT_DRIVER providers list
with: interrupt_register_provider().

Interrupt consumer can get their interrupt through DT data with
interrupt_dt_get(), interrupt_dt_get_by_index() or
interrupt_dt_get_by_name().

This change removes inclusion of interrupt.h from kernel/dt.h as it is
not needed and conflicts with inclusion of kernel/dt.h from
kernel/interrupt.h.

Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: arm64: preserve PSTATE.PAN when making SPSR

When setup_unwind_user_mode() prepares to resume execution after
syscall_sys_return() or when a thread is suspended a new SPSR is
fabricated base on

core: arm64: preserve PSTATE.PAN when making SPSR

When setup_unwind_user_mode() prepares to resume execution after
syscall_sys_return() or when a thread is suspended a new SPSR is
fabricated base on the current PSTATE.

Until now when remaining in S-EL1 to fabricate an SPSR only the
PSTATE.DAIF bits had to be taken into account. However, with PSTATE.PAN
there's yet another bit to consider. Since PSTATE has a few more bits
and more may be added as AArch64 evolves this problem is only going to
get worse. So implement this in a single internal C function to replace
current open codes C and assembly versions.

The AArch64 assembly versions of thread_rpc() are renamed to
thread_rpc_spsr() to indicate that SPSR is passed in the second argument
instead of having it open coded internally in the assembly function.

New C wrapper functions are added to preserve the old thread_rpc()
interface as needed.

handle_user_mode_panic() is still basing its created SPSR on the saved
SPSR from S-EL0, but now PAN bit is copied too.

Fixes: 6fa59c9a70dc ("arm64: Introduce permissive PAN implementation")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

core: Apply finer-grained PAN

Prior to this commit, the PAN was disabled for most of the time,
within the thread scall handler. After resolving all outstanding
missing unprivileged access functions,

core: Apply finer-grained PAN

Prior to this commit, the PAN was disabled for most of the time,
within the thread scall handler. After resolving all outstanding
missing unprivileged access functions, we can now enable finer-
grained PAN, where the unprivileged access is only allowed inside
handful of special user-access functions.

There are some exceptions where we toggle PAN to allow the OP-TEE
core to access user memory, instead of using user-access functions
or bounce buffers. Those are crypto services and ldelf syscall
handlers. Those are chosen to avoid potential large bounce buffer
allocations.

Signed-off-by: Seonghyun Park <seonghp@amazon.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

arm64: Introduce permissive PAN implementation

Privileged Access Never (PAN) is a part of ARMv8.1 extension that
restricts accesses to unprivileged memory from privileged mode
in order to prevent un

arm64: Introduce permissive PAN implementation

Privileged Access Never (PAN) is a part of ARMv8.1 extension that
restricts accesses to unprivileged memory from privileged mode
in order to prevent unintended accesses to potentially malicious
memory.

This introduces configuration of PAN and helper functions
enter_user_access() and exit_user_access() that toggles PSTATE.PAN
that controls the behavior of PAN.

Current OP-TEE impelmentation is not ready to apply strict PAN policy
due to missing user-access function uses, etc.

Hence, this patch takes a very permissive approach (yet better
than nothing), where PAN is deactivated in the entire lifetime of
thread_svc_handler (i.e., system call).

Signed-off-by: Seonghyun Park <seonghp@amazon.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: spmc: handle non-secure interrupts

Add FFA_INTERRUPT and FFA_RUN support for signaling non-secure
interrupts and for resuming to the secure world. If a secure partition
is preempted by a non-s

core: spmc: handle non-secure interrupts

Add FFA_INTERRUPT and FFA_RUN support for signaling non-secure
interrupts and for resuming to the secure world. If a secure partition
is preempted by a non-secure interrupt OP-TEE saves the SP's state and
sends an FFA_INTERRUPT to the normal world. After handling the interrupt
the normal world should send an FFA_RUN to OP-TEE so it can continue
running the SP.
If OP-TEE is the active FF-A endpoint (i.e. it is running TAs) the
non-secure interrupts are signaled by the existing
OPTEE_FFA_YIELDING_CALL_RETURN_INTERRUPT message instead of
FFA_INTERRUPT.

Signed-off-by: Imre Kis <imre.kis@arm.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: interrupt: rename itr_core_handler()

Renames itr_core_handler() to interrupt_main_handler() as a later
change will modify interrupt chip API functions using interrupt_
as prefix.

Reviewed-by:

core: interrupt: rename itr_core_handler()

Renames itr_core_handler() to interrupt_main_handler() as a later
change will modify interrupt chip API functions using interrupt_
as prefix.

Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

virt: rename CFG_VIRTUALIZATION to CFG_NS_VIRTUALIZATION

With the advent of virtualization support at S-EL2 in the Armv8.4-A
architecture, CFG_VIRTUALIZATION has become ambiguous. Let's rename
it to

virt: rename CFG_VIRTUALIZATION to CFG_NS_VIRTUALIZATION

With the advent of virtualization support at S-EL2 in the Armv8.4-A
architecture, CFG_VIRTUALIZATION has become ambiguous. Let's rename
it to CFG_NS_VIRTUALIZATION to indicate more clearly that it is about
supporting virtualization on the non-secure side.

This commit is the result of the following command:

$ for f in $(git grep -l -w CFG_VIRTUALIZATION); do \
sed -i -e 's/CFG_VIRTUALIZATION/CFG_NS_VIRTUALIZATION/g' $f; \
done

...plus the compatibility line in mk/config.mk:

CFG_NS_VIRTUALIZATION ?= $(CFG_VIRTUALIZATION)

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

show more ...

core: introduce scall layer from svc parts

Introduces a scall layer by renaming various thread_svc* names and
_*handle_svc() functions and function pointers as a first step in doing
architecture neu

core: introduce scall layer from svc parts

Introduces a scall layer by renaming various thread_svc* names and
_*handle_svc() functions and function pointers as a first step in doing
architecture neutral syscall processing.

The name scall is used instead of syscall since the syscall_ prefix is
reserved for the functions implementing the actual syscall. While scall
is the infrastructure used to reach the syscall functions.

No files are renamed and removed at this stage. This patch doesn't
change any behaviour.

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

arm32: kernel: Add common itr_core_handler() for NMFI workaround

Should we receive FIQ interrupt treat it always as panic to indicate to
platform developer that FIQ interrupts must be disabled in sy

arm32: kernel: Add common itr_core_handler() for NMFI workaround

Should we receive FIQ interrupt treat it always as panic to indicate to
platform developer that FIQ interrupts must be disabled in system level.

Function itr_core_handler() is defined without __weak to make sure that
there are no other function trying to handle the FIQ.

Signed-off-by: Vesa Jääskeläinen <vesa.jaaskelainen@vaisala.com>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: fix compile error with CFG_CORE_WORKAROUND_SPECTRE_BP_SEC=n

Prior to this patch there's a compile error when building with
CFG_CORE_WORKAROUND_SPECTRE_BP_SEC=n:
core/arch/arm/kernel/thread.c:

core: fix compile error with CFG_CORE_WORKAROUND_SPECTRE_BP_SEC=n

Prior to this patch there's a compile error when building with
CFG_CORE_WORKAROUND_SPECTRE_BP_SEC=n:
core/arch/arm/kernel/thread.c: In function 'select_vector_wa_spectre_bhb':
core/arch/arm/kernel/thread.c:644:48: error: 'thread_user_kdata_page' undeclared (first use in this function); did you mean 'thread_user_kcode_size'?
644 | struct thread_core_local *cl = (void *)thread_user_kdata_page;
| ^~~~~~~~~~~~~~~~~~~~~~
| thread_user_kcode_size
core/arch/arm/kernel/thread.c:644:48: note: each undeclared identifier is reported only once for each function it appears in
core/arch/arm/kernel/thread.c:646:27: error: 'struct thread_core_local' has no member named 'bhb_loop_count'
646 | cl[get_core_pos()].bhb_loop_count = loop_count;
| ^
core/arch/arm/kernel/thread.c:648:32: error: 'struct thread_core_local' has no member named 'bhb_loop_count'
648 | thread_get_core_local()->bhb_loop_count = loop_count;
| ^~

Fix this by disabling the unused code.

Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: add pointer authentication support

Previously pointer authentication was only supported for TAs. With this
patch add a configuration option CFG_CORE_PAUTH to enable support for
core. Each priv

core: add pointer authentication support

Previously pointer authentication was only supported for TAs. With this
patch add a configuration option CFG_CORE_PAUTH to enable support for
core. Each privileged thread has its own APIA key. There are also a
separate APIA key for each physical core used when handling an abort or
when using the tmp stack.

Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Jorge Ramirez-Ortiz <jorge@foundries.io>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: arm: spectre-bhb software workaround

Expands the config option CFG_CORE_WORKAROUND_SPECTRE_BP_SEC to cover
CVE-2022-23960 (aka Spectre-BHB) too since both have much in common.

Spectre-BHB is

core: arm: spectre-bhb software workaround

Expands the config option CFG_CORE_WORKAROUND_SPECTRE_BP_SEC to cover
CVE-2022-23960 (aka Spectre-BHB) too since both have much in common.

Spectre-BHB is another speculation attack on branch prediction. Further
details can be found at [1].

The software workaround added for CPUs vulnerable to Spectre-V2 covers
Spectre-BHB too. New software workaround is only needed for CPUs immune to
Spectre-V2, but not so to Spectre-BHB.

The Spectre-V2 workaround is to invalidate the entire branch predictor
table. Most new CPU immune to Spectre-V2 but vulnerable to Spectre-BHB
can avoid invalidating the entire branch predictor table, instead is
this invalidation replaced by a loop designed to exhaust the branch
predictor in a way that the exploit isn't possible any longer.

Link: [1] https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability/spectre-bhb

Fixes: CVE-2022-23960
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: refactor spectre-v2 workarounds

Refactors the Spectre-V2 workarounds to make room for further workarounds.

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wikl

core: refactor spectre-v2 workarounds

Refactors the Spectre-V2 workarounds to make room for further workarounds.

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: fix unused set_core_local_kcode_offset() warning

When compiling with CFG_CORE_UNMAP_CORE_AT_EL0=n there's a warning:
core/arch/arm/kernel/thread.c:529:13: error: ‘set_core_local_kcode_offset’

core: fix unused set_core_local_kcode_offset() warning

When compiling with CFG_CORE_UNMAP_CORE_AT_EL0=n there's a warning:
core/arch/arm/kernel/thread.c:529:13: error: ‘set_core_local_kcode_offset’ defined but not used [-Werror=unused-function]

Fix this with by adding a __maybe_unused to the function.

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: arm64: suppress text relocations caused by restore_mapping macro

Suppress the text relocations caused by 'ldr x0, =1f' in macro
restore_mapping when CFG_CORE_UNMAP_CORE_AT_EL0=y. Since this oc

core: arm64: suppress text relocations caused by restore_mapping macro

Suppress the text relocations caused by 'ldr x0, =1f' in macro
restore_mapping when CFG_CORE_UNMAP_CORE_AT_EL0=y. Since this occurs
when switching from the reduced kernel mapping to the full mapping, the
code offset needs to be loaded from somewhere readily accessible at that
point, that is the stack.

Signed-off-by: Jerome Forissier <jerome@forissier.org>
Suggested-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Sumit Garg <sumit.garg@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: move thread_defs.h into thread.h

Moves the defines in core/arch/arm/include/kernel/thread_defs.h into
core/include/kernel/thread.h.

Reviewed-by: Jerome Forissier <jerome@forissier.org>
Acked-

core: move thread_defs.h into thread.h

Moves the defines in core/arch/arm/include/kernel/thread_defs.h into
core/include/kernel/thread.h.

Reviewed-by: Jerome Forissier <jerome@forissier.org>
Acked-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: split core/arch/arm/kernel/thread.c

Splits core/arch/arm/kernel/thread.c into one generic and one
architecture specific file.

Acked-by: Jerome Forissier <jerome@forissier.org>
Acked-by: Marou

core: split core/arch/arm/kernel/thread.c

Splits core/arch/arm/kernel/thread.c into one generic and one
architecture specific file.

Acked-by: Jerome Forissier <jerome@forissier.org>
Acked-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: move thread_private.h to an include directory

Moves core/arch/arm/kernel/thread_private.h to a include directory to be
included as <kernel/thread_private.h>.

Reviewed-by: Jerome Forissier <je

core: move thread_private.h to an include directory

Moves core/arch/arm/kernel/thread_private.h to a include directory to be
included as <kernel/thread_private.h>.

Reviewed-by: Jerome Forissier <jerome@forissier.org>
Acked-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...