core: riscv: dynamic allocation of thread_core_local and its stacks

Refer to commit 59724f223500 ("core: dynamic allocation of
thread_core_local and its stacks"), we implement the code for RISC-V
ar

core: riscv: dynamic allocation of thread_core_local and its stacks

Refer to commit 59724f223500 ("core: dynamic allocation of
thread_core_local and its stacks"), we implement the code for RISC-V
architecture. With CFG_DYN_STACK_CONFIG enabled, the thread_core_local
and the two stacks, tmp_stack and abt_stack, are dynamically allocated.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

riscv: kernel: support booting non-contiguous non-zero-based hart IDs

Currently, OP-TEE assumes 0 <= hartid < CFG_TEE_CORE_NB_CORE,
and must be contiguous, which fails to accommodate different
CPU t

riscv: kernel: support booting non-contiguous non-zero-based hart IDs

Currently, OP-TEE assumes 0 <= hartid < CFG_TEE_CORE_NB_CORE,
and must be contiguous, which fails to accommodate different
CPU topologies. For example, some RISC-V platforms, such as
the HiFive Unmatched board, do not run Linux and OP-TEE on
hart0, as it is a monitor core without supervisor mode support.

To address this, introduce hart_index, which is used to index
per-hart structures, such as thread_core_local and root_pgt.
The hart_index will range from 0 to (CFG_TEE_CORE_NB_CORE - 1),
and the primary hart will have an index of 0.

Additionally, a new function, boot_primary_init_core_ids(),
is added to initialize secondary hart IDs for booting via
sbi_hsm_hart_start().

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: Remove thread_exit_user_mode()

Currently, the user mode abort and some system calls return to kernel
mode by thread_exit_user_mode(). Although this function creates a
shorter path to re

core: riscv: Remove thread_exit_user_mode()

Currently, the user mode abort and some system calls return to kernel
mode by thread_exit_user_mode(). Although this function creates a
shorter path to return to kernel mode, it leads to some problems because
the function does not update the core local flags. Especially when
CFG_CORE_DEBUG_CHECK_STACKS=y, some checks will fail due to wrong type
of stack recorded in the core local flags.

Fix it by removing thread_exit_user_mode(). So that the core local flags
can be correctly updated in the common trap handler.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Set exception return PC into XEPC for entering user mode

Instead of setting exception return PC into "ra" register and assign it
to XEPC, we should directly set exception return PC into

core: riscv: Set exception return PC into XEPC for entering user mode

Instead of setting exception return PC into "ra" register and assign it
to XEPC, we should directly set exception return PC into "XEPC" CSR to
improve code redability.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Rename and re-order parameters of the interrupt handlers

Rename thread_interrupt_handler() to thread_native_interrupt_handler(),
since we are going to support the foreign interrupt hand

core: riscv: Rename and re-order parameters of the interrupt handlers

Rename thread_interrupt_handler() to thread_native_interrupt_handler(),
since we are going to support the foreign interrupt handler.

The native interrupts are handled in OP-TEE side, while the foreign
interrupts are handled outside OP-TEE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: Simplify SP setup in setup_unwind_user_mode()

The parameter "regs" is the stack pointer which is allocated to store
system call registers when calling thread_scall_handler(). Thus, we c

core: riscv: Simplify SP setup in setup_unwind_user_mode()

The parameter "regs" is the stack pointer which is allocated to store
system call registers when calling thread_scall_handler(). Thus, we can
simply get the original stack pointer by "regs + 1" equation, and use it
to exit user mode.

The code is referenced from ARM's setup_unwind_user_mode().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: open-code thread_init_stack()

The implementations of thread_init_stack() are identical and trivial for
both arm and riscv. So simplify code further and open-code it where it's
called from in c

core: open-code thread_init_stack()

The implementations of thread_init_stack() are identical and trivial for
both arm and riscv. So simplify code further and open-code it where it's
called from in core/kernel/thread.c.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: Improve the trap handler

In current RISC-V trap handler, we always save the context into struct
thread_trap_regs first, and copy the saved context into another specific
structure to han

core: riscv: Improve the trap handler

In current RISC-V trap handler, we always save the context into struct
thread_trap_regs first, and copy the saved context into another specific
structure to handle that trap. For example, thread_user_ecall_handler()
copies the struct thread_trap_regs to be struct thread_scall_regs before
handling this system call. In fact, such copies may drop the performance
and they should be eliminated.

In this commit, the specific data structure used to save the context is
determined early in the trap handler by checking the type of the trap
from CSR XCAUSE. Thus, the copies between context structures are
eliminated.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Tested-by: Yu Chien Peter Lin <peterlin@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Apply STATUS helper for RPC resume

Since RPC resume is a kind of exception return, we invoke
xstatus_for_xret() to prepare the CSR STATUS for exception return. But
the actual value of S

core: riscv: Apply STATUS helper for RPC resume

Since RPC resume is a kind of exception return, we invoke
xstatus_for_xret() to prepare the CSR STATUS for exception return. But
the actual value of STATUS when calling thread_rpc() is still saved in
stack. This is to unify the behavior between RPC suspend and resume.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Refine thread enter/exit user mode

Now when thread is in user mode, the kernel TP is saved into CSR SCRATCH
instead of into kernel stack. The IE is also considered since it
contains mas

core: riscv: Refine thread enter/exit user mode

Now when thread is in user mode, the kernel TP is saved into CSR SCRATCH
instead of into kernel stack. The IE is also considered since it
contains masks of different exceptions.

Apply exception return to thread_exit_user_mode() to let hart correctly
back to kernel mode from exception.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Apply exception return to resume thread

In current implementation, the thread is resumed by function return. It
is not suitable for all scenarios, especially when the thread should be
r

core: riscv: Apply exception return to resume thread

In current implementation, the thread is resumed by function return. It
is not suitable for all scenarios, especially when the thread should be
resumed to user mode. The kernel mode can not return to user mode by
pure function return.

This commit applies exception return to resume the thread. The EPC and
IE are added into thread context. The xstatus_for_xret() helper function
is added to prepare the value of CSR STATUS for exception return.
Currently we only consider PIE(previous interrupt-enable) and
PP(previous privilege mode) for exception return.

We clear thread context when the context is reinitialized, enable native
interrupt, and setup kernel GP/TP. The thread_resume() now takes care of
restoring CSR EPC, STATUS, IE, SCRATCH and all general-purpose
registers. Finally it executes exception return to target privilege mode
encoded in CSR STATUS. The registers GP and TP are also restored since
user mode may use them.

This commit also modify the usage of CSR SCRATCH. In current
implementation the SCRATCH is used to save kernel stack pointer when the
thread is in user mode. The value of TP, which stores thread_core_local
structure, is saved into kernel stack before entering user mode. The
trap handler can then get TP(thread_core_local) from kernel stack. This
is not suitable for SMP system, since the thread might be resumed to
another core, and that core gets wrong TP from kernel stack. Fix it by
directly storing TP into CSR SCRATCH.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Include kernel/interrupt.h for thread_arch.c

Otherwise, the compiler can not find the declaration of
interrupt_main_handler().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-

core: riscv: Include kernel/interrupt.h for thread_arch.c

Otherwise, the compiler can not find the declaration of
interrupt_main_handler().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: Fix logic of thread_{get/set}_exceptions()

In ARM, the bits in DAIF register are used to mask the interrupts. While
in RISC-V, the bits in CSR XIE are used to enable(unmask) correspondi

core: riscv: Fix logic of thread_{get/set}_exceptions()

In ARM, the bits in DAIF register are used to mask the interrupts. While
in RISC-V, the bits in CSR XIE are used to enable(unmask) corresponding
interrupt sources.

To not modify the function of thread_get_exceptions(), we invert the
bits after reading the value of CSR XIE, as mask.

To not modify the function of thread_set_exceptions(), we invert the
bits in given "exceptions" before writing "exceptions" into CSR
XIE. Therefore, the intended masked exception bits will be cleared
when we write the final value into CSR XIE to mask those interrupts.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: Implement panic_at_abi_return as guard of ABI call

The ABI call to REE domain should not return. We implement
panic_at_abi_return macro as guard of ABI call. When the ABI call return
il

core: riscv: Implement panic_at_abi_return as guard of ABI call

The ABI call to REE domain should not return. We implement
panic_at_abi_return macro as guard of ABI call. When the ABI call return
illegally, the system will enter panic or an infinite loop.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: Fix condition of is_from_user()

RISC-V defines that xPP(previous privilege mode) field of CSR status
indicates the previous privilege level prior to the trap. Since the
encoding of user

core: riscv: Fix condition of is_from_user()

RISC-V defines that xPP(previous privilege mode) field of CSR status
indicates the previous privilege level prior to the trap. Since the
encoding of user mode is 0, we should compare the xPP field with 0 here
to know that the trap is from user mode.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: thread_arch.c: Use of ABI structure instead of SMC structure

SMC is an ARM-related keyword, make use thread_std_abi_entry instead of
thread_std_smc_entry.

Signed-off-by: Marouene Bouba

core: riscv: thread_arch.c: Use of ABI structure instead of SMC structure

SMC is an ARM-related keyword, make use thread_std_abi_entry instead of
thread_std_smc_entry.

Signed-off-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: Support Privileged Access Never by CSR status.SUM bit

The SUM (Supervisor User Memory access) bit modifies the privilege with
which S-mode loads and stores the user virtual memory. When

core: riscv: Support Privileged Access Never by CSR status.SUM bit

The SUM (Supervisor User Memory access) bit modifies the privilege with
which S-mode loads and stores the user virtual memory. When SUM bit is
0, S-mode accesses to pages whose U bit of corresponding PTE is set will
fault. When SUM bit is 1, these accesses are permitted.

When CFG_PAN is disabled in RISC-V architecture, the status.SUM bit is
initialized as 1 by default. Therefore all accesses to user pages will
succeed. When CFG_PAN is enabled, the status.SUM bit is initialized as
0, and only set to 1 when kernel needs to access user pages.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Fix width of status CSR

Since we also support RV64 with 64-bit register width, fix the width of
status CSR by declaring it as "unsigned long" and encoding it by general
bit-wise operati

core: riscv: Fix width of status CSR

Since we also support RV64 with 64-bit register width, fix the width of
status CSR by declaring it as "unsigned long" and encoding it by general
bit-wise operations instead of invoking fixed-width API.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: fix interrupt_main_handler() reference

Fixes itr_core_handler() reference in RiscV architecture that was renamed
interrupt_main_handler() in commit referred below.

Fixes: 358bf47c0612

core: riscv: fix interrupt_main_handler() reference

Fixes itr_core_handler() reference in RiscV architecture that was renamed
interrupt_main_handler() in commit referred below.

Fixes: 358bf47c0612 ("core: interrupt: rename itr_core_handler()")
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

show more ...

virt: rename CFG_VIRTUALIZATION to CFG_NS_VIRTUALIZATION

With the advent of virtualization support at S-EL2 in the Armv8.4-A
architecture, CFG_VIRTUALIZATION has become ambiguous. Let's rename
it to

virt: rename CFG_VIRTUALIZATION to CFG_NS_VIRTUALIZATION

With the advent of virtualization support at S-EL2 in the Armv8.4-A
architecture, CFG_VIRTUALIZATION has become ambiguous. Let's rename
it to CFG_NS_VIRTUALIZATION to indicate more clearly that it is about
supporting virtualization on the non-secure side.

This commit is the result of the following command:

$ for f in $(git grep -l -w CFG_VIRTUALIZATION); do \
sed -i -e 's/CFG_VIRTUALIZATION/CFG_NS_VIRTUALIZATION/g' $f; \
done

...plus the compatibility line in mk/config.mk:

CFG_NS_VIRTUALIZATION ?= $(CFG_VIRTUALIZATION)

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Acked-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

show more ...

core: riscv: implement thread management routines in thread_arch.c

This commit implements an initial thread management for RISC-V. It covers
the following routines:

- Exceptions un/masking
- Trap h

core: riscv: implement thread management routines in thread_arch.c

This commit implements an initial thread management for RISC-V. It covers
the following routines:

- Exceptions un/masking
- Trap handling, including syscalls handling.
- Thread allocation, execution, suspension, freeing with slight changes
to set RISC-V registers such as CSRs.
- RPC.

Pending routines:
- Floatting point support F/D/Q/L extensions and software FP.
- Abort mode.

Signed-off-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...