core: riscv: refactor MMU enablement code

Replace the set_satp macro with a proper enable_mmu function
to handle the transition to randomized virtual addresses. The
function executes from the identi

core: riscv: refactor MMU enablement code

Replace the set_satp macro with a proper enable_mmu function
to handle the transition to randomized virtual addresses. The
function executes from the identity mapped section to maintain
execution continuity during the VA->PA transition. It adjusts
the stack pointer, global pointer, thread pointer and ra register
with the ASLR offset.

The console is reinitialized after ASLR mapping is active since
the registered addresses need to be updated.

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Co-developed-by: Alvin Chang <alvinga@andestech.com>
Signed-off-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: dynamic allocation of thread_core_local and its stacks

Refer to commit 59724f223500 ("core: dynamic allocation of
thread_core_local and its stacks"), we implement the code for RISC-V
ar

core: riscv: dynamic allocation of thread_core_local and its stacks

Refer to commit 59724f223500 ("core: dynamic allocation of
thread_core_local and its stacks"), we implement the code for RISC-V
architecture. With CFG_DYN_STACK_CONFIG enabled, the thread_core_local
and the two stacks, tmp_stack and abt_stack, are dynamically allocated.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

riscv: kernel: support booting non-contiguous non-zero-based hart IDs

Currently, OP-TEE assumes 0 <= hartid < CFG_TEE_CORE_NB_CORE,
and must be contiguous, which fails to accommodate different
CPU t

riscv: kernel: support booting non-contiguous non-zero-based hart IDs

Currently, OP-TEE assumes 0 <= hartid < CFG_TEE_CORE_NB_CORE,
and must be contiguous, which fails to accommodate different
CPU topologies. For example, some RISC-V platforms, such as
the HiFive Unmatched board, do not run Linux and OP-TEE on
hart0, as it is a monitor core without supervisor mode support.

To address this, introduce hart_index, which is used to index
per-hart structures, such as thread_core_local and root_pgt.
The hart_index will range from 0 to (CFG_TEE_CORE_NB_CORE - 1),
and the primary hart will have an index of 0.

Additionally, a new function, boot_primary_init_core_ids(),
is added to initialize secondary hart IDs for booting via
sbi_hsm_hart_start().

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: Improve thread user mode record

Make the asm definitions be more human-readable.

Besides, it's unnecessary to save and restore kernel SP and GP into
thread_user_mode_rec, since they wi

core: riscv: Improve thread user mode record

Make the asm definitions be more human-readable.

Besides, it's unnecessary to save and restore kernel SP and GP into
thread_user_mode_rec, since they will be setup by system call trap
handler before executing thread_unwind_user_mode().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Improve the trap handler

In current RISC-V trap handler, we always save the context into struct
thread_trap_regs first, and copy the saved context into another specific
structure to han

core: riscv: Improve the trap handler

In current RISC-V trap handler, we always save the context into struct
thread_trap_regs first, and copy the saved context into another specific
structure to handle that trap. For example, thread_user_ecall_handler()
copies the struct thread_trap_regs to be struct thread_scall_regs before
handling this system call. In fact, such copies may drop the performance
and they should be eliminated.

In this commit, the specific data structure used to save the context is
determined early in the trap handler by checking the type of the trap
from CSR XCAUSE. Thus, the copies between context structures are
eliminated.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Tested-by: Yu Chien Peter Lin <peterlin@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Prepare SATP for each hart

To support multiple harts environment, we have allocated root page table
for each hart. Further more, we need to prepare value of CSR SATP, which
holds the ph

core: riscv: Prepare SATP for each hart

To support multiple harts environment, we have allocated root page table
for each hart. Further more, we need to prepare value of CSR SATP, which
holds the physical page number (PPN) of the root page table, for each
hart.

This commit enlarges the "struct core_mmu_config" for RISC-V
architecture to hold the value of CSR SATP for all the harts. In early
boot stage, each hart should initialize its CSR SATP from "struct
core_mmu_config".

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Refine thread trap handler

In order to support SMP, we made change on CSR SCRATCH from kernel stack
pointer to be kernel TP(thread_core_local). So that we can get TP from
SCRATCH easily

core: riscv: Refine thread trap handler

In order to support SMP, we made change on CSR SCRATCH from kernel stack
pointer to be kernel TP(thread_core_local). So that we can get TP from
SCRATCH easily in trap handler when the thread is in user mode. We also
save/restore CSR IE, kernel GP and SP so that we can handle task
migration to another hart.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Apply exception return to resume thread

In current implementation, the thread is resumed by function return. It
is not suitable for all scenarios, especially when the thread should be
r

core: riscv: Apply exception return to resume thread

In current implementation, the thread is resumed by function return. It
is not suitable for all scenarios, especially when the thread should be
resumed to user mode. The kernel mode can not return to user mode by
pure function return.

This commit applies exception return to resume the thread. The EPC and
IE are added into thread context. The xstatus_for_xret() helper function
is added to prepare the value of CSR STATUS for exception return.
Currently we only consider PIE(previous interrupt-enable) and
PP(previous privilege mode) for exception return.

We clear thread context when the context is reinitialized, enable native
interrupt, and setup kernel GP/TP. The thread_resume() now takes care of
restoring CSR EPC, STATUS, IE, SCRATCH and all general-purpose
registers. Finally it executes exception return to target privilege mode
encoded in CSR STATUS. The registers GP and TP are also restored since
user mode may use them.

This commit also modify the usage of CSR SCRATCH. In current
implementation the SCRATCH is used to save kernel stack pointer when the
thread is in user mode. The value of TP, which stores thread_core_local
structure, is saved into kernel stack before entering user mode. The
trap handler can then get TP(thread_core_local) from kernel stack. This
is not suitable for SMP system, since the thread might be resumed to
another core, and that core gets wrong TP from kernel stack. Fix it by
directly storing TP into CSR SCRATCH.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Implement thread_vector_table for ABI and FIQ entries

Implement thread_vector_table which only includes entries for standard
ABI, fast ABI, and foreign interrupts. Most of code is refer

core: riscv: Implement thread_vector_table for ABI and FIQ entries

Implement thread_vector_table which only includes entries for standard
ABI, fast ABI, and foreign interrupts. Most of code is referenced from
ARM architecture. The thread_vector_table will be registered into higher
privileged software, such as M-mode firmware. The higher privileged
software can jump(mret) to OP-TEE based on this vector table.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: Load register TP from thread_user_mode_rec in trap handler

RISC-V kernel uses TP register to store thread_core_local structure.
When the thread enters user mode, the value of TP is used

core: riscv: Load register TP from thread_user_mode_rec in trap handler

RISC-V kernel uses TP register to store thread_core_local structure.
When the thread enters user mode, the value of TP is used by user mode.
Therefore, when CPU enters trap handler, it needs to restore TP to get
thread_core_local structure. In previous implementation, the value of TP
is saved under kernel SP before entering user mode, and the trap handler
restores TP from that stack location. However, the value of TP has
already been saved into the thread_user_mode_rec structure, which is
also upon kernel SP, before entering user mode. So the value of TP can
be restored just from thread_user_mode_rec, instead of saving into
another location which is under the kernel SP.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

riscv: kernel: asm-defines.c: add defines for struct core_mmu_config

Add CORE_MMU_CONFIG_SIZE and CORE_MMU_CONFIG_SATP defines
to asm-defines.c

Signed-off-by: Marouene Boubakri <marouene.boubakri@n

riscv: kernel: asm-defines.c: add defines for struct core_mmu_config

Add CORE_MMU_CONFIG_SIZE and CORE_MMU_CONFIG_SATP defines
to asm-defines.c

Signed-off-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: Save and restore register s0 when trap occurs

In RISC-V the frame pointer is stored into register s0. It is necessary
to save/restore s0 when we enable CFG_UNWIND to process stack unwin

core: riscv: Save and restore register s0 when trap occurs

In RISC-V the frame pointer is stored into register s0. It is necessary
to save/restore s0 when we enable CFG_UNWIND to process stack unwinding.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: asm-defines.c: create and populate with thread-related defines

Create core/arch/riscv/kernel/asm-defines.c and add defines for thread_ctx,
thread_core_local, thread_ctx_regs, thread_use

core: riscv: asm-defines.c: create and populate with thread-related defines

Create core/arch/riscv/kernel/asm-defines.c and add defines for thread_ctx,
thread_core_local, thread_ctx_regs, thread_user_mode_rec, thread_trap_regs
and thread_scall_regs.

Signed-off-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...