riscv: kernel: support booting non-contiguous non-zero-based hart IDs

Currently, OP-TEE assumes 0 <= hartid < CFG_TEE_CORE_NB_CORE,
and must be contiguous, which fails to accommodate different
CPU t

riscv: kernel: support booting non-contiguous non-zero-based hart IDs

Currently, OP-TEE assumes 0 <= hartid < CFG_TEE_CORE_NB_CORE,
and must be contiguous, which fails to accommodate different
CPU topologies. For example, some RISC-V platforms, such as
the HiFive Unmatched board, do not run Linux and OP-TEE on
hart0, as it is a monitor core without supervisor mode support.

To address this, introduce hart_index, which is used to index
per-hart structures, such as thread_core_local and root_pgt.
The hart_index will range from 0 to (CFG_TEE_CORE_NB_CORE - 1),
and the primary hart will have an index of 0.

Additionally, a new function, boot_primary_init_core_ids(),
is added to initialize secondary hart IDs for booting via
sbi_hsm_hart_start().

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

riscv: kernel: sbi: introduce sbi_hsm_hart_get_status() function

Introduce sbi_hsm_hart_get_status() function and add comment for
sbi_hsm_hart_start().

Signed-off-by: Yu-Chien Peter Lin <peter.lin@

riscv: kernel: sbi: introduce sbi_hsm_hart_get_status() function

Introduce sbi_hsm_hart_get_status() function and add comment for
sbi_hsm_hart_start().

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

riscv: plat-virt: move stack alignment definition

The 16-byte stack alignment is a RISC-V ABI requirement
that applies to all RISC-V platforms. Move this definition
from the virt platform configurat

riscv: plat-virt: move stack alignment definition

The 16-byte stack alignment is a RISC-V ABI requirement
that applies to all RISC-V platforms. Move this definition
from the virt platform configuration to riscv.h.

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

riscv: kernel: entry.S: remove unused boot_args array

The boot_args array is unused anywhere and its contents
are cleared during the zeroing of the .bss section, so it
serves no purpose. Removing it

riscv: kernel: entry.S: remove unused boot_args array

The boot_args array is unused anywhere and its contents
are cleared during the zeroing of the .bss section, so it
serves no purpose. Removing it simplifies the code.

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Zong Li <zong.li@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

ta: enable ubsan support for TAs

Introduce CFG_TA_SANITIZE_UNDEFINED config to sanitize trusted
applications. If CFG_TA_SANITIZE_UNDEFINED is set sanitize flags are
propagated to internal TAs (avb,

ta: enable ubsan support for TAs

Introduce CFG_TA_SANITIZE_UNDEFINED config to sanitize trusted
applications. If CFG_TA_SANITIZE_UNDEFINED is set sanitize flags are
propagated to internal TAs (avb, pkcs11, remoteproc, trusted_keys) and
external TAs, which are built with the devkit.

Signed-off-by: Aleksandr Iashchenko <aleksandr.iashchenko@linutronix.de>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: add nex_dyn_vaspace and tee_dyn_vaspace areas

Add MEM_AREA_NEX_DYN_VASPACE and MEM_AREA_TEE_DYN_VASPACE areas for
dynamic Nexus and TEE memory mapping. This will be used to map
additional heap

core: add nex_dyn_vaspace and tee_dyn_vaspace areas

Add MEM_AREA_NEX_DYN_VASPACE and MEM_AREA_TEE_DYN_VASPACE areas for
dynamic Nexus and TEE memory mapping. This will be used to map
additional heap and the stacks in later patches.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: mm: Support dynamic allocation of translation tables

With CFG_DYN_CONFIG enabled allocate translation tables using the
boot_mem_*() functions. Static allocation from global variables is

core: riscv: mm: Support dynamic allocation of translation tables

With CFG_DYN_CONFIG enabled allocate translation tables using the
boot_mem_*() functions. Static allocation from global variables is still
used with CFG_DYN_CONFIG disabled.

This commit is referenced from ARM introduced in commit a28e4a0fe48d
("core: arm: mm: dynamic allocation of LPAE translation tables").

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: mm: Minor refactoring and add missing code

Add core_pos as argument of core_mmu_get_root_pgt_va() and clean up
relevant code. Add missing code of printing memory map into
core_init_mmu_

core: riscv: mm: Minor refactoring and add missing code

Add core_pos as argument of core_mmu_get_root_pgt_va() and clean up
relevant code. Add missing code of printing memory map into
core_init_mmu_prtn_tee().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: Add boot mem paddings to the heap

Add the paddings added due to requested alignment in boot mem
allocations to the heap.

This commit is referenced from ARM architecture introduced in c

core: riscv: Add boot mem paddings to the heap

Add the paddings added due to requested alignment in boot mem
allocations to the heap.

This commit is referenced from ARM architecture introduced in commit
0799b137207b ("core: arm: add boot mem paddings to the heap").

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: Preparation to support CFG_BOOT_MEM

Refer to commit d461c892a15a ("core: arm: enable CFG_BOOT_MEM
unconditionally") and commit f12843460d47 ("core: mm: allocate
temporary memory map arr

core: riscv: Preparation to support CFG_BOOT_MEM

Refer to commit d461c892a15a ("core: arm: enable CFG_BOOT_MEM
unconditionally") and commit f12843460d47 ("core: mm: allocate
temporary memory map array"), call the boot_mem_*() functions as
needed from entry.S and boot.c for RISC-V architecture.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

riscv: plat-virt: add APLIC and IMSIC support for QEMU virt platform

- Add APLIC and IMSIC configurations for the QEMU virt platform.
- Override the interrupt controller initialization and interrupt

riscv: plat-virt: add APLIC and IMSIC support for QEMU virt platform

- Add APLIC and IMSIC configurations for the QEMU virt platform.
- Override the interrupt controller initialization and interrupt handler
functions when using APLIC or IMSIC.

Signed-off-by: Huang Borong <huangborong@bosc.ac.cn>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

drivers: add RISC-V IMSIC interrupt driver

The RISC-V Advanced Interrupt Architecture (AIA) specification introduces
the IMSIC as a new external interrupt controller. An IMSIC receives and
records i

drivers: add RISC-V IMSIC interrupt driver

The RISC-V Advanced Interrupt Architecture (AIA) specification introduces
the IMSIC as a new external interrupt controller. An IMSIC receives and
records incoming message-signaled interrupts (MSIs).

This commit enables the initialization of the IMSIC based on the device
tree and adds control and status registers (CSRs) for indirect access to
the IMSIC as well as for reading interrupt identities.

Use the `CFG_RISCV_IMSIC` flag to control whether to build this driver.

For more details, see: https://github.com/riscv/riscv-aia

Signed-off-by: Huang Borong <huangborong@bosc.ac.cn>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Call call_driver_initcalls() late

Calls call_early_initcalls() and call_service_initcalls() directly
instead of call_initcalls() from init_tee_runtime().

This commit is to synchronize

core: riscv: Call call_driver_initcalls() late

Calls call_early_initcalls() and call_service_initcalls() directly
instead of call_initcalls() from init_tee_runtime().

This commit is to synchronize the initcalls with ARM architecture,
introduced in 27ed6973 (core: arm: call call_driver_initcalls() late).

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Introduce boot_init_primary_final()

Introduce boot_init_primary_final() and move the call to
call_finalcalls() into that function.

This commit is to synchronize the boot stages with AR

core: riscv: Introduce boot_init_primary_final()

Introduce boot_init_primary_final() and move the call to
call_finalcalls() into that function.

This commit is to synchronize the boot stages with ARM architecture,
introduced in d0c23684 (core: arm: introduce boot_init_primary_final()).

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

riscv: plat-virt: Let console devices be build time configurable

Currently RISC-V virtual platform enforces 16550 UART to be console
device. However, there are other console devices which can be cho

riscv: plat-virt: Let console devices be build time configurable

Currently RISC-V virtual platform enforces 16550 UART to be console
device. However, there are other console devices which can be chose by
developer. Thus, we allow the configurations for console device to be
overridden at build time while keeping the default value enabled.

Besides, fix CFG_SBI_CONSOLE to be CFG_RISCV_SBI_CONSOLE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

riscv: plat-virt: Let CFG_RISCV_PLIC be build time configurable

RISC-V has several standard interrupt controllers supported by QEMU
virtual platform. Thus, we allow CFG_RISCV_PLIC to be overridden a

riscv: plat-virt: Let CFG_RISCV_PLIC be build time configurable

RISC-V has several standard interrupt controllers supported by QEMU
virtual platform. Thus, we allow CFG_RISCV_PLIC to be overridden at
build time while keeping the default value enabled.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
al

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
all CPU cores too. This fixes an issue when the VA of
MEM_AREA_RES_VASPACE or MEM_AREA_SHM_VASPACE is not in a same 1GB region
with other memory regions.

Link: https://github.com/OP-TEE/optee_os/issues/7275
Signed-off-by: Mark Zhang <markz@nvidia.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: add VCORE_FREE_{PA,SZ,END_PA}

Add VCORE_FREE_{PA,SZ,END_PA} defines to identify the unused and free
memory range at the end of TEE_RAM_START..(TEE_RAM_START +
TEE_RAM_VA_SIZE).

VCORE_FREE_SZ

core: add VCORE_FREE_{PA,SZ,END_PA}

Add VCORE_FREE_{PA,SZ,END_PA} defines to identify the unused and free
memory range at the end of TEE_RAM_START..(TEE_RAM_START +
TEE_RAM_VA_SIZE).

VCORE_FREE_SZ is 0 in a pager configuration since all the memory is
used by the pager.

The VCORE_FREE range is excluded from the TEE_RAM_RW area for
CFG_NS_VIRTUALIZATION=y and instead put in a separate NEX_RAM_RW area.
This makes each partition use a bit less memory and leaves the
VCORE_FREE range available for the Nexus.

The VCORE_FREE range is added to the TEE_RAM_RW area for the normal
configuration with CFG_NS_VIRTUALIZATION=n and CFG_WITH_PAGER=n. It's in
practice unchanged behaviour in this configuration.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Improve macros for set/clear bits CSR operations

Rename `set_csr` to `read_set_csr` and `clear_csr` to `read_clear_csr`
because they perform atomic reads and set/clear bits in the CSR.

core: riscv: Improve macros for set/clear bits CSR operations

Rename `set_csr` to `read_set_csr` and `clear_csr` to `read_clear_csr`
because they perform atomic reads and set/clear bits in the CSR. These
two macros will return the previous value of the CSR.

Introduce new macros `set_csr` and `clear_csr`: `set_csr` uses the
RISC-V `csrs` assembler pseudoinstruction to set bits in the CSR when
the old value is not needed, while `clear_csr` uses the `csrc`
pseudoinstruction to clear bits in the CSR, also discarding the old
value.

Signed-off-by: Huang Borong <huangborong@bosc.ac.cn>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: Disable traps by clearing XIE CSR

Ensure we disable traps by clearing XIE CSR instead of clearing
XSTATUS.IE which is global interrupt enable bit.

Signed-off-by: Alvin Chang <alvinga@a

core: riscv: Disable traps by clearing XIE CSR

Ensure we disable traps by clearing XIE CSR instead of clearing
XSTATUS.IE which is global interrupt enable bit.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Improve thread user mode record

Make the asm definitions be more human-readable.

Besides, it's unnecessary to save and restore kernel SP and GP into
thread_user_mode_rec, since they wi

core: riscv: Improve thread user mode record

Make the asm definitions be more human-readable.

Besides, it's unnecessary to save and restore kernel SP and GP into
thread_user_mode_rec, since they will be setup by system call trap
handler before executing thread_unwind_user_mode().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Ensure XSTATUS is restored before XIE

In previous implementation, we found some accidental interrupts during
entering user mode and resuming of thread. We fixed it by clearing
XSTATUS.X

core: riscv: Ensure XSTATUS is restored before XIE

In previous implementation, we found some accidental interrupts during
entering user mode and resuming of thread. We fixed it by clearing
XSTATUS.XIE first, which is global interrupt enable bit, to ensure there
are no interrupts during those operations.

Now we found the better solution: restore XSTATUS before restoring XIE.
This can ensure the global interrupt bit in XSTATUS is cleared before we
restore the individual interrupt bits in XIE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Remove thread_exit_user_mode()

Currently, the user mode abort and some system calls return to kernel
mode by thread_exit_user_mode(). Although this function creates a
shorter path to re

core: riscv: Remove thread_exit_user_mode()

Currently, the user mode abort and some system calls return to kernel
mode by thread_exit_user_mode(). Although this function creates a
shorter path to return to kernel mode, it leads to some problems because
the function does not update the core local flags. Especially when
CFG_CORE_DEBUG_CHECK_STACKS=y, some checks will fail due to wrong type
of stack recorded in the core local flags.

Fix it by removing thread_exit_user_mode(). So that the core local flags
can be correctly updated in the common trap handler.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Use sp as base register of load instructions

Use sp as base register of load instructions can reduce code size if RVC
extension is enabled to generate 16-bit instructions. The following

core: riscv: Use sp as base register of load instructions

Use sp as base register of load instructions can reduce code size if RVC
extension is enabled to generate 16-bit instructions. The following code
shows the difference after applying this commit.

Before:
f10009da: 0d053d83 ld s11,208(a0)
f10009de: 0c853d03 ld s10,200(a0)
f10009e2: 0c053c83 ld s9,192(a0)
f10009e6: 0b853c03 ld s8,184(a0)
f10009ea: 0b053b83 ld s7,176(a0)
f10009ee: 0a853b03 ld s6,168(a0)
f10009f2: 0a053a83 ld s5,160(a0)
f10009f6: 09853a03 ld s4,152(a0)
f10009fa: 09053983 ld s3,144(a0)
f10009fe: 08853903 ld s2,136(a0)

After:
f10009a6: 6dce ld s11,208(sp)
f10009a8: 6d2e ld s10,200(sp)
f10009aa: 6c8e ld s9,192(sp)
f10009ac: 7c6a ld s8,184(sp)
f10009ae: 7bca ld s7,176(sp)
f10009b0: 7b2a ld s6,168(sp)
f10009b2: 7a8a ld s5,160(sp)
f10009b4: 6a6a ld s4,152(sp)
f10009b6: 69ca ld s3,144(sp)
f10009b8: 692a ld s2,136(sp)

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Fix misconfiguration of XSCRATCH when XRET to kernel mode

When the program wants to XRET to kernel mode, the value of XSCRATCH
must be cleared to zero.

Signed-off-by: Alvin Chang <alvi

core: riscv: Fix misconfiguration of XSCRATCH when XRET to kernel mode

When the program wants to XRET to kernel mode, the value of XSCRATCH
must be cleared to zero.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...