core: riscv: mm: Support dynamic allocation of translation tables

With CFG_DYN_CONFIG enabled allocate translation tables using the
boot_mem_*() functions. Static allocation from global variables is

core: riscv: mm: Support dynamic allocation of translation tables

With CFG_DYN_CONFIG enabled allocate translation tables using the
boot_mem_*() functions. Static allocation from global variables is still
used with CFG_DYN_CONFIG disabled.

This commit is referenced from ARM introduced in commit a28e4a0fe48d
("core: arm: mm: dynamic allocation of LPAE translation tables").

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: mm: Minor refactoring and add missing code

Add core_pos as argument of core_mmu_get_root_pgt_va() and clean up
relevant code. Add missing code of printing memory map into
core_init_mmu_

core: riscv: mm: Minor refactoring and add missing code

Add core_pos as argument of core_mmu_get_root_pgt_va() and clean up
relevant code. Add missing code of printing memory map into
core_init_mmu_prtn_tee().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: Add boot mem paddings to the heap

Add the paddings added due to requested alignment in boot mem
allocations to the heap.

This commit is referenced from ARM architecture introduced in c

core: riscv: Add boot mem paddings to the heap

Add the paddings added due to requested alignment in boot mem
allocations to the heap.

This commit is referenced from ARM architecture introduced in commit
0799b137207b ("core: arm: add boot mem paddings to the heap").

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: Preparation to support CFG_BOOT_MEM

Refer to commit d461c892a15a ("core: arm: enable CFG_BOOT_MEM
unconditionally") and commit f12843460d47 ("core: mm: allocate
temporary memory map arr

core: riscv: Preparation to support CFG_BOOT_MEM

Refer to commit d461c892a15a ("core: arm: enable CFG_BOOT_MEM
unconditionally") and commit f12843460d47 ("core: mm: allocate
temporary memory map array"), call the boot_mem_*() functions as
needed from entry.S and boot.c for RISC-V architecture.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

riscv: plat-virt: add APLIC and IMSIC support for QEMU virt platform

- Add APLIC and IMSIC configurations for the QEMU virt platform.
- Override the interrupt controller initialization and interrupt

riscv: plat-virt: add APLIC and IMSIC support for QEMU virt platform

- Add APLIC and IMSIC configurations for the QEMU virt platform.
- Override the interrupt controller initialization and interrupt handler
functions when using APLIC or IMSIC.

Signed-off-by: Huang Borong <huangborong@bosc.ac.cn>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

drivers: add RISC-V IMSIC interrupt driver

The RISC-V Advanced Interrupt Architecture (AIA) specification introduces
the IMSIC as a new external interrupt controller. An IMSIC receives and
records i

drivers: add RISC-V IMSIC interrupt driver

The RISC-V Advanced Interrupt Architecture (AIA) specification introduces
the IMSIC as a new external interrupt controller. An IMSIC receives and
records incoming message-signaled interrupts (MSIs).

This commit enables the initialization of the IMSIC based on the device
tree and adds control and status registers (CSRs) for indirect access to
the IMSIC as well as for reading interrupt identities.

Use the `CFG_RISCV_IMSIC` flag to control whether to build this driver.

For more details, see: https://github.com/riscv/riscv-aia

Signed-off-by: Huang Borong <huangborong@bosc.ac.cn>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Call call_driver_initcalls() late

Calls call_early_initcalls() and call_service_initcalls() directly
instead of call_initcalls() from init_tee_runtime().

This commit is to synchronize

core: riscv: Call call_driver_initcalls() late

Calls call_early_initcalls() and call_service_initcalls() directly
instead of call_initcalls() from init_tee_runtime().

This commit is to synchronize the initcalls with ARM architecture,
introduced in 27ed6973 (core: arm: call call_driver_initcalls() late).

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Introduce boot_init_primary_final()

Introduce boot_init_primary_final() and move the call to
call_finalcalls() into that function.

This commit is to synchronize the boot stages with AR

core: riscv: Introduce boot_init_primary_final()

Introduce boot_init_primary_final() and move the call to
call_finalcalls() into that function.

This commit is to synchronize the boot stages with ARM architecture,
introduced in d0c23684 (core: arm: introduce boot_init_primary_final()).

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

riscv: plat-virt: Let console devices be build time configurable

Currently RISC-V virtual platform enforces 16550 UART to be console
device. However, there are other console devices which can be cho

riscv: plat-virt: Let console devices be build time configurable

Currently RISC-V virtual platform enforces 16550 UART to be console
device. However, there are other console devices which can be chose by
developer. Thus, we allow the configurations for console device to be
overridden at build time while keeping the default value enabled.

Besides, fix CFG_SBI_CONSOLE to be CFG_RISCV_SBI_CONSOLE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

riscv: plat-virt: Let CFG_RISCV_PLIC be build time configurable

RISC-V has several standard interrupt controllers supported by QEMU
virtual platform. Thus, we allow CFG_RISCV_PLIC to be overridden a

riscv: plat-virt: Let CFG_RISCV_PLIC be build time configurable

RISC-V has several standard interrupt controllers supported by QEMU
virtual platform. Thus, we allow CFG_RISCV_PLIC to be overridden at
build time while keeping the default value enabled.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
al

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
all CPU cores too. This fixes an issue when the VA of
MEM_AREA_RES_VASPACE or MEM_AREA_SHM_VASPACE is not in a same 1GB region
with other memory regions.

Link: https://github.com/OP-TEE/optee_os/issues/7275
Signed-off-by: Mark Zhang <markz@nvidia.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: add VCORE_FREE_{PA,SZ,END_PA}

Add VCORE_FREE_{PA,SZ,END_PA} defines to identify the unused and free
memory range at the end of TEE_RAM_START..(TEE_RAM_START +
TEE_RAM_VA_SIZE).

VCORE_FREE_SZ

core: add VCORE_FREE_{PA,SZ,END_PA}

Add VCORE_FREE_{PA,SZ,END_PA} defines to identify the unused and free
memory range at the end of TEE_RAM_START..(TEE_RAM_START +
TEE_RAM_VA_SIZE).

VCORE_FREE_SZ is 0 in a pager configuration since all the memory is
used by the pager.

The VCORE_FREE range is excluded from the TEE_RAM_RW area for
CFG_NS_VIRTUALIZATION=y and instead put in a separate NEX_RAM_RW area.
This makes each partition use a bit less memory and leaves the
VCORE_FREE range available for the Nexus.

The VCORE_FREE range is added to the TEE_RAM_RW area for the normal
configuration with CFG_NS_VIRTUALIZATION=n and CFG_WITH_PAGER=n. It's in
practice unchanged behaviour in this configuration.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Improve macros for set/clear bits CSR operations

Rename `set_csr` to `read_set_csr` and `clear_csr` to `read_clear_csr`
because they perform atomic reads and set/clear bits in the CSR.

core: riscv: Improve macros for set/clear bits CSR operations

Rename `set_csr` to `read_set_csr` and `clear_csr` to `read_clear_csr`
because they perform atomic reads and set/clear bits in the CSR. These
two macros will return the previous value of the CSR.

Introduce new macros `set_csr` and `clear_csr`: `set_csr` uses the
RISC-V `csrs` assembler pseudoinstruction to set bits in the CSR when
the old value is not needed, while `clear_csr` uses the `csrc`
pseudoinstruction to clear bits in the CSR, also discarding the old
value.

Signed-off-by: Huang Borong <huangborong@bosc.ac.cn>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: riscv: Disable traps by clearing XIE CSR

Ensure we disable traps by clearing XIE CSR instead of clearing
XSTATUS.IE which is global interrupt enable bit.

Signed-off-by: Alvin Chang <alvinga@a

core: riscv: Disable traps by clearing XIE CSR

Ensure we disable traps by clearing XIE CSR instead of clearing
XSTATUS.IE which is global interrupt enable bit.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Improve thread user mode record

Make the asm definitions be more human-readable.

Besides, it's unnecessary to save and restore kernel SP and GP into
thread_user_mode_rec, since they wi

core: riscv: Improve thread user mode record

Make the asm definitions be more human-readable.

Besides, it's unnecessary to save and restore kernel SP and GP into
thread_user_mode_rec, since they will be setup by system call trap
handler before executing thread_unwind_user_mode().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Ensure XSTATUS is restored before XIE

In previous implementation, we found some accidental interrupts during
entering user mode and resuming of thread. We fixed it by clearing
XSTATUS.X

core: riscv: Ensure XSTATUS is restored before XIE

In previous implementation, we found some accidental interrupts during
entering user mode and resuming of thread. We fixed it by clearing
XSTATUS.XIE first, which is global interrupt enable bit, to ensure there
are no interrupts during those operations.

Now we found the better solution: restore XSTATUS before restoring XIE.
This can ensure the global interrupt bit in XSTATUS is cleared before we
restore the individual interrupt bits in XIE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Remove thread_exit_user_mode()

Currently, the user mode abort and some system calls return to kernel
mode by thread_exit_user_mode(). Although this function creates a
shorter path to re

core: riscv: Remove thread_exit_user_mode()

Currently, the user mode abort and some system calls return to kernel
mode by thread_exit_user_mode(). Although this function creates a
shorter path to return to kernel mode, it leads to some problems because
the function does not update the core local flags. Especially when
CFG_CORE_DEBUG_CHECK_STACKS=y, some checks will fail due to wrong type
of stack recorded in the core local flags.

Fix it by removing thread_exit_user_mode(). So that the core local flags
can be correctly updated in the common trap handler.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Use sp as base register of load instructions

Use sp as base register of load instructions can reduce code size if RVC
extension is enabled to generate 16-bit instructions. The following

core: riscv: Use sp as base register of load instructions

Use sp as base register of load instructions can reduce code size if RVC
extension is enabled to generate 16-bit instructions. The following code
shows the difference after applying this commit.

Before:
f10009da: 0d053d83 ld s11,208(a0)
f10009de: 0c853d03 ld s10,200(a0)
f10009e2: 0c053c83 ld s9,192(a0)
f10009e6: 0b853c03 ld s8,184(a0)
f10009ea: 0b053b83 ld s7,176(a0)
f10009ee: 0a853b03 ld s6,168(a0)
f10009f2: 0a053a83 ld s5,160(a0)
f10009f6: 09853a03 ld s4,152(a0)
f10009fa: 09053983 ld s3,144(a0)
f10009fe: 08853903 ld s2,136(a0)

After:
f10009a6: 6dce ld s11,208(sp)
f10009a8: 6d2e ld s10,200(sp)
f10009aa: 6c8e ld s9,192(sp)
f10009ac: 7c6a ld s8,184(sp)
f10009ae: 7bca ld s7,176(sp)
f10009b0: 7b2a ld s6,168(sp)
f10009b2: 7a8a ld s5,160(sp)
f10009b4: 6a6a ld s4,152(sp)
f10009b6: 69ca ld s3,144(sp)
f10009b8: 692a ld s2,136(sp)

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Fix misconfiguration of XSCRATCH when XRET to kernel mode

When the program wants to XRET to kernel mode, the value of XSCRATCH
must be cleared to zero.

Signed-off-by: Alvin Chang <alvi

core: riscv: Fix misconfiguration of XSCRATCH when XRET to kernel mode

When the program wants to XRET to kernel mode, the value of XSCRATCH
must be cleared to zero.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

core: riscv: Set exception return PC into XEPC for entering user mode

Instead of setting exception return PC into "ra" register and assign it
to XEPC, we should directly set exception return PC into

core: riscv: Set exception return PC into XEPC for entering user mode

Instead of setting exception return PC into "ra" register and assign it
to XEPC, we should directly set exception return PC into "XEPC" CSR to
improve code redability.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>

show more ...

riscv: plat-virt: allow enabling CFG_TEE_CORE_DEBUG for virt machine

Allow enabling CFG_TEE_CORE_DEBUG to make assertions useful.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed

riscv: plat-virt: allow enabling CFG_TEE_CORE_DEBUG for virt machine

Allow enabling CFG_TEE_CORE_DEBUG to make assertions useful.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mm: core_mmu: add core_mmu_user_va_range_is_defined() for RISC-V

The function hasn't been implemented for RISC-V, so move the
core_mmu_user_va_range_is_defined() definition to generic
core_mmu

core: mm: core_mmu: add core_mmu_user_va_range_is_defined() for RISC-V

The function hasn't been implemented for RISC-V, so move the
core_mmu_user_va_range_is_defined() definition to generic
core_mmu.h and function implementations to arch-specific files.

Also, update the assertions where checks if user va range is defined.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: core_mmu_arch: fix compile error for bit_test()

Fix the compile error in the bit_test() macro, which mistakenly
uses the address of g_asid as the parameter.

Signed-off-by: Yu Chien Pet

core: riscv: core_mmu_arch: fix compile error for bit_test()

Fix the compile error in the bit_test() macro, which mistakenly
uses the address of g_asid as the parameter.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: mm: fix arch_va2pa_helper() on superpage translation

Any level of PTE may be a leaf PTE in RISC-V page table, if the
page is not 4KiB, the page offset should be extended to VPN fields
o

core: riscv: mm: fix arch_va2pa_helper() on superpage translation

Any level of PTE may be a leaf PTE in RISC-V page table, if the
page is not 4KiB, the page offset should be extended to VPN fields
of virtual address.

e.g. on Sv39, if there is a leaf PTE on level-1 (2MiB megapage),
it maps to physical page with (va[20:12] | va[11:0]) as the page
offset.

Sv39 Virtual address:
|<--- superpage offset --->|
38_______30_29______21|20______12_11____________0|
| VPN[2] | VPN[1] | VPN[0] | page offset |
‾‾‾‾9‾‾‾‾‾‾‾‾‾‾9‾‾‾‾‾|‾‾‾‾9‾‾‾‾‾‾‾‾‾‾‾‾12‾‾‾‾‾‾‾|
| |
Physical address: | |
| |
55___________30_29______21|20______12_11____________0|
| PPN[2] | PPN[1] | PPN[0] | page offset |
‾‾‾‾‾‾26‾‾‾‾‾‾‾‾‾‾‾9‾‾‾‾‾'‾‾‾‾9‾‾‾‾‾‾‾‾‾‾‾‾12‾‾‾‾‾‾‾'

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Let platform choose native and foreign interrupts

Platform may want to choose different types of native and foreign
interrupts. This commit adds two definitions for the platforms:
1. PL

core: riscv: Let platform choose native and foreign interrupts

Platform may want to choose different types of native and foreign
interrupts. This commit adds two definitions for the platforms:
1. PLAT_THREAD_EXCP_FOREIGN_INTR: to define platform specific foreign
interrupts
2. PLAT_THREAD_EXCP_NATIVE_INTR: to define platform specific native
interrupts.

For RISC-V virt machine, we define all external/mtimer/software
interrupts as foreign interrupts. For RISC-V spike platform, we define
external interrupts as foreign interrupts, and mtimer/software
interrupts as native interrupts.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...