core: riscv: implement ASLR base address calculation

Implement arch_aslr_base_addr() for RISC-V to compute the ASLR base
address using the provided seed.

The implementation maps addresses across th

core: riscv: implement ASLR base address calculation

Implement arch_aslr_base_addr() for RISC-V to compute the ASLR base
address using the provided seed.

The implementation maps addresses across the full virtual address space:
- When MSB of new based address is set, the address is mapped
to the upper half by extending MSB to 64-bit
- Otherwise, it's mapped to the lower half

This approach utilizes the entire available virtual address space for
ASLR (e.g. 512 GiB for Sv39).

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

riscv: mm: fix relocation of pointers in user_vpn2_table_va[]

The user_vpn2_table_va[] stores several pointers pointing to the level 2
page tables used by user TA. When CFG_CORE_ASLR=y, these pointe

riscv: mm: fix relocation of pointers in user_vpn2_table_va[]

The user_vpn2_table_va[] stores several pointers pointing to the level 2
page tables used by user TA. When CFG_CORE_ASLR=y, these pointers must
be relocated to ASLR VA as well. To fix this issue, we call
boot_mem_add_reloc() onto each element of user_vpn2_table_va[], so that
the pointers have a chance to be relocated by boot_mem_relocate() later.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Suggested-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

riscv: mm: Fix VA base for canonical addresses

RISC-V defines the following virtual address rules:
- For Sv39 (39-bit VA), bits 63–39 all equal to bit 38 of VA
- For Sv48 (48-bit VA), bits 63–48 all

riscv: mm: Fix VA base for canonical addresses

RISC-V defines the following virtual address rules:
- For Sv39 (39-bit VA), bits 63–39 all equal to bit 38 of VA
- For Sv48 (48-bit VA), bits 63–48 all equal to bit 47 of VA
- For Sv57 (57-bit VA), bits 63–57 all equal to bit 56 of VA

In other words, the most-significant bits of VA base must be all one if
the highest SvXX address bit of VA is one.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: mm: add per-mapping VA range validation

Replace single maximum VA check with individual VA
range validation for each memory map entry during MMU
initialization, providing earlier detect

core: riscv: mm: add per-mapping VA range validation

Replace single maximum VA check with individual VA
range validation for each memory map entry during MMU
initialization, providing earlier detection of invalid
mappings.

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: mm: refactor ASLR mapping for architecture support

To allow adding RISC-V ASLR support, add arch_aslr_base_addr()
which will be used to apply architecture specific ASLR base
calculation.

Sign

core: mm: refactor ASLR mapping for architecture support

To allow adding RISC-V ASLR support, add arch_aslr_base_addr()
which will be used to apply architecture specific ASLR base
calculation.

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Suggested-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

riscv: mm: Fix core_mmu_entry_is_branch()

We must also check V bit to determine non-leaf PTE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.co

riscv: mm: Fix core_mmu_entry_is_branch()

We must also check V bit to determine non-leaf PTE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

riscv: mm: support Sv48 and Sv57 address translation for core and TA

Add the macros for Sv57 address translation mode. Add CFG_RISCV_MMU_MODE
into riscv.mk and set it as 39 by default to enable Sv39

riscv: mm: support Sv48 and Sv57 address translation for core and TA

Add the macros for Sv57 address translation mode. Add CFG_RISCV_MMU_MODE
into riscv.mk and set it as 39 by default to enable Sv39 virtual address
translation scheme.

Currently, TA virtual memory occupies 1GB space, and TAs page table
should be an entry inside a level 2 (VPN[2]) page table, which is
decided by user_va_idx variable. For Sv39 translation scheme, it starts
from VPN[2], so nothing to do. For Sv48 translation scheme, we need to
allocate entry 0 of level 3 (VPN[3]) page table, and let it point to the
level 2 page table used by TA. For Sv57 translation scheme, we need to
further allocate entry 0 of level 4 (VPN[4]) page table, and let it
point to the level 3 page table.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

riscv: mm: Explicitly set user VA range inside L2(RV64) and L1(RV32) VPN

Add macros for level and bit shift of virtual page number (VPN) encoded
in RISC-V virtual address. Explicitly set range of us

riscv: mm: Explicitly set user VA range inside L2(RV64) and L1(RV32) VPN

Add macros for level and bit shift of virtual page number (VPN) encoded
in RISC-V virtual address. Explicitly set range of user virtual address
inside level 2 VPN by giving CORE_MMU_VPN2_SHIFT since the
CORE_MMU_BASE_TABLE_SHIFT is not always based on level 2 VPN if the MMU
scheme is not Sv39.

For RV32, there is only two-level VPN. The user VA range would be inside
level 1 VPN.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

riscv: mm: clean up macro definitions

Rename RISCV_SATP_ASID_SIZE to RISCV_SATP_ASID_WIDTH since it is used to
represent width of bits. Also remove redundant RISCV_MMU_ASID_WIDTH
since we already ha

riscv: mm: clean up macro definitions

Rename RISCV_SATP_ASID_SIZE to RISCV_SATP_ASID_WIDTH since it is used to
represent width of bits. Also remove redundant RISCV_MMU_ASID_WIDTH
since we already have RISCV_SATP_ASID_WIDTH. Fix a minor compiler
warning due to inconsistent data types on variable comparison.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: mm: Support dynamic allocation of translation tables

With CFG_DYN_CONFIG enabled allocate translation tables using the
boot_mem_*() functions. Static allocation from global variables is

core: riscv: mm: Support dynamic allocation of translation tables

With CFG_DYN_CONFIG enabled allocate translation tables using the
boot_mem_*() functions. Static allocation from global variables is still
used with CFG_DYN_CONFIG disabled.

This commit is referenced from ARM introduced in commit a28e4a0fe48d
("core: arm: mm: dynamic allocation of LPAE translation tables").

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: riscv: mm: Minor refactoring and add missing code

Add core_pos as argument of core_mmu_get_root_pgt_va() and clean up
relevant code. Add missing code of printing memory map into
core_init_mmu_

core: riscv: mm: Minor refactoring and add missing code

Add core_pos as argument of core_mmu_get_root_pgt_va() and clean up
relevant code. Add missing code of printing memory map into
core_init_mmu_prtn_tee().

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
al

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
all CPU cores too. This fixes an issue when the VA of
MEM_AREA_RES_VASPACE or MEM_AREA_SHM_VASPACE is not in a same 1GB region
with other memory regions.

Link: https://github.com/OP-TEE/optee_os/issues/7275
Signed-off-by: Mark Zhang <markz@nvidia.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: mm: core_mmu: add core_mmu_user_va_range_is_defined() for RISC-V

The function hasn't been implemented for RISC-V, so move the
core_mmu_user_va_range_is_defined() definition to generic
core_mmu

core: mm: core_mmu: add core_mmu_user_va_range_is_defined() for RISC-V

The function hasn't been implemented for RISC-V, so move the
core_mmu_user_va_range_is_defined() definition to generic
core_mmu.h and function implementations to arch-specific files.

Also, update the assertions where checks if user va range is defined.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: core_mmu_arch: fix compile error for bit_test()

Fix the compile error in the bit_test() macro, which mistakenly
uses the address of g_asid as the parameter.

Signed-off-by: Yu Chien Pet

core: riscv: core_mmu_arch: fix compile error for bit_test()

Fix the compile error in the bit_test() macro, which mistakenly
uses the address of g_asid as the parameter.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: mm: fix arch_va2pa_helper() on superpage translation

Any level of PTE may be a leaf PTE in RISC-V page table, if the
page is not 4KiB, the page offset should be extended to VPN fields
o

core: riscv: mm: fix arch_va2pa_helper() on superpage translation

Any level of PTE may be a leaf PTE in RISC-V page table, if the
page is not 4KiB, the page offset should be extended to VPN fields
of virtual address.

e.g. on Sv39, if there is a leaf PTE on level-1 (2MiB megapage),
it maps to physical page with (va[20:12] | va[11:0]) as the page
offset.

Sv39 Virtual address:
|<--- superpage offset --->|
38_______30_29______21|20______12_11____________0|
| VPN[2] | VPN[1] | VPN[0] | page offset |
‾‾‾‾9‾‾‾‾‾‾‾‾‾‾9‾‾‾‾‾|‾‾‾‾9‾‾‾‾‾‾‾‾‾‾‾‾12‾‾‾‾‾‾‾|
| |
Physical address: | |
| |
55___________30_29______21|20______12_11____________0|
| PPN[2] | PPN[1] | PPN[0] | page offset |
‾‾‾‾‾‾26‾‾‾‾‾‾‾‾‾‾‾9‾‾‾‾‾'‾‾‾‾9‾‾‾‾‾‾‾‾‾‾‾‾12‾‾‾‾‾‾‾'

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: core_mmu_arch: fix next level page-table translation

If arch_va2pa_helper() and core_mmu_find_table() are called when
the MMU is disabled, we don't need to convert the next level page
b

core: riscv: core_mmu_arch: fix next level page-table translation

If arch_va2pa_helper() and core_mmu_find_table() are called when
the MMU is disabled, we don't need to convert the next level page
base address with phys_to_virt(). Add core_mmu_xlat_table_entry_pa2va()
to handle this address translation.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: riscv: core_mmu_arch: remove address translation when initializing SATP

Fix the handling of the page table base address (pgt) by removing
the unnecessary virt_to_phys(). The pgt is already a p

core: riscv: core_mmu_arch: remove address translation when initializing SATP

Fix the handling of the page table base address (pgt) by removing
the unnecessary virt_to_phys(). The pgt is already a physical address,
and thus does not require translation.

Additionally, since the ASID always set to 0, replaced the redundant
assertions with a explicit check to ensure the MMU is disabled in the
context.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: mm: introduce struct memory_map

Introduce struct memory_map to keep track of the array of struct
tee_mmap_region, covering number of used entries and number of allocated
entries.

core_mmap_is

core: mm: introduce struct memory_map

Introduce struct memory_map to keep track of the array of struct
tee_mmap_region, covering number of used entries and number of allocated
entries.

core_mmap_is_end_of_table() and MEM_AREA_END are now unused so remove
them.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

core: riscv: core_mmu_arch: fix PPN field extraction from PTE

The upper bits of page table entry may contain other fields
introduced since Priv. ISA spec. v1.11 (20211203), such as PBMT
and N bits,

core: riscv: core_mmu_arch: fix PPN field extraction from PTE

The upper bits of page table entry may contain other fields
introduced since Priv. ISA spec. v1.11 (20211203), such as PBMT
and N bits, thus PPN field should be masked out with PTE_PPN.

Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Translate to PA when allocating PGT with MMU enabled

If MMU is enabled, core_mmu_pgt_alloc() returns virtual address of PGT
instead of physical address. Thus, it leads to some errors wh

core: riscv: Translate to PA when allocating PGT with MMU enabled

If MMU is enabled, core_mmu_pgt_alloc() returns virtual address of PGT
instead of physical address. Thus, it leads to some errors when we
invoke pa_to_ppn() with returned PGT which is actually the virtual
address of that PGT.

Fix it by checking whether MMU is enabled or not. If MMU is enabled, we
translate the returned PGT to its physical address.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Prepare SATP for each hart

To support multiple harts environment, we have allocated root page table
for each hart. Further more, we need to prepare value of CSR SATP, which
holds the ph

core: riscv: Prepare SATP for each hart

To support multiple harts environment, we have allocated root page table
for each hart. Further more, we need to prepare value of CSR SATP, which
holds the physical page number (PPN) of the root page table, for each
hart.

This commit enlarges the "struct core_mmu_config" for RISC-V
architecture to hold the value of CSR SATP for all the harts. In early
boot stage, each hart should initialize its CSR SATP from "struct
core_mmu_config".

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Allocate root page table for each hart

To support multiple hart environment, each hart must have its dedicated
root page table. This commit enlarges the root page table. Also, when
the

core: riscv: Allocate root page table for each hart

To support multiple hart environment, each hart must have its dedicated
root page table. This commit enlarges the root page table. Also, when
the primary hart initializes the page table, we also copy the contents
of its root page table to the secondary harts' root page tables.
Therefore, all the harts have initial page tables at the boot time.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: riscv: Apply mask/unmask exceptions when operating page table

Add missing thread_{mask/unmask}_exceptions() when we operate the page
table. This is referenced from ARM architecture.

Signed-of

core: riscv: Apply mask/unmask exceptions when operating page table

Add missing thread_{mask/unmask}_exceptions() when we operate the page
table. This is referenced from ARM architecture.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Tested-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Fix PTE creation when freeing PTE

The core_mmu_pte_create() is also called when MM core frees the pages,
which means the PTE should be zero. Current implementation always sets
valid bit

core: riscv: Fix PTE creation when freeing PTE

The core_mmu_pte_create() is also called when MM core frees the pages,
which means the PTE should be zero. Current implementation always sets
valid bit (V), which is not proper way when clearing PTE. Fix it by only
honoring pte_bits parameter, which may be constructed in
mattr_to_pte_bits().

The core_mmu_ptp_create() is used to create non-leaf PTE, which points
to the next level of the page table. According to RISC-V privilege Spec,
non-leaf PTE only needs V bit. Therefore, we just give the V bit to
core_mmu_pte_create() when we want to create non-leaf PTE.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...

core: riscv: Rename mattr_to_perms() to mattr_to_pte_bits()

This function not only constructs permission bits (R, W, X) of PTE, but
also other bits such as valid bit (V), user bit (U), global bit (G

core: riscv: Rename mattr_to_perms() to mattr_to_pte_bits()

This function not only constructs permission bits (R, W, X) of PTE, but
also other bits such as valid bit (V), user bit (U), global bit (G),
accessed bit (A), and dirty bit (D). Rename it to mattr_to_pte_bits()
for greater readability.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Marouene Boubakri <marouene.boubakri@nxp.com>

show more ...