core: support dynamic protected memory lending

With CFG_CORE_DYN_PROTMEM=y support dynamic protected memory lending.

A new internal struct mobj_ffa_rsm is added to handle dynamic protected
memory f

core: support dynamic protected memory lending

With CFG_CORE_DYN_PROTMEM=y support dynamic protected memory lending.

A new internal struct mobj_ffa_rsm is added to handle dynamic protected
memory for FF-A.

A new internal struct mobj_protmem is add to handle dynamic protected
memory without FF-A.

Lending non-secure memory to OP-TEE to use it as protected memory means
that it should to become inaccessible by the normal world as part of the
process. This part is currently not supported, since it must be done in
a platform specific way for platforms that support that. QEMU don't
support that.

Adding two platform specific functions, plat_get_protmem_config() and
plat_set_protmem_range() for dynamic protected memory. The functions has
__weak implementation to allow easier testing. However,
plat_set_protmem_range() requires CFG_INSECURE=y since it doesn't change
memory protection.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

mmu: ignore VA spaces in core_mmu_get_type_by_pa

VA spaces have no valid PA addresses stored in memory map, so they are
not valid return values for core_mmu_get_type_by_pa() function.

This issues w

mmu: ignore VA spaces in core_mmu_get_type_by_pa

VA spaces have no valid PA addresses stored in memory map, so they are
not valid return values for core_mmu_get_type_by_pa() function.

This issues was discovered when OP-TEE tried to access a device tree
that was stored at the very beginning of physical address space. In
may case it had PA address 0x112C0, which was "covered" by
RES_VASPACE:

D/TC:0 0 dump_mmap_table:838 type RES_VASPACE va 0x1d800000..0x1e1fffff pa 0x00000000..0x009fffff size 0x00a00000 (pgdir)

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

show more ...

core: mm: fix zero-length access check inconsistency

Fix vm_check_access_rights() so it handles zero-length memory ranges
consistently. Previously, the function had inconsistent behavior for
zero-le

core: mm: fix zero-length access check inconsistency

Fix vm_check_access_rights() so it handles zero-length memory ranges
consistently. Previously, the function had inconsistent behavior for
zero-length checks:
- For page-aligned addresses: Would skip the page checking loop entirely
and return TEE_SUCCESS.
- For unaligned addresses: Would round uaddr down to page boundary and
return a result based on that page.

With this change flags = SECURE | NON_SECURE will still fail to preserve
the sanity checking, but all other zero-length ranges result in
TEE_SUCCESS.

Specifically this was required due to an interaction between OP-TEE and
Rust where Keymint [0] would call the Teaclave [1] wrapper around
TEE_MACComputeFinal with an empty temporary slice (pointer + length
pair) as the final message (&[]). Rust always requires the pointer to be
non-null, but allows it to dangle when length is zero. As a result the
arguments passed to TEE_MACComputeFinal were message=(void *)1,
messageLen=0. These arguments are passed unmodified to
vm_check_access_rights regardless of the length and presumably relied
on the page-aligned case to handle NULL + 0.

[0] https://android.googlesource.com/tee/optee/ta/keymint/
[1] https://github.com/apache/teaclave-trustzone-sdk

Signed-off-by: Martin Nyhus <martin@nyhus.dev>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: fix asan for CFG_WITH_PAGER=n

Some fixes are needed to make CFG_CORE_SANITIZE_KADDRESS=y work both
with and without CFG_DYN_CONFIG=y.

Sanitizing stack addresses aren't supported with CFG_DYN_

core: fix asan for CFG_WITH_PAGER=n

Some fixes are needed to make CFG_CORE_SANITIZE_KADDRESS=y work both
with and without CFG_DYN_CONFIG=y.

Sanitizing stack addresses aren't supported with CFG_DYN_CONFIG=y
since it requires extensive changes in the ASAN framework.

The VCORE_FREE area is moved right before the .asan_shadow area.

init_asan() calls boot_mem_init_asan() to tag access to already
allocated boot memory.

entry_a32.S is updated to skip allowing access to stacks in the
.asan_shadow area for CFG_DYN_CONFIG=y since stacks are stored
elsewhere in that configuration.

entry_a64.S is updated to initialize the .asan_shadow area in the same
way as in entry_a32.S.

The .asan_shadow area is mapped explicitly in collect_mem_ranges()
instead of relying on the now non-existent coverage of
MEM_AREA_TEE_RAM_RW.

CFG_DYN_CONFIG=y and CFG_WITH_PAGER=y is not yet known to work.

Fixes: 1c1f8b65b5c6 ("core: mm: unify secure core and TA memory")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: boot_mem: enable asan support

Add boot_mem_init_asan() to tag all allocated memory with
asan_tag_access().

Allocations with boot_mem_alloc() and boot_mem_alloc_tmp() are tagged
with asan_tag_

core: boot_mem: enable asan support

Add boot_mem_init_asan() to tag all allocated memory with
asan_tag_access().

Allocations with boot_mem_alloc() and boot_mem_alloc_tmp() are tagged
with asan_tag_access().

boot_mem_foreach_padding() temporarily allow access to paddings for the
callback and restores no-access if the callback returns false to tell
that the padding wasn't consumed.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mm: factor out virtual address range validation to arch code

Move virtual address range validation into architecture-specific
code since different architectures have different constraints on
v

core: mm: factor out virtual address range validation to arch code

Move virtual address range validation into architecture-specific
code since different architectures have different constraints on
valid VA ranges:

- For ARM, addresses must be within the VA width supported by the MMU
- For RISC-V, additional checks are needed on RV64 to ensure addresses
are canonically valid

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Reviewed-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: mm: refactor ASLR mapping for architecture support

To allow adding RISC-V ASLR support, add arch_aslr_base_addr()
which will be used to apply architecture specific ASLR base
calculation.

Sign

core: mm: refactor ASLR mapping for architecture support

To allow adding RISC-V ASLR support, add arch_aslr_base_addr()
which will be used to apply architecture specific ASLR base
calculation.

Signed-off-by: Yu-Chien Peter Lin <peter.lin@sifive.com>
Suggested-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Alvin Chang <alvinga@andestech.com>

show more ...

core: mm: shared xlat tables for NEX_DYN_VASPACE

Mappings in MEM_AREA_NEX_DYN_VASPACE belong to the nexus and are must to
be the same for all partitions. Since these mappings must be updated in
the

core: mm: shared xlat tables for NEX_DYN_VASPACE

Mappings in MEM_AREA_NEX_DYN_VASPACE belong to the nexus and are must to
be the same for all partitions. Since these mappings must be updated in
the partitions after the MMU has been enabled. Partitions share
translation tables for this mappings, so we only need to update in one
translation table when adding or removing mappings.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: fix discovered ns-mem check

When discovering or assigning available non-secure physical memory it's
checked against overlaps with other memory types. Memory types reserving
virtual memory spac

core: fix discovered ns-mem check

When discovering or assigning available non-secure physical memory it's
checked against overlaps with other memory types. Memory types reserving
virtual memory space should be excluded including the two recently added
types MEM_AREA_NEX_DYN_VASPACE and MEM_AREA_TEE_DYN_VASPACE. This was
missed when the memory types where added so add the check to exclude
them now.

This fixes an error like:
E/TC:0   check_phys_mem_is_outside:455 Non-sec mem (0:0x60000000) overlaps map (type 10 0:0x100000)
E/TC:0   Panic at core/mm/core_mmu.c:459 <check_phys_mem_is_outside>

Fixes: 96f43358c593 ("core: add nex_dyn_vaspace and tee_dyn_vaspace areas")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mm: add virt_page_alloc()

Add virt_page_alloc() to allocate memory from physical pool and map it
in a virtual address pool.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-

core: mm: add virt_page_alloc()

Add virt_page_alloc() to allocate memory from physical pool and map it
in a virtual address pool.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mm: add phys_mem_alloc_flags()

Add phys_mem_alloc_flags() taking MAF_* flags to control memory
allocation. The new flag MAF_CORE_MEM behaves like
{nex_,}phys_mem_core_alloc(), if the flag is

core: mm: add phys_mem_alloc_flags()

Add phys_mem_alloc_flags() taking MAF_* flags to control memory
allocation. The new flag MAF_CORE_MEM behaves like
{nex_,}phys_mem_core_alloc(), if the flag is absent it becomes
{nex_,}phys_mem_ta_alloc().

The MAF_NEX flag selects Nexus memory.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: add tee_mm_alloc_flags()

Add tee_mm_alloc_flags() taking a flags field to passed to
malloc_flags() when allocating the tee_mm_entry_t.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org

core: add tee_mm_alloc_flags()

Add tee_mm_alloc_flags() taking a flags field to passed to
malloc_flags() when allocating the tee_mm_entry_t.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: tee_mm.c: use malloc_flags() and free_flags()

Use malloc_flags() and free_flags() to simplify memory allocations
with regards to the MAF_NEX/TEE_MM_POOL_NEX_MALLOC flag.

Signed-off-by: Jens W

core: tee_mm.c: use malloc_flags() and free_flags()

Use malloc_flags() and free_flags() to simplify memory allocations
with regards to the MAF_NEX/TEE_MM_POOL_NEX_MALLOC flag.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: add nex_dyn_vaspace and tee_dyn_vaspace areas

Add MEM_AREA_NEX_DYN_VASPACE and MEM_AREA_TEE_DYN_VASPACE areas for
dynamic Nexus and TEE memory mapping. This will be used to map
additional heap

core: add nex_dyn_vaspace and tee_dyn_vaspace areas

Add MEM_AREA_NEX_DYN_VASPACE and MEM_AREA_TEE_DYN_VASPACE areas for
dynamic Nexus and TEE memory mapping. This will be used to map
additional heap and the stacks in later patches.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mmu: fix dynamic VA region dummy mapping

The commit 873f5f6c7201 ("core: mmu: Add dynamic VA regions' mapping to
page table") populated page tables so all are available later when
needed. Howe

core: mmu: fix dynamic VA region dummy mapping

The commit 873f5f6c7201 ("core: mmu: Add dynamic VA regions' mapping to
page table") populated page tables so all are available later when
needed. However, it also mapped physical address 0 in all those ranges.
So fix this by setting attributes to 0 when the physical address is 0.

Fixes: 873f5f6c7201 ("core: mmu: Add dynamic VA regions' mapping to page table")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: Implicitly enable CFG_BOOT_MEM

Now both ARM and RISC-V architectures support and enable CFG_BOOT_MEM by
default. It's unnecessary to define CFG_BOOT_MEM. This commit removes
CFG_BOOT_MEM and r

core: Implicitly enable CFG_BOOT_MEM

Now both ARM and RISC-V architectures support and enable CFG_BOOT_MEM by
default. It's unnecessary to define CFG_BOOT_MEM. This commit removes
CFG_BOOT_MEM and relevant dead code.

Signed-off-by: Alvin Chang <alvinga@andestech.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Yu-Chien Peter Lin <peter.lin@sifive.com>

show more ...

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
al

core: mmu: Add dynamic VA regions' mapping to page table

When optee boots, the initial mapping for MEM_AREA_RES_VASPACE and
MEM_AREA_SHM_VASPACE should be added into page tables and replicated to
all CPU cores too. This fixes an issue when the VA of
MEM_AREA_RES_VASPACE or MEM_AREA_SHM_VASPACE is not in a same 1GB region
with other memory regions.

Link: https://github.com/OP-TEE/optee_os/issues/7275
Signed-off-by: Mark Zhang <markz@nvidia.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: mm: fix carve_out_phys_mem()

carve_out_phys_mem() is prior to this patch not handling cases where the
memory to be carved out isn't covered entirely by the physical memory.
So fix carve_out_ph

core: mm: fix carve_out_phys_mem()

carve_out_phys_mem() is prior to this patch not handling cases where the
memory to be carved out isn't covered entirely by the physical memory.
So fix carve_out_phys_mem() to handle carving out memory that may only
overlap partially with the physical memory.

Add debug prints in core_mmu_set_discovered_nsec_ddr() to list the
non-secure RAM areas.

Fixes: 941dec3a7f6f ("core: adjust nsec ddr memory size correctly")
Fixes: 490c50dfdb33 ("core: assign non-sec DDR configuration from DT")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mm: fix panic with TEE_SDP_TEST_MEM

The commit 2f2f69df5afe ("core: mm: replace MEM_AREA_TA_RAM") uses
MEM_AREA_SEC_RAM_OVERALL to map practically all secure memory. This
conflicts with TEE_SD

core: mm: fix panic with TEE_SDP_TEST_MEM

The commit 2f2f69df5afe ("core: mm: replace MEM_AREA_TA_RAM") uses
MEM_AREA_SEC_RAM_OVERALL to map practically all secure memory. This
conflicts with TEE_SDP_TEST_MEM where MEM_AREA_SEC_RAM_OVERALL covers
TEE_SDP_TEST_MEM and triggers a panic in verify_special_mem_areas().

The commit 1c1f8b65b5c6 ("core: mm: unify secure core and TA memory")
changed to use vaddr_to_phys() to find the physical address for
TEE_SDP_TEST_MEM_BASE. This isn't right since it refers to physical
memory only.

So fix these problems.

Fixes: 2f2f69df5afe ("core: mm: replace MEM_AREA_TA_RAM")
Fixes: 1c1f8b65b5c6 ("core: mm: unify secure core and TA memory")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: fix partially unmapped MEM_AREA_TEE_RAM_RW

The commit 06a258064a92 ("core: mm: allow unmapping VCORE_FREE") allows
unmapping pages from the VCORE_FREE virtual memory range, but no
bookkeeping

core: fix partially unmapped MEM_AREA_TEE_RAM_RW

The commit 06a258064a92 ("core: mm: allow unmapping VCORE_FREE") allows
unmapping pages from the VCORE_FREE virtual memory range, but no
bookkeeping is added apart from what's recorded in the translation
tables. Later, the commit 7c9b85432343 ("core: allow partially unmapped
MEM_AREA_TEE_RAM_RW") does lookups the translation tables using
arch_va2pa_helper() to find out if pages in the VCORE_FREE virtual
memory range are mapped. This works well on arm, but not on riscv which
must traverse the translation tables in software and then is caught in
an infinite recursive loop.

Fix this problem by updating the memory regions in the struct memory_map
(splitting, shrinking, and removing) as needed.

Reported-by: Huang Borong <huangborong@bosc.ac.cn>
Closes: https://github.com/OP-TEE/optee_os/issues/7237
Fixes: 06a258064a92 ("core: mm: allow unmapping VCORE_FREE")
Fixes: 7c9b85432343 ("core: allow partially unmapped MEM_AREA_TEE_RAM_RW")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: mmu: fix memory regions found from ff-a manifest

Fix the 5th parameter of add_phys_mem() in collect_device_mem_ranges()
that has to be the size of memory region and not the end address of the

core: mmu: fix memory regions found from ff-a manifest

Fix the 5th parameter of add_phys_mem() in collect_device_mem_ranges()
that has to be the size of memory region and not the end address of the
region.

Fixes: b8ef8d0b6ff4 ("core: mm: introduce struct memory_map")
Signed-off-by: Sungbae Yoo <sungbaey@nvidia.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: mm: extend temporary dummy memory map

core_init_mmu_map() uses a temporary dummy memory map for the
virt_to_phys() and phys_to_virt() conversions to avoid asserting while
setting up translatio

core: mm: extend temporary dummy memory map

core_init_mmu_map() uses a temporary dummy memory map for the
virt_to_phys() and phys_to_virt() conversions to avoid asserting while
setting up translation tables before the MMU is enabled. CFG_DYN_CONFIG
will need a larger range of memory since translation tables might not be
allocated from .nozi memory only. So for CFG_DYN_CONFIG extend of end of
the unused memory range that the boot_mem_*() functions allocate memory
from.

Introduce CFG_DYN_CONFIG, enabled by default if CFG_BOOT_MEM is enabled
and CFG_WITH_PAGER disabled. CFG_DYN_CONFIG conflicts with
CFG_WITH_PAGER since the pager uses a different mechanism for memory
allocation.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: boot_mem: keep track of padding

When boot_mem_alloc() allocates memory up to alignment - 1 number of
bytes may have be skipped to satisfy the required alignment of the
returned pointer. If the

core: boot_mem: keep track of padding

When boot_mem_alloc() allocates memory up to alignment - 1 number of
bytes may have be skipped to satisfy the required alignment of the
returned pointer. If the skipped bytes, or padding, is large enough,
it's recorded in a list of padding. The list of paddings can be
processed and consumed with boot_mem_foreach_padding(). This allows
sufficiently large paddings to be added to for instance the heap instead
of being wasted.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: boot_mem: allow NULL pointers while relocating

In boot_mem_relocate() when relocating registered pointers, allow the
pointer to be NULL. NULL pointers are not relocated.

Signed-off-by: Jens W

core: boot_mem: allow NULL pointers while relocating

In boot_mem_relocate() when relocating registered pointers, allow the
pointer to be NULL. NULL pointers are not relocated.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: allow partially unmapped MEM_AREA_TEE_RAM_RW

Add special checks in phys_to_virt_tee_ram() to see that a virtual
address indeed is mapped before return the address if the memory area is
MEM_ARE

core: allow partially unmapped MEM_AREA_TEE_RAM_RW

Add special checks in phys_to_virt_tee_ram() to see that a virtual
address indeed is mapped before return the address if the memory area is
MEM_AREA_TEE_RAM_RW since the VCORE_FREE may be unmapped.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...