Merge changes from topic "NUMA_AWARE_PER_CPU" into integration

* changes:
docs(maintainers): add per-cpu framework into maintainers.rst
feat(per-cpu): add documentation for per-cpu framework
f

Merge changes from topic "NUMA_AWARE_PER_CPU" into integration

* changes:
docs(maintainers): add per-cpu framework into maintainers.rst
feat(per-cpu): add documentation for per-cpu framework
feat(rdv3): enable numa aware per-cpu for RD-V3-Cfg2
feat(per-cpu): migrate amu_ctx to per-cpu framework
feat(per-cpu): migrate spm_core_context to per-cpu framework
feat(per-cpu): migrate psci_ns_context to per-cpu framework
feat(per-cpu): migrate psci_cpu_pd_nodes to per-cpu framework
feat(per-cpu): migrate rmm_context to per-cpu framework
feat(per-cpu): integrate per-cpu framework into BL31/BL32
feat(per-cpu): introduce framework accessors/definers
feat(per-cpu): introduce linker changes for NUMA aware per-cpu framework
docs(changelog): add scope for per-cpu framework

show more ...

Merge changes from topic "ar/pre-idte3" into integration

* changes:
feat(el3-runtime): add flags argument to handle_sysreg_trap
fix(cpufeat): remove unused FEAT_NV feature functions

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clear

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clears them
during early C runtime initialization. For platforms that enable
NUMA_AWARE_PER_CPU, invokes a platform hook to zero-initialize
node-specific per-cpu regions.

Cache maintenance: Extends the BL31 exit path to clean dcache lines
covering the per-cpu region, ensuring data written by the primary core
is visible to secondary cores.

tpidr_el3 setup: Initializes tpidr_el3 with the base address of the
current CPU’s per-cpu section. This allows per-cpu framework to
resolve local cpu accesses efficiently.

The percpu_data object is currently stored in tpidr_el3. Since the
per-cpu framework will use tpidr_el3 for this-cpu access, percpu_data
must be migrated to avoid conflict. This commit moves percpu_data to
the per-cpu framework.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: Iff0c2e1f8c0ebd25c4bb0b09bfe15dd4fbe20561

show more ...

feat(el3-runtime): add flags argument to handle_sysreg_trap

Extend handle_sysreg_trap() to accept an additional flags
parameter indicating the caller’s security state.
The EL3 synchronous exception

feat(el3-runtime): add flags argument to handle_sysreg_trap

Extend handle_sysreg_trap() to accept an additional flags
parameter indicating the caller’s security state.
The EL3 synchronous exception handler now passes this value
when dispatching trapped system register accesses.
This allows handle_sysreg_trap() to handle traps based on
the originating security context.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: Ic6e4a13297bf0b3afec3b83e1696c03286615560

show more ...

Merge changes from topic "bk/simpler_panic" into integration

* changes:
fix(aarch64): do not print EL1 registers on EL3 panic
refactor(el3-runtime): streamline cpu_data assembly offsets using th

Merge changes from topic "bk/simpler_panic" into integration

* changes:
fix(aarch64): do not print EL1 registers on EL3 panic
refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

show more ...

refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

The cpu_data structure, just like cpu_ops, is collection of disparate
data that must be accessible from both C

refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

The cpu_data structure, just like cpu_ops, is collection of disparate
data that must be accessible from both C and assembly. Achieving this is
tricky as there is no way to export structure offsets from C directly so
they must be manually recreated with `#define`s and asserts. However,
the cpu_data structure is quite old and the assembly offsets are a
patchwork of additions and extremely difficult to reason with and
modify. In fact, certain currently unused builds with
ENABLE_RUNTIME_INSTRUMENTATION=1 fail to build.

To untangle this, convert the assembly offsets to the pattern used for
the cpu_ops structure. That is, first define the sizes of every member,
as generically as possible, and then chain their offsets one after the
other. To make sure this is always correct, add a CASSERT for the offset
of every member. This makes it easy to modify the structure and fixes
the build failures.

Change-Id: I61aeb55e9c494896663a3c719c10e3c072f56349
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes Id942c20c,Idd286bea,I8917a26e,Iec8c3477,If3c25dcd, ... into integration

* changes:
feat(cpufeat): enable FEAT_PAuth to FEAT_STATE_CHECKED
perf(cpufeat): centralise PAuth key saving

Merge changes Id942c20c,Idd286bea,I8917a26e,Iec8c3477,If3c25dcd, ... into integration

* changes:
feat(cpufeat): enable FEAT_PAuth to FEAT_STATE_CHECKED
perf(cpufeat): centralise PAuth key saving
refactor(cpufeat): convert FEAT_PAuth setup to C
refactor(cpufeat): prepare FEAT_PAuth for FEATURE_DETECTION
chore(cpufeat): remove PAuth presence checks
feat(cpufeat): enable FEAT_BTI to FEAT_STATE_CHECKED

show more ...

perf(cpufeat): centralise PAuth key saving

prepare_el3_entry() is meant to be the one-stop shop for all the context
we must fiddle with to enter EL3 proper. However, PAuth is the one
exception, happ

perf(cpufeat): centralise PAuth key saving

prepare_el3_entry() is meant to be the one-stop shop for all the context
we must fiddle with to enter EL3 proper. However, PAuth is the one
exception, happening right after. Absorb it into prepare_el3_entry(),
handling the BL1/BL31 difference.

This is a good time to also move the key saving into the enable
function, also to centralise. With this it becomes apparent that saving
keys just before CPU_SUSPEND is redundant as they will be reinitialised
when the core wakes up.

Note that the key loading, now in save_gp_pmcr_pauth_regs, does not end
in an isb. The effects of the key change are not needed until the isb
in the caller, so this isb is not needed.

Change-Id: Idd286bea91140c106ab4c933c5c44b0bc2050ca2
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(context-mgmt): introduce EL3/root context" into integration

feat(context-mgmt): introduce EL3/root context

* This patch adds root context procedure to restore/configure
the registers, which are of importance during EL3 execution.

* EL3/Root context is a s

feat(context-mgmt): introduce EL3/root context

* This patch adds root context procedure to restore/configure
the registers, which are of importance during EL3 execution.

* EL3/Root context is a simple restore operation that overwrites
the following bits: (MDCR_EL3.SDD, SCR_EL3.{EA, SIF}, PMCR_EL0.DP
PSTATE.DIT) while the execution is in EL3.

* It ensures EL3 world maintains its own settings distinct
from other worlds (NS/Realm/SWd). With this in place, the EL3
system register settings is no longer influenced by settings of
incoming worlds. This allows the EL3/Root world to access features
for its own execution at EL3 (eg: Pauth).

* It should be invoked at cold and warm boot entry paths and also
at all the possible exception handlers routing to EL3 at runtime.
Cold and warm boot paths are handled by including setup_el3_context
function in "el3_entrypoint_common" macro, which gets invoked in
both the entry paths.

* At runtime, el3_context is setup at the stage, while we get prepared
to enter into EL3 via "prepare_el3_entry" routine.

Change-Id: I5c090978c54a53bc1c119d1bc5fa77cd8813cdc2
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

Merge changes I9eba2e34,Iab2a2a2f into integration

* changes:
refactor(cpus): replace adr with adr_l
refactor(build): introduce adr_l macro

refactor(cpus): replace adr with adr_l

Replace "adr" with "adr_l" to handle symbols or labels that exceeds 1MB
access range. This modification resolves the link error.

Change-Id: I9eba2e34c0a303b40

refactor(cpus): replace adr with adr_l

Replace "adr" with "adr_l" to handle symbols or labels that exceeds 1MB
access range. This modification resolves the link error.

Change-Id: I9eba2e34c0a303b40e4c7b3ea7c5b113f4c6d989
Signed-off-by: Hsin-Hsiung Wang <hsin-hsiung.wang@mediatek.com>

show more ...

Merge changes from topic "mp/undef_injection" into integration

* changes:
feat(el3-runtime): introduce UNDEF injection to lower EL
feat(cpufeat): added few helper functions

feat(el3-runtime): introduce UNDEF injection to lower EL

For a feature to be used at lower ELs, EL3 generally needs to disable
the trap so that lower ELs can access the system registers associated
w

feat(el3-runtime): introduce UNDEF injection to lower EL

For a feature to be used at lower ELs, EL3 generally needs to disable
the trap so that lower ELs can access the system registers associated
with the feature. Lower ELs generally check ID registers to dynamically
detect if a feature is present (in HW) or not while EL3 Firmware relies
statically on feature build macros to enable a feature.

If a lower EL accesses a system register for a feature that EL3 FW is
unaware of, EL3 traps the access and panics. This happens mostly with
EL2 but sometimes VMs can also cause EL3 panic.

To provide platforms with capability to mitigate this problem, UNDEF
injection support has been introduced which injects a synchronous
exception into the lower EL which is supposed to handle the
synchronous exception.

The current support is only provided for aarch64.

The implementation does the following on encountering sys reg trap

- Get the target EL, which can be either EL2 or EL1
- Update ELR_ELx with ELR_EL3, so that after UNDEF handling in lower EL
control returns to original location.
- ESR_ELx with EC_UNKNOWN
- Update ELR_EL3 with vector address of sync exception handler with
following possible causes
- Current EL with SP0
- Current EL with SPx
- Lower EL using AArch64
- Re-create SPSR_EL3 which will be used to generate PSTATE at ERET

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I1b7bf6c043ce7aec1ee4fc1121c389b490b7bfb7

show more ...

Merge changes from topic "mp/exceptions" into integration

* changes:
docs(ras): update RAS documentation
docs(el3-runtime): update BL31 exception vector handling
fix(el3-runtime): restrict low

Merge changes from topic "mp/exceptions" into integration

* changes:
docs(ras): update RAS documentation
docs(el3-runtime): update BL31 exception vector handling
fix(el3-runtime): restrict lower el EA handlers in FFH mode
fix(ras): remove RAS_FFH_SUPPORT and introduce FFH_SUPPORT
fix(ras): restrict ENABLE_FEAT_RAS to have only two states
feat(ras): use FEAT_IESB for error synchronization
feat(el3-runtime): modify vector entry paths

show more ...

fix(el3-runtime): restrict lower el EA handlers in FFH mode

This patch does following changes to restrict handling of lower EL
EA's only if FFH mode is enabled.

- Compile ea_delegate.S only if FFH

fix(el3-runtime): restrict lower el EA handlers in FFH mode

This patch does following changes to restrict handling of lower EL
EA's only if FFH mode is enabled.

- Compile ea_delegate.S only if FFH mode is enabled.
- For Sync exception from lower ELs if the EC is not SMC or SYS reg
trap it was assumed that it is an EA, which is not correct. Move
the known Sync exceptions (EL3 Impdef) out of sync EA handler.
- Report unhandled exceptions if there are SError from lower EL in
KFH mode, as this is unexpected.
- Move code out of ea_delegate.S which are used for KFH mode.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I577089677d0ec8cde7c20952172bee955573d2ed

show more ...

fix(ras): remove RAS_FFH_SUPPORT and introduce FFH_SUPPORT

This patch removes RAS_FFH_SUPPORT macro which is the combination of
ENABLE_FEAT_RAS and HANDLE_EA_EL3_FIRST_NS. Instead introduce an
inter

fix(ras): remove RAS_FFH_SUPPORT and introduce FFH_SUPPORT

This patch removes RAS_FFH_SUPPORT macro which is the combination of
ENABLE_FEAT_RAS and HANDLE_EA_EL3_FIRST_NS. Instead introduce an
internal macro FFH_SUPPORT which gets enabled when platforms wants
to enable lower EL EA handling at EL3. The internal macro FFH_SUPPORT
will be automatically enabled if HANDLE_EA_EL3_FIRST_NS is enabled.
FFH_SUPPORT along with ENABLE_FEAT_RAS will be used in source files
to provide equivalent check which was provided by RAS_FFH_SUPPORT
earlier. In generic code we needed a macro which could abstract both
HANDLE_EA_EL3_FIRST_NS and RAS_FFH_SUPPORT macros that had limitations.
Former was tied up with NS world only while the latter was tied to RAS
feature.

This is to allow Secure/Realm world to have their own FFH macros
in future.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: Ie5692ccbf462f5dcc3f005a5beea5aa35124ac73

show more ...

feat(ras): use FEAT_IESB for error synchronization

For synchronization of errors at exception boundries TF-A uses "esb"
instruction with FEAT_RAS or "dsb" and "isb" otherwise. The problem
with esb i

feat(ras): use FEAT_IESB for error synchronization

For synchronization of errors at exception boundries TF-A uses "esb"
instruction with FEAT_RAS or "dsb" and "isb" otherwise. The problem
with esb instruction is, along with synching errors it might also
consume the error, which is not ideal in all scenarios. On the other
hand we can't use dsb always as its in the hot path.

To solve above mentioned problem the best way is to use FEAT_IESB
feature which provides controls to insert an implicit Error
synchronization event at exception entry and exception return.

Assumption in TF-A is, if RAS Extension is present then FEAT_IESB will
also be present and enabled.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: Ie5861eec5da4028a116406bb4d1fea7dac232456

show more ...

feat(el3-runtime): modify vector entry paths

Vector entries in EL3 from lower ELs, first check for any pending
async EAs from lower EL before handling the original exception.
This happens when there

feat(el3-runtime): modify vector entry paths

Vector entries in EL3 from lower ELs, first check for any pending
async EAs from lower EL before handling the original exception.
This happens when there is an error (EA) in the system which is not
yet signaled to PE while executing at lower EL. During entry into EL3
the errors (EA) are synchronized causing async EA to pend at EL3.

On detecting the pending EA (via ISR_EL1.A) EL3 either reflects it back
to lower EL (KFH) or handles it in EL3 (FFH) based on EA routing model.

In case of Firmware First handling mode (FFH), EL3 handles the pended
EA first before returing back to handle the original exception.

While in case of Kernel First handling mode (KFH), EL3 will return back
to lower EL without handling the original exception. On returing to
lower EL, EA will be pended. In KFH mode there is a risk of back and
forth between EL3 and lower EL if the EA is masked at lower EL or
priority of EA is lower than that of original exception. This is a
limitation in current architecture but can be solved in future if EL3
gets a capability to inject virtual SError.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I3a2a31de7cf454d9d690b1ef769432a5b24f6c11

show more ...

Merge "refactor(cm): move EL3 registers to global context" into integration

refactor(cm): move EL3 registers to global context

Currently, EL3 context registers are duplicated per-world per-cpu.
Some registers have the same value across all CPUs, so this patch
moves these re

refactor(cm): move EL3 registers to global context

Currently, EL3 context registers are duplicated per-world per-cpu.
Some registers have the same value across all CPUs, so this patch
moves these registers out into a per-world context to reduce
memory usage.

Change-Id: I91294e3d5f4af21a58c23599af2bdbd2a747c54a
Signed-off-by: Elizabeth Ho <elizabeth.ho@arm.com>
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

Merge "refactor(el3-runtime): move interrupt exception handler from macro to a function" into integration

refactor(el3-runtime): move interrupt exception handler from macro to a function

interrupt exception handler in vector entry is used as a asm macro
(added as inline code) instead of a function call.

refactor(el3-runtime): move interrupt exception handler from macro to a function

interrupt exception handler in vector entry is used as a asm macro
(added as inline code) instead of a function call. Since we have limited
space (0x80) for a vector entry there is a chance that it may overflow
in the future.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: Ieb59f249c58b52e56e0217268fa4dc40b420f8d3

show more ...

Merge "fix: pass SMCCCv1.3 SVE hint to internal flags" into integration

fix: pass SMCCCv1.3 SVE hint to internal flags

This change fixes the initial support for SMCCCv1.3 SVE hint bit [1].
In the aarch64 smc handler, the FID[16] bit is improperly extracted
and results i

fix: pass SMCCCv1.3 SVE hint to internal flags

This change fixes the initial support for SMCCCv1.3 SVE hint bit [1].
In the aarch64 smc handler, the FID[16] bit is improperly extracted
and results in the corresponding flags bit to be always set.
Fix by doing the proper masking and set into the flags register.

[1] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/17511

Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Change-Id: I62b8e211d48a50f28e184ff27cd718f51d8d56bf

show more ...