Merge changes I2485d583,I1374c482,I07e29dbb,I949e6486 into integration

* changes:
feat(qemu): enable ENABLE_FEAT_RAS and ENABLE_FEAT_SB
feat(cpufeat): update FEAT_SB's FEAT_STATE_CHECKED status

Merge changes I2485d583,I1374c482,I07e29dbb,I949e6486 into integration

* changes:
feat(qemu): enable ENABLE_FEAT_RAS and ENABLE_FEAT_SB
feat(cpufeat): update FEAT_SB's FEAT_STATE_CHECKED status
feat(cpufeat): advertise support for FEAT_RASv2
feat(cpufeat): enable FEAT_RAS for FEAT_STATE_CHECKED again

show more ...

feat(cpufeat): enable FEAT_RAS for FEAT_STATE_CHECKED again

FEAT_RAS was originally converted to FEAT_STATE_CHECKED in 6503ff291.
However, the ability to use it was removed with 970a4a8d8 by simply

feat(cpufeat): enable FEAT_RAS for FEAT_STATE_CHECKED again

FEAT_RAS was originally converted to FEAT_STATE_CHECKED in 6503ff291.
However, the ability to use it was removed with 970a4a8d8 by simply
saying it impacts execution at EL3. That's true, but FEAT_STATE_CHECKED
can still be allowed by being a bit clever about it.

First, the remainder of common code can be converted to use the
is_feat_ras_supported() helper instead of the `#if FEATURE` pattern.
There are no corner cases to consider there. The feature is either
present (and appropriate action must be taken) or the feature is not (so
we can skip RAS code).

A conscious choice is taken to check the RAS code in synchronize_errors
despite it being in a hot path. Any fixed platform that seeks to be
performant should be setting features to 0 or 1. Then, the
SCTLR_EL3.IESB bit is always set if ENABLE_FEAT_RAS != 0 since we expect
FEAT_IESB to be present if FEAT_RAS is (despite the architecture not
guaranteeing it). If FEAT_RAS isn't present then we don't particularly
care about the status of FEAT_IESB.

Second, platforms that don't set ENABLE_FEAT_RAS must continue to work.
This is true out of the box with the is_feat_xyz_supported() helpers, as
they make sure to fully disable code within them.

Third, platforms that do set ENABLE_FEAT_RAS=1 must continue to work.
This is also true out of the box and no logical change is undertaken in
common code.

Finally, ENABLE_FEAT_RAS is set to 2 on FVP. Having RAS implies that the
whole handling machinery will be built-in and registered as appropriate.
However, when RAS is built-in but not present in hardware, these
registrations can still happen, they will only never be invoked at
runtime.

Change-Id: I949e648601dc0951ef9c2b217f34136b6ea4b3dc
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cpufeat): enable access to extended BRPs/WRPs

Access to Extended Breakpoints(BRPs) and Watchpoints(WRPs) are enabled
through EBWE bit and this available from DebugV8P9. So enable access to
mode

fix(cpufeat): enable access to extended BRPs/WRPs

Access to Extended Breakpoints(BRPs) and Watchpoints(WRPs) are enabled
through EBWE bit and this available from DebugV8P9. So enable access to
mode select register default from lower EL's.

Though this bit RES0 when we have less than 16 BRPs/WRPs the Mode select
register is also RAZ/WI. So having EBWE write by default is harmless.
And will avoid trap to EL3 when enable access to bank selection when we
have more than 16 BRPs/WRPs.

Change-Id: Ib308be758c0beedde05a5558b0d24a161b79273a
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to their low
resolution and general availability of the data elsewhere. As such, they
are used for critical performance tuning and are expected to never be
turned off or context switched when switching worlds.

The upper 2 counters, INST_RETIRED and STALL_BACKEND_MEM, are different.
The data they provide is non-critical and expose new information that
could be used as a timing side channel, especially of Secure world. This
patch adds context switching of these two counters to prevent any such
side channel.

This is not done for group 1 auxiliary counters as those are IMP DEF and
are inaccessible by default unless overriden by the platform (with
AMU_RESTRICT_COUNTERS).

Change-Id: Ib4b946abb810e36736cabb9b84cd837308b4e761
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cpufeat): disable FEAT_AMU counters on context restore

The FEAT_AMU counters observe UNPREDICTABLE behaviour if written to
while counting so they must be disabled first. Further, the save happen

fix(cpufeat): disable FEAT_AMU counters on context restore

The FEAT_AMU counters observe UNPREDICTABLE behaviour if written to
while counting so they must be disabled first. Further, the save happens
on the PE's powerdown path and the restore happens on the wakeup path so
any disable will likely get lost on wakeup.

So add a disable from to the restore path. The restore path will usually
have the AMU reset and as such all counters disabled. There is a chance
though that the AMU might not have reset with the PE (which is IMPDEF)
or a pabandon might have happened so also add a check to skip disabling
the counters if they already are.

Even though reading AMU counters while they are enabled is perfectly
permissible, keep the disable so that the snapshot of saved values is
coherent. Otherwise, over many saves and restores, the values of the
later read counters could get out of sync with the ones read earlier.

Change-Id: Iefe6de44f09d8659a6118d5fea40abf82c44be16
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(per-cpu): migrate AArch32 amu_ctx to per-cpu framework

Brings it in line with AArch64.

Change-Id: I9333ea9cf07679735da169dae0fe90a8856d9801
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.

feat(per-cpu): migrate AArch32 amu_ctx to per-cpu framework

Brings it in line with AArch64.

Change-Id: I9333ea9cf07679735da169dae0fe90a8856d9801
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_UINJ

FEAT_UINJ allows higher ELs to inject Undefined Instruction exceptions
into lower ELs by setting SPSR_ELx.UINJ, which updates PSTATE.UINJ on
exception return

feat(cpufeat): add support for FEAT_UINJ

FEAT_UINJ allows higher ELs to inject Undefined Instruction exceptions
into lower ELs by setting SPSR_ELx.UINJ, which updates PSTATE.UINJ on
exception return. When PSTATE.UINJ is set, instruction execution at the
lower EL raises an Undefined Instruction exception (EC=0b000000).

This patch introduces support for FEAT_UINJ by updating the
inject_undef64() to use hardware undef injection if supported.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I48ad56a58eaab7859d508cfa8dfe81130b873b6b

show more ...

fix(cpufeat): require FEAT_AMUv1p1 to enable the auxiliary counters

The auxiliary counters are a feature of FEAT_AMUv1p1 but it's possible
to enable them (ENABLE_AMU_AUXILIARY_COUNTERS=1) without en

fix(cpufeat): require FEAT_AMUv1p1 to enable the auxiliary counters

The auxiliary counters are a feature of FEAT_AMUv1p1 but it's possible
to enable them (ENABLE_AMU_AUXILIARY_COUNTERS=1) without enabling
FEAT_AMUv1p1. As a result, the AMU_RESTRICT_COUNTERS may not take
effect, making this configuration potentially insecure.

Fix this by adding a constraints and rejigging auxiliary counter enables
such that they only happen when FEAT_AMUv1p1 has been enabled so that's
more apparent.

Change-Id: I5b5061d603013598f07d70401d68915c016a1a1b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless t

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless trapped to EL2). IDTE3 allows EL3 to
modify the view of ID registers for lower ELs, and this capability
is used to disable fields of ID registers tied to disabled features.

The ID registers are initially read as-is and stored in context.
Then, based on the feature enablement status for each world, if a
particular feature is disabled, its corresponding field in the
cached ID register is set to Res0. When lower ELs attempt to read
an ID register, the cached ID register value is returned. This
allows EL3 to prevent lower ELs from accessing feature-specific
system registers that are disabled in EL3, even though the hardware
implements them.

The emulated ID register values are stored primarily in per-world
context, except for certain debug-related ID registers such as
ID_AA64DFR0_EL1 and ID_AA64DFR1_EL1, which are stored in the
cpu_data and are unique to each PE. This is done to support feature
asymmetry that is commonly seen in debug features.

FEAT_IDTE3 traps all Group 3 ID registers in the range
op0 == 3, op1 == 0, CRn == 0, CRm == {2–7}, op2 == {0–7} and the
Group 5 GMID_EL1 register. However, only a handful of ID registers
contain fields used to detect features enabled in EL3. Hence, we
only cache those ID registers, while the rest are transparently
returned as is to the lower EL.

This patch updates the CREATE_FEATURE_FUNCS macro to generate
update_feat_xyz_idreg_field() functions that disable ID register
fields on a per-feature basis. The enabled_worlds scope is used to
disable ID register fields for security states where the feature is
not enabled.

This EXPERIMENTAL feature is controlled by the ENABLE_FEAT_IDTE3
build flag and is currently disabled by default.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I5f998eeab81bb48c7595addc5595313a9ebb96d5

show more ...

feat(per-cpu): migrate amu_ctx to per-cpu framework

migrate amu_ctx object to the NUMA-aware per-cpu framework
to optimize memory access and to efficiently utilize memory.

Signed-off-by: Sammit Jos

feat(per-cpu): migrate amu_ctx to per-cpu framework

migrate amu_ctx object to the NUMA-aware per-cpu framework
to optimize memory access and to efficiently utilize memory.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: I55e5e923ddd7307039e0cd9a35e91d908f910d10

show more ...

fix(cpufeat): extend FEAT_EBEP handling to delegate PMU control to EL2

Currently, the FEAT_EBEP feature presence check is only used for UNDEF
injection into lower ELs. However, this feature also aff

fix(cpufeat): extend FEAT_EBEP handling to delegate PMU control to EL2

Currently, the FEAT_EBEP feature presence check is only used for UNDEF
injection into lower ELs. However, this feature also affects the access
behavior of MDCR_EL2. Specifically, if the PMEE bits in MDCR_EL3 are not
set to 0b01, then the MDCR_EL2.PMEE bits cannot be configured by EL2.

This patch extends the use of FEAT_EBEP to delegate PMU IRQ and
profiling exception control to EL2 by setting MDCR_EL3.PMEE = 0b01.This
ensures that lower ELs can manage PMU configuration.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: Ib7e1d5c72f017b8ffc2131fc57309dd9d811c973

show more ...

fix(cpufeat): use of additional breakpoints

Extended Breakpoints access through mdcr_el3.ebwe is available
only when there are greater than 16 breakpoints implemented.
Otherwise the EBWE Bit is RES0

fix(cpufeat): use of additional breakpoints

Extended Breakpoints access through mdcr_el3.ebwe is available
only when there are greater than 16 breakpoints implemented.
Otherwise the EBWE Bit is RES0 and we could skip enabling Extended
Breakpoint access.

Ref: https://developer.arm.com/documentation/111107/2025-09/AArch64-Registers/MDCR-EL3--Monitor-Debug-Configuration-Register--EL3-?lang=en

Change-Id: I2b2147e83d65ee9b0492d3cf3adafd5c8cbe17f5
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never writ

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never written to and so zeroes end up in registers, which is not
necessarily correct.

This patch gets BL1 to call cm_manage_extensions_per_world() whenever
BL2 runs in a lower EL. This allows the per-world registers to have the
reset values we intend. An accompanying call to
cm_manage_extensions_el3() is also added for completeness.

Doing this shows a small deficiency in cptr_el3 - bits TFP and TCPAC
change a lot. This patch makes them consistent by always setting TCPAC
and TFP to 0 which unconditionally enable access to CPTR_EL2 and FPCR by
default as they are always accessible. Other places that manipulate the
TFP bit are removed.

A nice side effect of all of this is that we're now in a position to
enable and use any architectural extension in BL2.

Change-Id: I070d62bbf8e9d9b472caf7e2c931c303523be308
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cpufeat): don't overwrite PAuth keys with an erroneous cache clean

Accessing cpu_data when TF-A is built with HW_ASSISTED_COHERENCY=1 is
simple. Caching (SCTLR_EL3.C) is enabled along with the M

fix(cpufeat): don't overwrite PAuth keys with an erroneous cache clean

Accessing cpu_data when TF-A is built with HW_ASSISTED_COHERENCY=1 is
simple. Caching (SCTLR_EL3.C) is enabled along with the MMU and we can
rely on all accesses being coherent. However, this is not the case when
HW_ASSISTED_COHERENCY=0. Most of EL3's initialisation (especially on
warm boot) happens with the MMU on but with caching being off. Caches
are only enabled deep into CPU_ON processing when we can be certain the
core has entered coherency. This latter case is the subject of this
patch.

Prior to this patch, the way to work around that was to clean the
apiakey cpu_data storage right after writing it. The write would have
gone straight to memory as caches were off and the clean asserted that
nothing would be in the caches which were assumed to be invalid since
we've just came out of reset.

The problem with this is that we cannot assume that ALL caches are
invalid when coming out of reset. We can reasonably assume those private
to the core to be (so the L1 and/or the L2; those are guaranteed to be
invalidated out of reset for every Arm core) but that is not the case
for shared caches (eg an L2/L3 DSU cache) which can be on when a core
powers down. So the old keys could still be live in the shared cache, we
write new ones to memory and clean the old to memory too, undoing the
work.

So the correct thing to do is to clean and invalidate the cache prior to
writing the keys to memory and invalidate it after. This ensures that if
there is any other data after the apiakey, which shares the cache line,
it will be safely forwarded to memory and the caches will be invalid
when caching is turned on.

It is important to note at this point that this was never observed in
practice - every known configuration that uses PAuth has the apiakey as
the very last member of the cpu_data struct which is padded up to a
cache line and the usage of the apiakey is such that it was never
allocated into the shared caches. So the clean would effectively perform
an invalidate of only the apiakey and all worked well. This was only
spotted with a proposed patch that added data after the apiakey
(https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/39698/7).

Change-Id: I8493221dff53114c5c56dd73fbfd2a3301e2542c
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): enable FEAT_CPA2 for EL3

FEAT_CPA2 enables checked pointer arithmetic, which in the event
of an arithmetic overflow in pointer generation will result in a
non-canonical pointer being

feat(cpufeat): enable FEAT_CPA2 for EL3

FEAT_CPA2 enables checked pointer arithmetic, which in the event
of an arithmetic overflow in pointer generation will result in a
non-canonical pointer being generated and subsequent address fault.

Note that FEAT_CPA is a trivial implementation that exists in
some hardware purely so it can run CPA2-enabled instructions
without crashing but they don't actually have checked arithmetic,
so FEAT_CPA is not explicitly enabled in TF-A.

Change-Id: I6d2ca7a7e4b986bb9e917aa8baf8091a271c168b
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

Merge "fix(cpufeat): configure CPTR_EL2.ZEN and CPTR_EL2.TZ to match Linux" into integration

fix(cpufeat): configure CPTR_EL2.ZEN and CPTR_EL2.TZ to match Linux

Linux Documentation/arch/arm64/booting.rst states that:
"
For CPUs with the Scalable Vector Extension (FEAT_SVE) present:
...
-

fix(cpufeat): configure CPTR_EL2.ZEN and CPTR_EL2.TZ to match Linux

Linux Documentation/arch/arm64/booting.rst states that:
"
For CPUs with the Scalable Vector Extension (FEAT_SVE) present:
...
- If the kernel is entered at EL1 and EL2 is present:
- CPTR_EL2.TZ (bit 8) must be initialised to 0b0.
- CPTR_EL2.ZEN (bits 17:16) must be initialised to 0b11.
"
Without these settings, Linux kernel hangs on boot when trying
to use SVE. Adjust the register settings to match Linux kernel
expectations.

Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Change-Id: I9a72810dd902b08f9c61f157cc31e603aad2f73a

show more ...

refactor: unify blx_setup() and blx_main()

All BLs have a bl_setup() for things that need to happen early, a fall
back into assembly and then bl_main() for the main functionality. This
was necessary

refactor: unify blx_setup() and blx_main()

All BLs have a bl_setup() for things that need to happen early, a fall
back into assembly and then bl_main() for the main functionality. This
was necessary in order to fiddle with PAuth related things that tend to
break C calls. Since then PAuth's enablement has seen a lot of
refactoring and this is now worked around cleanly so the distinction can
be removed. The only tradeoff is that this requires pauth to not be used
for the top-level main function.

There are two main benefits to doing this: First, code is easier to
understand as it's all together and the entrypoint is smaller. Second,
the compiler gets to see more of the code and apply optimisations
(importantly LTO).

Change-Id: Iddb93551115a2048988017547eb7b8db441dbd37
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to EL3.

When INIT_UNUSED_NS_EL2=1, configure MPAMBW2_EL2 in EL3 so
that MPAM_PE_BW_CTRL accesses from EL0/EL1 do not trap to EL2.

At this stage, PE-side MPAM bandwidth controls remain disabled
in EL3.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I8e359b0eb912cff3bdda109b21727a627cac3a7e

show more ...

fix(lib): modify function to have single return

This corrects the MISRA violation C2012-15.5:
function should have a single point of exit at the end.
Introduced a temporary variable to store the ret

fix(lib): modify function to have single return

This corrects the MISRA violation C2012-15.5:
function should have a single point of exit at the end.
Introduced a temporary variable to store the return value to
ensure single return for the function

Change-Id: Ib9941afe23f5988e5c569171563169f8f10f2b94
Signed-off-by: Saivardhan Thatikonda <saivardhan.thatikonda@amd.com>

show more ...

fix(cm): disable SPE/TRBE correctly

SPE and TRBE are unusual features. They have multi-bit enables whose
function is not immediately apparent and disabling them is not
straightforward.

While attemp

fix(cm): disable SPE/TRBE correctly

SPE and TRBE are unusual features. They have multi-bit enables whose
function is not immediately apparent and disabling them is not
straightforward.

While attempting to figure this out, the disables were made a mess of.
Patch fc7dca72b began changing the owning security state of SPE and
TRBE. This was first used in patch 79c0c7fac0 with calls to
spe_disable() and trbe_disbale(). However, patch 13f4a2525 reverted the
security state ownership, making the spe_disable() and trbe_disable()
redundant and their comments incorrect - the DoS protection is achieved
by the psb/tsb barriers on context switch, introduces separately in
f80887337 and 73d98e375.

Those patches got the behaviour full circle to what it was in fc7dca72b
so the disables can be fully removed for clarity.

However, the original method for disabling these features is not fully
correct - letting the "disabled" state be all zeroes made the features
seem enabled for secure world but they would trap. That is not a problem
while secure world doesn't use them, but could lead to some confusing
debugging in the future. NS and Realm worlds were not affected. This
patch fully establishes the pattern for SPE and TRBE's enablement,
documents it, and implements it such.

The description comments in the features boil down to 2 rules. There is
a third rule possible:
3. To enable TRBE/SPE for world X with a dirty buffer:
* world X owns the buffer
* trapping enabled
This is not listed as it would not work correctly with
SMCCC_ARCH_FEATURE_AVAILABILITY which relies on trapping to be disabled
to report correctly. If that is ever implemented, the SMCCC
implementation should be considered too.

Change-Id: I5588a3d5fc074c2445470954d8c3b172bec77d43
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(spe): add support for FEAT_SPE_FDS" into integration

Merge changes from topic "fix_pmuv3p9_test" into integration

* changes:
fix(smccc): properly set RAS feature bit
fix(trng): allow FEAT_RNG_TRAP in dynamic fashion
feat(smccc): add FEAT_TWED to

Merge changes from topic "fix_pmuv3p9_test" into integration

* changes:
fix(smccc): properly set RAS feature bit
fix(trng): allow FEAT_RNG_TRAP in dynamic fashion
feat(smccc): add FEAT_TWED to ARCH_FEATURE_AVAILABILITY
feat(cpufeat): add support for PMUv3p9

show more ...

feat(spe): add support for FEAT_SPE_FDS

Allow access to PMSDSFR_EL1 register at NS-EL1 or NS-EL2 when
FEAT_SPE_FDS is implemented.

Change-Id: I538577cbfa5b5f242d5dbaeeace7b8e4ee6ffd03
Signed-off-by

feat(spe): add support for FEAT_SPE_FDS

Allow access to PMSDSFR_EL1 register at NS-EL1 or NS-EL2 when
FEAT_SPE_FDS is implemented.

Change-Id: I538577cbfa5b5f242d5dbaeeace7b8e4ee6ffd03
Signed-off-by: James Clark <james.clark2@arm.com>

show more ...

feat(cpufeat): add support for FEAT_PAUTH_LR

This patch enables FEAT_PAUTH_LR at EL3 on systems that support it when
the new ENABLE_FEAT_PAUTH_LR flag is set.

Currently, PAUTH_LR is only supported

feat(cpufeat): add support for FEAT_PAUTH_LR

This patch enables FEAT_PAUTH_LR at EL3 on systems that support it when
the new ENABLE_FEAT_PAUTH_LR flag is set.

Currently, PAUTH_LR is only supported by arm clang compiler and not GCC.

Change-Id: I7db1e34b661ed95cad75850b62878ac5d98466ea
Signed-off-by: John Powell <john.powell@arm.com>

show more ...