Merge "fix(cpufeat): configure CPTR_EL2.ZEN and CPTR_EL2.TZ to match Linux" into integration

fix(cpufeat): configure CPTR_EL2.ZEN and CPTR_EL2.TZ to match Linux

Linux Documentation/arch/arm64/booting.rst states that:
"
For CPUs with the Scalable Vector Extension (FEAT_SVE) present:
...
-

fix(cpufeat): configure CPTR_EL2.ZEN and CPTR_EL2.TZ to match Linux

Linux Documentation/arch/arm64/booting.rst states that:
"
For CPUs with the Scalable Vector Extension (FEAT_SVE) present:
...
- If the kernel is entered at EL1 and EL2 is present:
- CPTR_EL2.TZ (bit 8) must be initialised to 0b0.
- CPTR_EL2.ZEN (bits 17:16) must be initialised to 0b11.
"
Without these settings, Linux kernel hangs on boot when trying
to use SVE. Adjust the register settings to match Linux kernel
expectations.

Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Change-Id: I9a72810dd902b08f9c61f157cc31e603aad2f73a

show more ...

refactor: unify blx_setup() and blx_main()

All BLs have a bl_setup() for things that need to happen early, a fall
back into assembly and then bl_main() for the main functionality. This
was necessary

refactor: unify blx_setup() and blx_main()

All BLs have a bl_setup() for things that need to happen early, a fall
back into assembly and then bl_main() for the main functionality. This
was necessary in order to fiddle with PAuth related things that tend to
break C calls. Since then PAuth's enablement has seen a lot of
refactoring and this is now worked around cleanly so the distinction can
be removed. The only tradeoff is that this requires pauth to not be used
for the top-level main function.

There are two main benefits to doing this: First, code is easier to
understand as it's all together and the entrypoint is smaller. Second,
the compiler gets to see more of the code and apply optimisations
(importantly LTO).

Change-Id: Iddb93551115a2048988017547eb7b8db441dbd37
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(bl2): unify the BL2 EL3 and RME entrypoints

BL2 has 3(!) entrypoints:
1) the regular EL1 entrypoint (once per AArch)
2) an EL3 entrypoint
3) an EL3 entrypoint with RME

The EL1 and EL3 entryp

fix(bl2): unify the BL2 EL3 and RME entrypoints

BL2 has 3(!) entrypoints:
1) the regular EL1 entrypoint (once per AArch)
2) an EL3 entrypoint
3) an EL3 entrypoint with RME

The EL1 and EL3 entrypoints are quite distinct so it's useful to keep
them separate. But the EL3 and RME entrypoints are conceptually
identical just configured differently and having slightly different
assumptions (eg whether we can rely on BL1). So put them together with
only the configuration as a difference. This has a few benefits:
* makes the naming consistent - BL2 always runs at EL1, BL2_EL3 always
runs at EL3. This is most important for the linker script.
* paves the way for ENABLE_RME and RESET_TO_BL2 to coexist.
* allows for more general refactors

Currently, ENABLE_RME and RESET_TO_BL2 are mutually exclusive (from a
makefile constraint) so the checks are simplified to one or the other as
there is no danger of their simultaneous use.

Change-Id: Iecffab2ff3a0bd7823f8277d9f66e22e4f42cc8c
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to EL3.

When INIT_UNUSED_NS_EL2=1, configure MPAMBW2_EL2 in EL3 so
that MPAM_PE_BW_CTRL accesses from EL0/EL1 do not trap to EL2.

At this stage, PE-side MPAM bandwidth controls remain disabled
in EL3.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I8e359b0eb912cff3bdda109b21727a627cac3a7e

show more ...

fix(lib): use 64-bit constants in MDCR_EL2 bit macros

Updated bitmask definitions in MDCR_EL2 macros to use ULL(1) instead
of U(1), ensuring the left operand in shift operations is of an
appropriate

fix(lib): use 64-bit constants in MDCR_EL2 bit macros

Updated bitmask definitions in MDCR_EL2 macros to use ULL(1) instead
of U(1), ensuring the left operand in shift operations is of an
appropriate width. This avoids MISRA C:2012 Rule 12.2 violations,
which prohibit shifting a value by more than 7 bits when the left-
hand operand is an unsigned int.

Change-Id: I279fb33c2ef714367953d53e61456490cd7c798b
Signed-off-by: Saivardhan Thatikonda <saivardhan.thatikonda@amd.com>

show more ...

fix(cm): disable SPE/TRBE correctly

SPE and TRBE are unusual features. They have multi-bit enables whose
function is not immediately apparent and disabling them is not
straightforward.

While attemp

fix(cm): disable SPE/TRBE correctly

SPE and TRBE are unusual features. They have multi-bit enables whose
function is not immediately apparent and disabling them is not
straightforward.

While attempting to figure this out, the disables were made a mess of.
Patch fc7dca72b began changing the owning security state of SPE and
TRBE. This was first used in patch 79c0c7fac0 with calls to
spe_disable() and trbe_disbale(). However, patch 13f4a2525 reverted the
security state ownership, making the spe_disable() and trbe_disable()
redundant and their comments incorrect - the DoS protection is achieved
by the psb/tsb barriers on context switch, introduces separately in
f80887337 and 73d98e375.

Those patches got the behaviour full circle to what it was in fc7dca72b
so the disables can be fully removed for clarity.

However, the original method for disabling these features is not fully
correct - letting the "disabled" state be all zeroes made the features
seem enabled for secure world but they would trap. That is not a problem
while secure world doesn't use them, but could lead to some confusing
debugging in the future. NS and Realm worlds were not affected. This
patch fully establishes the pattern for SPE and TRBE's enablement,
documents it, and implements it such.

The description comments in the features boil down to 2 rules. There is
a third rule possible:
3. To enable TRBE/SPE for world X with a dirty buffer:
* world X owns the buffer
* trapping enabled
This is not listed as it would not work correctly with
SMCCC_ARCH_FEATURE_AVAILABILITY which relies on trapping to be disabled
to report correctly. If that is ever implemented, the SMCCC
implementation should be considered too.

Change-Id: I5588a3d5fc074c2445470954d8c3b172bec77d43
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cm): unify RMM context

setup_realm_context() is the de facto place to put any code that relates
to the RMM's context. It is frequently updated and contains the vast
majority of code. manage

refactor(cm): unify RMM context

setup_realm_context() is the de facto place to put any code that relates
to the RMM's context. It is frequently updated and contains the vast
majority of code. manage_extensions_realm() on the other hand is out of
date and obscure.

So absorb manage_extensions_realm() and rmm_el2_context_init() into
setup_realm_context().

We can also combine the write to sctlr_el2 for all worlds as they should
all observe the RES1 values.

Finally, the SPSR_EL2.PAN comment in the realm copy is updated.

Change-Id: I21dccad0c13301e3249db6f6e292beb5d853563e
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_FGWTE3

Enable write traps for key EL3 system registers as per FEAT_FGWTE3,
ensuring their values remain unchanged after boot.

Excluded Registers:
MDCR_EL3 and MP

feat(cpufeat): add support for FEAT_FGWTE3

Enable write traps for key EL3 system registers as per FEAT_FGWTE3,
ensuring their values remain unchanged after boot.

Excluded Registers:
MDCR_EL3 and MPAM3_EL3: Not trapped as they are part of the EL3 context.
SCTLR_EL3: Not trapped since it is overwritten during
powerdown sequence(Included when HW_ASSISTED_COHERENCY=1)

TPIDR_EL3: Excluded due to its use in crash reporting(It is included
when CRASH_REPORTING=0)

Reference:
https://developer.arm.com/documentation/ddi0601/2025-06/AArch64-Registers/FGWTE3-EL3--Fine-Grained-Write-Traps-EL3

Change-Id: Idcb32aaac7d65a0b0e5c90571af00e01a4e9edb1
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>

show more ...

feat(dsu): support power control and autonomous powerdown config

This patch allows platforms to enable certain DSU settings
to ensure memory retention and control over
cache power requests. We also

feat(dsu): support power control and autonomous powerdown config

This patch allows platforms to enable certain DSU settings
to ensure memory retention and control over
cache power requests. We also move the driver out of css
into drivers/arm. Platforms can configure the
CLUSTERPWRCTLR and CLUSTERPWRDN registers [1] to improve
power efficiency.

These registers enable finer-grained control of
DSU power state transitions, including
powerdown and retention.

IMP_CLUSTERPWRCTLR_EL1 provides:
- Functional retention: Allows configuration of the
duration of inactivity before the DSU uses
CLUSTERPACTIVE to request functional retention.

- Cache power request: These bits are output on
CLUSTERPACTIVE[19:16] to indicate to the power controller
which cache portions must remain powered.

IMP_CLUSTERPWRDN_EL1 includes:
- Powerdown: Triggers full cluster powerdown, including
control logic.

- Memory retention: Requests memory retention mode,
keeping L3 RAM contents while powering off
the rest of the DSU.

The DSU-120 TRM [2] provides the full field definitions,
which are used as references in the `dsu_driver_data` structure.

References:
[1]: https://developer.arm.com/documentation/100453/latest/
[2]: https://developer.arm.com/documentation/102547/0201/?lang=en

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I2eba808b8f2a27797782a333c65dd092b03208fe

show more ...

Merge "perf(bl31): convert cpu_data fetching to C" into integration

perf(bl31): convert cpu_data fetching to C

The assembly routines are opaque to the compiler and it can't inline
them. There is also no requirement for them to be called without a
stack - each of the

perf(bl31): convert cpu_data fetching to C

The assembly routines are opaque to the compiler and it can't inline
them. There is also no requirement for them to be called without a
stack - each of their calls has a stack available. So convert them to C
so that the compiler can do its inlining magic.

On AArch32 we need to be able to call _cpu_data from the entrypoint so
it has to stay as a slight exception.

We can also straighten out the type of the cpu_ops_ptr member so we
don't have to cast it everywhere.

Change-Id: I9c2939a955b396edf26b99ef36318eebeaab13e6
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(gicv5): assign interrupt sources to appropriate security states

Assign the PPI interrupts we commonly have in the device tree to the NS
domain. This is a short-term solution that allows Linux t

feat(gicv5): assign interrupt sources to appropriate security states

Assign the PPI interrupts we commonly have in the device tree to the NS
domain. This is a short-term solution that allows Linux to fully boot.
This is expected to be fully replaced with context management when
adding world switching support as some of these are expected to be
shared between worlds.

Change-Id: I59a7b5a63f878c9a717ef81e977be7133a402f3f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(gicv5): add a barebones GICv5 driver

This is the absolute minimum that's needed to compile an NS-only build
end exit out of EL3. The GIC is not used and/or configured in any way
but all the nec

feat(gicv5): add a barebones GICv5 driver

This is the absolute minimum that's needed to compile an NS-only build
end exit out of EL3. The GIC is not used and/or configured in any way
but all the necessary hooks are populated.

Notably, SCR_EL3.FIQ becomes RES1 as GICv5 behaves in a similar manner
to a GICv3 with FIQ set.

Change-Id: Idae52b9df97f4ca2996b2dcd1e5efc45478a43f2
Co-developed-by: Achin Gupta <achin.gupta@arm.com>
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(gicv5): add support for building with gicv5

The Generic Interrupt Controller v5 (GICv5) is the next generation of
Arm interrupt controllers. It is a clean slate design and has native
support fo

feat(gicv5): add support for building with gicv5

The Generic Interrupt Controller v5 (GICv5) is the next generation of
Arm interrupt controllers. It is a clean slate design and has native
support for the latest Armv9 features. As such it is entirely backwards
incompatible with GICv3/v4.

This patch adds the necessary boilerplate to select a build with GICv5.
The GIC has always had two parts. BL31 deals directly with the CPU
interface while platform code is responsible for managing the IRI. In v5
this split is formalised and the CPU interface, FEAT_GCIE, may be
implemented on its own. So reflect this split in our code with
ENABLE_FEAT_GCIE which only affects BL31 and the GICv5 IRI lies in the
generic GIC driver.

No actual functionality yet.

Change-Id: I97a0c3ba708877c213e50e7ef148e3412aa2af90
Co-developed-by: Achin Gupta <achin.gupta@arm.com>
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(spe): add support for FEAT_SPE_FDS" into integration

Merge changes from topic "fix_pmuv3p9_test" into integration

* changes:
fix(smccc): properly set RAS feature bit
fix(trng): allow FEAT_RNG_TRAP in dynamic fashion
feat(smccc): add FEAT_TWED to

Merge changes from topic "fix_pmuv3p9_test" into integration

* changes:
fix(smccc): properly set RAS feature bit
fix(trng): allow FEAT_RNG_TRAP in dynamic fashion
feat(smccc): add FEAT_TWED to ARCH_FEATURE_AVAILABILITY
feat(cpufeat): add support for PMUv3p9

show more ...

Merge "feat(cpufeat): add support for FEAT_PAUTH_LR" into integration

fix(cpufeat): replace "bti" mnemonic with hint instructions

Older GNU binutils version require to specify at least "armv8.5-a" for
the ARM architecture revision to accept "bti" instructions in the
a

fix(cpufeat): replace "bti" mnemonic with hint instructions

Older GNU binutils version require to specify at least "armv8.5-a" for
the ARM architecture revision to accept "bti" instructions in the
assembly code. Binutils v2.35 have relaxed this, since "bti" is in the
hint space, so is ignored on older cores and does NOT require a BTI
enabled core to execute.

To not exclude those older binutils versions (as shipped with Ubuntu
20.04), use the "hint" encoding for the "bti" instructions, which are
accepted regardless of the minimum architecture revision. Hide this
encoding in a macro, to make the "bti" usage more readable in the
source code.

Change-Id: I005586efd8974a3f2c7202896c881bb5fed07eea
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

fix(cpufeat): improve xpaci wrapper

Older toolchains seem to have trouble parsing a directive on the same
line as an instruction, so they choke on the separating semicolon:

/tmp/ccBwXW8I.s:93: Erro

fix(cpufeat): improve xpaci wrapper

Older toolchains seem to have trouble parsing a directive on the same
line as an instruction, so they choke on the separating semicolon:

/tmp/ccBwXW8I.s:93: Error: unknown architecture `armv8.3-a;'

Put the .arch directive on a line of its own, so that the assembler can
digest it.

Drop the comment on the way: xpaci is a FEAT_PAUTH instruction not
encoded in the hint space, so *every* toolchain needs to be allowed
ARMv8.3 level instructions, not just older ones.

Also we do not need to force x0 as the register, the compiler will pick
it automatically, so drop the register variable.

Change-Id: I0d4d74bcdac04aa86c565ad0455f62adf81febb5
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

feat(spe): add support for FEAT_SPE_FDS

Allow access to PMSDSFR_EL1 register at NS-EL1 or NS-EL2 when
FEAT_SPE_FDS is implemented.

Change-Id: I538577cbfa5b5f242d5dbaeeace7b8e4ee6ffd03
Signed-off-by

feat(spe): add support for FEAT_SPE_FDS

Allow access to PMSDSFR_EL1 register at NS-EL1 or NS-EL2 when
FEAT_SPE_FDS is implemented.

Change-Id: I538577cbfa5b5f242d5dbaeeace7b8e4ee6ffd03
Signed-off-by: James Clark <james.clark2@arm.com>

show more ...

feat(cpufeat): add support for FEAT_PAUTH_LR

This patch enables FEAT_PAUTH_LR at EL3 on systems that support it when
the new ENABLE_FEAT_PAUTH_LR flag is set.

Currently, PAUTH_LR is only supported

feat(cpufeat): add support for FEAT_PAUTH_LR

This patch enables FEAT_PAUTH_LR at EL3 on systems that support it when
the new ENABLE_FEAT_PAUTH_LR flag is set.

Currently, PAUTH_LR is only supported by arm clang compiler and not GCC.

Change-Id: I7db1e34b661ed95cad75850b62878ac5d98466ea
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

feat(cpufeat): enable FEAT_PAuth to FEAT_STATE_CHECKED

FEAT_PAuth is the second to last feature to be a boolean choice - it's
either unconditionally compiled in and must be present in hardware or
it

feat(cpufeat): enable FEAT_PAuth to FEAT_STATE_CHECKED

FEAT_PAuth is the second to last feature to be a boolean choice - it's
either unconditionally compiled in and must be present in hardware or
it's not compiled in. FEAT_PAuth is architected to be backwards
compatible - a subset of the branch guarding instructions (pacia/autia)
execute as NOPs when PAuth is not present. That subset is used with
`-mbranch-protection=standard` and -march pre-8.3. This patch adds the
necessary logic to also check accesses of the non-backward compatible
registers and allow a fully checked implementation.

Note that a checked support requires -march to be pre 8.3, as otherwise
the compiler will include branch protection instructions that are not
NOPs without PAuth (eg retaa) which cannot be checked.

Change-Id: Id942c20cae9d15d25b3d72b8161333642574ddaa
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cpufeat): prepare FEAT_PAuth for FEATURE_DETECTION

Convert the old style is_armv8_3_pauth_present() to the new style
is_feat_pauth_{present, supported}() helpers and hook FEATURE_DETECTION

refactor(cpufeat): prepare FEAT_PAuth for FEATURE_DETECTION

Convert the old style is_armv8_3_pauth_present() to the new style
is_feat_pauth_{present, supported}() helpers and hook FEATURE_DETECTION
into it. This is in preparation for converting FEAT_PAuth to FEAT_STATE.

Change-Id: Iec8c3477fafb2cdae67d39ae4da2cca76a67511a
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): enable FEAT_BTI to FEAT_STATE_CHECKED

Introduce the is_feat_bti_{supported, present}() helpers and replace
checks for ENABLE_BTI with it. Also factor out the setting of
SCTLR_EL3.BT o

feat(cpufeat): enable FEAT_BTI to FEAT_STATE_CHECKED

Introduce the is_feat_bti_{supported, present}() helpers and replace
checks for ENABLE_BTI with it. Also factor out the setting of
SCTLR_EL3.BT out of the PAuth enablement and place it in the respective
entrypoints where we initialise SCTLR_EL3. This makes PAuth
self-contained and SCTLR_EL3 initialisation centralised.

Change-Id: I0c0657ff1e78a9652cd2cf1603478283dc01f17b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...