Merge "feat(cpufeat): enable USE_SPINLOCK_CAS to FEAT_STATE_CHECKED" into integration

feat(cpufeat): enable USE_SPINLOCK_CAS to FEAT_STATE_CHECKED

The FEAT_LSE enablement predates the FEAT_STATE framework and has never
been converted. Since the introduction of USE_SPINLOCK_CAS we've

feat(cpufeat): enable USE_SPINLOCK_CAS to FEAT_STATE_CHECKED

The FEAT_LSE enablement predates the FEAT_STATE framework and has never
been converted. Since the introduction of USE_SPINLOCK_CAS we've gained
lots of quality of life features that allow for better feature
enablement. This patch converts USE_SPINLOCK_CAS to tri-state and adds
it to FEATURE_DETECTION to align with all other features.

Instead of introducing the assembly checking for tri-state, this patch
translates all locking routines to C inline assembly and uses the
standard C helpers. The main benefit is that this gives greater
visibility to the compiler about what the functions are doing and lets
it optimise better. Namely, it is able to allocate registers itself and
inline the functions when LTO is enabled.

An unsuccessful attempt was made to use the instructions directly and
have even flow control in C. This, however, made code very complicated
and less efficient in the tight loops of the spinlock.

The last use of ARM_ARCH_AT_LEAST goes away with this change and so this
macro is removed. It has now been fully superseded by the FEAT_STATE
framework.

This change exposes a limitation - RME_GPT_BITLOCK_BLOCK requires
USE_SPINLOCK_CAS. This patch does not address this in any way but makes
the relationship explicit.

Change-Id: I580081549aceded2dca3e0f4564ee7510a7e56ae
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "ar/feat_uinj" into integration

* changes:
feat(cpufeat): add support for FEAT_UINJ
feat(cpufeat): enable mandatory Armv9.4–Armv9.6 features by default
fix(cpufeat): u

Merge changes from topic "ar/feat_uinj" into integration

* changes:
feat(cpufeat): add support for FEAT_UINJ
feat(cpufeat): enable mandatory Armv9.4–Armv9.6 features by default
fix(cpufeat): update feature names and comments
fix(cpufeat): simplify AArch32 feature disablement

show more ...

feat(cpufeat): add support for FEAT_UINJ

FEAT_UINJ allows higher ELs to inject Undefined Instruction exceptions
into lower ELs by setting SPSR_ELx.UINJ, which updates PSTATE.UINJ on
exception return

feat(cpufeat): add support for FEAT_UINJ

FEAT_UINJ allows higher ELs to inject Undefined Instruction exceptions
into lower ELs by setting SPSR_ELx.UINJ, which updates PSTATE.UINJ on
exception return. When PSTATE.UINJ is set, instruction execution at the
lower EL raises an Undefined Instruction exception (EC=0b000000).

This patch introduces support for FEAT_UINJ by updating the
inject_undef64() to use hardware undef injection if supported.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I48ad56a58eaab7859d508cfa8dfe81130b873b6b

show more ...

fix(cpufeat): simplify AArch32 feature disablement

Remove redundant conditional checks for unsupported features
(SPE, SVE, MPAM) in aarch32 builds and set them unconditionally
to 0. Add correspondin

fix(cpufeat): simplify AArch32 feature disablement

Remove redundant conditional checks for unsupported features
(SPE, SVE, MPAM) in aarch32 builds and set them unconditionally
to 0. Add corresponding constraint checks to ensure these
features are not enabled when ARCH=aarch32.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I6486b54c69bf0c273371235d1661fafbcb7abb8c

show more ...

Merge changes from topic "ar/idte3" into integration

* changes:
feat(cpufeat): add support for FEAT_IDTE3
feat(cpufeat): include enabled security state scope
feat(cpufeat): add ID register def

Merge changes from topic "ar/idte3" into integration

* changes:
feat(cpufeat): add support for FEAT_IDTE3
feat(cpufeat): include enabled security state scope
feat(cpufeat): add ID register defines and read helpers

show more ...

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless t

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless trapped to EL2). IDTE3 allows EL3 to
modify the view of ID registers for lower ELs, and this capability
is used to disable fields of ID registers tied to disabled features.

The ID registers are initially read as-is and stored in context.
Then, based on the feature enablement status for each world, if a
particular feature is disabled, its corresponding field in the
cached ID register is set to Res0. When lower ELs attempt to read
an ID register, the cached ID register value is returned. This
allows EL3 to prevent lower ELs from accessing feature-specific
system registers that are disabled in EL3, even though the hardware
implements them.

The emulated ID register values are stored primarily in per-world
context, except for certain debug-related ID registers such as
ID_AA64DFR0_EL1 and ID_AA64DFR1_EL1, which are stored in the
cpu_data and are unique to each PE. This is done to support feature
asymmetry that is commonly seen in debug features.

FEAT_IDTE3 traps all Group 3 ID registers in the range
op0 == 3, op1 == 0, CRn == 0, CRm == {2–7}, op2 == {0–7} and the
Group 5 GMID_EL1 register. However, only a handful of ID registers
contain fields used to detect features enabled in EL3. Hence, we
only cache those ID registers, while the rest are transparently
returned as is to the lower EL.

This patch updates the CREATE_FEATURE_FUNCS macro to generate
update_feat_xyz_idreg_field() functions that disable ID register
fields on a per-feature basis. The enabled_worlds scope is used to
disable ID register fields for security states where the feature is
not enabled.

This EXPERIMENTAL feature is controlled by the ENABLE_FEAT_IDTE3
build flag and is currently disabled by default.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I5f998eeab81bb48c7595addc5595313a9ebb96d5

show more ...

Merge changes from topic "NUMA_AWARE_PER_CPU" into integration

* changes:
docs(maintainers): add per-cpu framework into maintainers.rst
feat(per-cpu): add documentation for per-cpu framework
f

Merge changes from topic "NUMA_AWARE_PER_CPU" into integration

* changes:
docs(maintainers): add per-cpu framework into maintainers.rst
feat(per-cpu): add documentation for per-cpu framework
feat(rdv3): enable numa aware per-cpu for RD-V3-Cfg2
feat(per-cpu): migrate amu_ctx to per-cpu framework
feat(per-cpu): migrate spm_core_context to per-cpu framework
feat(per-cpu): migrate psci_ns_context to per-cpu framework
feat(per-cpu): migrate psci_cpu_pd_nodes to per-cpu framework
feat(per-cpu): migrate rmm_context to per-cpu framework
feat(per-cpu): integrate per-cpu framework into BL31/BL32
feat(per-cpu): introduce framework accessors/definers
feat(per-cpu): introduce linker changes for NUMA aware per-cpu framework
docs(changelog): add scope for per-cpu framework

show more ...

feat(per-cpu): introduce linker changes for NUMA aware per-cpu framework

This commit introduces linker changes for NUMA aware per-cpu objects in
the BL31 and BL32 images. The per-cpu framework is de

feat(per-cpu): introduce linker changes for NUMA aware per-cpu framework

This commit introduces linker changes for NUMA aware per-cpu objects in
the BL31 and BL32 images. The per-cpu framework is designed to minimise
cache thrashing, and the linker layout ensures each CPU’s per-cpu data
is placed on a separate cache line. This isolation is expected to
improve performance when the per-cpu framework is enabled.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: Ie4d8b4e444971adbd9dba0446d1ab8cafaca1556

show more ...

Merge changes I54152fbb,I9b70c3a2 into integration

* changes:
feat(cpufeat): introduce FEAT_RME_GDI support
feat(cpufeat): add support for FEAT_RME_GPC2

feat(cpufeat): introduce FEAT_RME_GDI support

This patch adds a new build flag ENABLE_FEAT_RME_GDI to enable this
feature, along with defining various related register fields. At this
point, when en

feat(cpufeat): introduce FEAT_RME_GDI support

This patch adds a new build flag ENABLE_FEAT_RME_GDI to enable this
feature, along with defining various related register fields. At this
point, when enabled, this feature enables the SA and NSP GPI encodings
by setting the corresponding bits in GPCCR_EL3.

Change-Id: I54152fbb3d19b176264e5d16acbcc866725dc290
Signed-off-by: John Powell <john.powell@arm.com>
Signed-off-by: Timothy Hayes <timothy.hayes@arm.com>

show more ...

Merge changes Ie8453359,Icd58a49c into integration

* changes:
docs: deprecate SPM_MM build option
docs: deprecate NS_TIMER_SWITCH build option

docs: deprecate SPM_MM build option

Following the ML post [1] deprecating the SPM-MM build option.

[1] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/Z6GAD7OG

docs: deprecate SPM_MM build option

Following the ML post [1] deprecating the SPM-MM build option.

[1] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/Z6GAD7OGKNDPNKECJ63HQZ4XEYUJXTNM/

Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Change-Id: Ie845335989a6b11382ebe2f32962f534ad1bf8c6

show more ...

Merge changes Ia8b52237,Id4b7714e into integration

* changes:
fix(build): put the -target definitions in toolchain.mk
fix(build): align the cpu-ops flags with all others

fix(build): align the cpu-ops flags with all others

Since the cpu-ops file was created we've obtained the constraints.mk and
cflags.mk files and we also have the defaults.mk. The cpu-ops file is
not

fix(build): align the cpu-ops flags with all others

Since the cpu-ops file was created we've obtained the constraints.mk and
cflags.mk files and we also have the defaults.mk. The cpu-ops file is
not much different to these three, just much more complex. This patch
keeps the complicated bit in cpu-ops.mk but it makes it behave like
defaults.mk. The non-complicated bits (like cross referencing and
compiler flags) go to their corresponding files. This centralises
responsibilities and makes it nicer to keep track of.

The reason for doing this untangling is that the order of defaulting,
cross referencing, and compiler flag selection is significant and we can
run into problems where seemingly identical lines of code produce
different outcomes.

Change-Id: Id4b7714e432a0d628d33412836fd5c93f0488970
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(cpufeat): fix build with ENABLE_FEAT_PAUTH_LR=1" into integration

feat(cpufeat): fix build with ENABLE_FEAT_PAUTH_LR=1

Since build system underwent some refactoring, the build has
been failing with ENABLE_FEAT_PAUTH_LR=1 so this patch fixes it

Change-Id: I1a3faad

feat(cpufeat): fix build with ENABLE_FEAT_PAUTH_LR=1

Since build system underwent some refactoring, the build has
been failing with ENABLE_FEAT_PAUTH_LR=1 so this patch fixes it

Change-Id: I1a3faad53b508d760a65656dad78057027a7d69c
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

Merge "fix(cm): deprecate use of NS_TIMER_SWITCH" into integration

fix(cm): deprecate use of NS_TIMER_SWITCH

On AArch64, secure world has it's own EL3 physical timer registers
accessible to secure EL1 in absence of S-EL2. With S-EL2 there is
virtualized view availa

fix(cm): deprecate use of NS_TIMER_SWITCH

On AArch64, secure world has it's own EL3 physical timer registers
accessible to secure EL1 in absence of S-EL2. With S-EL2 there is
virtualized view available for EL1 timer registers. So it is
unreasonable for secure world to use non-secure EL1 physical timer
registers. Moreover, the non-secure operating system (Linux in our case)
relies heavily on these EL1 physical timer registers for scheduling
decisions. If NS_TIMER_SWITCH is enabled, it simply breaks the preemption
model of the non-secure world by disabling non-secure timer interrupts
leading to RCU stalls being observed on long running secure world tasks.

The only arch timer register which will benefit from context management
is cntkctl_el1: Counter-timer Kernel Control Register. This enables the
secure and non-secure worlds to independently control accesses to EL0
for counter-timer registers. This is something that OP-TEE uses to
enable ftrace feature for Trusted Applications and SPM_MM uses for EL0
access as well.

Lets enable context management of cntkctl_el1 by default and deprecate
conditional context management of non-secure EL1 physical timer
registers for whom there isn't any upstream user. With that deprecate
this NS_TIMER_SWITCH build option which just adds confusion for the
platform maintainers. It will be eventually dropped following
deprecation policy of TF-A.

Reported-by: Stauffer Thomas MTANA <thomas.stauffer@mt.com>
Reported-by: Andrew Davis <afd@ti.com>
Change-Id: Ifb3a919dc0bf8c05c38895352de5fe94b4f4387e
Signed-off-by: Sumit Garg <sumit.garg@oss.qualcomm.com>

show more ...

Merge "refactor(el3-runtime): extract cpu_data limitations to top-level constraints" into integration

Merge "feat(cpufeat): enable FEAT_CPA2 for EL3" into integration

feat(cpufeat): enable FEAT_CPA2 for EL3

FEAT_CPA2 enables checked pointer arithmetic, which in the event
of an arithmetic overflow in pointer generation will result in a
non-canonical pointer being

feat(cpufeat): enable FEAT_CPA2 for EL3

FEAT_CPA2 enables checked pointer arithmetic, which in the event
of an arithmetic overflow in pointer generation will result in a
non-canonical pointer being generated and subsequent address fault.

Note that FEAT_CPA is a trivial implementation that exists in
some hardware purely so it can run CPA2-enabled instructions
without crashing but they don't actually have checked arithmetic,
so FEAT_CPA is not explicitly enabled in TF-A.

Change-Id: I6d2ca7a7e4b986bb9e917aa8baf8091a271c168b
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

refactor(el3-runtime): extract cpu_data limitations to top-level constraints

CRASH_REPORTING is checked via an `#error` statement in the header,
while EL3_EXCEPTION_HANDLING is carefully carved out

refactor(el3-runtime): extract cpu_data limitations to top-level constraints

CRASH_REPORTING is checked via an `#error` statement in the header,
while EL3_EXCEPTION_HANDLING is carefully carved out when not supported.
However, both are only used on AArch64 builds and never on AArch32. We
can promote both to proper make constraints and keep the cpu_data
implementation a little bit simpler.

Change-Id: Ia164e046f953a552dc6e6cf624961a90669eaeeb
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support" into integration

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to EL3.

When INIT_UNUSED_NS_EL2=1, configure MPAMBW2_EL2 in EL3 so
that MPAM_PE_BW_CTRL accesses from EL0/EL1 do not trap to EL2.

At this stage, PE-side MPAM bandwidth controls remain disabled
in EL3.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I8e359b0eb912cff3bdda109b21727a627cac3a7e

show more ...