fix(cm): do not check for AArch32 support to enable features

EL3 is running in AArch64 mode so it is always able to enable all
features. Some features are not implemented for AArch32 but that will b

fix(cm): do not check for AArch32 support to enable features

EL3 is running in AArch64 mode so it is always able to enable all
features. Some features are not implemented for AArch32 but that will be
handled architecturally on changing exception levels so we don't need to
worry about it. Always enable all features (checked on the FEAT_STATE
flag of course) and save ourselves the check.

To prevent confusion in future, the SCR bits that actually affect
AArch32 execution and must always be checked are moved up in the file
and grouped together, rather than being straddled by feature enablement.

Change-Id: I154957405befb750c03738d1989bfb12696fc79d
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless t

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless trapped to EL2). IDTE3 allows EL3 to
modify the view of ID registers for lower ELs, and this capability
is used to disable fields of ID registers tied to disabled features.

The ID registers are initially read as-is and stored in context.
Then, based on the feature enablement status for each world, if a
particular feature is disabled, its corresponding field in the
cached ID register is set to Res0. When lower ELs attempt to read
an ID register, the cached ID register value is returned. This
allows EL3 to prevent lower ELs from accessing feature-specific
system registers that are disabled in EL3, even though the hardware
implements them.

The emulated ID register values are stored primarily in per-world
context, except for certain debug-related ID registers such as
ID_AA64DFR0_EL1 and ID_AA64DFR1_EL1, which are stored in the
cpu_data and are unique to each PE. This is done to support feature
asymmetry that is commonly seen in debug features.

FEAT_IDTE3 traps all Group 3 ID registers in the range
op0 == 3, op1 == 0, CRn == 0, CRm == {2–7}, op2 == {0–7} and the
Group 5 GMID_EL1 register. However, only a handful of ID registers
contain fields used to detect features enabled in EL3. Hence, we
only cache those ID registers, while the rest are transparently
returned as is to the lower EL.

This patch updates the CREATE_FEATURE_FUNCS macro to generate
update_feat_xyz_idreg_field() functions that disable ID register
fields on a per-feature basis. The enabled_worlds scope is used to
disable ID register fields for security states where the feature is
not enabled.

This EXPERIMENTAL feature is controlled by the ENABLE_FEAT_IDTE3
build flag and is currently disabled by default.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I5f998eeab81bb48c7595addc5595313a9ebb96d5

show more ...

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clear

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clears them
during early C runtime initialization. For platforms that enable
NUMA_AWARE_PER_CPU, invokes a platform hook to zero-initialize
node-specific per-cpu regions.

Cache maintenance: Extends the BL31 exit path to clean dcache lines
covering the per-cpu region, ensuring data written by the primary core
is visible to secondary cores.

tpidr_el3 setup: Initializes tpidr_el3 with the base address of the
current CPU’s per-cpu section. This allows per-cpu framework to
resolve local cpu accesses efficiently.

The percpu_data object is currently stored in tpidr_el3. Since the
per-cpu framework will use tpidr_el3 for this-cpu access, percpu_data
must be migrated to avoid conflict. This commit moves percpu_data to
the per-cpu framework.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: Iff0c2e1f8c0ebd25c4bb0b09bfe15dd4fbe20561

show more ...

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never writ

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never written to and so zeroes end up in registers, which is not
necessarily correct.

This patch gets BL1 to call cm_manage_extensions_per_world() whenever
BL2 runs in a lower EL. This allows the per-world registers to have the
reset values we intend. An accompanying call to
cm_manage_extensions_el3() is also added for completeness.

Doing this shows a small deficiency in cptr_el3 - bits TFP and TCPAC
change a lot. This patch makes them consistent by always setting TCPAC
and TFP to 0 which unconditionally enable access to CPTR_EL2 and FPCR by
default as they are always accessible. Other places that manipulate the
TFP bit are removed.

A nice side effect of all of this is that we're now in a position to
enable and use any architectural extension in BL2.

Change-Id: I070d62bbf8e9d9b472caf7e2c931c303523be308
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(context-mgmt): correct guards for PIE and POE registers" into integration

Merge changes from topic "bk/simpler_panic" into integration

* changes:
fix(aarch64): do not print EL1 registers on EL3 panic
refactor(el3-runtime): streamline cpu_data assembly offsets using th

Merge changes from topic "bk/simpler_panic" into integration

* changes:
fix(aarch64): do not print EL1 registers on EL3 panic
refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

show more ...

fix(context-mgmt): correct guards for PIE and POE registers

At the moment access to the POR_EL2 system register is guarded by
either FEAT_S1POE or FEAT_S2POE being implemented. However this
register

fix(context-mgmt): correct guards for PIE and POE registers

At the moment access to the POR_EL2 system register is guarded by
either FEAT_S1POE or FEAT_S2POE being implemented. However this
register exists only with FEAT_S1POE, and the two features are
independent from each other.

Similarly access to PIRE0_EL2 and PIR_EL2 is guarded by either
FEAT_S1PIE or FEAT_S2PIE yet these only exist with FEAT_S1PIE, with the
two features also independent of each other.

Change the guards to only look at FEAT_S1PIE/FEAT_S1POE, to fix crashes
when FEAT_S2PIE/FEAT_S2POE is implemented, but FEAT_S1PIE/FEAT_S1POE is
not.

Signed-off-by: Agathiyan Bragadeesh <agathiyan.bragadeesh2@arm.com>
Change-Id: I5d145c0f1b388e7bdade94920b6bf5f2d7eebfcd

show more ...

Merge "fix(cm): correctly restore BL2's context" into integration

feat(cpufeat): enable FEAT_PFAR support

Implement support for FEAT_PFAR, which introduces the PFAR_ELx system
register, recording the faulting physical address for some aborts.
Those system register

feat(cpufeat): enable FEAT_PFAR support

Implement support for FEAT_PFAR, which introduces the PFAR_ELx system
register, recording the faulting physical address for some aborts.
Those system registers are trapped by the SCR_EL3.PFARen bit, so set the
bit for the non-secure world context to allow OSes to use the feature.

This is controlled by the ENABLE_FEAT_PFAR build flag, which follows the
usual semantics of 2 meaning the feature being runtime detected.
Let the default for this flag be 0, but set it to 2 for the FVP.

Change-Id: I5c9ae750417e75792f693732df3869e02b6e4319
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

Merge "feat(cpufeat): enable FEAT_AIE support" into integration

fix(cm): correctly restore BL2's context

We have a single context management library but two BLs that use it very
differently - BL1 and BL31. Configuring it correctly is quite tricky as
in the BL1 +

fix(cm): correctly restore BL2's context

We have a single context management library but two BLs that use it very
differently - BL1 and BL31. Configuring it correctly is quite tricky as
in the BL1 + BL2 + BL31 + spmd case we have it operate on both EL1 and
EL2 context (for BL1 and BL31 respectively).

Welp, we've got that wrong - BL1 will skip EL1's register initialisation
leaving it at the mercy of UNKNOWN registers' values. If they aren't as
we expect, things don't go well.

This is not the end of the world as BL1 is expected to be the first
thing to run and UNKNOWN values generally reset to things we find
acceptable. But add the correct BL1 carveouts so they are set up
correctly.

Change-Id: Ia0d5522ac0c9f4616dd124b10f0c2fdc823a0d87
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(cm): deprecate use of NS_TIMER_SWITCH" into integration

feat(cpufeat): enable FEAT_AIE support

Implement support for FEAT_AIE, which introduces the AMAIR2_ELx and
MAIR2_ELx system registers, extending the memory attributes described
by {A}MAIR_ELx.
Those

feat(cpufeat): enable FEAT_AIE support

Implement support for FEAT_AIE, which introduces the AMAIR2_ELx and
MAIR2_ELx system registers, extending the memory attributes described
by {A}MAIR_ELx.
Those system registers are trapped by the SCR_EL3.AIEn bit, so set the
bit for the non-secure world context to allow OSes to use the feature.

This is controlled by the ENABLE_FEAT_AIE build flag, which follows the
usual semantics of 2 meaning the feature being runtime detected.
Let the default for this flag be 0, but set it to 2 for the FVP.

Change-Id: Iba2011719013a89f9cb3a4317bde18254f45cd25
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

Merge "fix(cm): remove unused macro" into integration

Merge "refactor(el3-runtime): move context security states to context.h" into integration

fix(cm): remove unused macro

It is never referenced.

Change-Id: I538b1f3d8660426faf5bafa68ecda2d637b0bc50
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

feat(cpufeat): enable FEAT_CPA2 for EL3

FEAT_CPA2 enables checked pointer arithmetic, which in the event
of an arithmetic overflow in pointer generation will result in a
non-canonical pointer being

feat(cpufeat): enable FEAT_CPA2 for EL3

FEAT_CPA2 enables checked pointer arithmetic, which in the event
of an arithmetic overflow in pointer generation will result in a
non-canonical pointer being generated and subsequent address fault.

Note that FEAT_CPA is a trivial implementation that exists in
some hardware purely so it can run CPA2-enabled instructions
without crashing but they don't actually have checked arithmetic,
so FEAT_CPA is not explicitly enabled in TF-A.

Change-Id: I6d2ca7a7e4b986bb9e917aa8baf8091a271c168b
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

The cpu_data structure, just like cpu_ops, is collection of disparate
data that must be accessible from both C

refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

The cpu_data structure, just like cpu_ops, is collection of disparate
data that must be accessible from both C and assembly. Achieving this is
tricky as there is no way to export structure offsets from C directly so
they must be manually recreated with `#define`s and asserts. However,
the cpu_data structure is quite old and the assembly offsets are a
patchwork of additions and extremely difficult to reason with and
modify. In fact, certain currently unused builds with
ENABLE_RUNTIME_INSTRUMENTATION=1 fail to build.

To untangle this, convert the assembly offsets to the pattern used for
the cpu_ops structure. That is, first define the sizes of every member,
as generically as possible, and then chain their offsets one after the
other. To make sure this is always correct, add a CASSERT for the offset
of every member. This makes it easy to modify the structure and fixes
the build failures.

Change-Id: I61aeb55e9c494896663a3c719c10e3c072f56349
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(el3-runtime): move context security states to context.h

The three security states (S, NS, RL) are architecturally quite
consistent - anything that uses them has the same numerical assignmen

refactor(el3-runtime): move context security states to context.h

The three security states (S, NS, RL) are architecturally quite
consistent - anything that uses them has the same numerical assignments
(0, 1, 2) and they are quite convenient for indexing. However, we're not
as consistent in tf-a and this is defined in a few places. Since
cpu_data has a dependency on the context management library, use its
security state convention in a few more places and take away this
responsibility from cpu_data.

Change-Id: Iec73b2be2eef91975554767557de72424d0031f1
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cm): deprecate use of NS_TIMER_SWITCH

On AArch64, secure world has it's own EL3 physical timer registers
accessible to secure EL1 in absence of S-EL2. With S-EL2 there is
virtualized view availa

fix(cm): deprecate use of NS_TIMER_SWITCH

On AArch64, secure world has it's own EL3 physical timer registers
accessible to secure EL1 in absence of S-EL2. With S-EL2 there is
virtualized view available for EL1 timer registers. So it is
unreasonable for secure world to use non-secure EL1 physical timer
registers. Moreover, the non-secure operating system (Linux in our case)
relies heavily on these EL1 physical timer registers for scheduling
decisions. If NS_TIMER_SWITCH is enabled, it simply breaks the preemption
model of the non-secure world by disabling non-secure timer interrupts
leading to RCU stalls being observed on long running secure world tasks.

The only arch timer register which will benefit from context management
is cntkctl_el1: Counter-timer Kernel Control Register. This enables the
secure and non-secure worlds to independently control accesses to EL0
for counter-timer registers. This is something that OP-TEE uses to
enable ftrace feature for Trusted Applications and SPM_MM uses for EL0
access as well.

Lets enable context management of cntkctl_el1 by default and deprecate
conditional context management of non-secure EL1 physical timer
registers for whom there isn't any upstream user. With that deprecate
this NS_TIMER_SWITCH build option which just adds confusion for the
platform maintainers. It will be eventually dropped following
deprecation policy of TF-A.

Reported-by: Stauffer Thomas MTANA <thomas.stauffer@mt.com>
Reported-by: Andrew Davis <afd@ti.com>
Change-Id: Ifb3a919dc0bf8c05c38895352de5fe94b4f4387e
Signed-off-by: Sumit Garg <sumit.garg@oss.qualcomm.com>

show more ...

refactor(aarch64): move BL31 specific setup out of the PSCI entrypoint

We've charged the PSCI entrypoint with doing BL31 specific things like
setting up the EL3 context and doing feature detection.

refactor(aarch64): move BL31 specific setup out of the PSCI entrypoint

We've charged the PSCI entrypoint with doing BL31 specific things like
setting up the EL3 context and doing feature detection. Well, this is
irrelevant for sp_min and not really appropriate for PSCI. So move it to
the bl31_warmboot() function to reflect this correctly and bring the
feature detection a bit earlier, hopefully spotting more errors.

This allows for a pair of minor cleanups - we can pass the core_pos to
psci_warmboot_entrypoint() without having to refetch it, and we can put
the pauth enablement in cm_manage_extensions_el3() along with all
others. The call of that function is kept after the MMU is turned on so
that we have nicer (coherent) access to cpu_data.

Change-Id: Id031cfa0e1d8fe98919a14f9db73eb5bc9e00f67
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support" into integration

Merge changes Id38d6f1b,I5fcfe8dd,I7f3b50e5 into integration

* changes:
fix(cpus): inform the compiler that struct cpu_ops is aligned
refactor(el3-runtime): move the initialisation of the cpu_op

Merge changes Id38d6f1b,I5fcfe8dd,I7f3b50e5 into integration

* changes:
fix(cpus): inform the compiler that struct cpu_ops is aligned
refactor(el3-runtime): move the initialisation of the cpu_ops_ptr to C
fix(aarch32): make get_cpu_ops_ptr() PCS compliant

show more ...

refactor(el3-runtime): move the initialisation of the cpu_ops_ptr to C

The difference between AArch32 and AArch64 is insignificant and the
usage is identical. The only thing that required the use of

refactor(el3-runtime): move the initialisation of the cpu_ops_ptr to C

The difference between AArch32 and AArch64 is insignificant and the
usage is identical. The only thing that required the use of assembly was
that the get_cpu_ops_ptr() function was not PCS compliant and needed a
wrapper to do that instead. That has now been fixed so move this to C so
it's more readable and more optimise-able by the compiler.

Change-Id: I5fcfe8ddb122dd35d58adc6d44a7484c5c595815
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to

feat(cpufeat): enable FEAT_MPAM_PE_BW_CTRL support

Implement support for FEAT_MPAM_PE_BW_CTRL, allowing lower
Exception Levels to access MPAM_PE_BW_CTRL control registers
by disabling their traps to EL3.

When INIT_UNUSED_NS_EL2=1, configure MPAMBW2_EL2 in EL3 so
that MPAM_PE_BW_CTRL accesses from EL0/EL1 do not trap to EL2.

At this stage, PE-side MPAM bandwidth controls remain disabled
in EL3.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I8e359b0eb912cff3bdda109b21727a627cac3a7e

show more ...