feat(crypto): enable floating point register traps in EL3

To prevent the leakage of EL3 information to lower ELs, access to
floating point registers needed to be traped to EL3 unless necessary
(e.g

feat(crypto): enable floating point register traps in EL3

To prevent the leakage of EL3 information to lower ELs, access to
floating point registers needed to be traped to EL3 unless necessary
(e.g the SIMD crypto extension, SIMD context save/restore).

Change-Id: I28a734c43d3e965de87ccc08e99f86669729871f
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

Merge "fix(context-mgmt): actually clear MDCR_EL3 bits" into integration

fix(morello): avoid capability tag fault on data access

TF-A runtime service at EL3 switches the stack pointer from SP_EL3
to SP_EL0. This creates a capability tag fault when the DDC_EL0 is
zeroed o

fix(morello): avoid capability tag fault on data access

TF-A runtime service at EL3 switches the stack pointer from SP_EL3
to SP_EL0. This creates a capability tag fault when the DDC_EL0 is
zeroed out (purecap user space) as any data accesses computes
tag/permission with DDC_EL0 value when SpSel is 0 and when EL3 is
in hybrid mode.

As a workaround, this patch creates a per cpu context variable
to store DDC_EL0 value so that when EL3 runtime is entered DDC_EL0
is saved on to stack. DDC_EL3 is then copied into DDC_EL0 after
switching SP to SP_EL0. Once the runtime finishes, during el3_exit,
the saved DDC_EL0 is restored from stack.

Signed-off-by: Selvarasu Ganesan <selvarasu.ganesan@arm.com>
Signed-off-by: Manoj Kumar <manoj.kumar3@arm.com>
Signed-off-by: Varshit Pandya <varshit.pandya@arm.com>
Change-Id: I4e4010f0e20913cb4e35b58fb49a177bdf26feb1

show more ...

feat(morello): add Morello capability enablement changes

This patch adds a build macro ENABLE_FEAT_MORELLO which when set will
compile BL31 firmware with changes required to boot capability
aware so

feat(morello): add Morello capability enablement changes

This patch adds a build macro ENABLE_FEAT_MORELLO which when set will
compile BL31 firmware with changes required to boot capability
aware software.

It also adds helper function in c and assmbly to check if morello
hardware is present and if morello capability is enabled or not.

CE field, bits [23:20] in ID_AA64PFR1_EL1 defines whether morello
architecture is present or not, 0b0000 indicates that it is absent
and 0b0001 indicates that it is present. While whether capabilities
are enabled or not is decided at runtime with ENABLE_FEAT_MORELLO build
option.

Reference: https://developer.arm.com/documentation/ddi0606/latest/

Signed-off-by: Manoj Kumar <manoj.kumar3@arm.com>
Signed-off-by: Varshit Pandya <varshit.pandya@arm.com>
Change-Id: Ib16877acbfcb72c4bd8c08e97e44edc0a3e46089

show more ...

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to their low
resolution and general availability of the data elsewhere. As such, they
are used for critical performance tuning and are expected to never be
turned off or context switched when switching worlds.

The upper 2 counters, INST_RETIRED and STALL_BACKEND_MEM, are different.
The data they provide is non-critical and expose new information that
could be used as a timing side channel, especially of Secure world. This
patch adds context switching of these two counters to prevent any such
side channel.

This is not done for group 1 auxiliary counters as those are IMP DEF and
are inaccessible by default unless overriden by the platform (with
AMU_RESTRICT_COUNTERS).

Change-Id: Ib4b946abb810e36736cabb9b84cd837308b4e761
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(context-mgmt): actually clear MDCR_EL3 bits

When setting up MDCR_EL3 for a given context, we need to set some bits,
but also clear some other bits. This was done in a single statement,
but using

fix(context-mgmt): actually clear MDCR_EL3 bits

When setting up MDCR_EL3 for a given context, we need to set some bits,
but also clear some other bits. This was done in a single statement,
but using the C "|=" operator, which would never clear any bits in the
left-hand side.

Split this into two statements, one for setting, the other for clearing
bits.

It seems that on the FVP the bits to clear already reset to 0, so this
never caused any issues so far, but the architecture declares those bits
as: "this field resets to an architecturally UNKNOWN value".

Change-Id: Id1e9e4c010167af2ea3d5820532704220aa7c647
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(el3-runtime): translate EL3 handled exceptions to C and always call prepare_el3_entry

Exception handling in BL31 is tricky business and to satisfy the varying
requirements of the different code

feat(el3-runtime): translate EL3 handled exceptions to C and always call prepare_el3_entry

Exception handling in BL31 is tricky business and to satisfy the varying
requirements of the different code paths it has thus far largely been
written in assembly. However, assembly is extremely tedious to read and
modify. Similar to context management, it is desirable to have as much
as possible in C. C code is generally easier to follow and can enable
the compiler to do more optimisations on surrounding code.

Most exceptions that BL31 deals with are the synchronous exceptions and
those are processed within BL31. They already get prepared for EL3 entry
and after the initial dispatch end up in C. So the dispatch can also be
converted in C. Interrupt exceptions are very similar so are converted
too. Finally, asynchronous external aborts share some code with
synchronous external aborts and may end up being processed deeper in
BL31. So they can safely be prepared for EL3 entry too and converted to
C so that they can share code properly.

The IMP DEF exceptions are not part of this refactor as their speed may
be important. There is currently little that uses them, but they can be
converted to C too once their use expands and usage allows it.

This refactor allows to expand the responsibilities of
prepare_el3_entry(). Its role is already to prepare context for
executing within EL3 but with this patch EL3 execution is synonymous
with C runtime execution. So it's given the responsibility of saving
spsr and elr as well as putting the runtime stack in.

When a synchronous exception happens, the only possible paths are to
enter the C EL3 runtime, exiting via el3_exit(), or to panic. In the EL3
runtime case, we always need prepare_el3_entry() and the runtime stack,
whereas in the panic case, this doesn't matter as we will never return.
So hoist the prepare_el3_entry() call and the changing of the stacks as
early as possible and make the rest of the code agnostic of this.

This patch also gets rid of smc_prohibited. It is an optimisation by
skipping prepare_el3_entry() when a bad smc call happens. However, speed
doesn't matter in this case as this is an erroneous case.

Change-Id: I411af9d17ef4046a736b1f4f5f8fbc9c28e66106
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "refactor(cm): reduce conditional compilation" into integration

fix(cpufeat): enable FEAT_FGWTE3 after FEAT_CPA

FEAT_CPA needs to write SCTLR2_EL3 which will be forbidden after
initialising FEAT_FGWTE3. Correct the order.

Change-Id: I3a0554d2a73f773b3ad672eb1e4

fix(cpufeat): enable FEAT_FGWTE3 after FEAT_CPA

FEAT_CPA needs to write SCTLR2_EL3 which will be forbidden after
initialising FEAT_FGWTE3. Correct the order.

Change-Id: I3a0554d2a73f773b3ad672eb1e4b0db0171d38bd
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cm): reduce conditional compilation

Context debug needs to switch between EL1 and EL2 context but it can
re-use its variables and function calls with a bit of clever naming.
Unify them to r

refactor(cm): reduce conditional compilation

Context debug needs to switch between EL1 and EL2 context but it can
re-use its variables and function calls with a bit of clever naming.
Unify them to reduce #if-s.

Change-Id: I401667c4bc07938c99163c035dbed1101d986859
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cm): do not check for AArch32 support to enable features

EL3 is running in AArch64 mode so it is always able to enable all
features. Some features are not implemented for AArch32 but that will b

fix(cm): do not check for AArch32 support to enable features

EL3 is running in AArch64 mode so it is always able to enable all
features. Some features are not implemented for AArch32 but that will be
handled architecturally on changing exception levels so we don't need to
worry about it. Always enable all features (checked on the FEAT_STATE
flag of course) and save ourselves the check.

To prevent confusion in future, the SCR bits that actually affect
AArch32 execution and must always be checked are moved up in the file
and grouped together, rather than being straddled by feature enablement.

Change-Id: I154957405befb750c03738d1989bfb12696fc79d
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless t

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless trapped to EL2). IDTE3 allows EL3 to
modify the view of ID registers for lower ELs, and this capability
is used to disable fields of ID registers tied to disabled features.

The ID registers are initially read as-is and stored in context.
Then, based on the feature enablement status for each world, if a
particular feature is disabled, its corresponding field in the
cached ID register is set to Res0. When lower ELs attempt to read
an ID register, the cached ID register value is returned. This
allows EL3 to prevent lower ELs from accessing feature-specific
system registers that are disabled in EL3, even though the hardware
implements them.

The emulated ID register values are stored primarily in per-world
context, except for certain debug-related ID registers such as
ID_AA64DFR0_EL1 and ID_AA64DFR1_EL1, which are stored in the
cpu_data and are unique to each PE. This is done to support feature
asymmetry that is commonly seen in debug features.

FEAT_IDTE3 traps all Group 3 ID registers in the range
op0 == 3, op1 == 0, CRn == 0, CRm == {2–7}, op2 == {0–7} and the
Group 5 GMID_EL1 register. However, only a handful of ID registers
contain fields used to detect features enabled in EL3. Hence, we
only cache those ID registers, while the rest are transparently
returned as is to the lower EL.

This patch updates the CREATE_FEATURE_FUNCS macro to generate
update_feat_xyz_idreg_field() functions that disable ID register
fields on a per-feature basis. The enabled_worlds scope is used to
disable ID register fields for security states where the feature is
not enabled.

This EXPERIMENTAL feature is controlled by the ENABLE_FEAT_IDTE3
build flag and is currently disabled by default.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I5f998eeab81bb48c7595addc5595313a9ebb96d5

show more ...

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clear

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clears them
during early C runtime initialization. For platforms that enable
NUMA_AWARE_PER_CPU, invokes a platform hook to zero-initialize
node-specific per-cpu regions.

Cache maintenance: Extends the BL31 exit path to clean dcache lines
covering the per-cpu region, ensuring data written by the primary core
is visible to secondary cores.

tpidr_el3 setup: Initializes tpidr_el3 with the base address of the
current CPU’s per-cpu section. This allows per-cpu framework to
resolve local cpu accesses efficiently.

The percpu_data object is currently stored in tpidr_el3. Since the
per-cpu framework will use tpidr_el3 for this-cpu access, percpu_data
must be migrated to avoid conflict. This commit moves percpu_data to
the per-cpu framework.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: Iff0c2e1f8c0ebd25c4bb0b09bfe15dd4fbe20561

show more ...

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never writ

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never written to and so zeroes end up in registers, which is not
necessarily correct.

This patch gets BL1 to call cm_manage_extensions_per_world() whenever
BL2 runs in a lower EL. This allows the per-world registers to have the
reset values we intend. An accompanying call to
cm_manage_extensions_el3() is also added for completeness.

Doing this shows a small deficiency in cptr_el3 - bits TFP and TCPAC
change a lot. This patch makes them consistent by always setting TCPAC
and TFP to 0 which unconditionally enable access to CPTR_EL2 and FPCR by
default as they are always accessible. Other places that manipulate the
TFP bit are removed.

A nice side effect of all of this is that we're now in a position to
enable and use any architectural extension in BL2.

Change-Id: I070d62bbf8e9d9b472caf7e2c931c303523be308
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(context-mgmt): correct guards for PIE and POE registers" into integration

Merge changes from topic "bk/simpler_panic" into integration

* changes:
fix(aarch64): do not print EL1 registers on EL3 panic
refactor(el3-runtime): streamline cpu_data assembly offsets using th

Merge changes from topic "bk/simpler_panic" into integration

* changes:
fix(aarch64): do not print EL1 registers on EL3 panic
refactor(el3-runtime): streamline cpu_data assembly offsets using the cpu_ops template

show more ...

fix(context-mgmt): correct guards for PIE and POE registers

At the moment access to the POR_EL2 system register is guarded by
either FEAT_S1POE or FEAT_S2POE being implemented. However this
register

fix(context-mgmt): correct guards for PIE and POE registers

At the moment access to the POR_EL2 system register is guarded by
either FEAT_S1POE or FEAT_S2POE being implemented. However this
register exists only with FEAT_S1POE, and the two features are
independent from each other.

Similarly access to PIRE0_EL2 and PIR_EL2 is guarded by either
FEAT_S1PIE or FEAT_S2PIE yet these only exist with FEAT_S1PIE, with the
two features also independent of each other.

Change the guards to only look at FEAT_S1PIE/FEAT_S1POE, to fix crashes
when FEAT_S2PIE/FEAT_S2POE is implemented, but FEAT_S1PIE/FEAT_S1POE is
not.

Signed-off-by: Agathiyan Bragadeesh <agathiyan.bragadeesh2@arm.com>
Change-Id: I5d145c0f1b388e7bdade94920b6bf5f2d7eebfcd

show more ...

Merge "fix(cm): correctly restore BL2's context" into integration

feat(cpufeat): enable FEAT_PFAR support

Implement support for FEAT_PFAR, which introduces the PFAR_ELx system
register, recording the faulting physical address for some aborts.
Those system register

feat(cpufeat): enable FEAT_PFAR support

Implement support for FEAT_PFAR, which introduces the PFAR_ELx system
register, recording the faulting physical address for some aborts.
Those system registers are trapped by the SCR_EL3.PFARen bit, so set the
bit for the non-secure world context to allow OSes to use the feature.

This is controlled by the ENABLE_FEAT_PFAR build flag, which follows the
usual semantics of 2 meaning the feature being runtime detected.
Let the default for this flag be 0, but set it to 2 for the FVP.

Change-Id: I5c9ae750417e75792f693732df3869e02b6e4319
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

Merge "feat(cpufeat): enable FEAT_AIE support" into integration

fix(cm): correctly restore BL2's context

We have a single context management library but two BLs that use it very
differently - BL1 and BL31. Configuring it correctly is quite tricky as
in the BL1 +

fix(cm): correctly restore BL2's context

We have a single context management library but two BLs that use it very
differently - BL1 and BL31. Configuring it correctly is quite tricky as
in the BL1 + BL2 + BL31 + spmd case we have it operate on both EL1 and
EL2 context (for BL1 and BL31 respectively).

Welp, we've got that wrong - BL1 will skip EL1's register initialisation
leaving it at the mercy of UNKNOWN registers' values. If they aren't as
we expect, things don't go well.

This is not the end of the world as BL1 is expected to be the first
thing to run and UNKNOWN values generally reset to things we find
acceptable. But add the correct BL1 carveouts so they are set up
correctly.

Change-Id: Ia0d5522ac0c9f4616dd124b10f0c2fdc823a0d87
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(cm): deprecate use of NS_TIMER_SWITCH" into integration

feat(cpufeat): enable FEAT_AIE support

Implement support for FEAT_AIE, which introduces the AMAIR2_ELx and
MAIR2_ELx system registers, extending the memory attributes described
by {A}MAIR_ELx.
Those

feat(cpufeat): enable FEAT_AIE support

Implement support for FEAT_AIE, which introduces the AMAIR2_ELx and
MAIR2_ELx system registers, extending the memory attributes described
by {A}MAIR_ELx.
Those system registers are trapped by the SCR_EL3.AIEn bit, so set the
bit for the non-secure world context to allow OSes to use the feature.

This is controlled by the ENABLE_FEAT_AIE build flag, which follows the
usual semantics of 2 meaning the feature being runtime detected.
Let the default for this flag be 0, but set it to 2 for the FVP.

Change-Id: Iba2011719013a89f9cb3a4317bde18254f45cd25
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

Merge "fix(cm): remove unused macro" into integration

Merge "refactor(el3-runtime): move context security states to context.h" into integration