feat(rmmd): replace ENABLE_RME with ENABLE_RMM

RME architectural requirements are now handled under the feature
detection option ENABLE_FEAT_RME. However, the existing ENABLE_RME build
option perfor

feat(rmmd): replace ENABLE_RME with ENABLE_RMM

RME architectural requirements are now handled under the feature
detection option ENABLE_FEAT_RME. However, the existing ENABLE_RME build
option performs RMM-specific tasks such as GPT setup, loading the RMM,
and enabling RMMD support.

Since ENABLE_RME now only controls RMM-related functionality, rename it
to ENABLE_RMM to better reflect its purpose and avoid confusion with
ENABLE_FEAT_RME.

For backward compatibility, setting the legacy ENABLE_RME=1 (until it is
deprecated) will automatically enable both ENABLE_FEAT_RME and
ENABLE_RMM.

Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>
Change-Id: Iac945bdffe5002161bf1161b81a5aa7abec68192

show more ...

fix(cm): initialise TCR2_EL2 and HAFGRTR_EL2 for compatibility

Older EL2 implementations that do not implement support for these
features but are run on hardware that has them will still experience

fix(cm): initialise TCR2_EL2 and HAFGRTR_EL2 for compatibility

Older EL2 implementations that do not implement support for these
features but are run on hardware that has them will still experience the
effects of these registers. When EL2 is the highest EL in the system,
they reset to safe working values (all 0) but when EL3 is highest it is
EL3's responsibility to do this.

Change-Id: Id23fe0b696b7f434de9522db447222d3f6af1c2d
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cm): make the REG_INIT_VAL init relationship explicit

For compatibility of older software, there is a short list of registers
that must get initialised to a sane value. These have to be wri

refactor(cm): make the REG_INIT_VAL init relationship explicit

For compatibility of older software, there is a short list of registers
that must get initialised to a sane value. These have to be written
either in context or in place. However, the actual code is spread very
far apart with no references to itself.

This patch fixes that by carving out setup_el2_regs() and putting it
right after the equivalent portion of setup_el2_context() and leaving a
recursive reference with a note to keep them in sync.

Another small thing this patch does is make the 2 pairs mutually
exclusive - if CTX_INCLUDE_EL2_REGS is in effect then the context values
will be written at the end and the in place writes are redundant.

Finally, the part where the EL2 context is used will inherit the zeroing
of context. As such, there is no need to write 0 to registers
individually. So drop that code and leave a comment so they aren't
missed when sycing up the 2 functions.

Change-Id: I1d347c129cf96a4ab1da3417d1ad2facc27eea05
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cm): propagate EL2 register compatibility init to all worlds

Currently, there is a short list of EL2 registers that are given known
safe values to aid running software without support for the l

feat(cm): propagate EL2 register compatibility init to all worlds

Currently, there is a short list of EL2 registers that are given known
safe values to aid running software without support for the latest
features on later than intended hardware. This is only done for NS
world, but the same problems would exist for other worlds too. So
propagate the init there too.

Also coalesce all EL2 context init just like we we do for EL1. Finally,
update the comments to reduce duplication and make them more concise.

Change-Id: I4a30850c6c9bba11c9b42f1aea46594be629fb8b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cm): always call setup_el1_context()

This patch simplifies the web of conditional compilation.

First always call setup_el1_context() but have the body of the function
conditionally compile

refactor(cm): always call setup_el1_context()

This patch simplifies the web of conditional compilation.

First always call setup_el1_context() but have the body of the function
conditionally compiled out instead of the caller - this unifies the
conditions.

Then, call setup_el1_context() from setup_context_common() - this
removes a call site. This does add the call for realm context. However,
FEAT_RME requires CTX_INCLUDE_EL2_REGS so this code path will never be
encountered in practice.

Change-Id: I25dcc8ad0a086b14e2eeadddb45b8405086cd3ed
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cm): don't context switch GICv3 registers on NS<->RL transitions

The GICv3 is architectured to solely manage interrupts targeted to
Normal and Secure world. It doesn't manage interrupts targetin

fix(cm): don't context switch GICv3 registers on NS<->RL transitions

The GICv3 is architectured to solely manage interrupts targeted to
Normal and Secure world. It doesn't manage interrupts targeting the
more recently introduced Realm world. Hence the new RMMv2.0
specification mandates that EL3 should not save and restore
the GIC registers on a world switch. This change is not backward
compatible with RMMv1.x ABI.

Note the change in implementation of cm_el2_sysregs_context_save()
and cm_el2_sysregs_context_restore() API as GIC state is not
managed by these APIs anymore.

Add new build flag RMM_V1_COMPAT to support backward compatibility
with RMMv1.x. This flag is currently enabled by default.

This patch is a reworked version of the original patch at:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/45658

NOTE: If RMM_V1_COMPAT is not enabled, then RMM_EL3_IFC_VERSION
is bumped to 1.0 which makes it incompatible with an RMM supporting
0.x.

Change-Id: If4c53b85ef31091c254b383ed7b32c39124f0dbb
Signed-off-by: Shruti Gupta <shruti.gupta@arm.com>

show more ...

revert(cm): don't context switch GICv3 registers on NS<->RL transitions

This reverts commit c84cf19308299de9ad68c340a4c4744a0fe2f18a.

Reason: Linux Boot in Realm test fails with this patch because

revert(cm): don't context switch GICv3 registers on NS<->RL transitions

This reverts commit c84cf19308299de9ad68c340a4c4744a0fe2f18a.

Reason: Linux Boot in Realm test fails with this patch because the
NS sets up ich_hcr_el2 which is not expected by RMMv1.0. The GICv3
context switch changes will need to wait till RMM migrates to
RMMv2.0.

Change-Id: I39185bd08f35bc0836d2ef199c5d930d62ee23d2
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

show more ...

fix(cm): don't context switch GICv3 registers on NS<->RL transitions

The GICv3 is architectured to solely manage interrupts targeted to
Normal and Secure world. It doesn't manage interrupts targetin

fix(cm): don't context switch GICv3 registers on NS<->RL transitions

The GICv3 is architectured to solely manage interrupts targeted to
Normal and Secure world. It doesn't manage interrupts targeting the
more recently introduced Realm world. Hence the new RMMv2.0
specification mandates that EL3 should not save and restore
the GIC registers on a world switch. This change is backward
compatible with RMMv1.x ABI.

NOTE: Change in implementation of cm_el2_sysregs_context_save()
and cm_el2_sysregs_context_restore() API as GIC state is not
managed by these APIs anymore.

Change-Id: I24d7fa26503ffad9d9fede21d8449f481e32984e
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Signed-off-by: Shruti Gupta <shruti.gupta@arm.com>

show more ...

feat(cpufeat): add support for FEAT_HACDBS

The Hardware accelerator for cleaning Dirty state feature also has two
register just like FEAT_HDBSS. They are guarded by a SCR_EL3 bit which
set for NS wo

feat(cpufeat): add support for FEAT_HACDBS

The Hardware accelerator for cleaning Dirty state feature also has two
register just like FEAT_HDBSS. They are guarded by a SCR_EL3 bit which
set for NS world only and are not context switched as a result. There is
no use for this feature at EL3.

Change-Id: Ica7a312d891a1671df8e9f2adbfe464d96bbcd4d
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_HDBSS

The Hardware Dirty state tracking structure feature has two registers to
enable tracking at lower ELs which are guarded by an SCR_EL3 bit. Set
that bit for

feat(cpufeat): add support for FEAT_HDBSS

The Hardware Dirty state tracking structure feature has two registers to
enable tracking at lower ELs which are guarded by an SCR_EL3 bit. Set
that bit for NS only and do not context switch the registers. There is
no use of the feature at EL3.

Change-Id: I174a256d70a99abfafc65eed3a2fbdaea5ea946d
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_STEP2

This feature only needs MDCR_EL3.EnSTEPOP to be written and mdstepop_el1
to be context switched when the next EL is EL1.

Change-Id: I70e2a488f4e50da4b181a0

feat(cpufeat): add support for FEAT_STEP2

This feature only needs MDCR_EL3.EnSTEPOP to be written and mdstepop_el1
to be context switched when the next EL is EL1.

Change-Id: I70e2a488f4e50da4b181a00648c4f608e1da451c
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(context-mgmt): actually clear MDCR_EL3 bits" into integration

fix(morello): avoid capability tag fault on data access

TF-A runtime service at EL3 switches the stack pointer from SP_EL3
to SP_EL0. This creates a capability tag fault when the DDC_EL0 is
zeroed o

fix(morello): avoid capability tag fault on data access

TF-A runtime service at EL3 switches the stack pointer from SP_EL3
to SP_EL0. This creates a capability tag fault when the DDC_EL0 is
zeroed out (purecap user space) as any data accesses computes
tag/permission with DDC_EL0 value when SpSel is 0 and when EL3 is
in hybrid mode.

As a workaround, this patch creates a per cpu context variable
to store DDC_EL0 value so that when EL3 runtime is entered DDC_EL0
is saved on to stack. DDC_EL3 is then copied into DDC_EL0 after
switching SP to SP_EL0. Once the runtime finishes, during el3_exit,
the saved DDC_EL0 is restored from stack.

Signed-off-by: Selvarasu Ganesan <selvarasu.ganesan@arm.com>
Signed-off-by: Manoj Kumar <manoj.kumar3@arm.com>
Signed-off-by: Varshit Pandya <varshit.pandya@arm.com>
Change-Id: I4e4010f0e20913cb4e35b58fb49a177bdf26feb1

show more ...

feat(morello): add Morello capability enablement changes

This patch adds a build macro ENABLE_FEAT_MORELLO which when set will
compile BL31 firmware with changes required to boot capability
aware so

feat(morello): add Morello capability enablement changes

This patch adds a build macro ENABLE_FEAT_MORELLO which when set will
compile BL31 firmware with changes required to boot capability
aware software.

It also adds helper function in c and assmbly to check if morello
hardware is present and if morello capability is enabled or not.

CE field, bits [23:20] in ID_AA64PFR1_EL1 defines whether morello
architecture is present or not, 0b0000 indicates that it is absent
and 0b0001 indicates that it is present. While whether capabilities
are enabled or not is decided at runtime with ENABLE_FEAT_MORELLO build
option.

Reference: https://developer.arm.com/documentation/ddi0606/latest/

Signed-off-by: Manoj Kumar <manoj.kumar3@arm.com>
Signed-off-by: Varshit Pandya <varshit.pandya@arm.com>
Change-Id: Ib16877acbfcb72c4bd8c08e97e44edc0a3e46089

show more ...

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to their low
resolution and general availability of the data elsewhere. As such, they
are used for critical performance tuning and are expected to never be
turned off or context switched when switching worlds.

The upper 2 counters, INST_RETIRED and STALL_BACKEND_MEM, are different.
The data they provide is non-critical and expose new information that
could be used as a timing side channel, especially of Secure world. This
patch adds context switching of these two counters to prevent any such
side channel.

This is not done for group 1 auxiliary counters as those are IMP DEF and
are inaccessible by default unless overriden by the platform (with
AMU_RESTRICT_COUNTERS).

Change-Id: Ib4b946abb810e36736cabb9b84cd837308b4e761
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(context-mgmt): actually clear MDCR_EL3 bits

When setting up MDCR_EL3 for a given context, we need to set some bits,
but also clear some other bits. This was done in a single statement,
but using

fix(context-mgmt): actually clear MDCR_EL3 bits

When setting up MDCR_EL3 for a given context, we need to set some bits,
but also clear some other bits. This was done in a single statement,
but using the C "|=" operator, which would never clear any bits in the
left-hand side.

Split this into two statements, one for setting, the other for clearing
bits.

It seems that on the FVP the bits to clear already reset to 0, so this
never caused any issues so far, but the architecture declares those bits
as: "this field resets to an architecturally UNKNOWN value".

Change-Id: Id1e9e4c010167af2ea3d5820532704220aa7c647
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(el3-runtime): translate EL3 handled exceptions to C and always call prepare_el3_entry

Exception handling in BL31 is tricky business and to satisfy the varying
requirements of the different code

feat(el3-runtime): translate EL3 handled exceptions to C and always call prepare_el3_entry

Exception handling in BL31 is tricky business and to satisfy the varying
requirements of the different code paths it has thus far largely been
written in assembly. However, assembly is extremely tedious to read and
modify. Similar to context management, it is desirable to have as much
as possible in C. C code is generally easier to follow and can enable
the compiler to do more optimisations on surrounding code.

Most exceptions that BL31 deals with are the synchronous exceptions and
those are processed within BL31. They already get prepared for EL3 entry
and after the initial dispatch end up in C. So the dispatch can also be
converted in C. Interrupt exceptions are very similar so are converted
too. Finally, asynchronous external aborts share some code with
synchronous external aborts and may end up being processed deeper in
BL31. So they can safely be prepared for EL3 entry too and converted to
C so that they can share code properly.

The IMP DEF exceptions are not part of this refactor as their speed may
be important. There is currently little that uses them, but they can be
converted to C too once their use expands and usage allows it.

This refactor allows to expand the responsibilities of
prepare_el3_entry(). Its role is already to prepare context for
executing within EL3 but with this patch EL3 execution is synonymous
with C runtime execution. So it's given the responsibility of saving
spsr and elr as well as putting the runtime stack in.

When a synchronous exception happens, the only possible paths are to
enter the C EL3 runtime, exiting via el3_exit(), or to panic. In the EL3
runtime case, we always need prepare_el3_entry() and the runtime stack,
whereas in the panic case, this doesn't matter as we will never return.
So hoist the prepare_el3_entry() call and the changing of the stacks as
early as possible and make the rest of the code agnostic of this.

This patch also gets rid of smc_prohibited. It is an optimisation by
skipping prepare_el3_entry() when a bad smc call happens. However, speed
doesn't matter in this case as this is an erroneous case.

Change-Id: I411af9d17ef4046a736b1f4f5f8fbc9c28e66106
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "refactor(cm): reduce conditional compilation" into integration

fix(cpufeat): enable FEAT_FGWTE3 after FEAT_CPA

FEAT_CPA needs to write SCTLR2_EL3 which will be forbidden after
initialising FEAT_FGWTE3. Correct the order.

Change-Id: I3a0554d2a73f773b3ad672eb1e4

fix(cpufeat): enable FEAT_FGWTE3 after FEAT_CPA

FEAT_CPA needs to write SCTLR2_EL3 which will be forbidden after
initialising FEAT_FGWTE3. Correct the order.

Change-Id: I3a0554d2a73f773b3ad672eb1e4b0db0171d38bd
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cm): reduce conditional compilation

Context debug needs to switch between EL1 and EL2 context but it can
re-use its variables and function calls with a bit of clever naming.
Unify them to r

refactor(cm): reduce conditional compilation

Context debug needs to switch between EL1 and EL2 context but it can
re-use its variables and function calls with a bit of clever naming.
Unify them to reduce #if-s.

Change-Id: I401667c4bc07938c99163c035dbed1101d986859
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(cm): do not check for AArch32 support to enable features

EL3 is running in AArch64 mode so it is always able to enable all
features. Some features are not implemented for AArch32 but that will b

fix(cm): do not check for AArch32 support to enable features

EL3 is running in AArch64 mode so it is always able to enable all
features. Some features are not implemented for AArch32 but that will be
handled architecturally on changing exception levels so we don't need to
worry about it. Always enable all features (checked on the FEAT_STATE
flag of course) and save ourselves the check.

To prevent confusion in future, the SCR bits that actually affect
AArch32 execution and must always be checked are moved up in the file
and grouped together, rather than being straddled by feature enablement.

Change-Id: I154957405befb750c03738d1989bfb12696fc79d
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless t

feat(cpufeat): add support for FEAT_IDTE3

This patch adds support for FEAT_IDTE3, which introduces support
for handling the trapping of Group 3 and Group 5 (only GMID_EL1)
registers to EL3 (unless trapped to EL2). IDTE3 allows EL3 to
modify the view of ID registers for lower ELs, and this capability
is used to disable fields of ID registers tied to disabled features.

The ID registers are initially read as-is and stored in context.
Then, based on the feature enablement status for each world, if a
particular feature is disabled, its corresponding field in the
cached ID register is set to Res0. When lower ELs attempt to read
an ID register, the cached ID register value is returned. This
allows EL3 to prevent lower ELs from accessing feature-specific
system registers that are disabled in EL3, even though the hardware
implements them.

The emulated ID register values are stored primarily in per-world
context, except for certain debug-related ID registers such as
ID_AA64DFR0_EL1 and ID_AA64DFR1_EL1, which are stored in the
cpu_data and are unique to each PE. This is done to support feature
asymmetry that is commonly seen in debug features.

FEAT_IDTE3 traps all Group 3 ID registers in the range
op0 == 3, op1 == 0, CRn == 0, CRm == {2–7}, op2 == {0–7} and the
Group 5 GMID_EL1 register. However, only a handful of ID registers
contain fields used to detect features enabled in EL3. Hence, we
only cache those ID registers, while the rest are transparently
returned as is to the lower EL.

This patch updates the CREATE_FEATURE_FUNCS macro to generate
update_feat_xyz_idreg_field() functions that disable ID register
fields on a per-feature basis. The enabled_worlds scope is used to
disable ID register fields for security states where the feature is
not enabled.

This EXPERIMENTAL feature is controlled by the ENABLE_FEAT_IDTE3
build flag and is currently disabled by default.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I5f998eeab81bb48c7595addc5595313a9ebb96d5

show more ...

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clear

feat(per-cpu): integrate per-cpu framework into BL31/BL32

Integrate per-cpu support into BL31/BL32 by extending the following
areas:

Zero-initialization: Treats per-cpu sections like .bss and clears them
during early C runtime initialization. For platforms that enable
NUMA_AWARE_PER_CPU, invokes a platform hook to zero-initialize
node-specific per-cpu regions.

Cache maintenance: Extends the BL31 exit path to clean dcache lines
covering the per-cpu region, ensuring data written by the primary core
is visible to secondary cores.

tpidr_el3 setup: Initializes tpidr_el3 with the base address of the
current CPU’s per-cpu section. This allows per-cpu framework to
resolve local cpu accesses efficiently.

The percpu_data object is currently stored in tpidr_el3. Since the
per-cpu framework will use tpidr_el3 for this-cpu access, percpu_data
must be migrated to avoid conflict. This commit moves percpu_data to
the per-cpu framework.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: Iff0c2e1f8c0ebd25c4bb0b09bfe15dd4fbe20561

show more ...

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never writ

fix(bl1): use per-world context correctly

Currently, the configuration with BL1 and BL2 at SEL1 will transition
via el3_exit which will restore per-world context. However, that context
is never written to and so zeroes end up in registers, which is not
necessarily correct.

This patch gets BL1 to call cm_manage_extensions_per_world() whenever
BL2 runs in a lower EL. This allows the per-world registers to have the
reset values we intend. An accompanying call to
cm_manage_extensions_el3() is also added for completeness.

Doing this shows a small deficiency in cptr_el3 - bits TFP and TCPAC
change a lot. This patch makes them consistent by always setting TCPAC
and TFP to 0 which unconditionally enable access to CPTR_EL2 and FPCR by
default as they are always accessible. Other places that manipulate the
TFP bit are removed.

A nice side effect of all of this is that we're now in a position to
enable and use any architectural extension in BL2.

Change-Id: I070d62bbf8e9d9b472caf7e2c931c303523be308
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(context-mgmt): correct guards for PIE and POE registers" into integration