feat(psci): remove cpu context init by index

Currently, the calling core (meaning the core which received the call to
CPU_ON or the powerdown path of CPU_SUSPEND on the same core) is in
charge of in

feat(psci): remove cpu context init by index

Currently, the calling core (meaning the core which received the call to
CPU_ON or the powerdown path of CPU_SUSPEND on the same core) is in
charge of initialising the context for the waking core (the warmboot
entrypoint for both). This is convenient because the calling core can
write the context while in coherency and the waking core will only need
the context after its entered coherency. This avoids any cache
maintenance and makes communication simple.

However, this has 3 main problems:
a) asymmetric feature support is problematic - the calling core has no
way of knowing the feature set of the waking core. If the two
diverge, the architectural feature discovery via ID registers breaks
down. We've thus far "fixed" this on a case by case basis which
doesn't scale and introduces redundancy.

b) powerdown abandon (pabandon) introduces a contradiction - the calling
core has to initialise the context for when the core wakes up, but
should the core not powerdown it needs its old context intact. The only
way to work around this is by keeping two copies of context which
incurs a runtime and memory overhead.

c) cm_prepare_el3_exit[_ns]() doesn't have access to the entrypoint but needs
it to make initialisation decisions. We can infer some of this from
registers that have already been written but this is awkwardly
limiting for what we can do. This also necessitates the split from
the context initialisation.

We can solve all three by a making a core be in full ownership of its
own context. The calling core then only writes entrypoint information
and nothing else. The waking core then initialises its own context as it
sees fit with full knowledge of the whole picture.

The only tricky bit is cache coherency - the waking core has to be able
to coherently observe its new entrypoint. Calling cores will write to
the shared region with coherent caches on. If we make sure to read the
context only after the waking core has entered coherency, then we can
avoid cache operations and let hardware handle everything.

We can skip the spsr check for FEAT_TCR2 as it doesn't make a
difference. We can also skip enabling it twice from generic code.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I86e7fe8b698191fc3b469e5ced1fd010f8754b0e

show more ...

fix(build): enable fp during fp save/restore

Newer compilers such as clang/LLVM 19 flag uses of floating point
instructions when the architecture does not allow for it. We can
temporarily enable the

fix(build): enable fp during fp save/restore

Newer compilers such as clang/LLVM 19 flag uses of floating point
instructions when the architecture does not allow for it. We can
temporarily enable the use of floating point operations where it
it is safe and necessary for the build to succeed.

Change-Id: I1a832f846915c35792684906c94aef81c1f72d63
Signed-off-by: Andrei Homescu <ahomescu@google.com>
Signed-off-by: Per Larsen <perlarsen@google.com>

show more ...

Merge changes from topic "xlnx_fix_gen_datatype_cast" into integration

* changes:
fix(psci): add const qualifier
fix(el3-runtime): add const qualifier
fix(bl31): add const qualifier
fix(cons

Merge changes from topic "xlnx_fix_gen_datatype_cast" into integration

* changes:
fix(psci): add const qualifier
fix(el3-runtime): add const qualifier
fix(bl31): add const qualifier
fix(console): typecast expressions to match data type
fix(arm-drivers): typecast expressions to match data type
fix(arm-drivers): align essential type categories
fix(arm-drivers): typecast expression to match data type

show more ...

Merge "fix(errata): workaround for Cortex-A510 erratum 2971420" into integration

fix(errata): workaround for Cortex-A510 erratum 2971420

Cortex-A510 erratum 2971420 applies to revisions r0p1, r0p2, r0p3,
r1p0, r1p1, r1p2 and r1p3, and is still open.

Under some conditions, data

fix(errata): workaround for Cortex-A510 erratum 2971420

Cortex-A510 erratum 2971420 applies to revisions r0p1, r0p2, r0p3,
r1p0, r1p1, r1p2 and r1p3, and is still open.

Under some conditions, data might be corrupted if Trace Buffer
Extension (TRBE) is enabled. The workaround is to disable trace
collection via TRBE by programming MDCR_EL3.NSTB[1] to the opposite
value of SCR_EL3.NS on a security state switch. Since we only enable
TRBE for non-secure world, the workaround is to disable TRBE by
setting the NSTB field to 00 so accesses are trapped to EL3 and
secure state owns the buffer.

SDEN: https://developer.arm.com/documentation/SDEN-1873361/latest/

Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Ia77051f6b64c726a8c50596c78f220d323ab7d97

show more ...

fix(cpufeat): add feat_hcx check before enabling FEAT_MOPS

This patch also checks for FEAT_HCX before enabling FEAT_MOPS
when INIT_UNUSED_NS_EL1 = 1 and adds build dependency check.

Signed-off-by:

fix(cpufeat): add feat_hcx check before enabling FEAT_MOPS

This patch also checks for FEAT_HCX before enabling FEAT_MOPS
when INIT_UNUSED_NS_EL1 = 1 and adds build dependency check.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: Iff4a068aa392fc8d29e2e4da7a2e7df0b3104e65

show more ...

Merge "feat(rmmd): add FEAT_MEC support" into integration

feat(rmmd): add FEAT_MEC support

This patch provides architectural support for further use of
Memory Encryption Contexts (MEC) by declaring the necessary
registers, bits, masks, helpers and values a

feat(rmmd): add FEAT_MEC support

This patch provides architectural support for further use of
Memory Encryption Contexts (MEC) by declaring the necessary
registers, bits, masks, helpers and values and modifying the
necessary registers to enable FEAT_MEC.

Signed-off-by: Tushar Khandelwal <tushar.khandelwal@arm.com>
Signed-off-by: Juan Pablo Conde <juanpablo.conde@arm.com>
Change-Id: I670dbfcef46e131dcbf3a0b927467ebf6f438fa4

show more ...

fix(el3-runtime): add const qualifier

This corrects the MISRA violation C2012-8.13:
A pointer should point to a const-qualified type whenever possible.
Added const qualifier to pointer in the functi

fix(el3-runtime): add const qualifier

This corrects the MISRA violation C2012-8.13:
A pointer should point to a const-qualified type whenever possible.
Added const qualifier to pointer in the function arguments.

Change-Id: Idf4b8ea7842304849242a06d6ada73f11afc8cde
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

fix(el3-runtime): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement

fix(el3-runtime): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body within the curly braces.

Change-Id: I14a69f79aba98e243fa29a50914431358efa2a49
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context

SVE and SME aren't enabled symmetrically for all worlds, but EL3 needs
to context switch them nonetheless. Previously,

perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context

SVE and SME aren't enabled symmetrically for all worlds, but EL3 needs
to context switch them nonetheless. Previously, this had to happen by
writing the enable bits just before reading/writing the relevant
context. But since the introduction of root context, this need not be
the case. We can have these enables always be present for EL3 and save
on some work (and ISBs!) on every context switch.

We can also hoist ZCR_EL3 to a never changing register, as we set its
value to be identical for every world, which happens to be the one we
want for EL3 too.

Change-Id: I3d950e72049a298008205ba32f230d5a5c02f8b0
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

perf(amu): greatly simplify AMU context management

The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness

perf(amu): greatly simplify AMU context management

The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness in that this is rather slow. A large part of it is written in
assembly, making it opaque to the compiler for optimisations. The
future proofness requires reading registers that are effectively
`volatile`, making it even harder for the compiler, as well as adding
lots of implicit barriers, making it hard for the microarchitecutre to
optimise as well.

We can make a few assumptions, checked by a few well placed asserts, and
remove a lot of this burden. For a start, at the moment there are 4
group 0 counters with static assignments. Contexting them is a trivial
affair that doesn't need a loop. Similarly, there can only be up to 16
group 1 counters. Contexting them is a bit harder, but we can do with a
single branch with a falling through switch. If/when both of these
change, we have a pair of asserts and the feature detection mechanism to
guard us against pretending that we support something we don't.

We can drop contexting of the offset registers. They are fully
accessible by EL2 and as such are its responsibility to preserve on
powerdown.

Another small thing we can do, is pass the core_pos into the hook.
The caller already knows which core we're running on, we don't need to
call this non-trivial function again.

Finally, knowing this, we don't really need the auxiliary AMUs to be
described by the device tree. Linux doesn't care at the moment, and any
information we need for EL3 can be neatly placed in a simple array.

All of this, combined with lifting the actual saving out of assembly,
reduces the instructions to save the context from 180 to 40, including a
lot fewer branches. The code is also much shorter and easier to read.

Also propagate to aarch32 so that the two don't diverge too much.

Change-Id: Ib62e6e9ba5be7fb9fb8965c8eee148d5598a5361
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the i

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the init_cpu_data_ptr function
perf(cpus): inline the reset function
perf(cpus): inline the cpu_get_rev_var call
perf(cpus): inline cpu_rev_var checks
refactor(cpus): register DSU errata with the errata framework's wrappers
refactor(cpus): convert checker functions to standard helpers
refactor(cpus): convert the Cortex-A65 to use the errata framework
fix(cpus): declare reset errata correctly

show more ...

perf(cpus): inline the init_cpu_data_ptr function

Similar to the reset function inline, inline this too to not do a costly
branch with no extra cost.

Change-Id: I54cc399e570e9d0f373ae13c7224d32dbdf

perf(cpus): inline the init_cpu_data_ptr function

Similar to the reset function inline, inline this too to not do a costly
branch with no extra cost.

Change-Id: I54cc399e570e9d0f373ae13c7224d32dbdfae1e5
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(rmm): add support for BRBCR_EL2 register for feat_brbe

Currently BRBE is being disabled for Realm world in EL3 by
switching the SBRBE bit in mdcr_el3 register to 0b00.
The patch removes the swit

fix(rmm): add support for BRBCR_EL2 register for feat_brbe

Currently BRBE is being disabled for Realm world in EL3 by
switching the SBRBE bit in mdcr_el3 register to 0b00.
The patch removes the switching of SBRBE bits, and adds
context switch of BRBCR_EL2 register.

Change-Id: I66ca13edefc37e40fa265fd438b0b66f7d09b4bb
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>

show more ...

Merge "fix(simd): fix base register in fpregs_context_*" into integration

refactor(cpufeat): add FGT2 and Debugv8p9 to realm state

Enable FEAT_FGT2 and FEAT_Debugv8p9 in Realm state as well.

Change-Id: Ib9cdde3af328ffdd8718b1ba404265757f2e542b
Signed-off-by: Sona Mathew

refactor(cpufeat): add FGT2 and Debugv8p9 to realm state

Enable FEAT_FGT2 and FEAT_Debugv8p9 in Realm state as well.

Change-Id: Ib9cdde3af328ffdd8718b1ba404265757f2e542b
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>

show more ...

fix(cpus): workaround for accessing ICH_VMCR_EL2

When ICH_VMCR_EL2.VBPR1 is written in Secure state (SCR_EL3.NS==0)
and then subsequently read in Non-secure state (SCR_EL3.NS==1), a
wrong value migh

fix(cpus): workaround for accessing ICH_VMCR_EL2

When ICH_VMCR_EL2.VBPR1 is written in Secure state (SCR_EL3.NS==0)
and then subsequently read in Non-secure state (SCR_EL3.NS==1), a
wrong value might be returned. The same issue exists in the opposite way.

Adding workaround in EL3 software that performs context save/restore
on a change of Security state to use a value of SCR_EL3.NS when
accessing ICH_VMCR_EL2 that reflects the Security state that owns the
data being saved or restored. For example, EL3 software should set
SCR_EL3.NS to 1 when saving or restoring the value ICH_VMCR_EL2 for
Non-secure(or Realm) state. EL3 software should clear
SCR_EL3.NS to 0 when saving or restoring the value ICH_VMCR_EL2 for
Secure state.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1775101/latest/

Change-Id: I9f0403601c6346276e925f02eab55908b009d957
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

fix(simd): fix base register in fpregs_context_*

The fpregs_state_* macros require the base register
to point to the start of the simd_regs_t structure.
The fpregs_context_* functions were passing t

fix(simd): fix base register in fpregs_context_*

The fpregs_state_* macros require the base register
to point to the start of the simd_regs_t structure.
The fpregs_context_* functions were passing the
address incorrectly shifted by 512 bytes.

Signed-off-by: Andrei Homescu <ahomescu@google.com>
Change-Id: I757a26f8910c2ab648116e001e06baa3deb2eec4

show more ...

Merge "feat(pmuv3): setup per world MDCR_EL3" into integration

feat(pmuv3): setup per world MDCR_EL3

MDCR_EL3 register will context switch across all worlds. Thus the pmuv3
init has to be part of context management initialization.

Change-Id: I10ef7a3071c0fc5c1

feat(pmuv3): setup per world MDCR_EL3

MDCR_EL3 register will context switch across all worlds. Thus the pmuv3
init has to be part of context management initialization.

Change-Id: I10ef7a3071c0fc5c11a93d3c9c2a95ec8c6493bf
Signed-off-by: Mateusz Sulimowicz <matsul@google.com>

show more ...

feat(mops): enable FEAT_MOPS in EL3 when INIT_UNUSED_NS_EL2=1

FEAT_MOPS, mandatory from Arm v8.8, is typically managed in EL2.
However, in configurations where NS_EL2 is not enabled,
EL3 must set th

feat(mops): enable FEAT_MOPS in EL3 when INIT_UNUSED_NS_EL2=1

FEAT_MOPS, mandatory from Arm v8.8, is typically managed in EL2.
However, in configurations where NS_EL2 is not enabled,
EL3 must set the HCRX_EL2.MSCEn bit to 1 to enable the feature.

This patch ensures FEAT_MOPS is enabled by setting HCRX_EL2.MSCEn to 1.

Change-Id: Ic4960e0cc14a44279156b79ded50de475b3b21c5
Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>

show more ...

refactor(cm): clean up per-world context

In preparation for SMCCC_ARCH_FEATURE_AVAILABILITY, it is useful for
context to be directly related to the underlying system. Currently,
certain bits like SC

refactor(cm): clean up per-world context

In preparation for SMCCC_ARCH_FEATURE_AVAILABILITY, it is useful for
context to be directly related to the underlying system. Currently,
certain bits like SCR_EL3.APK are always set with the understanding that
they will only take effect if the feature is present.

However, that is problematic for SMCCC_ARCH_FEATURE_AVAILABILITY (an
SMCCC call to report which features firmware enables), as simply reading
the enable bit may contradict the ID register, like the APK bit above
for a system with no Pauth present.

This patch is to clean up these cases. Add a check for PAuth's presence
so that the APK bit remains unset if not present. Also move SPE and TRBE
enablement to only the NS context. They already only enable the features
for NS only and disable them for Secure and Realm worlds. This change
only makes these worlds' context read 0 for easy bitmasking.

There's only a single snag on SPE and TRBE. Currently, their fields have
the same values and any world asymmetry is handled by hardware. Since we
don't want to do that, the buffers' ownership will change if we just set
the fields to 0 for non-NS worlds. Doing that, however, exposes Secure
state to a potential denial of service attack - a malicious NS can
enable profiling and call an SMC. Then, the owning security state will
change and since no SPE/TRBE registers are contexted, Secure state will
start generating records. Always have NS world own the buffers to
prevent this.

Finally, get rid of manage_extensions_common() as it's just a level of
indirection to enable a single feature.

Change-Id: I487bd4c70ac3e2105583917a0e5499e0ee248ed9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "bk/smccc_feature" into integration

* changes:
fix(trbe): add a tsb before context switching
fix(spe): add a psb before updating context and remove context saving

fix(cm): fix context management SYSREG128 write macros

This patch fixes a bug which was introduced in commit
3065513 related to improper saving of EL1 context in the
context management library code

fix(cm): fix context management SYSREG128 write macros

This patch fixes a bug which was introduced in commit
3065513 related to improper saving of EL1 context in the
context management library code when using 128-bit
system registers.

Bug explanation:
The function el1_sysregs_context_save still used the normal
macros that read all the system registers related to the EL1
context, which then involved casting them to uint64_t and
eventually writing them to a memory structure. This means that
the context management library was saving EL1-related SYSREG128
registers with the upper 64 bits zeroed out.

Alternative macros had previously been introduced for the EL2
context in the aforementioned commit, but not for EL1.

Some refactoring has also been done as part of this patch:
- Re-added "common" back to write_el2_ctx_common_sysreg128
- Added dummy SYSREG128 macros for cases when some features
are disabled
- Removed some newlines

Change-Id: I15aa2190794ac099a493e5f430220b1c81e1b558
Signed-off-by: Igor Podgainõi <igor.podgainoi@arm.com>

show more ...