feat(cpus): add support for Rosillo cpu

Add basic CPU library code to support Rosillo CPU

Change-Id: I0e11e511511562297e4dccd2745842ebcfa2bff4
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

Merge changes from topic "bk/simpler_panic" into integration

* changes:
refactor(aarch64): remove crash reporting's dependency on cpu_data
fix(el3-runtime): remove lower_el_panic()

fix(morello): avoid capability tag fault on data access

TF-A runtime service at EL3 switches the stack pointer from SP_EL3
to SP_EL0. This creates a capability tag fault when the DDC_EL0 is
zeroed o

fix(morello): avoid capability tag fault on data access

TF-A runtime service at EL3 switches the stack pointer from SP_EL3
to SP_EL0. This creates a capability tag fault when the DDC_EL0 is
zeroed out (purecap user space) as any data accesses computes
tag/permission with DDC_EL0 value when SpSel is 0 and when EL3 is
in hybrid mode.

As a workaround, this patch creates a per cpu context variable
to store DDC_EL0 value so that when EL3 runtime is entered DDC_EL0
is saved on to stack. DDC_EL3 is then copied into DDC_EL0 after
switching SP to SP_EL0. Once the runtime finishes, during el3_exit,
the saved DDC_EL0 is restored from stack.

Signed-off-by: Selvarasu Ganesan <selvarasu.ganesan@arm.com>
Signed-off-by: Manoj Kumar <manoj.kumar3@arm.com>
Signed-off-by: Varshit Pandya <varshit.pandya@arm.com>
Change-Id: I4e4010f0e20913cb4e35b58fb49a177bdf26feb1

show more ...

feat(morello): add capability load/store/track support to MMU

Morello architecture adds additional bits to TCR_EL3 and uses the
HWU bits of page/block descriptors to provision permission for
loading

feat(morello): add capability load/store/track support to MMU

Morello architecture adds additional bits to TCR_EL3 and uses the
HWU bits of page/block descriptors to provision permission for
loading, storing and tracking of valid capability tags.

This patch reserves bit 31 of the existing translation table
attribute field which can be used by the user to enable capability
load/store/track permission for a given memory region.

This patch also enables this permission for BL31 region.

Signed-off-by: Manoj Kumar <manoj.kumar3@arm.com>
Signed-off-by: Varshit Pandya <varshit.pandya@arm.com>
Change-Id: I1939c70aac3585969d74b0956529681e840d6f63

show more ...

Merge changes from topic "bk/amu_private" into integration

* changes:
fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds
fix(cpufeat): disable FEAT_AMU counters on conte

Merge changes from topic "bk/amu_private" into integration

* changes:
fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds
fix(cpufeat): disable FEAT_AMU counters on context restore
feat(per-cpu): migrate AArch32 amu_ctx to per-cpu framework

show more ...

refactor(aarch64): remove crash reporting's dependency on cpu_data

Crash reporting is useful as early as possible, even before most of the
runtime has been set up. This means that all of its depende

refactor(aarch64): remove crash reporting's dependency on cpu_data

Crash reporting is useful as early as possible, even before most of the
runtime has been set up. This means that all of its dependencies,
currently only cpu_data, must be set up as early as possible too. This
can be constraining as fiddling with the general EL3 runtime from the
early entrypoint is very difficult. So remove the cpu_data dependency.
Further benefits are that crash reporting will work even earlier (during
cpu reset functions!) and also in other BLs.

Change-Id: I92bb6b3921c6dec10560f8341b3bca5cdacfb492
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(el3-runtime): resolve essential-type mismatch

This resolves MISRA C:2012 Rule 10.7 violation
where a composite expression involved operands of differing
essential types, causing unintended impli

fix(el3-runtime): resolve essential-type mismatch

This resolves MISRA C:2012 Rule 10.7 violation
where a composite expression involved operands of differing
essential types, causing unintended implicit conversions.

The fix ensures all operands in the expression have matching
essential types by introducing explicit casts,
preventing unsafe or inconsistent arithmetic operations.

Change-Id: If01dfe78e7a5cffc8b0efa6ac969b262e236852b
Signed-off-by: Taticharla Venkata Sai <venkatasai.taticharla@amd.com>

show more ...

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to

fix(cpufeat): prevent FEAT_AMU counters 2 and 3 from counting across worlds

FEAT_AMU has 4 architected counters. The lower 2, CPU_CYCLES and
CNT_CYCLES, are not considered to be side channels due to their low
resolution and general availability of the data elsewhere. As such, they
are used for critical performance tuning and are expected to never be
turned off or context switched when switching worlds.

The upper 2 counters, INST_RETIRED and STALL_BACKEND_MEM, are different.
The data they provide is non-critical and expose new information that
could be used as a timing side channel, especially of Secure world. This
patch adds context switching of these two counters to prevent any such
side channel.

This is not done for group 1 auxiliary counters as those are IMP DEF and
are inaccessible by default unless overriden by the platform (with
AMU_RESTRICT_COUNTERS).

Change-Id: Ib4b946abb810e36736cabb9b84cd837308b4e761
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(psci): make CMOs target the whole psci_cpu_data_t

psci_cpu_data_t is tiny - on AArch64 it's 12 bytes. Cache maintenance
operations (CMOs) operate on cache lines which are much bigger - usua

refactor(psci): make CMOs target the whole psci_cpu_data_t

psci_cpu_data_t is tiny - on AArch64 it's 12 bytes. Cache maintenance
operations (CMOs) operate on cache lines which are much bigger - usually
64 bytes long. As such, issuing a cache clean for a member in the middle
of psci_cpu_data_t won't necessarily have the expected effect. The
member will be cleaned, sure, but so will the rest of the cache line
along with it. If the struct happens to straddle cache lines this will
lead to the next 52 bytes, most of which not belonging to
psci_cpu_data_t, being cleaned as well and the start of psci_cpu_data_t
not being cleaned at all.

This is not a problem because of the per-cpu (and cpu_data before it)
section - it is cache size aligned and all data within a single section
belongs to the same core so overdoing cache cleans won't have strange
side effects.

Regardless, this patch clarifies CMOs around psci_cpu_data_t by always
targeting the whole structure. To make sure there is never a situation
where it straddles cache lines and this causes weird side effect, its
alignment is set to the size of the structure to make sure it is always
on the same cache line.

Change-Id: I5d82ee6bb2ce0ed3c6a7e4abb7aa890f5e3bd0af
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

perf(cpus): reduce the footprint of errata reporting

Since the advent of spin_trylock() it's possible to combine the spinlock
with the errata_reported field. If the spinlock is only acquired with a

perf(cpus): reduce the footprint of errata reporting

Since the advent of spin_trylock() it's possible to combine the spinlock
with the errata_reported field. If the spinlock is only acquired with a
non-blocking call then a successful call means reporting should be done
and an unsuccessful one means that reporting would have been done by
whoever acquired it. This relies on the lock never being released which
this patch does. The effect is a smaller memory footprint and a smaller
runtime.

Change-Id: I215a84bd2c91e33703349c41fc59f654f7764b2f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes I1a57de22,If97ea5fd into integration

* changes:
feat(locks): make spin_trylock with exclusives spin until it knows the state of the lock
fix(locks): restore spin_trylock's ability

Merge changes I1a57de22,If97ea5fd into integration

* changes:
feat(locks): make spin_trylock with exclusives spin until it knows the state of the lock
fix(locks): restore spin_trylock's ability to acquire a lock

show more ...

Merge changes I411af9d1,I89813759 into integration

* changes:
feat(el3-runtime): translate EL3 handled exceptions to C and always call prepare_el3_entry
refactor(el3-runtime): factor out handler

Merge changes I411af9d1,I89813759 into integration

* changes:
feat(el3-runtime): translate EL3 handled exceptions to C and always call prepare_el3_entry
refactor(el3-runtime): factor out handler fetching code

show more ...

Merge changes I20c97011,Ia1facabb into integration

* changes:
fix(rk3576): shorten names to fit into the allocated space
fix(debugfs): allocate enough space to fit all names

Merge changes from topic "xl/x925-errata" into integration

* changes:
fix(cpus): workaround for Cortex-X925 erratum 3865185
fix(cpus): workaround for Cortex-X925 erratum 3730893
fix(cpus): wor

Merge changes from topic "xl/x925-errata" into integration

* changes:
fix(cpus): workaround for Cortex-X925 erratum 3865185
fix(cpus): workaround for Cortex-X925 erratum 3730893
fix(cpus): workaround for Cortex-X925 erratum 3692980
fix(cpus): workaround for Cortex-X925 erratum 3324334
fix(cpus): workaround for Cortex-X925 erratum 2933290
fix(cpus): workaround for Cortex-X925 erratum 2922378
fix(cpus): workaround for Cortex-X925 erratum 2921199

show more ...

fix(cpus): workaround for Cortex-X925 erratum 3865185

Cortex-X925 erratum 3865185 is a Cat B erratum that
applies to revisions r0p0 and r0p1, it is fixed in r0p2.

Load issued to Non-Cacheable or De

fix(cpus): workaround for Cortex-X925 erratum 3865185

Cortex-X925 erratum 3865185 is a Cat B erratum that
applies to revisions r0p0 and r0p1, it is fixed in r0p2.

Load issued to Non-Cacheable or Device GRE memory can
read stale data brought in by an earlier load to the
same cache-line thereby violating ordering requirements.
This erratum can be avoided by setting CPUACTLR2[22] to 1'b1,
which will disable linking multiple Non-Cacheable or Device
GRE loads to the same read request for the cache-line. This
might have a significant performance impact to Non-cacheable
and Device GRE read bandwidth for streaming scenarios.

SDEN documentation:
https://developer.arm.com/documentation/109180/latest/

Change-Id: Iff224ef82bd1cb9aff8d6b11451e2ac1d048149f
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

fix(cpus): workaround for Cortex-X925 erratum 3730893

Cortex-X925 erratum 3730893 is a Cat B erratum that
applies to revisions r0p0 and r0p1, it is fixed in r0p2.

PE executing a load instruction th

fix(cpus): workaround for Cortex-X925 erratum 3730893

Cortex-X925 erratum 3730893 is a Cat B erratum that
applies to revisions r0p0 and r0p1, it is fixed in r0p2.

PE executing a load instruction that accesses a memory
region which crosses a 4K boundary might cause a deadlock.
This erratum can be avoided by setting CPUACTLR_EL1[60:58]
to 3'b001, which has a small perf impact.

SDEN documentation:
https://developer.arm.com/documentation/109180/latest/

Change-Id: I0245183669255afb0d3ec71cafa058aa72129de0
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

fix(cpus): workaround for Cortex-X925 erratum 2922378

Cortex-X925 erratum 2922378 is a Cat B erratum that
applies to r0p0 and is fixed in r0p1.

Branch prediction history is not suppressed when swit

fix(cpus): workaround for Cortex-X925 erratum 2922378

Cortex-X925 erratum 2922378 is a Cat B erratum that
applies to r0p0 and is fixed in r0p1.

Branch prediction history is not suppressed when switching from low
to high EL, this erratum can be avoided by setting the CPUACTLR4[10]
to 1 and CPUACTLR4[11] to 1.

SDEN documentation:
https://developer.arm.com/documentation/109180/latest/

Change-Id: Ieb5fe278821d85382af60be25e9546e65ba9a629
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

Merge "feat(lib): add mmio_setbits_64 inline function" into integration

fix(cpus): workaround for Cortex-X925 erratum 2921199

Cortex-X925 erratum 2921199 is a Cat B erratum that
applies to r0p0 and is fixed in r0p1.

Under certain rare microarchitectural conditions, two

fix(cpus): workaround for Cortex-X925 erratum 2921199

Cortex-X925 erratum 2921199 is a Cat B erratum that
applies to r0p0 and is fixed in r0p1.

Under certain rare microarchitectural conditions, two or more STG
instructions that access the same cache line but different 32-bytes
might not write the MTE allocation tag to memory. This erratum can
be avoided by setting CPUACTLR5_EL1[14] to 1.

SDEN documentation:
https://developer.arm.com/documentation/109180/latest/

Change-Id: I8eb8bbdd6f99f69c8713400191ac66f55ffedc8b
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

feat(lib): add mmio_setbits_64 inline function

Add mmio 64 bits set utility function.

Change-Id: I35c54d1c0e981f6a68eb8fd1101947451dcf253f
Signed-off-by: Runyang Chen <runyang.chen@mediatek.com>

Merge changes from topic "xl/c1nano-errata" into integration

* changes:
fix(cpus): workaround for C1-Nano erratum 3754876
fix(cpus): workaround for C1-Nano erratum 3419531
fix(cpus): workaroun

Merge changes from topic "xl/c1nano-errata" into integration

* changes:
fix(cpus): workaround for C1-Nano erratum 3754876
fix(cpus): workaround for C1-Nano erratum 3419531
fix(cpus): workaround for C1-Nano erratum 3630925
fix(cpus): workaround for C1-Nano erratum 3616450
fix(cpus): workaround for C1-Nano erratum 3516455
fix(cpus): workaround for C1-Nano erratum 3437202
fix(cpus): workaround for C1-Nano erratum 3392149

show more ...

Merge changes from topic "v3_errata" into integration

* changes:
fix(cpus): workaround for Neoverse-V3 erratum 3312417
fix(cpus): workaround for Neoverse V3 erratum 3878291
fix(cpus): workarou

Merge changes from topic "v3_errata" into integration

* changes:
fix(cpus): workaround for Neoverse-V3 erratum 3312417
fix(cpus): workaround for Neoverse V3 erratum 3878291
fix(cpus): workaround for Neoverse V3 erratum 3864536
fix(cpus): workaround for Neoverse V3 erratum 3782181
fix(cpus): workaround for Neoverse V3 erratum 3734562
fix(cpus): workaround for Neoverse V3 erratum 3696307

show more ...

fix(cpus): workaround for C1-Nano erratum 3419531

C1-Nano erratum 3419531 is a Cat B erratum that applies
to revision r0p0, and is fixed in r0p1.

This errata can be avoided by setting IMP_CPUACTLR_

fix(cpus): workaround for C1-Nano erratum 3419531

C1-Nano erratum 3419531 is a Cat B erratum that applies
to revision r0p0, and is fixed in r0p1.

This errata can be avoided by setting IMP_CPUACTLR_EL1[27] to
1, which disable write streaming for MTE stores when MTE
feature is enabled.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-3273788/0800/

Change-Id: Ib5103483163a1f93cbb2df8c3b3fcfb2c6d487c6
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

fix(cpus): workaround for C1-Nano erratum 3630925

C1-Nano erratum 3630925 is a Cat B erratum that applies
to revision r0p0, and is fixed in r0p1.

This errata can be avoided by disable entering full

fix(cpus): workaround for C1-Nano erratum 3630925

C1-Nano erratum 3630925 is a Cat B erratum that applies
to revision r0p0, and is fixed in r0p1.

This errata can be avoided by disable entering full
retention mode by setting both IMP_CPUPWRCTLR_EL1[9:7] and
IMP_CPUPWRCTLR_EL1[6:4] to 3'b000.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-3273788/0800/

Change-Id: I61cdf21b50dfb534ce2a1e74c22b06bde9a7c0a7
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...

fix(cpus): workaround for C1-Nano erratum 3516455

C1-Nano erratum 3516455 is a Cat B erratum that applies
to revision r0p0, and is fixed in r0p1.

This errata might cause the core to deadlock in
str

fix(cpus): workaround for C1-Nano erratum 3516455

C1-Nano erratum 3516455 is a Cat B erratum that applies
to revision r0p0, and is fixed in r0p1.

This errata might cause the core to deadlock in
streaming mode when Non-SME instruction abort.
Which can be avoided by restricts address generation
based on speculatively produced data for vector
load/stores accessing 4 vector registers in streaming SVE
mode. The workaround can have a minor impact on
performance in heavy streaming SVE workloads, depending
on the density of the affected instructions

SDEN documentation:
https://developer.arm.com/documentation/SDEN-3273788/0800/

Change-Id: Id97fbfd1d76e9dc1a3488ce33e353c032c41e0f1
Signed-off-by: Xialin Liu <xialin.liu@arm.com>

show more ...