Merge changes I2c7c8da9,I9786ab88,Ia76ba243,Ifec40dee,Ifdd59c09, ... into integration

* changes:
fix(cpus): workaround for Cortex-A510 erratum 3704847
fix(cpus): workaround for Cortex-A510 errat

Merge changes I2c7c8da9,I9786ab88,Ia76ba243,Ifec40dee,Ifdd59c09, ... into integration

* changes:
fix(cpus): workaround for Cortex-A510 erratum 3704847
fix(cpus): workaround for Cortex-A510 erratum 3672349
fix(cpus): workaround for Cortex-A510 erratum 2420992
fix(cpus): workaround for Cortex-A510 erratum 2218134
fix(cpus): workaround for Cortex-A510 erratum 2169012
fix(cpus): workaround for Cortex-A510 erratum 2008766

show more ...

fix(cpus): workaround for Cortex-A510 erratum 3704847

Cortex-A510 erratum 3704847 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3, r1p0, r1p1, r1p2 and r1p3, and
is still open.

fix(cpus): workaround for Cortex-A510 erratum 3704847

Cortex-A510 erratum 3704847 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3, r1p0, r1p1, r1p2 and r1p3, and
is still open.

The workaround is to set bit 9 in CPUACTLR_EL1.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1873361/latest/

Change-Id: I2c7c8da9c66471115b5bf8fb5c87d4de46ca265c
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

fix(cpus): workaround for Cortex-A510 erratum 3672349

Cortex-A510 erratum 3672349 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3, r1p0, r1p1, r1p2 and r1p3, and
is still open.

fix(cpus): workaround for Cortex-A510 erratum 3672349

Cortex-A510 erratum 3672349 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3, r1p0, r1p1, r1p2 and r1p3, and
is still open.

The workaround is to clear the WFE_RET_CTRL and WFI_RET_CTRL fields
in CPUPWRCTLR_EL1 to disable full retention.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1873361/latest/

Change-Id: I9786ab8843a2eab45e650c6af50b6933481527ec
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

fix(cpus): workaround for Cortex-A510 erratum 2420992

Cortex-A510 erratum 2420992 is a Cat B erratum that applies only to
revisions r1p0 and r1p1, and is fixed in r1p1.

The workaround is to set bit

fix(cpus): workaround for Cortex-A510 erratum 2420992

Cortex-A510 erratum 2420992 is a Cat B erratum that applies only to
revisions r1p0 and r1p1, and is fixed in r1p1.

The workaround is to set bit 3 in CPUACTLR3_EL1 which will have no
performance impact, but will increase power consumption by 0.3-0.5%.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1873361/latest/

Change-Id: Ia76ba2431d76f14c08b95a998806986190d682c3
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

fix(cpus): workaround for Cortex-A510 erratum 2218134

Cortex-A510 erratum 2218134 is a Cat B erratum that applies only to
revision r1p0 and is fixed in r1p1.

The workaround is to set bit 43 in CPUA

fix(cpus): workaround for Cortex-A510 erratum 2218134

Cortex-A510 erratum 2218134 is a Cat B erratum that applies only to
revision r1p0 and is fixed in r1p1.

The workaround is to set bit 43 in CPUACTLR2_EL1 which will correct
the instruction fetch stream with no performance impact.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1873361/latest/

Change-Id: Ifec40dee2f7e42c56c9ed447b6b1997b170f9453
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

fix(cpus): workaround for Cortex-A510 erratum 2169012

Cortex-A510 erratum 2169012 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3 and r1p0, and is fixed in r1p1.

This erratum ha

fix(cpus): workaround for Cortex-A510 erratum 2169012

Cortex-A510 erratum 2169012 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3 and r1p0, and is fixed in r1p1.

This erratum has an identical workaround to 1922240 and resolves
a similar issue, but that erratum only applies to r0p0 which is
not used in any production hardware, so it has been removed.

This workaround has a negligible performance impact.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1873361/latest/

Change-Id: Ifdd59c09e84252dc292600630d81d32986fd6c0c
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

fix(cpus): workaround for Cortex-A510 erratum 2008766

Cortex-A510 erratum 2008766 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3, r1p0, r1p1, r1p2 and r1p3, and
is still open.

fix(cpus): workaround for Cortex-A510 erratum 2008766

Cortex-A510 erratum 2008766 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2, r0p3, r1p0, r1p1, r1p2 and r1p3, and
is still open.

The workaround is to clear the ERXCTLR_EL1.ED bit before power
down, which will cause any detected errors during power down to
be ignored.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-1873361/latest/

Change-Id: Id1aa0f2c518a055363c962f9abdb27e1ee8bff18
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

Merge changes from topic "bk/pabandon_cleanup" into integration

* changes:
feat(cpus): add pabandon support to the Alto cpu
feat(psci): optimise clock init on a pabandon
feat(psci): check that

Merge changes from topic "bk/pabandon_cleanup" into integration

* changes:
feat(cpus): add pabandon support to the Alto cpu
feat(psci): optimise clock init on a pabandon
feat(psci): check that CPUs handled a pabandon
feat(psci): make pabandon support generic
refactor(psci): unify coherency exit between AArch64 and AArch32
refactor(psci): absorb psci_power_down_wfi() into common code
refactor(platforms): remove usage of psci_power_down_wfi
fix(cm): disable SPE/TRBE correctly

show more ...

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called f

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called from
non-PSCI code as PSCI is smart enough to handle this after platform
hooks. So absorb the functions into common code.

This allows for a tiny bit of optimisation: there will be no branch
(that can be missed or non-cached) to a non-inlineable function. Then in
the terminal case we can call wfi() directly with the application of the
erratum before the loop. And finally in the wakeup case, we don't have
to explicitly clear the errata as that will happen automatically on the
second call of prepare_cpu_pwr_dwn().

The A510 erratum requires a tsb csync before the dsb+wfi combo to turn
the core off. We can do this a little bit earlier in the cpu hook and
relieve common code from the responsibility. EL3 is always a prohibited
region so the buffer will stay empty.

Change-Id: I5f950df3fb7b0736df4ce25a21f78b29896de215
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "chore(cpus): remove in-order checks" into integration

chore(cpus): remove in-order checks

Remove runtime in-order checks for Erratum and CVE's.
Fix out-of-order issues in CPU files found with CPU Erratum and CVE
static checker script run on entire fold

chore(cpus): remove in-order checks

Remove runtime in-order checks for Erratum and CVE's.
Fix out-of-order issues in CPU files found with CPU Erratum and CVE
static checker script run on entire folder `lib/cpus/aarch64/`.

Change-Id: Iee5a8cb49834e9f35c6c2f2a84065430ca1ec8a6
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

Merge "fix(errata): workaround for Cortex-A510 erratum 2971420" into integration

fix(errata): workaround for Cortex-A510 erratum 2971420

Cortex-A510 erratum 2971420 applies to revisions r0p1, r0p2, r0p3,
r1p0, r1p1, r1p2 and r1p3, and is still open.

Under some conditions, data

fix(errata): workaround for Cortex-A510 erratum 2971420

Cortex-A510 erratum 2971420 applies to revisions r0p1, r0p2, r0p3,
r1p0, r1p1, r1p2 and r1p3, and is still open.

Under some conditions, data might be corrupted if Trace Buffer
Extension (TRBE) is enabled. The workaround is to disable trace
collection via TRBE by programming MDCR_EL3.NSTB[1] to the opposite
value of SCR_EL3.NS on a security state switch. Since we only enable
TRBE for non-secure world, the workaround is to disable TRBE by
setting the NSTB field to 00 so accesses are trapped to EL3 and
secure state owns the buffer.

SDEN: https://developer.arm.com/documentation/SDEN-1873361/latest/

Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: Ia77051f6b64c726a8c50596c78f220d323ab7d97

show more ...

Merge changes I3d950e72,Id315a8fe,Ib62e6e9b,I1d0475b2 into integration

* changes:
perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context
perf(psci): get PMF timestamps wi

Merge changes I3d950e72,Id315a8fe,Ib62e6e9b,I1d0475b2 into integration

* changes:
perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context
perf(psci): get PMF timestamps with no cache flushes if possible
perf(amu): greatly simplify AMU context management
perf(mpmm): greatly simplify MPMM enablement

show more ...

perf(mpmm): greatly simplify MPMM enablement

MPMM is a core-specific microarchitectural feature. It has been present
in every Arm core since the Cortex-A510 and has been implemented in
exactly the s

perf(mpmm): greatly simplify MPMM enablement

MPMM is a core-specific microarchitectural feature. It has been present
in every Arm core since the Cortex-A510 and has been implemented in
exactly the same way. Despite that, it is enabled more like an
architectural feature with a top level enable flag. This utilised the
identical implementation.

This duality has left MPMM in an awkward place, where its enablement
should be generic, like an architectural feature, but since it is not,
it should also be core-specific if it ever changes. One choice to do
this has been through the device tree.

This has worked just fine so far, however, recent implementations expose
a weakness in that this is rather slow - the device tree has to be read,
there's a long call stack of functions with many branches, and system
registers are read. In the hot path of PSCI CPU powerdown, this has a
significant and measurable impact. Besides it being a rather large
amount of code that is difficult to understand.

Since MPMM is a microarchitectural feature, its correct placement is in
the reset function. The essence of the current enablement is to write
CPUPPMCR_EL3.MPMM_EN if CPUPPMCR_EL3.MPMMPINCTL == 0. Replacing the C
enablement with an assembly macro in each CPU's reset function achieves
the same effect with just a single close branch and a grand total of 6
instructions (versus the old 2 branches and 32 instructions).

Having done this, the device tree entry becomes redundant. Should a core
that doesn't support MPMM arise, this can cleanly be handled in the
reset function. As such, the whole ENABLE_MPMM_FCONF and platform hooks
mechanisms become obsolete and are removed.

Change-Id: I1d0475b21a1625bb3519f513ba109284f973ffdf
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the i

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the init_cpu_data_ptr function
perf(cpus): inline the reset function
perf(cpus): inline the cpu_get_rev_var call
perf(cpus): inline cpu_rev_var checks
refactor(cpus): register DSU errata with the errata framework's wrappers
refactor(cpus): convert checker functions to standard helpers
refactor(cpus): convert the Cortex-A65 to use the errata framework
fix(cpus): declare reset errata correctly

show more ...

perf(cpus): make reset errata do fewer branches

Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and cach

perf(cpus): make reset errata do fewer branches

Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.

We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.

Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.

The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.

Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cpus): register DSU errata with the errata framework's wrappers

The existing DSU errata workarounds hijack the errata framework's inner
workings to register with it. However, that is undesi

refactor(cpus): register DSU errata with the errata framework's wrappers

The existing DSU errata workarounds hijack the errata framework's inner
workings to register with it. However, that is undesirable as any change
to the framework may end up missing these workarounds. So convert the
checks and workarounds to macros and have them included with the
standard wrappers.

The only problem with this is the is_scu_present_in_dsu weak function.
Fortunately, it is only needed for 2 of the errata and only on 3 cores.
So drop it, assuming the default behaviour and have the callers handle
the exception.

Change-Id: Iefa36325804ea093e938f867b9a6f49a6984b8ae
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "clean-up-errata-compatibility" into integration

* changes:
refactor(cpus): remove cpu specific errata funcs
refactor(cpus): directly invoke errata reporter

refactor(cpus): remove cpu specific errata funcs

Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and remove

refactor(cpus): remove cpu specific errata funcs

Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.

Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>

show more ...

Merge "fix(cpus): workaround for Cortex-A510 erratum 2080326" into integration

fix(cpus): workaround for Cortex-A510 erratum 2080326

Cortex-A510 erratum 2080326 is a Cat B erratum that applies
to all revisions <= r0p2 and is fixed in r0p3.
The workaround sequence helps perform

fix(cpus): workaround for Cortex-A510 erratum 2080326

Cortex-A510 erratum 2080326 is a Cat B erratum that applies
to all revisions <= r0p2 and is fixed in r0p3.
The workaround sequence helps perform a DSB after each TLBI
instruction and can be applied only for version r0p2 and has
minimal performance impact.
The workaround is not applicable for versions < r0p2.

SDEN documentation:
https://developer.arm.com/documentation/SDEN1873361/latest

Change-Id: Ib9bce8b711c25a79f7b2f891ae6f8b366fc80ddd
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>

show more ...

Merge changes from topics "hm/errata-refactor", "jc/errata_refactor" into integration

* changes:
refactor(cpus): convert the Cortex-x2 to use cpu helpers
refactor(cpus): convert the Cortex-x2 to

Merge changes from topics "hm/errata-refactor", "jc/errata_refactor" into integration

* changes:
refactor(cpus): convert the Cortex-x2 to use cpu helpers
refactor(cpus): convert the Cortex-x2 to use the errata framework
refactor(cpus): reorder Cortex-x2 errata by ascending order
refactor(cpus): convert the Cortex-A65AE to use the errata framework
refactor(cpus): convert the Cortex-A510 to use cpu helpers
refactor(cpus): convert the Cortex-A510 to use the errata framework
refactor(cpus): reorder Cortex-A510 errata by ascending order
chore(fvp): add Aarch32 Cortex-A53 to the build
refactor(cpus): add Cortex-A53 errata framework information
feat(cpus): add errata framework helpers
chore(brcm): include cpu_helpers.S for bl2 build

show more ...

refactor(cpus): convert the Cortex-A510 to use cpu helpers

Change-Id: I6d26092525c2d5255a741515071ee7ed873aa52d
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

refactor(cpus): convert the Cortex-A510 to use the errata framework

This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <c

refactor(cpus): convert the Cortex-A510 to use the errata framework

This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive

It is important to note that the errata workaround sequences remain
unchanged and preserve their git blame.

Note: cortex_a510.S is applicable and being used only by arm_fpga platform.

However, to test the ported changes, below steps were carried out on the
fvp and the obtained results has been verified.

Testing was conducted by:
* Building for release with all errata flags enabled and running script
in change 19136 to compare output of objdump for each errata.

* Testing via script was not complete, as it directed to verify the
check and the workaround functions of few erratas manually.

* Manual comparison of disassembly of converted functions with non-
converted functions

aarch64-none-elf-objdump -D <trusted-firmware-a with conversion>/build/../release/bl31/bl31.elf
vs
aarch64-none-elf-objdump -D <trusted-firmware-a clean repo>/build/fvp/release/bl31/bl31.elf

* Manual comparison of disassembly of both both files(bl31.elf)
ensured, the ported changes were identical and hence verified.

* Build for release with all errata flags enabled and run default
tftf tests.

CROSS_COMPILE=aarch64-none-elf- \
make PLAT=fvp \
ARCH=aarch64 \
DEBUG=0 \
HW_ASSISTED_COHERENCY=1 \
USE_COHERENT_MEM=0 \
CTX_INCLUDE_AARCH32_REGS=0 \
ERRATA_A510_1922240=1 \
ERRATA_A510_2288014=1 \
ERRATA_A510_2042739=1 \
ERRATA_A510_2041909=1 \
ERRATA_A510_2250311=1 \
ERRATA_A510_2218950=1 \
ERRATA_A510_2172148=1 \
ERRATA_A510_2347730=1 \
ERRATA_A510_2371937=1 \
ERRATA_A510_2666669=1 \
ERRATA_A510_2684597=1 \
ERRATA_DSU_2313941=1 \
BL33=/home/jaychi01/tf_a/tf-a-tests/build/fvp/release/tftf.bin \
fip all -j12

* Build for debug with all errata enabled and step through ArmDS
at reset to ensure that if Errata are applicable then the
workaround functions are entered precisely.

Change-Id: Icf7aa25c0b3b30f5e2ad6db83953f7f4f0b201d9
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...