Merge changes from topic "ssbs_errata_catchup" into integration

* changes:
fix(cpus): workaround for Neoverse-N3 erratum 3456111
fix(cpus): workaround for Neoverse-N2 erratum 3324339
fix(cpus)

Merge changes from topic "ssbs_errata_catchup" into integration

* changes:
fix(cpus): workaround for Neoverse-N3 erratum 3456111
fix(cpus): workaround for Neoverse-N2 erratum 3324339
fix(cpus): workaround for Neoverse-N1 erratum 3324349

show more ...

fix(cpus): workaround for Neoverse-N1 erratum 3324349

Neoverse-N1 erratum 3324349 is a Cat B erratum that applies
to all revisions <= r4p1, and is still open.

This errata can be avoided by adding a

fix(cpus): workaround for Neoverse-N1 erratum 3324349

Neoverse-N1 erratum 3324349 is a Cat B erratum that applies
to all revisions <= r4p1, and is still open.

This errata can be avoided by adding a speculation barrier
instruction following writes to the SSBS register to
ensure the new value of PSTATE.SSBS affects the subsequent
instructions in the execution stream under speculation.

SDEN documentation:
https://developer.arm.com/documentation/SDEN-885747

Change-Id: I1f142027ed73135d78c368be926072c2f73eab46
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

Merge "fix(cpus): check minor revision before applying runtime errata" into integration

fix(cpus): check minor revision before applying runtime errata

Patch db9ee83432 removed cpu_rev checking for runtime errata
within cpu functions with the argument that if we're in the cpu file,
we'v

fix(cpus): check minor revision before applying runtime errata

Patch db9ee83432 removed cpu_rev checking for runtime errata
within cpu functions with the argument that if we're in the cpu file,
we've already check the MIDR and matched against the CPU. However, that
also removes the revision check which being in the cpu file does not
guarantee. Reintroduce the MIDR checking so that the revision check
happens and errata can be skipped if they don't apply.

Change-Id: I46b2ba8b524a073e02b4b5de641ae97795bc176b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(cpus): drop esb from the Neoverse N1" into integration

fix(cpus): drop esb from the Neoverse N1

An esb is only necessary when FEAT_IESB is not present in hardware but
FEAT_RAS is. When FEAT_RAS is present we rely on the fact that FEAT_IESB
will also be

fix(cpus): drop esb from the Neoverse N1

An esb is only necessary when FEAT_IESB is not present in hardware but
FEAT_RAS is. When FEAT_RAS is present we rely on the fact that FEAT_IESB
will also be present and an implicit esb will pre present on eret.

Well the N1 implements FEAT_RAS and FEAT_IESB and the platforms that use
it (n1sdp) enable the features in firmware. So the esb is redundant.
There are dynamic platforms where this may not necessarily be true,
however, the esb is in an erratum workaround which cannot be present in
these platforms.

Change-Id: I5775180ec61373cc5d1b9831e3fa0f2fbb19eab9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(errata-abi): add support for handling split workarounds" into integration

fix(errata-abi): add support for handling split workarounds

Certain erratum workarounds like Neoverse N1 1542419, need a part
of their mitigation done in EL3 and the rest in lower EL. But currently

fix(errata-abi): add support for handling split workarounds

Certain erratum workarounds like Neoverse N1 1542419, need a part
of their mitigation done in EL3 and the rest in lower EL. But currently
such workarounds return HIGHER_EL_MITIGATION which indicates that the
erratum has already been mitigated by a higher EL(EL3 in this case)
which causes the lower EL to not apply it's part of the mitigation.

This patch fixes this issue by adding support for split workarounds
so that on certain errata we return AFFECTED even though EL3 has
applied it's workaround. This is done by reusing the chosen field of
erratum_entry structure into a bitfield that has two bitfields -
Bit 0 indicates that the erratum has been enabled in build,
Bit 1 indicates that the erratum is a split workaround and should
return AFFECTED instead of HIGHER_EL_MITIGATION.

SDEN documentation:
https://developer.arm.com/documentation/SDEN885747/latest

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: Iec94d665b5f55609507a219a7d1771eb75e7f4a7

show more ...

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the i

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the init_cpu_data_ptr function
perf(cpus): inline the reset function
perf(cpus): inline the cpu_get_rev_var call
perf(cpus): inline cpu_rev_var checks
refactor(cpus): register DSU errata with the errata framework's wrappers
refactor(cpus): convert checker functions to standard helpers
refactor(cpus): convert the Cortex-A65 to use the errata framework
fix(cpus): declare reset errata correctly

show more ...

perf(cpus): make reset errata do fewer branches

Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and cach

perf(cpus): make reset errata do fewer branches

Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.

We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.

Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.

The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.

Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cpus): register DSU errata with the errata framework's wrappers

The existing DSU errata workarounds hijack the errata framework's inner
workings to register with it. However, that is undesi

refactor(cpus): register DSU errata with the errata framework's wrappers

The existing DSU errata workarounds hijack the errata framework's inner
workings to register with it. However, that is undesirable as any change
to the framework may end up missing these workarounds. So convert the
checks and workarounds to macros and have them included with the
standard wrappers.

The only problem with this is the is_scu_present_in_dsu weak function.
Fortunately, it is only needed for 2 of the errata and only on 3 cores.
So drop it, assuming the default behaviour and have the callers handle
the exception.

Change-Id: Iefa36325804ea093e938f867b9a6f49a6984b8ae
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "chore(cpus): optimise runtime errata applications" into integration

chore(cpus): optimise runtime errata applications

The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dw

chore(cpus): optimise runtime errata applications

The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.

Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.

Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.

Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "clean-up-errata-compatibility" into integration

* changes:
refactor(cpus): remove cpu specific errata funcs
refactor(cpus): directly invoke errata reporter

refactor(cpus): remove cpu specific errata funcs

Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and remove

refactor(cpus): remove cpu specific errata funcs

Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.

Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>

show more ...

Merge changes from topic "lw/errata_refactor" into integration

* changes:
refactor(cpus): convert Neoverse-N1 to use helpers
refactor(cpus): convert Neoverse-N1 to framework
refactor(cpus): re

Merge changes from topic "lw/errata_refactor" into integration

* changes:
refactor(cpus): convert Neoverse-N1 to use helpers
refactor(cpus): convert Neoverse-N1 to framework
refactor(cpus): reorder Neoverse-N1 .S file
refactor(cpus): convert Neoverse-E1 to framework

show more ...

refactor(cpus): convert Neoverse-N1 to use helpers

Conversion to use CPU helpers for Neoverse-N1 testing done with
framework adaptation patch.

Change-Id: I2103f6e64daf0ee4c7b756083e5bf485f15c0e21
S

refactor(cpus): convert Neoverse-N1 to use helpers

Conversion to use CPU helpers for Neoverse-N1 testing done with
framework adaptation patch.

Change-Id: I2103f6e64daf0ee4c7b756083e5bf485f15c0e21
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

show more ...

refactor(cpus): convert Neoverse-N1 to framework

For N1, this involves replacing:
- The reset_func with the standard cpu_reset_func_{start,end}
to apply errata automatically
- The <cpu>_erra

refactor(cpus): convert Neoverse-N1 to framework

For N1, this involves replacing:
- The reset_func with the standard cpu_reset_func_{start,end}
to apply errata automatically
- The <cpu>_errata_report with the errata_report_shim to
report errata automatically
And for each erratum:
- The prologue with the workaround_<type>_start to do the checks and
framework registration automatically at reset or runtime
- The epilogue with the workaround_<type>_end
- The checker function with the check_erratum_<type> to check whether
the erratum applies on the revision of the CPU.

Testing was conducted by:
- Manual comparison of disassembly of converted functions with non-
converted functions:

aarch64-none-elf-objdump -D
<trusted-firmware-a with conversion>/build/fvp/release/bl31/bl31.elf
vs
aarch64-none-elf-objdump -D
<trusted-firmware-a clean repo>/build/fvp/release/bl31/bl31.elf

- Build for debug with all errata enabled and step through ArmDS
to ensure all functions are entered and the path remains the same
as before conversion to the new framework.

Change-Id: I2ea612d4c197dd73835fadda81f59732c19534f8
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

show more ...

refactor(cpus): reorder Neoverse-N1 .S file

Moving neoverse_n1_disable_speculative_loads function before reset
function to maintain git blame with refactor to new framework.

Change-Id: I79a4de9955a

refactor(cpus): reorder Neoverse-N1 .S file

Moving neoverse_n1_disable_speculative_loads function before reset
function to maintain git blame with refactor to new framework.

Change-Id: I79a4de9955a6f37e289456a743b946c0c4c8c27f
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

show more ...

Merge "fix(cpus): do not put RAS check before using esb" into integration

fix(cpus): do not put RAS check before using esb

If RAS Extension is not implemented esb instruction executes as a NOP.
No need to have a check for RAS presence in the code.
Also, The handler is rel

fix(cpus): do not put RAS check before using esb

If RAS Extension is not implemented esb instruction executes as a NOP.
No need to have a check for RAS presence in the code.
Also, The handler is related to a synchronous exceptions which
implicitly is part of BL31 image only, so remove that check too.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: If4264504cba9f0642b7b9c581ae66cd4deace32b

show more ...

Merge changes from topic "feat_amu_rework" into integration

* changes:
refactor(amu): use new AMU feature check routines
refactor(amu): unify ENABLE_AMU and ENABLE_FEAT_AMUv1

refactor(amu): unify ENABLE_AMU and ENABLE_FEAT_AMUv1

So far we have the ENABLE_AMU build option to include AMU register
handling code for enabling and context switch. There is also an
ENABLE_FEAT_A

refactor(amu): unify ENABLE_AMU and ENABLE_FEAT_AMUv1

So far we have the ENABLE_AMU build option to include AMU register
handling code for enabling and context switch. There is also an
ENABLE_FEAT_AMUv1 option, solely to protect the HAFGRTR_EL2 system
register handling. The latter needs some alignment with the new feature
scheme, but it conceptually overlaps with the ENABLE_AMU option.

Since there is no real need for two separate options, unify both into a
new ENABLE_FEAT_AMU name in a first step. This is mostly just renaming at
this point, a subsequent patch will make use of the new feature handling
scheme.

Change-Id: I97d8a55bdee2ed1e1509fa9f2b09fd0bdd82736e
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

Merge changes from topic "errata" into integration

* changes:
fix(cpus): workaround for Cortex-A76 erratum 2743102
fix(cpus): workaround for Neoverse N1 erratum 2743102

fix(cpus): workaround for Neoverse N1 erratum 2743102

Neoverse N1 erratum 2743102 is a Cat B erratum that applies to
all revisions <=r4p1 and is still open. The workaround is to
insert a dsb before

fix(cpus): workaround for Neoverse N1 erratum 2743102

Neoverse N1 erratum 2743102 is a Cat B erratum that applies to
all revisions <=r4p1 and is still open. The workaround is to
insert a dsb before the isb in the power down sequence.

SDEN documentation:
https://developer.arm.com/documentation/SDEN885747/latest

Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I81a8793c1a118764df3ac97b67f5e088f56f6a20

show more ...