Merge "fix(cpus): check minor revision before applying runtime errata" into integration

fix(cpus): check minor revision before applying runtime errata

Patch db9ee83432 removed cpu_rev checking for runtime errata
within cpu functions with the argument that if we're in the cpu file,
we'v

fix(cpus): check minor revision before applying runtime errata

Patch db9ee83432 removed cpu_rev checking for runtime errata
within cpu functions with the argument that if we're in the cpu file,
we've already check the MIDR and matched against the CPU. However, that
also removes the revision check which being in the cpu file does not
guarantee. Reintroduce the MIDR checking so that the revision check
happens and errata can be skipped if they don't apply.

Change-Id: I46b2ba8b524a073e02b4b5de641ae97795bc176b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "ar/cvereorder" into integration

* changes:
chore(cpus): rearrange the errata and cve order in Neoverse-N2
chore(cpus): rearrange cve in order in Cortex-X1
chore(cpus)

Merge changes from topic "ar/cvereorder" into integration

* changes:
chore(cpus): rearrange the errata and cve order in Neoverse-N2
chore(cpus): rearrange cve in order in Cortex-X1
chore(cpus): fix cve order in Neoverse-V1
chore(cpus): fix cve order in Cortex-X2
chore(cpus): fix cve order in Cortex-A78C
chore(cpus): fix cve order in Cortex-A78_AE
chore(cpus): fix cve order in Cortex-A78
chore(cpus): fix cve order in Cortex-A77

show more ...

chore(cpus): fix cve order in Cortex-A77

This patch rearranges CVE-2024-5660 apply order in Cortex-A77.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I41d76268ce2248bfd36

chore(cpus): fix cve order in Cortex-A77

This patch rearranges CVE-2024-5660 apply order in Cortex-A77.

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I41d76268ce2248bfd3600bbf6b89d16b6bdce8f0

show more ...

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the i

Merge changes from topic "bk/errata_speed" into integration

* changes:
refactor(cpus): declare runtime errata correctly
perf(cpus): make reset errata do fewer branches
perf(cpus): inline the init_cpu_data_ptr function
perf(cpus): inline the reset function
perf(cpus): inline the cpu_get_rev_var call
perf(cpus): inline cpu_rev_var checks
refactor(cpus): register DSU errata with the errata framework's wrappers
refactor(cpus): convert checker functions to standard helpers
refactor(cpus): convert the Cortex-A65 to use the errata framework
fix(cpus): declare reset errata correctly

show more ...

perf(cpus): make reset errata do fewer branches

Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and cach

perf(cpus): make reset errata do fewer branches

Errata application is painful for performance. For a start, it's done
when the core has just come out of reset, which means branch predictors
and caches will be empty so a branch to a workaround function must be
fetched from memory and that round trip is very slow. Then it also runs
with the I-cache off, which means that the loop to iterate over the
workarounds must also be fetched from memory on each iteration.

We can remove both branches. First, we can simply apply every erratum
directly instead of defining a workaround function and jumping to it.
Currently, no errata that need to be applied at both reset and runtime,
with the same workaround function, exist. If the need arose in future,
this should be achievable with a reset + runtime wrapper combo.

Then, we can construct a function that applies each erratum linearly
instead of looping over the list. If this function is part of the reset
function, then the only "far" branches at reset will be for the checker
functions. Importantly, this mitigates the slowdown even when an erratum
is disabled.

The result is ~50% speedup on N1SDP and ~20% on AArch64 Juno on wakeup
from PSCI calls that end in powerdown. This is roughly back to the
baseline of v2.9, before the errata framework regressed on performance
(or a little better). It is important to note that there are other
slowdowns since then that remain unknown.

Change-Id: Ie4d5288a331b11fd648e5c4a0b652b74160b07b9
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

perf(cpus): inline cpu_rev_var checks

We strive to apply errata as close to reset as possible with as few
things enabled as possible. Importantly, the I-cache will not be
enabled. This means that re

perf(cpus): inline cpu_rev_var checks

We strive to apply errata as close to reset as possible with as few
things enabled as possible. Importantly, the I-cache will not be
enabled. This means that repeated branches to these tiny functions must
be re-fetched all the way from memory each time which has glacial speed.
Cores are allowed to fetch things ahead of time though as long as
execution is fairly linear. So we can trade a little bit of space (3 to
7 instructions per erratum) to keep things linear and not have to go to
memory.

While we're at it, optimise the the cpu_rev_var_{ls, hs, range}
functions to take up less space. Dropping the moves allows for a bit of
assembly magic that produces the same result in 2 and 3 instructions
respectively.

Change-Id: I51608352f23b2244ea7a99e76c10892d257f12bf
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "sm/fix_erratum" into integration

* changes:
fix(cpus): workaround for CVE-2024-5660 for Cortex-X925
fix(cpus): workaround for CVE-2024-5660 for Cortex-X2
fix(cpus): w

Merge changes from topic "sm/fix_erratum" into integration

* changes:
fix(cpus): workaround for CVE-2024-5660 for Cortex-X925
fix(cpus): workaround for CVE-2024-5660 for Cortex-X2
fix(cpus): workaround for CVE-2024-5660 for Cortex-A77
fix(cpus): workaround for CVE-2024-5660 for Neoverse-V1
fix(cpus): workaround for CVE-2024-5660 for Cortex-A78_AE
fix(cpus): workaround for CVE-2024-5660 for Cortex-A78C
fix(cpus): workaround for CVE-2024-5660 for Cortex-A78
fix(cpus): workaround for CVE-2024-5660 for Cortex-X1
fix(cpus): workaround for CVE-2024-5660 for Neoverse-N2
fix(cpus): workaround for CVE-2024-5660 for Cortex-A710
fix(cpus): workaround for CVE-2024-5660 for Neoverse-V2
fix(cpus): workaround for CVE-2024-5660 for Cortex-X3
fix(cpus): workaround for CVE-2024-5660 for Neoverse-V3
fix(cpus): workaround for CVE-2024-5660 for Cortex-X4

show more ...

fix(cpus): workaround for CVE-2024-5660 for Cortex-A77

Implements mitigation for CVE-2024-5660 that affects Cortex-A77
revisions r0p0, r1p0, r1p1.
The workaround is to disable the hardware page aggr

fix(cpus): workaround for CVE-2024-5660 for Cortex-A77

Implements mitigation for CVE-2024-5660 that affects Cortex-A77
revisions r0p0, r1p0, r1p1.
The workaround is to disable the hardware page aggregation at
EL3 by setting CPUECTLR_EL1[46] = 1'b1.

Public Documentation:
https://developer.arm.com/Arm%20Security%20Center/Arm%20CPU%20Vulnerability%20CVE-2024-5660

Change-Id: Ic71b163883ea624e9f2f77deb8b30c69612938b9
Signed-off-by: Sona Mathew <sonarebecca.mathew@arm.com>

show more ...

Merge "chore(cpus): optimise runtime errata applications" into integration

chore(cpus): optimise runtime errata applications

The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dw

chore(cpus): optimise runtime errata applications

The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.

Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.

Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.

Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "clean-up-errata-compatibility" into integration

* changes:
refactor(cpus): remove cpu specific errata funcs
refactor(cpus): directly invoke errata reporter

refactor(cpus): remove cpu specific errata funcs

Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and remove

refactor(cpus): remove cpu specific errata funcs

Errata printing is done directly via generic_errata_report.
This commit removes the unused \_cpu\()_errata_report
functions for all cores, and removes errata_func from cpu_ops.

Change-Id: I04fefbde5f0ff63b1f1cd17c864557a14070d68c
Signed-off-by: Ryan Everett <ryan.everett@arm.com>

show more ...

Merge changes from topic "sm_bk/errata_refactor" into integration

* changes:
refactor(cpus): convert the Cortex-A78AE to use cpu helpers
refactor(cpus): convert the Denver cpu to use the errata

Merge changes from topic "sm_bk/errata_refactor" into integration

* changes:
refactor(cpus): convert the Cortex-A78AE to use cpu helpers
refactor(cpus): convert the Denver cpu to use the errata framework
refactor(cpus): convert the Cortex-A78AE to use the errata framework
refactor(cpus): convert the Cortex-A5 to use the errata framework
refactor(cpus): convert the Cortex-A77 to use the bit set helpers
refactor(cpus): convert the Cortex-A77 to use the errata framework
refactor(cpus): reorder Cortex-A77 errata by ascending order

show more ...

refactor(cpus): convert the Cortex-A77 to use the bit set helpers

This makes the implementation itself much more readable. At this point
all errata have been tested with a script [1] to make sure th

refactor(cpus): convert the Cortex-A77 to use the bit set helpers

This makes the implementation itself much more readable. At this point
all errata have been tested with a script [1] to make sure the migration
kept everything the same. It reported 1508412, 1946167, and
CVE_2022_23960 as having some mismatch. The first has a small
non-trivial change that results in identical behaviour. The second is
non-trivial to compare, but manual inspection shows it is identical. The
CVE had no workaround function previously, however, the instructions are
indeed identical. All errata have been checked that they get invoked.

The script's commandline looks like:
./script.py cortex_a77 /path/to/tf-a-with-changes /path/to/tf-a-clean/

[1]: the script:
import re
import subprocess
import sys

def full_cpu_name():
return sys.argv[1]

def old_cpu_name():
return sys.argv[1].split('_')[1]

def new_build():
return sys.argv[2]

def old_build():
return sys.argv[3]

def get_dump(root_dir, symbol):
# bl31 includes more stuff
raw_dump = subprocess.run([
'aarch64-none-elf-objdump', f'--disassemble={symbol}',
root_dir + '/build/fvp/release/bl31/bl31.elf'
], capture_output=True, encoding='ascii'
).stdout

# get rid of objdump verbosity
raw_dump = raw_dump.split('\n')[7:-1]
# split arguments and remove addresses at the start
return [line.split('\t')[2:] for line in raw_dump]

def check_identical(new, old):
if old and old[-1][0] == 'isb':
old = old[:-1]
print(' NOTE: dropped trailing isb (ok on reset)')

if not new or not old or len(new) != len(old):
return False

for newi, oldi in zip(new, old):
if newi[0] == oldi[0] == 'b':
# ignore the address, compare just the name
if newi[1].split(' ')[1] != newi[1].split(' ')[1]:
return False
continue # identical, proceed

if newi != oldi:
return False
return True

FLAG_RE = r'report_errata (.*?), '
cpu_path = old_build() + '/lib/cpus/aarch64/' + full_cpu_name() + '.S'
with open(cpu_path) as cpu_src:
errata_flags = re.findall(FLAG_RE, cpu_src.read())
errata_ids = [flg.split('_')[-1] for flg in errata_flags]

print('List of flags to build with:')
print(' '.join([flg + '=1' for flg in errata_flags]))
input((
'Press enter when your patch in argv[2] and '
'the top of master in argv[3] are both built for release...'
))

for id in errata_ids:
new_check = get_dump(new_build(),
f'check_erratum_{full_cpu_name()}_{id}')
old_check = get_dump(old_build(), f'check_errata_{id}')
new_wa = get_dump(new_build(), f'erratum_{full_cpu_name()}_{id}_wa')
old_wa = get_dump(old_build(), f'errata_{old_cpu_name()}_{id}_wa')

# remove the boilerplate for each (mov, bl, cbz, ret)
new_wa = new_wa[4:-3]
old_wa = old_wa[3:-1]

print(f'Checking {id} . . .')
if not check_identical(new_check, old_check):
print(f' Check {id} check function manually!')
if not check_identical(new_wa, old_wa):
print(f' Check {id} workaround manually!')

print('All previous errata checked against their migrations')

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I987ded7962f3449344feda47e314994f400e85b8

show more ...

refactor(cpus): convert the Cortex-A77 to use the errata framework

This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cp

refactor(cpus): convert the Cortex-A77 to use the errata framework

This involves replacing:
* the reset_func with the standard cpu_reset_func_{start,end} to apply
errata automatically
* the <cpu>_errata_report with the errata_report_shim to report errata
automatically
...and for each erratum:
* the prologue with the workaround_<type>_start to do the checks and
framework registration automatically
* the epilogue with the workaround_<type>_end
* the checker function with the check_erratum_<type> to make it more
descriptive

It is important to note that the errata workaround sequences remain
unchanged and preserve their git blame.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I5b74bf56eee95f54a1fb2fc6d3eccd86e26b522e

show more ...

refactor(cpus): reorder Cortex-A77 errata by ascending order

Errata report order is enforced to be in ascending order. To achieve
this with the errata framework this has to be done at the definition

refactor(cpus): reorder Cortex-A77 errata by ascending order

Errata report order is enforced to be in ascending order. To achieve
this with the errata framework this has to be done at the definition
level.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: Ica348d2c81e204eae2e08e9ccf677807e02efef9

show more ...

Merge "fix(cpus): workaround for Cortex-A77 erratum 2743100" into integration

fix(cpus): workaround for Cortex-A77 erratum 2743100

Cortex-A77 erratum 2743100 is a Cat B erratum that applies to revisions
r0p0, r1p0, r1p1, and is still open. The workaround is to insert a dsb
be

fix(cpus): workaround for Cortex-A77 erratum 2743100

Cortex-A77 erratum 2743100 is a Cat B erratum that applies to revisions
r0p0, r1p0, r1p1, and is still open. The workaround is to insert a dsb
before the isb in the power down sequence.

SDEN can be found here:
https://developer.arm.com/documentation/SDEN1152370/latest

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I8e49a2dac8611f31ace249a17ae7a90cd60e742a

show more ...

Merge "revert(cpus): "Revert workaround for A77 erratum 1800714"" into integration

revert(cpus): "Revert workaround for A77 erratum 1800714"

Reinstate the workaround introduced in commit
9bbc03a6e0608a949d66d9da6db12a455b452bfb. The cited change to the SDEN
could not be found and

revert(cpus): "Revert workaround for A77 erratum 1800714"

Reinstate the workaround introduced in commit
9bbc03a6e0608a949d66d9da6db12a455b452bfb. The cited change to the SDEN
could not be found and there are no known problems with the workaround.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: Iec9938f173e7565024aca798f224df339de90806

show more ...

Merge "fix(errata): workaround for Cortex-A77 erratum 2356587" into integration

fix(errata): workaround for Cortex-A77 erratum 2356587

Cortex-A77 erratum 2356587 is a cat B erratum that applies to revisions
r0p0 - r1p1 and is still open. The workaround is to set bit[0] of
CPUAC

fix(errata): workaround for Cortex-A77 erratum 2356587

Cortex-A77 erratum 2356587 is a cat B erratum that applies to revisions
r0p0 - r1p1 and is still open. The workaround is to set bit[0] of
CPUACTLR2_EL1 to force PLDW/PFRM ST to behave like PLD/PRFM LD and not
cause invalidations to other PE caches.

SDEN can be found here:
https://developer.arm.com/documentation/SDEN1152370/latest

Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I243cfd587bca06ffd2a7be5bce28f8d2c5e68230

show more ...

Merge "fix(security): workaround for CVE-2022-23960" into integration

fix(security): workaround for CVE-2022-23960

Implements the loop workaround for Cortex-A77, Cortex-A78,
Cortex-A710, Cortex-X2, Neoverse N1, Neoverse N2 and Neoverse V1
CPUs.

Signed-off-by: Bipin R

fix(security): workaround for CVE-2022-23960

Implements the loop workaround for Cortex-A77, Cortex-A78,
Cortex-A710, Cortex-X2, Neoverse N1, Neoverse N2 and Neoverse V1
CPUs.

Signed-off-by: Bipin Ravi <bipin.ravi@arm.com>
Change-Id: I11d342df7a2068a15e18f4974c645af3b341235b

show more ...