Merge changes from topic "bk/pabandon_cleanup" into integration

* changes:
feat(cpus): add pabandon support to the Alto cpu
feat(psci): optimise clock init on a pabandon
feat(psci): check that

Merge changes from topic "bk/pabandon_cleanup" into integration

* changes:
feat(cpus): add pabandon support to the Alto cpu
feat(psci): optimise clock init on a pabandon
feat(psci): check that CPUs handled a pabandon
feat(psci): make pabandon support generic
refactor(psci): unify coherency exit between AArch64 and AArch32
refactor(psci): absorb psci_power_down_wfi() into common code
refactor(platforms): remove usage of psci_power_down_wfi
fix(cm): disable SPE/TRBE correctly

show more ...

refactor(psci): unify coherency exit between AArch64 and AArch32

The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't

refactor(psci): unify coherency exit between AArch64 and AArch32

The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't, then we
must turn data caches off, handle the confusion that causes with the
stack, and call into the cpu driver which will flush the caches that
need flushing.

On AArch32 the above happens in common code. On AArch64, however, the
turning off of the caches happens in the cpu driver. Since we're dealing
with the stack, we must exercise control over it and implement this in
assembly. But as the two implementations are nominally different (in the
ordering of operations), the part that is in assembly is quite large as
jumping back to C to handle the difference might involve the stack.

Presumably, the AArch difference was introduced in order to cater for a
possible implementation where turning off the caches requires an IMP DEF
sequence. Well, Arm no longer makes cores without hardware assisted
coherency, so this eventually is not possible.

So take this part out of the cpu driver and put it into common code,
just like in AArch32. With this, there is no longer a need call
prepare_cpu_pwr_dwn() in a different order either - we can delay it a
bit to happen after the stack management. So the two AArch-s flows
become identical. We can convert prepare_cpu_pwr_dwn() to C and leave
psci_do_pwrdown_cache_maintenance() only to exercise control over stack.

Change-Id: Ie4759ebe20bb74b60533c6a47dbc2b101875900f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called f

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called from
non-PSCI code as PSCI is smart enough to handle this after platform
hooks. So absorb the functions into common code.

This allows for a tiny bit of optimisation: there will be no branch
(that can be missed or non-cached) to a non-inlineable function. Then in
the terminal case we can call wfi() directly with the application of the
erratum before the loop. And finally in the wakeup case, we don't have
to explicitly clear the errata as that will happen automatically on the
second call of prepare_cpu_pwr_dwn().

The A510 erratum requires a tsb csync before the dsb+wfi combo to turn
the core off. We can do this a little bit earlier in the cpu hook and
relieve common code from the responsibility. EL3 is always a prohibited
region so the buffer will stay empty.

Change-Id: I5f950df3fb7b0736df4ce25a21f78b29896de215
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes I765a7fa0,Ic33f0b6d,I8d1a88c7,I381f96be,I698fa849, ... into integration

* changes:
fix(cpus): clear CPUPWRCTLR_EL1.CORE_PWRDN_EN_BIT on reset
chore(docs): drop the "wfi" from `pwr_

Merge changes I765a7fa0,Ic33f0b6d,I8d1a88c7,I381f96be,I698fa849, ... into integration

* changes:
fix(cpus): clear CPUPWRCTLR_EL1.CORE_PWRDN_EN_BIT on reset
chore(docs): drop the "wfi" from `pwr_domain_pwr_down_wfi`
chore(psci): drop skip_wfi variable
feat(arm): convert arm platforms to expect a wakeup
fix(cpus): avoid SME related loss of context on powerdown
feat(psci): allow cores to wake up from powerdown
refactor: panic after calling psci_power_down_wfi()
refactor(cpus): undo errata mitigations
feat(cpus): add sysreg_bit_toggle

show more ...

feat(psci): allow cores to wake up from powerdown

The simplistic view of a core's powerdown sequence is that power is
atomically cut upon calling `wfi`. However, it turns out that it has
lots to do

feat(psci): allow cores to wake up from powerdown

The simplistic view of a core's powerdown sequence is that power is
atomically cut upon calling `wfi`. However, it turns out that it has
lots to do - it has to talk to the interconnect to exit coherency, clean
caches, check for RAS errors, etc. These take significant amounts of
time and are certainly not atomic. As such there is a significant window
of opportunity for external events to happen. Many of these steps are
not destructive to context, so theoretically, the core can just "give
up" half way (or roll certain actions back) and carry on running. The
point in this sequence after which roll back is not possible is called
the point of no return.

One of these actions is the checking for RAS errors. It is possible for
one to happen during this lengthy sequence, or at least remain
undiscovered until that point. If the core were to continue powerdown
when that happens, there would be no (easy) way to inform anyone about
it. Rejecting the powerdown and letting software handle the error is the
best way to implement this.

Arm cores since at least the a510 have included this exact feature. So
far it hasn't been deemed necessary to account for it in firmware due to
the low likelihood of this happening. However, events like GIC wakeup
requests are much more probable. Older cores will powerdown and
immediately power back up when this happens. Travis and Gelas include a
feature similar to the RAS case above, called powerdown abandon. The
idea is that this will improve the latency to service the interrupt by
saving on work which the core and software need to do.

So far firmware has relied on the `wfi` being the point of no return and
if it doesn't explicitly detect a pending interrupt quite early on, it
will embark onto a sequence that it expects to end with shutdown. To
accommodate for it not being a point of no return, we must undo all of
the system management we did, just like in the warm boot entrypoint.

To achieve that, the pwr_domain_pwr_down_wfi hook must not be terminal.
Most recent platforms do some platform management and finish on the
standard `wfi`, followed by a panic or an endless loop as this is
expected to not return. To make this generic, any platform that wishes
to support wakeups must instead let common code call
`psci_power_down_wfi()` right after. Besides wakeups, this lets common
code handle powerdown errata better as well.

Then, the CPU_OFF case is simple - PSCI does not allow it to return. So
the best that can be done is to attempt the `wfi` a few times (the
choice of 32 is arbitrary) in the hope that the wakeup is transient. If
it isn't, the only choice is to panic, as the system is likely to be in
a bad state, eg. interrupts weren't routed away. The same applies for
SYSTEM_OFF, SYSTEM_RESET, and SYSTEM_RESET2. There the panic won't
matter as the system is going offline one way or another. The RAS case
will be considered in a separate patch.

Now, the CPU_SUSPEND case is more involved. First, to powerdown it must
wipe its context as it is not written on warm boot. But it cannot be
overwritten in case of a wakeup. To avoid the catch 22, save a copy that
will only be used if powerdown fails. That is about 500 bytes on the
stack so it hopefully doesn't tip anyone over any limits. In future that
can be avoided by having a core manage its own context.

Second, when the core wakes up, it must undo anything it did to prepare
for poweroff, which for the cores we care about, is writing
CPUPWRCTLR_EL1.CORE_PWRDN_EN. The least intrusive for the cpu library
way of doing this is to simply call the power off hook again and have
the hook toggle the bit. If in the future there need to be more complex
sequences, their direction can be advised on the value of this bit.

Third, do the actual "resume". Most of the logic is already there for
the retention suspend, so that only needs a small touch up to apply to
the powerdown case as well. The missing bit is the powerdown specific
state management. Luckily, the warmboot entrypoint does exactly that
already too, so steal that and we're done.

All of this is hidden behind a FEAT_PABANDON flag since it has a large
memory and runtime cost that we don't want to burden non pabandon cores
with.

Finally, do some function renaming to better reflect their purpose and
make names a little bit more consistent.

Change-Id: I2405b59300c2e24ce02e266f91b7c51474c1145f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cpus): undo errata mitigations

The workarounds introduced in the three patches starting at
888eafa00b99aa06b4ff688407336811a7ff439a assumed that any powerdown
request will be (forced to be)

refactor(cpus): undo errata mitigations

The workarounds introduced in the three patches starting at
888eafa00b99aa06b4ff688407336811a7ff439a assumed that any powerdown
request will be (forced to be) terminal. This assumption can no longer
be the case for new CPUs so there is a need to revisit these older
cores. Since we may wake up, we now need to respect the workaround's
recommendation that the workaround needs to be reverted on wakeup. So do
exactly that.

Introduce a new helper to toggle bits in assembly. This allows us to
call the workaround twice, with the first call setting the workaround
and second undoing it. This is also used for gelas' an travis' powerdown
routines. This is so the same function can be called again

Also fix the condition in the cpu helper macro as it was subtly wrong

Change-Id: Iff9e5251dc9d8670d085d88c070f78991955e7c3
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "chore(cpus): optimise runtime errata applications" into integration

chore(cpus): optimise runtime errata applications

The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dw

chore(cpus): optimise runtime errata applications

The errata framework has a helper to invoke workarounds, complete with a
cpu rev_var check. We can use that directly instead of the
apply_cpu_pwr_dwn_errata to save on some code, as well as an extra
branch. It's also more readable.

Also, apply_erratum invocation in cpu files don't need to check the
rev_var as that was already done by the cpu_ops dispatcher for us to end
up in the file.

Finally, X2 erratum 2768515 only applies in the powerdown sequence, i.e.
at runtime. It doesn't achieve anything at reset, so we can label it
accordingly.

Change-Id: I02f9dd7d0619feb54c870938ea186be5e3a6ca7b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "chore: update to use Arm word across TF-A" into integration

chore: update to use Arm word across TF-A

Align entire TF-A to use Arm in copyright header.

Change-Id: Ief9992169efdab61d0da6bd8c5180de7a4bc2244
Signed-off-by: Govindraj Raja <govindraj.raja@arm.co

chore: update to use Arm word across TF-A

Align entire TF-A to use Arm in copyright header.

Change-Id: Ief9992169efdab61d0da6bd8c5180de7a4bc2244
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

Merge "fix(cpus): workaround for Cortex-A510 erratum 2684597" into integration

Merge "fix(psci): tighten psci_power_down_wfi behaviour" into integration

fix(cpus): workaround for Cortex-A510 erratum 2684597

Cortex-A510 erratum 2684597 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3, r1p0, r1p1 and r1p2. It is fixed in r1p3. The
w

fix(cpus): workaround for Cortex-A510 erratum 2684597

Cortex-A510 erratum 2684597 is a Cat B erratum that applies to revisions
r0p0, r0p1, r0p2, r0p3, r1p0, r1p1 and r1p2. It is fixed in r1p3. The
workaround is to execute a TSB CSYNC and DSB before executing WFI for
power down.

SDEN can be found here:
https://developer.arm.com/documentation/SDEN1873361/latest
https://developer.arm.com/documentation/SDEN1873351/latest

Change-Id: Ic0b24b600bc013eb59c797401fbdc9bda8058d6d
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>

show more ...

fix(psci): tighten psci_power_down_wfi behaviour

A processing element should never return from a wfi, however, due to a
hardware bug, certain CPUs may wake up because of an external event.
This patc

fix(psci): tighten psci_power_down_wfi behaviour

A processing element should never return from a wfi, however, due to a
hardware bug, certain CPUs may wake up because of an external event.
This patch tightens the behaviour of the common power down sequence, it
ensures the routine never returns by entering a wfi loop at its end. It
aligns with the behaviour of the platform implementations.

Change-Id: I36d8b0c64eccb71035bf164b4cd658d66ed7beb4
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>

show more ...

Merge pull request #1726 from antonio-nino-diaz-arm/an/includes

Sanitise includes across codebase

Sanitise includes across codebase

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- inclu

Sanitise includes across codebase

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928d5b7 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a282 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca33988b9 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

show more ...

Merge pull request #1587 from antonio-nino-diaz-arm/an/deprecated

Remove deprecated interfaces for all platforms

Remove all other deprecated interfaces and files

Change-Id: Icd1cdd42afdc78895a9be6c46b414b0a155cfa63
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

PSCI: Remove deprecated file plat_psci_common.c

Change-Id: I9fd8016527ad7706494f34356fdae8efacef5f72
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Merge pull request #1513 from antonio-nino-diaz-arm/an/xlat-caches

xlat v2: Cleanup and dcache coherency bug fix

xlat v2: Flush xlat tables after being modified

During cold boot, the initial translation tables are created with data
caches disabled, so all modifications go to memory directly. After the
MMU is e

xlat v2: Flush xlat tables after being modified

During cold boot, the initial translation tables are created with data
caches disabled, so all modifications go to memory directly. After the
MMU is enabled and data cache is enabled, any modification to the tables
goes to data cache, and eventually may get flushed to memory.

If CPU0 modifies the tables while CPU1 is off, CPU0 will have the
modified tables in its data cache. When CPU1 is powered on, the MMU is
enabled, then it enables coherency, and then it enables the data cache.
Until this is done, CPU1 isn't in coherency, and the translation tables
it sees can be outdated if CPU0 still has some modified entries in its
data cache.

This can be a problem in some cases. For example, the warm boot code
uses only the tables mapped during cold boot, which don't normally
change. However, if they are modified (and a RO page is made RW, or a XN
page is made executable) the CPU will see the old attributes and crash
when it tries to access it.

This doesn't happen in systems with HW_ASSISTED_COHERENCY or
WARMBOOT_ENABLE_DCACHE_EARLY. In these systems, the data cache is
enabled at the same time as the MMU. As soon as this happens, the CPU is
in coherency.

There was an attempt of a fix in psci_helpers.S, but it didn't solve the
problem. That code has been deleted. The code was introduced in commit
<264410306381> ("Invalidate TLB entries during warm boot").

Now, during a map or unmap operation, the memory associated to each
modified table is flushed. Traversing a table will also flush it's
memory, as there is no way to tell in the current implementation if the
table that has been traversed has also been modified.

Change-Id: I4b520bca27502f1018878061bc5fb82af740bb92
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

show more ...

Merge pull request #1286 from antonio-nino-diaz-arm/an/mmu-mismatch

Clarify comments in xlat tables lib and fixes related to the TLB

Invalidate TLB entries during warm boot

During the warm boot sequence:

1. The MMU is enabled with the data cache disabled. The MMU table walker
is set up to access the translation tables as in c

Invalidate TLB entries during warm boot

During the warm boot sequence:

1. The MMU is enabled with the data cache disabled. The MMU table walker
is set up to access the translation tables as in cacheable memory,
but its accesses are non-cacheable because SCTLR_EL3.C controls them
as well.
2. The interconnect is set up and the CPU enters coherency with the
rest of the system.
3. The data cache is enabled.

If the support for dynamic translation tables is enabled and another CPU
makes changes to a region, the changes may only be present in the data
cache, not in RAM. The CPU that is booting isn't in coherency with the
rest of the system, so the table walker of that CPU isn't either. This
means that it may read old entries from RAM and it may have invalid TLB
entries corresponding to the dynamic mappings.

This is not a problem for the boot code because the mapping is 1:1 and
the regions are static. However, the code that runs after the boot
sequence may need to access the dynamically mapped regions.

This patch invalidates all TLBs during warm boot when the dynamic
translation tables support is enabled to prevent this problem.

Change-Id: I80264802dc0aa1cb3edd77d0b66b91db6961af3d
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

show more ...

Merge pull request #925 from dp-arm/dp/spdx

Use SPDX license identifiers

Use SPDX license identifiers

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by

Use SPDX license identifiers

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/

Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

show more ...