Merge changes from topic "NUMA_AWARE_PER_CPU" into integration

* changes:
docs(maintainers): add per-cpu framework into maintainers.rst
feat(per-cpu): add documentation for per-cpu framework
f

Merge changes from topic "NUMA_AWARE_PER_CPU" into integration

* changes:
docs(maintainers): add per-cpu framework into maintainers.rst
feat(per-cpu): add documentation for per-cpu framework
feat(rdv3): enable numa aware per-cpu for RD-V3-Cfg2
feat(per-cpu): migrate amu_ctx to per-cpu framework
feat(per-cpu): migrate spm_core_context to per-cpu framework
feat(per-cpu): migrate psci_ns_context to per-cpu framework
feat(per-cpu): migrate psci_cpu_pd_nodes to per-cpu framework
feat(per-cpu): migrate rmm_context to per-cpu framework
feat(per-cpu): integrate per-cpu framework into BL31/BL32
feat(per-cpu): introduce framework accessors/definers
feat(per-cpu): introduce linker changes for NUMA aware per-cpu framework
docs(changelog): add scope for per-cpu framework

show more ...

feat(per-cpu): migrate psci_cpu_pd_nodes to per-cpu framework

migrate psci_cpu_pd_nodes object to the NUMA-aware per-cpu
framework to optimize memory access and to efficiently
utilize memory.

Signe

feat(per-cpu): migrate psci_cpu_pd_nodes to per-cpu framework

migrate psci_cpu_pd_nodes object to the NUMA-aware per-cpu
framework to optimize memory access and to efficiently
utilize memory.

Signed-off-by: Sammit Joshi <sammit.joshi@arm.com>
Signed-off-by: Rohit Mathew <rohit.mathew@arm.com>
Change-Id: Idec3e3b74ecf03b420b339a183be2b9e00f8a78f

show more ...

Merge changes from topic "bk/no_blx_setup" into integration

* changes:
fix: replace stray BL2_AT_EL3 with RESET_TO_BL2
refactor(aarch64): move BL31 specific setup out of the PSCI entrypoint
re

Merge changes from topic "bk/no_blx_setup" into integration

* changes:
fix: replace stray BL2_AT_EL3 with RESET_TO_BL2
refactor(aarch64): move BL31 specific setup out of the PSCI entrypoint
refactor: unify blx_setup() and blx_main()
fix(bl2): unify the BL2 EL3 and RME entrypoints

show more ...

refactor(aarch64): move BL31 specific setup out of the PSCI entrypoint

We've charged the PSCI entrypoint with doing BL31 specific things like
setting up the EL3 context and doing feature detection.

refactor(aarch64): move BL31 specific setup out of the PSCI entrypoint

We've charged the PSCI entrypoint with doing BL31 specific things like
setting up the EL3 context and doing feature detection. Well, this is
irrelevant for sp_min and not really appropriate for PSCI. So move it to
the bl31_warmboot() function to reflect this correctly and bring the
feature detection a bit earlier, hopefully spotting more errors.

This allows for a pair of minor cleanups - we can pass the core_pos to
psci_warmboot_entrypoint() without having to refetch it, and we can put
the pauth enablement in cm_manage_extensions_el3() along with all
others. The call of that function is kept after the MMU is turned on so
that we have nicer (coherent) access to cpu_data.

Change-Id: Id031cfa0e1d8fe98919a14f9db73eb5bc9e00f67
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "fix(psci): add missing curly braces" into integration

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body w

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body within the curly braces.

Change-Id: Ida2460b7fe6f27b23382a1259a5ac93fe36bd48d
Signed-off-by: Saivardhan Thatikonda <saivardhan.thatikonda@amd.com>
Signed-off-by: Suraj Kakade <suraj.hanumantkakade@amd.com>

show more ...

Merge changes I3e086865,I47f05a9f,Iee495571 into integration

* changes:
fix(cpufeat): update FEAT_PAUTH's feat detect line to tri-state
fix(cpufeat): do feature detection before feature enableme

Merge changes I3e086865,I47f05a9f,Iee495571 into integration

* changes:
fix(cpufeat): update FEAT_PAUTH's feat detect line to tri-state
fix(cpufeat): do feature detection before feature enablement
feat(cpufeat): do feature detection on secondary cores too

show more ...

feat(cpufeat): do feature detection on secondary cores too

Feature detection currently only happens on the boot core, however, it
is possible to have asymmetry between cores. TF-A supports limited s

feat(cpufeat): do feature detection on secondary cores too

Feature detection currently only happens on the boot core, however, it
is possible to have asymmetry between cores. TF-A supports limited such
configurations so it should check secondary cores too.

Change-Id: Iee4955714685be9ae6a017af4a6c284e835ff299
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "bk/pabandon_cleanup" into integration

* changes:
feat(cpus): add pabandon support to the Alto cpu
feat(psci): optimise clock init on a pabandon
feat(psci): check that

Merge changes from topic "bk/pabandon_cleanup" into integration

* changes:
feat(cpus): add pabandon support to the Alto cpu
feat(psci): optimise clock init on a pabandon
feat(psci): check that CPUs handled a pabandon
feat(psci): make pabandon support generic
refactor(psci): unify coherency exit between AArch64 and AArch32
refactor(psci): absorb psci_power_down_wfi() into common code
refactor(platforms): remove usage of psci_power_down_wfi
fix(cm): disable SPE/TRBE correctly

show more ...

feat(psci): check that CPUs handled a pabandon

Up to now PSCI assumed that if a pabandon happened then the CPU driver
will have handled it. This patch adds a simple protocol to make sure
that this i

feat(psci): check that CPUs handled a pabandon

Up to now PSCI assumed that if a pabandon happened then the CPU driver
will have handled it. This patch adds a simple protocol to make sure
that this is indeed the case. The chosen method is with a return value
that is highly unlikely on cores that are unaware of pabandon (x0 will
be primed with 1 and if used should be overwritten with the value of
CPUPWRCTLR_EL1 which should have its last bit set to power off and its
top bits RES0; the ACK value is chosen to be the exact opposite). An
alternative method would have been to add a field in cpu_ops, however
that would have required more major refactoring across many cpus and
would have taken up more memory on older platforms, so it was not
chosen.

Change-Id: I5826c0e4802e104d295c4ecbd80b5f676d2cd871
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(psci): make pabandon support generic

Support for aborted powerdowns does not require much dedicated code.
Rather, it is largely a matter of orchestrating things to happen in the
right order.

T

feat(psci): make pabandon support generic

Support for aborted powerdowns does not require much dedicated code.
Rather, it is largely a matter of orchestrating things to happen in the
right order.

The only exception to this are older secure world dispatchers, which
assume that a CPU_SUSPEND call will be terminal and therefore can
clobber context. This was patched over in common code and hidden behind
a flag. This patch moves this to the dispatchers themselves.

Dispatchers that don't register svc_suspend{_finish} are unaffected.
Those that do must save the NS context before clobbering it and
restoring in only in case of a pabandon. Due to this operation being
non-trivial, this patch makes the assumption that these dispatchers will
only be present on hardware that does not support pabandon and therefore
does not add any contexting for them. In case this assumption ever
changes, asserts are added that should alert us of this change.

Change-Id: I94a907515b782b4d2136c0d274246cfe1d567c0e
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(psci): unify coherency exit between AArch64 and AArch32

The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't

refactor(psci): unify coherency exit between AArch64 and AArch32

The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't, then we
must turn data caches off, handle the confusion that causes with the
stack, and call into the cpu driver which will flush the caches that
need flushing.

On AArch32 the above happens in common code. On AArch64, however, the
turning off of the caches happens in the cpu driver. Since we're dealing
with the stack, we must exercise control over it and implement this in
assembly. But as the two implementations are nominally different (in the
ordering of operations), the part that is in assembly is quite large as
jumping back to C to handle the difference might involve the stack.

Presumably, the AArch difference was introduced in order to cater for a
possible implementation where turning off the caches requires an IMP DEF
sequence. Well, Arm no longer makes cores without hardware assisted
coherency, so this eventually is not possible.

So take this part out of the cpu driver and put it into common code,
just like in AArch32. With this, there is no longer a need call
prepare_cpu_pwr_dwn() in a different order either - we can delay it a
bit to happen after the stack management. So the two AArch-s flows
become identical. We can convert prepare_cpu_pwr_dwn() to C and leave
psci_do_pwrdown_cache_maintenance() only to exercise control over stack.

Change-Id: Ie4759ebe20bb74b60533c6a47dbc2b101875900f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called f

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called from
non-PSCI code as PSCI is smart enough to handle this after platform
hooks. So absorb the functions into common code.

This allows for a tiny bit of optimisation: there will be no branch
(that can be missed or non-cached) to a non-inlineable function. Then in
the terminal case we can call wfi() directly with the application of the
erratum before the loop. And finally in the wakeup case, we don't have
to explicitly clear the errata as that will happen automatically on the
second call of prepare_cpu_pwr_dwn().

The A510 erratum requires a tsb csync before the dsb+wfi combo to turn
the core off. We can do this a little bit earlier in the cpu hook and
relieve common code from the responsibility. EL3 is always a prohibited
region so the buffer will stay empty.

Change-Id: I5f950df3fb7b0736df4ce25a21f78b29896de215
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(psci): add API to get number of CPUs currently in ON state" into integration

feat(psci): add API to get number of CPUs currently in ON state

Introduce a new PSCI helper function `psci_num_cpus_running_on_safe()`
to return the number of CPUs that are currently in the ON state

feat(psci): add API to get number of CPUs currently in ON state

Introduce a new PSCI helper function `psci_num_cpus_running_on_safe()`
to return the number of CPUs that are currently in the ON state.
This API locks the PSCI power state data to ensure consistency and
is safe to call in concurrent environments.

This utility can assist components that need to reason about system
CPU activity or coordinate operations based on active CPUs.

Change-Id: Ie15aa4bd393a5f01e7cd80ae8a9b28707fde7c53
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

show more ...

Merge "feat(psci): remove cpu context init by index" into integration

feat(psci): remove cpu context init by index

Currently, the calling core (meaning the core which received the call to
CPU_ON or the powerdown path of CPU_SUSPEND on the same core) is in
charge of in

feat(psci): remove cpu context init by index

Currently, the calling core (meaning the core which received the call to
CPU_ON or the powerdown path of CPU_SUSPEND on the same core) is in
charge of initialising the context for the waking core (the warmboot
entrypoint for both). This is convenient because the calling core can
write the context while in coherency and the waking core will only need
the context after its entered coherency. This avoids any cache
maintenance and makes communication simple.

However, this has 3 main problems:
a) asymmetric feature support is problematic - the calling core has no
way of knowing the feature set of the waking core. If the two
diverge, the architectural feature discovery via ID registers breaks
down. We've thus far "fixed" this on a case by case basis which
doesn't scale and introduces redundancy.

b) powerdown abandon (pabandon) introduces a contradiction - the calling
core has to initialise the context for when the core wakes up, but
should the core not powerdown it needs its old context intact. The only
way to work around this is by keeping two copies of context which
incurs a runtime and memory overhead.

c) cm_prepare_el3_exit[_ns]() doesn't have access to the entrypoint but needs
it to make initialisation decisions. We can infer some of this from
registers that have already been written but this is awkwardly
limiting for what we can do. This also necessitates the split from
the context initialisation.

We can solve all three by a making a core be in full ownership of its
own context. The calling core then only writes entrypoint information
and nothing else. The waking core then initialises its own context as it
sees fit with full knowledge of the whole picture.

The only tricky bit is cache coherency - the waking core has to be able
to coherently observe its new entrypoint. Calling cores will write to
the shared region with coherent caches on. If we make sure to read the
context only after the waking core has entered coherency, then we can
avoid cache operations and let hardware handle everything.

We can skip the spsr check for FEAT_TCR2 as it doesn't make a
difference. We can also skip enabling it twice from generic code.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I86e7fe8b698191fc3b469e5ced1fd010f8754b0e

show more ...

Merge changes from topic "xlnx_fix_gen_uniq_var" into integration

* changes:
fix(psci): avoid altering function parameters
fix(services): avoid altering function parameters
fix(common): ignore

Merge changes from topic "xlnx_fix_gen_uniq_var" into integration

* changes:
fix(psci): avoid altering function parameters
fix(services): avoid altering function parameters
fix(common): ignore the unused function return value
fix(psci): modify variable conflicting with external function
fix(delay-timer): create unique variable name

show more ...

Merge changes from topic "xlnx_fix_gen_datatype_cast" into integration

* changes:
fix(psci): add const qualifier
fix(el3-runtime): add const qualifier
fix(bl31): add const qualifier
fix(cons

Merge changes from topic "xlnx_fix_gen_datatype_cast" into integration

* changes:
fix(psci): add const qualifier
fix(el3-runtime): add const qualifier
fix(bl31): add const qualifier
fix(console): typecast expressions to match data type
fix(arm-drivers): typecast expressions to match data type
fix(arm-drivers): align essential type categories
fix(arm-drivers): typecast expression to match data type

show more ...

fix(psci): avoid altering function parameters

This corrects the MISRA violation C2012-17.8:
A function parameter should not be modified.
Local variable is declared and used to process the value
from

fix(psci): avoid altering function parameters

This corrects the MISRA violation C2012-17.8:
A function parameter should not be modified.
Local variable is declared and used to process the value
from the argument.

Change-Id: Ia757db4903132794623dbf92ff8cecc9b40f170d
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

fix(psci): add const qualifier

This corrects the MISRA violation C2012-8.13:
A pointer should point to a const-qualified type whenever possible.
Added const qualifier to pointer in the function argu

fix(psci): add const qualifier

This corrects the MISRA violation C2012-8.13:
A pointer should point to a const-qualified type whenever possible.
Added const qualifier to pointer in the function arguments.

Change-Id: Id3d4aa528f275973a37c0b9af04495632cb2dda3
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

Merge changes from topic "xlnx_fix_gen_missing_brace" into integration

* changes:
fix(platforms): modify function to have single return
fix(el3-runtime): add missing curly braces
fix(locks): a

Merge changes from topic "xlnx_fix_gen_missing_brace" into integration

* changes:
fix(platforms): modify function to have single return
fix(el3-runtime): add missing curly braces
fix(locks): add missing curly braces
fix(psci): add missing curly braces
fix(bl31): add missing curly braces
fix(console): add missing curly braces
fix(arm-drivers): add missing curly braces
fix(common): add missing curly braces
fix(platforms): add missing curly braces

show more ...

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body w

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body within the curly braces.

Change-Id: I8b656f59b445e914dd3f47e3dde83735481a3640
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

Merge changes I3d950e72,Id315a8fe,Ib62e6e9b,I1d0475b2 into integration

* changes:
perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context
perf(psci): get PMF timestamps wi

Merge changes I3d950e72,Id315a8fe,Ib62e6e9b,I1d0475b2 into integration

* changes:
perf(cm): drop ZCR_EL3 saving and some ISBs and replace them with root context
perf(psci): get PMF timestamps with no cache flushes if possible
perf(amu): greatly simplify AMU context management
perf(mpmm): greatly simplify MPMM enablement

show more ...

perf(amu): greatly simplify AMU context management

The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness

perf(amu): greatly simplify AMU context management

The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness in that this is rather slow. A large part of it is written in
assembly, making it opaque to the compiler for optimisations. The
future proofness requires reading registers that are effectively
`volatile`, making it even harder for the compiler, as well as adding
lots of implicit barriers, making it hard for the microarchitecutre to
optimise as well.

We can make a few assumptions, checked by a few well placed asserts, and
remove a lot of this burden. For a start, at the moment there are 4
group 0 counters with static assignments. Contexting them is a trivial
affair that doesn't need a loop. Similarly, there can only be up to 16
group 1 counters. Contexting them is a bit harder, but we can do with a
single branch with a falling through switch. If/when both of these
change, we have a pair of asserts and the feature detection mechanism to
guard us against pretending that we support something we don't.

We can drop contexting of the offset registers. They are fully
accessible by EL2 and as such are its responsibility to preserve on
powerdown.

Another small thing we can do, is pass the core_pos into the hook.
The caller already knows which core we're running on, we don't need to
call this non-trivial function again.

Finally, knowing this, we don't really need the auxiliary AMUs to be
described by the device tree. Linux doesn't care at the moment, and any
information we need for EL3 can be neatly placed in a simple array.

All of this, combined with lifting the actual saving out of assembly,
reduces the instructions to save the context from 180 to 40, including a
lot fewer branches. The code is also much shorter and easier to read.

Also propagate to aarch32 so that the two don't diverge too much.

Change-Id: Ib62e6e9ba5be7fb9fb8965c8eee148d5598a5361
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...