feat(psci): check that CPUs handled a pabandon

Up to now PSCI assumed that if a pabandon happened then the CPU driver
will have handled it. This patch adds a simple protocol to make sure
that this i

feat(psci): check that CPUs handled a pabandon

Up to now PSCI assumed that if a pabandon happened then the CPU driver
will have handled it. This patch adds a simple protocol to make sure
that this is indeed the case. The chosen method is with a return value
that is highly unlikely on cores that are unaware of pabandon (x0 will
be primed with 1 and if used should be overwritten with the value of
CPUPWRCTLR_EL1 which should have its last bit set to power off and its
top bits RES0; the ACK value is chosen to be the exact opposite). An
alternative method would have been to add a field in cpu_ops, however
that would have required more major refactoring across many cpus and
would have taken up more memory on older platforms, so it was not
chosen.

Change-Id: I5826c0e4802e104d295c4ecbd80b5f676d2cd871
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(psci): make pabandon support generic

Support for aborted powerdowns does not require much dedicated code.
Rather, it is largely a matter of orchestrating things to happen in the
right order.

T

feat(psci): make pabandon support generic

Support for aborted powerdowns does not require much dedicated code.
Rather, it is largely a matter of orchestrating things to happen in the
right order.

The only exception to this are older secure world dispatchers, which
assume that a CPU_SUSPEND call will be terminal and therefore can
clobber context. This was patched over in common code and hidden behind
a flag. This patch moves this to the dispatchers themselves.

Dispatchers that don't register svc_suspend{_finish} are unaffected.
Those that do must save the NS context before clobbering it and
restoring in only in case of a pabandon. Due to this operation being
non-trivial, this patch makes the assumption that these dispatchers will
only be present on hardware that does not support pabandon and therefore
does not add any contexting for them. In case this assumption ever
changes, asserts are added that should alert us of this change.

Change-Id: I94a907515b782b4d2136c0d274246cfe1d567c0e
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(psci): unify coherency exit between AArch64 and AArch32

The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't

refactor(psci): unify coherency exit between AArch64 and AArch32

The procedure is fairly simple: if we have hardware assisted coherency,
call into the cpu driver and let it do its thing. If we don't, then we
must turn data caches off, handle the confusion that causes with the
stack, and call into the cpu driver which will flush the caches that
need flushing.

On AArch32 the above happens in common code. On AArch64, however, the
turning off of the caches happens in the cpu driver. Since we're dealing
with the stack, we must exercise control over it and implement this in
assembly. But as the two implementations are nominally different (in the
ordering of operations), the part that is in assembly is quite large as
jumping back to C to handle the difference might involve the stack.

Presumably, the AArch difference was introduced in order to cater for a
possible implementation where turning off the caches requires an IMP DEF
sequence. Well, Arm no longer makes cores without hardware assisted
coherency, so this eventually is not possible.

So take this part out of the cpu driver and put it into common code,
just like in AArch32. With this, there is no longer a need call
prepare_cpu_pwr_dwn() in a different order either - we can delay it a
bit to happen after the stack management. So the two AArch-s flows
become identical. We can convert prepare_cpu_pwr_dwn() to C and leave
psci_do_pwrdown_cache_maintenance() only to exercise control over stack.

Change-Id: Ie4759ebe20bb74b60533c6a47dbc2b101875900f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called f

refactor(psci): absorb psci_power_down_wfi() into common code

The AArch64 and AArch32 variants are not that different so there is no
need for them to be in assembly. They should also not be called from
non-PSCI code as PSCI is smart enough to handle this after platform
hooks. So absorb the functions into common code.

This allows for a tiny bit of optimisation: there will be no branch
(that can be missed or non-cached) to a non-inlineable function. Then in
the terminal case we can call wfi() directly with the application of the
erratum before the loop. And finally in the wakeup case, we don't have
to explicitly clear the errata as that will happen automatically on the
second call of prepare_cpu_pwr_dwn().

The A510 erratum requires a tsb csync before the dsb+wfi combo to turn
the core off. We can do this a little bit earlier in the cpu hook and
relieve common code from the responsibility. EL3 is always a prohibited
region so the buffer will stay empty.

Change-Id: I5f950df3fb7b0736df4ce25a21f78b29896de215
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(psci): initialise variable to default zero

This corrects the MISRA violation C2012-9.1:

The value of an object with automatic storage duration shall
not be read before it has been set.Initializ

fix(psci): initialise variable to default zero

This corrects the MISRA violation C2012-9.1:

The value of an object with automatic storage duration shall
not be read before it has been set.Initialized the variable
to default value zero.

Change-Id: Ib3a2a853b82ce3c3ba1f518705d7ac2e01130e37
Signed-off-by: Prasad Kummari <prasad.kummari@amd.com>
Signed-off-by: Saivardhan Thatikonda <saivardhan.thatikonda@amd.com>

show more ...

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:

The body of an iteration-statement or a selection-statement
shall be a compound-statement.Enclosed statement body w

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:

The body of an iteration-statement or a selection-statement
shall be a compound-statement.Enclosed statement body within
the curly braces.

Change-Id: Ibd402a52e44e59d1c37613d8925aac8e115aa31c
Signed-off-by: Prasad Kummari <prasad.kummari@amd.com>
Signed-off-by: Saivardhan Thatikonda <saivardhan.thatikonda@amd.com>

show more ...

fix(psci): do not modify higher levels for standby in OSI mode

The OSI mode for the CPU standby fast path updates the requested power
states incorrectly. psci_update_req_local_pwr_states() will not

fix(psci): do not modify higher levels for standby in OSI mode

The OSI mode for the CPU standby fast path updates the requested power
states incorrectly. psci_update_req_local_pwr_states() will not bother
saving level 0. It will also clamp the level it accesses to the
end_pwrlvl argument. Well, in the fast path, this argument is always 0
(as a condition of entering the fast path). The result is that it reads
the level 0 power state (retention) and writes it to the level 1 power
state, even though by definition, the fast path will not update higher
power levels.

The fix is to pass PLAT_MAX_PWR_LVL instead of target_pwrlvl to
psci_update_req_local_pwr_states(). However, doing that makes it
apparent that this call will not change anything and therefore there is
nothing to save - the level 0 power state is written by
psci_set_cpu_local_state(). So the save + restore are redundant. Remove
them to save on a bunch of work.

Another thing that is not considered is locking. No locks are held in
the fast path, so these updates are not safe.

Change-Id: Icc785df87b294b938b59705116369abd80ccf4da
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(psci): typecast operands to match data type

This corrects the MISRA violation C2012-10.3:
The value of an expression shall not be assigned to an object with a
narrower essential type or of a dif

fix(psci): typecast operands to match data type

This corrects the MISRA violation C2012-10.3:
The value of an expression shall not be assigned to an object with a
narrower essential type or of a different essential type category.
The condition is explicitly checked against 0U, appending 'U' and
typecasting for unsigned comparison.

Change-Id: Ic4a109cb55fdf9b60c8d26df68f61811b59a1a9f
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

Merge "perf(bl31): convert cpu_data fetching to C" into integration

perf(bl31): convert cpu_data fetching to C

The assembly routines are opaque to the compiler and it can't inline
them. There is also no requirement for them to be called without a
stack - each of the

perf(bl31): convert cpu_data fetching to C

The assembly routines are opaque to the compiler and it can't inline
them. There is also no requirement for them to be called without a
stack - each of their calls has a stack available. So convert them to C
so that the compiler can do its inlining magic.

On AArch32 we need to be able to call _cpu_data from the entrypoint so
it has to stay as a slight exception.

We can also straighten out the type of the cpu_ops_ptr member so we
don't have to cast it everywhere.

Change-Id: I9c2939a955b396edf26b99ef36318eebeaab13e6
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(psci): add API to get number of CPUs currently in ON state

Introduce a new PSCI helper function `psci_num_cpus_running_on_safe()`
to return the number of CPUs that are currently in the ON state

feat(psci): add API to get number of CPUs currently in ON state

Introduce a new PSCI helper function `psci_num_cpus_running_on_safe()`
to return the number of CPUs that are currently in the ON state.
This API locks the PSCI power state data to ensure consistency and
is safe to call in concurrent environments.

This utility can assist components that need to reason about system
CPU activity or coordinate operations based on active CPUs.

Change-Id: Ie15aa4bd393a5f01e7cd80ae8a9b28707fde7c53
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

show more ...

refactor(gic): promote most of the GIC driver to common code

More often than not, Arm based systems include some revision of a GIC.
There are two ways of adding support for them in platform code - c

refactor(gic): promote most of the GIC driver to common code

More often than not, Arm based systems include some revision of a GIC.
There are two ways of adding support for them in platform code - calling
the top-level helpers from plat/arm/common/arm_gicvX.c or by using the
driver directly. Both of these methods allow for a high degree of
customisation - most functions are defined to be weak and there are no
calls to any of them in generic code.

As it turns out, requirements around those GICs are largely the same.
Platforms that use arm_gicvX.c use the helpers identically among each
other. Platforms that use the driver directly tend to end up with calls
that look a lot like the arm_gicvX.c helpers and the weakness of the
functions are never exercised.

All of this results in a lot of code duplication to do what is
essentially the same thing. Even though it's not a lot of code, when
multiplied among many platforms it becomes significant and makes
refactoring it quite difficult. It's also bug prone since the steps are
a little convoluted and things are likely to work even with subtle
errors (see 50009f61177421118f42d6a000611ba0e613d54b).

So promote as much of the GIC to be called from common code. Do the
setup in bl31_main() and have every PSCI method do the state management
directly instead of delegating it to the platform hooks. We can base
this implementation on arm_gicvX.c since they already offer logical
names and have worked quite well so far with minimal changes.

The main benefit of doing this is reduced code duplication. If we assume
that, outside of some platform setup, GIC management is identical, then
a platform can add support by telling the build system, regardless of
GIC revision. The other benefit is performance - BL31 and PSCI already
know the core_pos and they can pass it as an argument instead of having
to call plat_my_core_pos(). Now, the only platform specific GIC actions
necessary are the saving and restoring of context on entering and
exiting a power domain. The PSCI library does not keep track of this so
it is unable perform it itself. The routines themselves are also
provided.

For compatibility all of this is hidden behind a build flag. Platforms
are encouraged to adopt this driver, but it would not be practical to
convert and validate every GIC based platform.

This patch renames the functions in question to follow the
gic_<function>() convention. This allows the names to be version
agnostic.

Finally, drop the weak definitions - they are unused, likely to remain
so, and can be added back if the need arises.

Change-Id: I5b5267f4b72f633fb1096400ec8e4b208694135f
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes Id942c20c,Idd286bea,I8917a26e,Iec8c3477,If3c25dcd, ... into integration

* changes:
feat(cpufeat): enable FEAT_PAuth to FEAT_STATE_CHECKED
perf(cpufeat): centralise PAuth key saving

Merge changes Id942c20c,Idd286bea,I8917a26e,Iec8c3477,If3c25dcd, ... into integration

* changes:
feat(cpufeat): enable FEAT_PAuth to FEAT_STATE_CHECKED
perf(cpufeat): centralise PAuth key saving
refactor(cpufeat): convert FEAT_PAuth setup to C
refactor(cpufeat): prepare FEAT_PAuth for FEATURE_DETECTION
chore(cpufeat): remove PAuth presence checks
feat(cpufeat): enable FEAT_BTI to FEAT_STATE_CHECKED

show more ...

perf(cpufeat): centralise PAuth key saving

prepare_el3_entry() is meant to be the one-stop shop for all the context
we must fiddle with to enter EL3 proper. However, PAuth is the one
exception, happ

perf(cpufeat): centralise PAuth key saving

prepare_el3_entry() is meant to be the one-stop shop for all the context
we must fiddle with to enter EL3 proper. However, PAuth is the one
exception, happening right after. Absorb it into prepare_el3_entry(),
handling the BL1/BL31 difference.

This is a good time to also move the key saving into the enable
function, also to centralise. With this it becomes apparent that saving
keys just before CPU_SUSPEND is redundant as they will be reinitialised
when the core wakes up.

Note that the key loading, now in save_gp_pmcr_pauth_regs, does not end
in an isb. The effects of the key change are not needed until the isb
in the caller, so this isb is not needed.

Change-Id: Idd286bea91140c106ab4c933c5c44b0bc2050ca2
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "feat(psci): remove cpu context init by index" into integration

feat(psci): remove cpu context init by index

Currently, the calling core (meaning the core which received the call to
CPU_ON or the powerdown path of CPU_SUSPEND on the same core) is in
charge of in

feat(psci): remove cpu context init by index

Currently, the calling core (meaning the core which received the call to
CPU_ON or the powerdown path of CPU_SUSPEND on the same core) is in
charge of initialising the context for the waking core (the warmboot
entrypoint for both). This is convenient because the calling core can
write the context while in coherency and the waking core will only need
the context after its entered coherency. This avoids any cache
maintenance and makes communication simple.

However, this has 3 main problems:
a) asymmetric feature support is problematic - the calling core has no
way of knowing the feature set of the waking core. If the two
diverge, the architectural feature discovery via ID registers breaks
down. We've thus far "fixed" this on a case by case basis which
doesn't scale and introduces redundancy.

b) powerdown abandon (pabandon) introduces a contradiction - the calling
core has to initialise the context for when the core wakes up, but
should the core not powerdown it needs its old context intact. The only
way to work around this is by keeping two copies of context which
incurs a runtime and memory overhead.

c) cm_prepare_el3_exit[_ns]() doesn't have access to the entrypoint but needs
it to make initialisation decisions. We can infer some of this from
registers that have already been written but this is awkwardly
limiting for what we can do. This also necessitates the split from
the context initialisation.

We can solve all three by a making a core be in full ownership of its
own context. The calling core then only writes entrypoint information
and nothing else. The waking core then initialises its own context as it
sees fit with full knowledge of the whole picture.

The only tricky bit is cache coherency - the waking core has to be able
to coherently observe its new entrypoint. Calling cores will write to
the shared region with coherent caches on. If we make sure to read the
context only after the waking core has entered coherency, then we can
avoid cache operations and let hardware handle everything.

We can skip the spsr check for FEAT_TCR2 as it doesn't make a
difference. We can also skip enabling it twice from generic code.

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I86e7fe8b698191fc3b469e5ced1fd010f8754b0e

show more ...

fix(psci): initialise variables

When building with LTO, GCC is uncomfortable that these variables are
uninitialised and complains that they may be used before they are
initialised. Set them to 0 as

fix(psci): initialise variables

When building with LTO, GCC is uncomfortable that these variables are
uninitialised and complains that they may be used before they are
initialised. Set them to 0 as there are plenty of asserts to make sure
these branches cannot be taken.

Change-Id: Ic1f05e77252e93bdafab033dcb24ad42856ebf9a
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes from topic "xlnx_fix_gen_uniq_var" into integration

* changes:
fix(psci): avoid altering function parameters
fix(services): avoid altering function parameters
fix(common): ignore

Merge changes from topic "xlnx_fix_gen_uniq_var" into integration

* changes:
fix(psci): avoid altering function parameters
fix(services): avoid altering function parameters
fix(common): ignore the unused function return value
fix(psci): modify variable conflicting with external function
fix(delay-timer): create unique variable name

show more ...

fix(psci): avoid altering function parameters

This corrects the MISRA violation C2012-17.8:
A function parameter should not be modified.
Local variable is declared and used to process the value
from

fix(psci): avoid altering function parameters

This corrects the MISRA violation C2012-17.8:
A function parameter should not be modified.
Local variable is declared and used to process the value
from the argument.

Change-Id: Ia757db4903132794623dbf92ff8cecc9b40f170d
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

fix(psci): modify variable conflicting with external function

This corrects the MISRA violation C2012-5.8:
Identifiers that define objects or functions with
external linkage shall be unique.
Modify

fix(psci): modify variable conflicting with external function

This corrects the MISRA violation C2012-5.8:
Identifiers that define objects or functions with
external linkage shall be unique.
Modify the variable name to prevent conflict with
external function declaration

Change-Id: I2f109242b6dd3b3c5e9289881e3dd5466c74fcb5
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

fix(psci): add const qualifier

This corrects the MISRA violation C2012-8.13:
A pointer should point to a const-qualified type whenever possible.
Added const qualifier to pointer in the function argu

fix(psci): add const qualifier

This corrects the MISRA violation C2012-8.13:
A pointer should point to a const-qualified type whenever possible.
Added const qualifier to pointer in the function arguments.

Change-Id: Id3d4aa528f275973a37c0b9af04495632cb2dda3
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body w

fix(psci): add missing curly braces

This corrects the MISRA violation C2012-15.6:
The body of an iteration-statement or a selection-statement shall
be a compound-statement.
Enclosed statement body within the curly braces.

Change-Id: I8b656f59b445e914dd3f47e3dde83735481a3640
Signed-off-by: Nithin G <nithing@amd.com>
Signed-off-by: Maheedhar Bollapalli <maheedharsai.bollapalli@amd.com>

show more ...

perf(amu): greatly simplify AMU context management

The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness

perf(amu): greatly simplify AMU context management

The current code is incredibly resilient to updates to the spec and
has worked quite well so far. However, recent implementations expose a
weakness in that this is rather slow. A large part of it is written in
assembly, making it opaque to the compiler for optimisations. The
future proofness requires reading registers that are effectively
`volatile`, making it even harder for the compiler, as well as adding
lots of implicit barriers, making it hard for the microarchitecutre to
optimise as well.

We can make a few assumptions, checked by a few well placed asserts, and
remove a lot of this burden. For a start, at the moment there are 4
group 0 counters with static assignments. Contexting them is a trivial
affair that doesn't need a loop. Similarly, there can only be up to 16
group 1 counters. Contexting them is a bit harder, but we can do with a
single branch with a falling through switch. If/when both of these
change, we have a pair of asserts and the feature detection mechanism to
guard us against pretending that we support something we don't.

We can drop contexting of the offset registers. They are fully
accessible by EL2 and as such are its responsibility to preserve on
powerdown.

Another small thing we can do, is pass the core_pos into the hook.
The caller already knows which core we're running on, we don't need to
call this non-trivial function again.

Finally, knowing this, we don't really need the auxiliary AMUs to be
described by the device tree. Linux doesn't care at the moment, and any
information we need for EL3 can be neatly placed in a simple array.

All of this, combined with lifting the actual saving out of assembly,
reduces the instructions to save the context from 180 to 40, including a
lot fewer branches. The code is also much shorter and easier to read.

Also propagate to aarch32 so that the two don't diverge too much.

Change-Id: Ib62e6e9ba5be7fb9fb8965c8eee148d5598a5361
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

chore(docs): drop the "wfi" from `pwr_domain_pwr_down_wfi`

To allow for generic handling of a wakeup, this hook is no longer
expected to call wfi itself. Update the name everywhere to reflect this
e

chore(docs): drop the "wfi" from `pwr_domain_pwr_down_wfi`

To allow for generic handling of a wakeup, this hook is no longer
expected to call wfi itself. Update the name everywhere to reflect this
expectation so that future platform implementers don't get misled.

Change-Id: Ic33f0b6da74592ad6778fd802c2f0b85223af614
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

chore(psci): drop skip_wfi variable

There is now a convent place at the end of the function to jump to when
we shouldn't power down. This eliminates the need for skip_wfi.

Change-Id: I8d1a88c77463e

chore(psci): drop skip_wfi variable

There is now a convent place at the end of the function to jump to when
we shouldn't power down. This eliminates the need for skip_wfi.

Change-Id: I8d1a88c77463e8ee6765b185adbe3d23d9fce101
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...