fix(cpufeat): don't overwrite PAuth keys with an erroneous cache clean

Accessing cpu_data when TF-A is built with HW_ASSISTED_COHERENCY=1 is
simple. Caching (SCTLR_EL3.C) is enabled along with the M

fix(cpufeat): don't overwrite PAuth keys with an erroneous cache clean

Accessing cpu_data when TF-A is built with HW_ASSISTED_COHERENCY=1 is
simple. Caching (SCTLR_EL3.C) is enabled along with the MMU and we can
rely on all accesses being coherent. However, this is not the case when
HW_ASSISTED_COHERENCY=0. Most of EL3's initialisation (especially on
warm boot) happens with the MMU on but with caching being off. Caches
are only enabled deep into CPU_ON processing when we can be certain the
core has entered coherency. This latter case is the subject of this
patch.

Prior to this patch, the way to work around that was to clean the
apiakey cpu_data storage right after writing it. The write would have
gone straight to memory as caches were off and the clean asserted that
nothing would be in the caches which were assumed to be invalid since
we've just came out of reset.

The problem with this is that we cannot assume that ALL caches are
invalid when coming out of reset. We can reasonably assume those private
to the core to be (so the L1 and/or the L2; those are guaranteed to be
invalidated out of reset for every Arm core) but that is not the case
for shared caches (eg an L2/L3 DSU cache) which can be on when a core
powers down. So the old keys could still be live in the shared cache, we
write new ones to memory and clean the old to memory too, undoing the
work.

So the correct thing to do is to clean and invalidate the cache prior to
writing the keys to memory and invalidate it after. This ensures that if
there is any other data after the apiakey, which shares the cache line,
it will be safely forwarded to memory and the caches will be invalid
when caching is turned on.

It is important to note at this point that this was never observed in
practice - every known configuration that uses PAuth has the apiakey as
the very last member of the cpu_data struct which is padded up to a
cache line and the usage of the apiakey is such that it was never
allocated into the shared caches. So the clean would effectively perform
an invalidate of only the apiakey and all worked well. This was only
spotted with a proposed patch that added data after the apiakey
(https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/39698/7).

Change-Id: I8493221dff53114c5c56dd73fbfd2a3301e2542c
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor: unify blx_setup() and blx_main()

All BLs have a bl_setup() for things that need to happen early, a fall
back into assembly and then bl_main() for the main functionality. This
was necessary

refactor: unify blx_setup() and blx_main()

All BLs have a bl_setup() for things that need to happen early, a fall
back into assembly and then bl_main() for the main functionality. This
was necessary in order to fiddle with PAuth related things that tend to
break C calls. Since then PAuth's enablement has seen a lot of
refactoring and this is now worked around cleanly so the distinction can
be removed. The only tradeoff is that this requires pauth to not be used
for the top-level main function.

There are two main benefits to doing this: First, code is easier to
understand as it's all together and the entrypoint is smaller. Second,
the compiler gets to see more of the code and apply optimisations
(importantly LTO).

Change-Id: Iddb93551115a2048988017547eb7b8db441dbd37
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): add support for FEAT_PAUTH_LR

This patch enables FEAT_PAUTH_LR at EL3 on systems that support it when
the new ENABLE_FEAT_PAUTH_LR flag is set.

Currently, PAUTH_LR is only supported

feat(cpufeat): add support for FEAT_PAUTH_LR

This patch enables FEAT_PAUTH_LR at EL3 on systems that support it when
the new ENABLE_FEAT_PAUTH_LR flag is set.

Currently, PAUTH_LR is only supported by arm clang compiler and not GCC.

Change-Id: I7db1e34b661ed95cad75850b62878ac5d98466ea
Signed-off-by: John Powell <john.powell@arm.com>

show more ...

perf(cpufeat): centralise PAuth key saving

prepare_el3_entry() is meant to be the one-stop shop for all the context
we must fiddle with to enter EL3 proper. However, PAuth is the one
exception, happ

perf(cpufeat): centralise PAuth key saving

prepare_el3_entry() is meant to be the one-stop shop for all the context
we must fiddle with to enter EL3 proper. However, PAuth is the one
exception, happening right after. Absorb it into prepare_el3_entry(),
handling the BL1/BL31 difference.

This is a good time to also move the key saving into the enable
function, also to centralise. With this it becomes apparent that saving
keys just before CPU_SUSPEND is redundant as they will be reinitialised
when the core wakes up.

Note that the key loading, now in save_gp_pmcr_pauth_regs, does not end
in an isb. The effects of the key change are not needed until the isb
in the caller, so this isb is not needed.

Change-Id: Idd286bea91140c106ab4c933c5c44b0bc2050ca2
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

refactor(cpufeat): convert FEAT_PAuth setup to C

An oversimplified view of FEAT_PAuth is that it's a symmetric encryption
of the LR. PAC instructions execute as NOPs until explicitly turned on.
So i

refactor(cpufeat): convert FEAT_PAuth setup to C

An oversimplified view of FEAT_PAuth is that it's a symmetric encryption
of the LR. PAC instructions execute as NOPs until explicitly turned on.
So in a function that turns PAuth on, the signing would have executed as
a NOP and the authentication will encrypt the address, leading to a
failure. That's why enablement is in assembly - we have full control of
when pointer authentications happen.

However, assembly is hard to read, is opaque to the compiler for
optimisations, and we need to call into C anyway for the platform hook
to get the key. So convert it to C. We can instruct the compiler to not
generate branch protection for the enable function only and as long as
the caller doesn't do branch protection (and all callers are entrypoints
written in assembly) everything will work.

Change-Id: I8917a26e1293033c910e3058664e3ca9207359b7
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

feat(cpufeat): enable FEAT_BTI to FEAT_STATE_CHECKED

Introduce the is_feat_bti_{supported, present}() helpers and replace
checks for ENABLE_BTI with it. Also factor out the setting of
SCTLR_EL3.BT o

feat(cpufeat): enable FEAT_BTI to FEAT_STATE_CHECKED

Introduce the is_feat_bti_{supported, present}() helpers and replace
checks for ENABLE_BTI with it. Also factor out the setting of
SCTLR_EL3.BT out of the PAuth enablement and place it in the respective
entrypoints where we initialise SCTLR_EL3. This makes PAuth
self-contained and SCTLR_EL3 initialisation centralised.

Change-Id: I0c0657ff1e78a9652cd2cf1603478283dc01f17b
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

chore(pauth): remove redundant pauth_disable_el3() call

Both bl2_main and bl2_run_next_image call pauth_disable_el3. However,
bl2_main is the only caller of bl2_run_next_image so it doesn't need to

chore(pauth): remove redundant pauth_disable_el3() call

Both bl2_main and bl2_run_next_image call pauth_disable_el3. However,
bl2_main is the only caller of bl2_run_next_image so it doesn't need to
call it

Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Change-Id: I91769b2994ad643d2259c211936dbac4ef010d25

show more ...