feat(cpufeat): add ENABLE_FEAT_LS64_ACCDATA

Armv8.6 introduced the FEAT_LS64 extension, which provides a 64 *byte*
store instruction. A related instruction is ST64BV0, which will replace
the lowest

feat(cpufeat): add ENABLE_FEAT_LS64_ACCDATA

Armv8.6 introduced the FEAT_LS64 extension, which provides a 64 *byte*
store instruction. A related instruction is ST64BV0, which will replace
the lowest 32 bits of the data with a value taken from the ACCDATA_EL1
system register (so that EL0 cannot alter them).
Using that ST64BV0 instruction and accessing the ACCDATA_EL1 system
register is guarded by two SCR_EL3 bits, which we should set to avoid a
trap into EL3, when lower ELs use one of those.

Add the required bits and pieces to make this feature usable:
- Add the ENABLE_FEAT_LS64_ACCDATA build option (defaulting to 0).
- Add the CPUID and SCR_EL3 bit definitions associated with FEAT_LS64.
- Add a feature check to check for the existing four variants of the
LS64 feature and detect future extensions.
- Add code to save and restore the ACCDATA_EL1 register on
secure/non-secure context switches.
- Enable the feature with runtime detection for FVP and Arm FPGA.

Please note that the *basic* FEAT_LS64 feature does not feature any trap
bits, it's only the addition of the ACCDATA_EL1 system register that
adds these traps and the SCR_EL3 bits.

Change-Id: Ie3e2ca2d9c4fbbd45c0cc6089accbb825579138a
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

show more ...

feat(d128): add support for FEAT_D128

This patch disables trapping to EL3 when the FEAT_D128
specific registers are accessed by setting the SCR_EL3.D128En bit.

If FEAT_D128 is implemented, then FEA

feat(d128): add support for FEAT_D128

This patch disables trapping to EL3 when the FEAT_D128
specific registers are accessed by setting the SCR_EL3.D128En bit.

If FEAT_D128 is implemented, then FEAT_SYSREG128 is implemented.
With FEAT_SYSREG128 certain system registers are treated as 128-bit,
so we should be context saving and restoring 128-bits instead of 64-bit
when FEAT_D128 is enabled.

FEAT_SYSREG128 adds support for MRRS and MSRR instruction which
helps us to read write to 128-bit system register.
Refer to Arm Architecture Manual for further details.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I1a53db5eac29e56c8fbdcd4961ede3abfcb2411a
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

Merge "fix(el3-runtime): correct CASSERT for cpu data size" into integration

fix(el3-runtime): correct CASSERT for cpu data size

Build breaks when EL3_EXCEPTION_HANDLING option is enabled. The
CPU_DATA_SIZE macro does not consider the size required to save the
ehf_data field

fix(el3-runtime): correct CASSERT for cpu data size

Build breaks when EL3_EXCEPTION_HANDLING option is enabled. The
CPU_DATA_SIZE macro does not consider the size required to save the
ehf_data field of cpu_data structure.

include/lib/el3_runtime/cpu_data.h:163:17: error: size of array
'assert_cpu_data_size_mismatch' is negative
assert_cpu_data_size_mismatch);

This patch adds support to consider ehf_data field to calculate the
CPU_DATA_SIZE macro. Also adds relevant checks and asserts if the
ehf_data field is not considered.

Signed-off-by: Omkar Anand Kulkarni <omkar.kulkarni@arm.com>
Change-Id: I3c11b2982f4a612ce28e46848b5c5035a8f8efc2

show more ...

feat(sctlr2): add support for FEAT_SCTLR2

Arm v8.9 introduces FEAT_SCTLR2, adding SCTLR2_ELx registers.
Support this, context switching the registers and disabling
traps so lower ELs can access the

feat(sctlr2): add support for FEAT_SCTLR2

Arm v8.9 introduces FEAT_SCTLR2, adding SCTLR2_ELx registers.
Support this, context switching the registers and disabling
traps so lower ELs can access the new registers.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I0c4cba86917b6b065a7e8dd6af7daf64ee18dcda
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

feat(the): add support for FEAT_THE

Arm v8.9 introduces FEAT_THE, adding Translation Hardening Extension
Read-Check-Write mask registers, RCWMASK_EL1 and RCWSMASK_EL1.
Support this, context switchin

feat(the): add support for FEAT_THE

Arm v8.9 introduces FEAT_THE, adding Translation Hardening Extension
Read-Check-Write mask registers, RCWMASK_EL1 and RCWSMASK_EL1.
Support this, context switching the registers and disabling
traps so lower ELs can access the new registers.

Change the FVP platform to default to handling this as a dynamic option
so the right decision can be made by the code at runtime.

Change-Id: I8775787f523639b39faf61d046ef482f73b2a562
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...

refactor(cm): remove el1 context when SPMD_SPM_AT_SEL2=1

* Currently, EL1 context is included in cpu_context_t by default
for all the build configurations.
As part of the cpu context structure,

refactor(cm): remove el1 context when SPMD_SPM_AT_SEL2=1

* Currently, EL1 context is included in cpu_context_t by default
for all the build configurations.
As part of the cpu context structure, we hold a copy of EL1, EL2
system registers, per world per PE. This context structure is
enormous and will continue to grow bigger with the addition of
new features incorporating new registers.

* Ideally, EL3 should save and restore the system registers at its next
lower exception level, which is EL2 in majority of the configurations.

* This patch aims at optimising the memory allocation in cases, when
the members from the context structure are unused. So el1 system
register context must be omitted when lower EL is always x-EL2.

* "CTX_INCLUDE_EL2_REGS" is the internal build flag which gets set,
when SPD=spmd and SPMD_SPM_AT_SEL2=1 or ENABLE_RME=1.
It indicates, the system registers at EL2 are context switched for
the respective build configuration. Here, there is no need to save
and restore EL1 system registers, while x-EL2 is enabled.

Henceforth, this patch addresses this issue, by taking out the EL1
context at all possible places, while EL2 (CTX_INCLUDE_EL2_REGS) is
enabled, there by saving memory.

Change-Id: Ifddc497d3c810e22a15b1c227a731bcc133c2f4a
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

chore(cm): reorganise sctlr_el1 and tcr_el1 ctx code

SCTLR_EL1 and TCR_EL1 regs are included either as part of errata
"ERRATA_SPECULATIVE_AT" or under el1_sysregs_t context structure.
The code to wr

chore(cm): reorganise sctlr_el1 and tcr_el1 ctx code

SCTLR_EL1 and TCR_EL1 regs are included either as part of errata
"ERRATA_SPECULATIVE_AT" or under el1_sysregs_t context structure.
The code to write and read into these context entries, looks
repetitive and is invoked at most places.
This section is refactored to bring them under a static procedure,
keeping the code neat and easier to maintain.

Change-Id: Ib0d8c51bee09e1600c5baaa7f9745083dca9fee1
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

Merge changes from topic "mp/simd_ctxt_mgmt" into integration

* changes:
feat(fvp): allow SIMD context to be put in TZC DRAM
docs(simd): introduce CTX_INCLUDE_SVE_REGS build flag
feat(fvp): ad

Merge changes from topic "mp/simd_ctxt_mgmt" into integration

* changes:
feat(fvp): allow SIMD context to be put in TZC DRAM
docs(simd): introduce CTX_INCLUDE_SVE_REGS build flag
feat(fvp): add Cactus partition manifest for EL3 SPMC
chore(simd): remove unused macros and utilities for FP
feat(el3-spmc): support simd context management upon world switch
feat(trusty): switch to simd_ctx_save/restore apis
feat(pncd): switch to simd_ctx_save/restore apis
feat(spm-mm): switch to simd_ctx_save/restore APIs
feat(simd): add rules to rationalize simd ctxt mgmt
feat(simd): introduce simd context helper APIs
feat(simd): add routines to save, restore sve state
feat(simd): add sve state to simd ctxt struct
feat(simd): add data struct for simd ctxt management

show more ...

chore(simd): remove unused macros and utilities for FP

Now that we have switched to simd context save/restore apis
which uses simd_context_t data structures, we can safely remove
any macros, helpers

chore(simd): remove unused macros and utilities for FP

Now that we have switched to simd context save/restore apis
which uses simd_context_t data structures, we can safely remove
any macros, helpers and utilities used in the old way of
FPU context management.

Change-Id: I27a636dd07bf5c4cb99fd25b9a204d55b525b677
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

show more ...

feat(simd): introduce simd context helper APIs

This patch adds the common API to save and restore FP and SVE. When SVE
is enabled we save and restore SVE which automatically covers FP. If FP
is enab

feat(simd): introduce simd context helper APIs

This patch adds the common API to save and restore FP and SVE. When SVE
is enabled we save and restore SVE which automatically covers FP. If FP
is enabled while SVE is not, then we save and restore FP only.

The patch uses simd_ctx_t to save and restore both FP and SVE which
means developers need not use fp or sve routines directly. Once all the
calls to fpregs_context_* are replaced with simd_ctx_*, we can remove
fp_regs_t data structure and macros (taken care in a following patch).

simd_ctx_t is currently allocated in section of its own. This will go
into BSS section by default but platform will have option of relocating
it to a different section by overriding in plat.ld.S.

Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
Signed-off-by: Okash Khawaja <okash@google.com>
Change-Id: I090f8b8fa3862e527b6c40385249adc69256bf24

show more ...

feat(simd): add sve state to simd ctxt struct

This patch extends the simd data structure to hold the SVE state. A new
build flag CTX_INCLUDE_SVE_REGS is introduced in this patch to enable
SVE contex

feat(simd): add sve state to simd ctxt struct

This patch extends the simd data structure to hold the SVE state. A new
build flag CTX_INCLUDE_SVE_REGS is introduced in this patch to enable
SVE context management.

Necessary precautions are taken such as ensuring the field offsets are
not changed and necessary padding is added for alignment reasons.

Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
Signed-off-by: Okash Khawaja <okash@google.com>
Change-Id: I54f5222c7d8c68638105003f68caa511d347cd60

show more ...

feat(simd): add data struct for simd ctxt management

From a context management perspective, we refer to SVE and FP state
as SIMD. SVE can be considered as a superset of FP/NEON functionality
and the

feat(simd): add data struct for simd ctxt management

From a context management perspective, we refer to SVE and FP state
as SIMD. SVE can be considered as a superset of FP/NEON functionality
and there is a architectural dependency between the two due to the fact
that FP registers occupy the bottom 128 bits of SVE Z vector registers.

This patch introduces a new data structure to hold the FP state of a CPU
as needed for context management operations during world switch events.
It will evolve in next patches to capture the SVE state as well.

Having this unified data structure is necessary for us to implement
SIMD context management while catering to various build configurations.

Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>
Signed-off-by: Okash Khawaja <okash@google.com>
Change-Id: If2a9a7526883780df49bbd3dade2f09f7c3d95ae

show more ...

feat(cm): support for asymmetric feature among cores

TF-A assumes that all the cores in a platform has architecture feature
parity, this is evident by the fact that primary sets up the
Non-secure co

feat(cm): support for asymmetric feature among cores

TF-A assumes that all the cores in a platform has architecture feature
parity, this is evident by the fact that primary sets up the
Non-secure context of secondary cores.

With changing landscape of platforms (e.g. big/little/mid cores), we are
seeing more and more platforms which has feature asymmetry among cores.
There is also a scenario where certain CPU erratum only applies to one
type of cores and requires a feature to be disabled even it supports
the feature.

To handle these scenarios, introduce a hook in warmboot path which would
be called on the running CPU to override any feature disparity in the
NS context stashed up by primary. Note that, re-checking of feature for
Secure/Realm context is not required as the context is created on
running core itself.

Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Change-Id: I5a01dbda528fa8481a00fdd098b58a7463ed0e22

show more ...

refactor(cm): convert el1-ctx assembly offset entries to c structure

Currently the EL1 part of the context structure (el1_sysregs_t),
is coupled with feature flags reducing the context memory alloca

refactor(cm): convert el1-ctx assembly offset entries to c structure

Currently the EL1 part of the context structure (el1_sysregs_t),
is coupled with feature flags reducing the context memory allocation
for platforms, that don't enable/support all the architectural
features at once.

Similar to the el2 context optimization commit-"d6af234" this patch
further improves this section by converting the assembly context-offset
entries into a c structure. It relies on garbage collection of the
linker removing unreferenced structures from memory, as well as aiding
in readability and future maintenance. Additionally, it eliminates
the #ifs usage in 'context_mgmt.c' source file.

Change-Id: If6075931cec994bc89231241337eccc7042c5ede
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

feat(cm): add explicit context entries for ERRATA_SPECULATIVE_AT

* Currently, "ERRATA_SPECUALTIVE_AT" errata is enabled by default
for few cores and they need context entries for saving and
rest

feat(cm): add explicit context entries for ERRATA_SPECULATIVE_AT

* Currently, "ERRATA_SPECUALTIVE_AT" errata is enabled by default
for few cores and they need context entries for saving and
restoring EL1 regs "SCTLR_EL1 and TCR_EL1" registers at all times.

* This prevents the mechanism of decoupling EL1 and EL2 registers,
as EL3 firmware shouldn't be handling both simultaneously.

* Depending on the build configuration either EL1 or EL2 context
structures need to included, which would result in saving a good
amount of context memory.

* In order to achieve this it's essential to have explicit context
entries for registers supporting "ERRATA_SPECULATIVE_AT".

* This patch adds two context entries under "errata_speculative_at"
structure to assist this errata and thereby allows decoupling
EL1 and EL2 context structures.

Change-Id: Ia50626eea8fb64899a2e2d81622adbe07fe77d65
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

feat(fgt2): add support for FEAT_FGT2

This patch disables trapping to EL3 when the FEAT_FGT2
specific trap registers are accessed by setting the
SCR_EL3.FGTEn2 bit

Signed-off-by: Arvind Ram Prakash

feat(fgt2): add support for FEAT_FGT2

This patch disables trapping to EL3 when the FEAT_FGT2
specific trap registers are accessed by setting the
SCR_EL3.FGTEn2 bit

Signed-off-by: Arvind Ram Prakash <arvind.ramprakash@arm.com>
Change-Id: I6d2b614affb9067b2bc3d7bf0ae7d169d031592a

show more ...

feat(cm): context switch MDCR_EL3 register

Currently MDCR_EL3 register value is same for all the
worlds(Non-secure, Secure, Realm and Root).

With this approach, features enable/disable settings
rem

feat(cm): context switch MDCR_EL3 register

Currently MDCR_EL3 register value is same for all the
worlds(Non-secure, Secure, Realm and Root).

With this approach, features enable/disable settings
remain same across all the worlds. This is not ideal as
there must be flexibility in controlling feature as per
the requirements for individual world.

The patch addresses this by providing MDCR_EL3 a per world
value. Features with identical values for all the worlds are
grouped under ``manage_extensions_common`` API.

Change-Id: Ibc068d985fe165d8cb6d0ffb84119bffd743b3d1
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

refactor(cm): move mpam registers into el2 context

* FEAT_MPAM related EL2 registers are placed explicitly outside
the EL2 context in the cpu_context_t structure.

* With EL2 registers now coupled

refactor(cm): move mpam registers into el2 context

* FEAT_MPAM related EL2 registers are placed explicitly outside
the EL2 context in the cpu_context_t structure.

* With EL2 registers now coupled with dependent features, this
patch moves them to the el2_context structure "el2_sysregs_t".

* Further, converting the assembly context-offset entries into a
c structure. It relies on garbage collection of the linker
removing unreferenced structures from memory, as well as aiding
in readability and future maintenance.

Change-Id: Ib784bc8d2fbe35a8a47a569426d8663282ec06aa
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

fix(cm): add more feature registers to EL1 context mgmt

The following system registers are made part of save and restore
operations for EL1 context:

TRFCR_EL1
SCXTNUM_EL0
SCXTNUM_EL1
GCSCR_EL1
GCSC

fix(cm): add more feature registers to EL1 context mgmt

The following system registers are made part of save and restore
operations for EL1 context:

TRFCR_EL1
SCXTNUM_EL0
SCXTNUM_EL1
GCSCR_EL1
GCSCRE0_EL1
GCSPR_EL1
GCSPR_EL0

Change-Id: I1077112bdc29a6c9cd39b9707d6cf10b95fa15e3
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

show more ...

Merge changes from topic "lto-fixes" into integration

* changes:
fix(bl1): add missing `__RW_{START,END}__` symbols
fix(fvp): don't check MPIDRs with the power controller in BL1
fix(arm): only

Merge changes from topic "lto-fixes" into integration

* changes:
fix(bl1): add missing `__RW_{START,END}__` symbols
fix(fvp): don't check MPIDRs with the power controller in BL1
fix(arm): only expose `arm_bl2_dyn_cfg_init` to BL2
fix(cm): hide `cm_init_context_by_index` from BL1
fix(bl1): add missing spinlock dependency

show more ...

fix(cm): remove ENABLE_FEAT_MTE usage

commit@c282384dbb45b6185b4aba14efebbad110d18e49
removed ENABLE_FEAT_MTE but missed its usage in
context structure declaration path.

All mte regs that are curre

fix(cm): remove ENABLE_FEAT_MTE usage

commit@c282384dbb45b6185b4aba14efebbad110d18e49
removed ENABLE_FEAT_MTE but missed its usage in
context structure declaration path.

All mte regs that are currently context saved/restored
are needed only when FEAT_MTE2 is enabled, so move to
usage of FEAT_MTE2 and remove FEAT_MTE usage

Change-Id: I6b4417485fa6b7f52a31045562600945e48e81b7
Signed-off-by: Jayanth Dodderi Chidanand <jayanthdodderi.chidanand@arm.com>

show more ...

fix(cm): hide `cm_init_context_by_index` from BL1

BL1 requires the context management library but does not use or
implement `cm_init_context_by_index`. This change ensures that is not
compiled into

fix(cm): hide `cm_init_context_by_index` from BL1

BL1 requires the context management library but does not use or
implement `cm_init_context_by_index`. This change ensures that is not
compiled into BL1, as linking with LTO enabled causes an undefined
reference for this function.

Change-Id: I4a4602843bd75bc4f47b3e0c4c5a6efce1514ef6
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

fix(cm): add more system registers to EL1 context mgmt

The following system registers are made part of save and restore
operations for EL1 context:
MDCCINT_EL1
MDSCR_EL1
DISR_EL1
PIRE0_EL1

fix(cm): add more system registers to EL1 context mgmt

The following system registers are made part of save and restore
operations for EL1 context:
MDCCINT_EL1
MDSCR_EL1
DISR_EL1
PIRE0_EL1
PIR_EL1
POR_EL1
S2POR_EL1
TCR2_EL1

Some of these registers are available as part of core Armv8-A
architecture while others are made available through various
architectural extensions.

Change-Id: I507dccb9053ba177e1b98100fceccd1f32bdfc5c
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

show more ...

refactor(mte): remove mte, mte_perm

Currently both FEAT_MTE and FEAT_MTE_PERM aren't used for enabling
of any feature bits in EL3. So remove both FEAT handling.

All mte regs that are currently cont

refactor(mte): remove mte, mte_perm

Currently both FEAT_MTE and FEAT_MTE_PERM aren't used for enabling
of any feature bits in EL3. So remove both FEAT handling.

All mte regs that are currently context saved/restored are needed
only when FEAT_MTE2 is enabled, so move to usage of FEAT_MTE2 and
remove FEAT_MTE usage.

BREAKING CHANGE: Any platform or downstream code trying to use
SCR_EL3.ATA bit(26) will see failures as this is now moved to be
used only with FEAT_MTE2 with
commit@ef0d0e5478a3f19cbe70a378b9b184036db38fe2

Change-Id: Id01e154156571f7792135639e17dc5c8d0e17cf8
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>

show more ...