1Secure Partition Manager 2************************ 3 4.. contents:: 5 6.. toctree:: 7 ffa-manifest-binding 8 9Acronyms 10======== 11 12+--------+--------------------------------------+ 13| CoT | Chain of Trust | 14+--------+--------------------------------------+ 15| DMA | Direct Memory Access | 16+--------+--------------------------------------+ 17| DTB | Device Tree Blob | 18+--------+--------------------------------------+ 19| DTS | Device Tree Source | 20+--------+--------------------------------------+ 21| EC | Execution Context | 22+--------+--------------------------------------+ 23| FIP | Firmware Image Package | 24+--------+--------------------------------------+ 25| FF-A | Firmware Framework for Arm A-profile | 26+--------+--------------------------------------+ 27| IPA | Intermediate Physical Address | 28+--------+--------------------------------------+ 29| JOP | Jump-Oriented Programming | 30+--------+--------------------------------------+ 31| NWd | Normal World | 32+--------+--------------------------------------+ 33| ODM | Original Design Manufacturer | 34+--------+--------------------------------------+ 35| OEM | Original Equipment Manufacturer | 36+--------+--------------------------------------+ 37| PA | Physical Address | 38+--------+--------------------------------------+ 39| PE | Processing Element | 40+--------+--------------------------------------+ 41| PM | Power Management | 42+--------+--------------------------------------+ 43| PVM | Primary VM | 44+--------+--------------------------------------+ 45| ROP | Return-Oriented Programming | 46+--------+--------------------------------------+ 47| SMMU | System Memory Management Unit | 48+--------+--------------------------------------+ 49| SP | Secure Partition | 50+--------+--------------------------------------+ 51| SPD | Secure Payload Dispatcher | 52+--------+--------------------------------------+ 53| SPM | Secure Partition Manager | 54+--------+--------------------------------------+ 55| SPMC | SPM Core | 56+--------+--------------------------------------+ 57| SPMD | SPM Dispatcher | 58+--------+--------------------------------------+ 59| SiP | Silicon Provider | 60+--------+--------------------------------------+ 61| SWd | Secure World | 62+--------+--------------------------------------+ 63| TLV | Tag-Length-Value | 64+--------+--------------------------------------+ 65| TOS | Trusted Operating System | 66+--------+--------------------------------------+ 67| VM | Virtual Machine | 68+--------+--------------------------------------+ 69 70Foreword 71======== 72 73Three implementations of a Secure Partition Manager co-exist in the TF-A 74codebase: 75 76#. S-EL2 SPMC based on the FF-A specification `[1]`_, enabling virtualization in 77 the secure world, managing multiple S-EL1 or S-EL0 partitions. 78#. EL3 SPMC based on the FF-A specification, managing a single S-EL1 partition 79 without virtualization in the secure world. 80#. EL3 SPM based on the MM specification, legacy implementation managing a 81 single S-EL0 partition `[2]`_. 82 83These implementations differ in their respective SW architecture and only one 84can be selected at build time. This document: 85 86- describes the implementation from bullet 1. when the SPMC resides at S-EL2. 87- is not an architecture specification and it might provide assumptions 88 on sections mandated as implementation-defined in the specification. 89- covers the implications to TF-A used as a bootloader, and Hafnium used as a 90 reference code base for an S-EL2/SPMC secure firmware on platforms 91 implementing the FEAT_SEL2 architecture extension. 92 93Terminology 94----------- 95 96- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 97 (or partitions) in the normal world. 98- The term SPMC refers to the S-EL2 component managing secure partitions in 99 the secure world when the FEAT_SEL2 architecture extension is implemented. 100- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 101 partition and implementing the FF-A ABI on platforms not implementing the 102 FEAT_SEL2 architecture extension. 103- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 104- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 105 106Support for legacy platforms 107---------------------------- 108 109The SPM is split into a dispatcher and a core component (respectively SPMD and 110SPMC) residing at different exception levels. To permit the FF-A specification 111adoption and a smooth migration, the SPMD supports an SPMC residing either at 112S-EL1 or S-EL2: 113 114- The SPMD is located at EL3 and mainly relays the FF-A protocol from NWd 115 (Hypervisor or OS kernel) to the SPMC. 116- The same SPMD component is used for both S-EL1 and S-EL2 SPMC configurations. 117- The SPMC exception level is a build time choice. 118 119TF-A supports both cases: 120 121- S-EL1 SPMC for platforms not supporting the FEAT_SEL2 architecture 122 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 123- S-EL2 SPMC for platforms implementing the FEAT_SEL2 architecture 124 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 125 126Sample reference stack 127====================== 128 129The following diagram illustrates a possible configuration when the 130FEAT_SEL2 architecture extension is implemented, showing the SPMD 131and SPMC, one or multiple secure partitions, with an optional 132Hypervisor: 133 134.. image:: ../resources/diagrams/ff-a-spm-sel2.png 135 136TF-A build options 137================== 138 139This section explains the TF-A build options involved in building with 140support for an FF-A based SPM where the SPMD is located at EL3 and the 141SPMC located at S-EL1, S-EL2 or EL3: 142 143- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 144 protocol from NWd to SWd back and forth. It is not possible to 145 enable another Secure Payload Dispatcher when this option is chosen. 146- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 147 level to being at S-EL2. It defaults to enabled (value 1) when 148 SPD=spmd is chosen. 149- **SPMC_AT_EL3**: this option adjusts the SPMC exception level to being 150 at EL3. 151- If neither ``SPMD_SPM_AT_SEL2`` or ``SPMC_AT_EL3`` are enabled the SPMC 152 exception level is set to S-EL1. 153- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 154 restoring) the EL2 system register context before entering (resp. 155 after leaving) the SPMC. It is mandatorily enabled when 156 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 157 and exhaustive list of registers is visible at `[4]`_. 158- **SP_LAYOUT_FILE**: this option specifies a text description file 159 providing paths to SP binary images and manifests in DTS format 160 (see `Describing secure partitions`_). It 161 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 162 secure partitions are to be loaded by BL2 on behalf of the SPMC. 163 164+---------------+----------------------+------------------+-------------+ 165| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | SPMC_AT_EL3 | 166+---------------+----------------------+------------------+-------------+ 167| SPMC at S-EL1 | 0 | 0 | 0 | 168+---------------+----------------------+------------------+-------------+ 169| SPMC at S-EL2 | 1 | 1 (default when | 0 | 170| | | SPD=spmd) | | 171+---------------+----------------------+------------------+-------------+ 172| SPMC at EL3 | 0 | 0 | 1 | 173+---------------+----------------------+------------------+-------------+ 174 175Other combinations of such build options either break the build or are not 176supported. 177 178Notes: 179 180- Only Arm's FVP platform is supported to use with the TF-A reference software 181 stack. 182- When ``SPMD_SPM_AT_SEL2=1``, the reference software stack assumes enablement 183 of FEAT_PAuth, FEAT_BTI and FEAT_MTE architecture extensions. 184- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 185 barely saving/restoring EL2 registers from an Arm arch perspective. As such 186 it is decoupled from the ``SPD=spmd`` option. 187- BL32 option is re-purposed to specify the SPMC image. It can specify either 188 the Hafnium binary path (built for the secure world) or the path to a TEE 189 binary implementing FF-A interfaces. 190- BL33 option can specify the TFTF binary or a normal world loader 191 such as U-Boot or the UEFI framework payload. 192 193Sample TF-A build command line when the SPMC is located at S-EL1 194(e.g. when the FEAT_SEL2 architecture extension is not implemented): 195 196.. code:: shell 197 198 make \ 199 CROSS_COMPILE=aarch64-none-elf- \ 200 SPD=spmd \ 201 SPMD_SPM_AT_SEL2=0 \ 202 BL32=<path-to-tee-binary> \ 203 BL33=<path-to-bl33-binary> \ 204 PLAT=fvp \ 205 all fip 206 207Sample TF-A build command line when FEAT_SEL2 architecture extension is 208implemented and the SPMC is located at S-EL2: 209.. code:: shell 210 211 make \ 212 CROSS_COMPILE=aarch64-none-elf- \ 213 PLAT=fvp \ 214 SPD=spmd \ 215 CTX_INCLUDE_EL2_REGS=1 \ 216 ARM_ARCH_MINOR=5 \ 217 BRANCH_PROTECTION=1 \ 218 CTX_INCLUDE_PAUTH_REGS=1 \ 219 CTX_INCLUDE_MTE_REGS=1 \ 220 BL32=<path-to-hafnium-binary> \ 221 BL33=<path-to-bl33-binary> \ 222 SP_LAYOUT_FILE=sp_layout.json \ 223 all fip 224 225Sample TF-A build command line when FEAT_SEL2 architecture extension is 226implemented, the SPMC is located at S-EL2, and enabling secure boot: 227.. code:: shell 228 229 make \ 230 CROSS_COMPILE=aarch64-none-elf- \ 231 PLAT=fvp \ 232 SPD=spmd \ 233 CTX_INCLUDE_EL2_REGS=1 \ 234 ARM_ARCH_MINOR=5 \ 235 BRANCH_PROTECTION=1 \ 236 CTX_INCLUDE_PAUTH_REGS=1 \ 237 CTX_INCLUDE_MTE_REGS=1 \ 238 BL32=<path-to-hafnium-binary> \ 239 BL33=<path-to-bl33-binary> \ 240 SP_LAYOUT_FILE=sp_layout.json \ 241 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 242 TRUSTED_BOARD_BOOT=1 \ 243 COT=dualroot \ 244 ARM_ROTPK_LOCATION=devel_rsa \ 245 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 246 GENERATE_COT=1 \ 247 all fip 248 249Sample TF-A build command line when the SPMC is located at EL3: 250 251.. code:: shell 252 253 make \ 254 CROSS_COMPILE=aarch64-none-elf- \ 255 SPD=spmd \ 256 SPMD_SPM_AT_SEL2=0 \ 257 SPMC_AT_EL3=1 \ 258 BL32=<path-to-tee-binary> \ 259 BL33=<path-to-bl33-binary> \ 260 PLAT=fvp \ 261 all fip 262 263FVP model invocation 264==================== 265 266The FVP command line needs the following options to exercise the S-EL2 SPMC: 267 268+---------------------------------------------------+------------------------------------+ 269| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 270| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 271+---------------------------------------------------+------------------------------------+ 272| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 273| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 274| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 275| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 276| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 277| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 278| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 279| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 280+---------------------------------------------------+------------------------------------+ 281| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 282| - cluster1.has_branch_target_exception=1 | | 283+---------------------------------------------------+------------------------------------+ 284| - cluster0.has_pointer_authentication=2 | Implements FEAT_PAuth | 285| - cluster1.has_pointer_authentication=2 | | 286+---------------------------------------------------+------------------------------------+ 287| - cluster0.memory_tagging_support_level=2 | Implements FEAT_MTE2 | 288| - cluster1.memory_tagging_support_level=2 | | 289| - bp.dram_metadata.is_enabled=1 | | 290+---------------------------------------------------+------------------------------------+ 291 292Sample FVP command line invocation: 293 294.. code:: shell 295 296 <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \ 297 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 298 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 299 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 300 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 301 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 302 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 \ 303 -C cluster0.has_pointer_authentication=2 -C cluster1.has_pointer_authentication=2 \ 304 -C cluster0.has_branch_target_exception=1 -C cluster1.has_branch_target_exception=1 \ 305 -C cluster0.memory_tagging_support_level=2 -C cluster1.memory_tagging_support_level=2 \ 306 -C bp.dram_metadata.is_enabled=1 \ 307 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B \ 308 -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 \ 309 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 \ 310 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 311 312Boot process 313============ 314 315Loading Hafnium and secure partitions in the secure world 316--------------------------------------------------------- 317 318TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 319 320SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 321Thus they are supplied as distinct signed entities within the FIP flash 322image. The FIP image itself is not signed hence this provides the ability 323to upgrade SPs in the field. 324 325Booting through TF-A 326-------------------- 327 328SP manifests 329~~~~~~~~~~~~ 330 331An SP manifest describes SP attributes as defined in `[1]`_ 332(partition manifest at virtual FF-A instance) in DTS format. It is 333represented as a single file associated with the SP. A sample is 334provided by `[5]`_. A binding document is provided by `[6]`_. 335 336Secure Partition packages 337~~~~~~~~~~~~~~~~~~~~~~~~~ 338 339Secure partitions are bundled as independent package files consisting 340of: 341 342- a header 343- a DTB 344- an image payload 345 346The header starts with a magic value and offset values to SP DTB and 347image payload. Each SP package is loaded independently by BL2 loader 348and verified for authenticity and integrity. 349 350The SP package identified by its UUID (matching FF-A uuid property) is 351inserted as a single entry into the FIP at end of the TF-A build flow 352as shown: 353 354.. code:: shell 355 356 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 357 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 358 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 359 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 360 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 361 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 362 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 363 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 364 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 365 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 366 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 367 368.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 369 370Describing secure partitions 371~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 372 373A json-formatted description file is passed to the build flow specifying paths 374to the SP binary image and associated DTS partition manifest file. The latter 375is processed by the dtc compiler to generate a DTB fed into the SP package. 376Optionally, the partition's json description can contain offsets for both 377the image and partition manifest within the SP package. Both offsets need to be 3784KB aligned, because it is the translation granule supported by Hafnium SPMC. 379These fields can be leveraged to support SPs with S1 translation granules that 380differ from 4KB, and to configure the regions allocated within the SP package, 381as well as to comply with the requirements for the implementation of the boot 382information protocol (see `Passing boot data to the SP`_ for more details). In 383case the offsets are absent in their json node, they default to 0x1000 and 3840x4000 for the manifest offset and image offset respectively. 385This file also specifies the SP owner (as an optional field) identifying the 386signing domain in case of dual root CoT. 387The SP owner can either be the silicon or the platform provider. The 388corresponding "owner" field value can either take the value of "SiP" or "Plat". 389In absence of "owner" field, it defaults to "SiP" owner. 390The UUID of the partition can be specified as a field in the description file or 391if it does not exist there the UUID is extracted from the DTS partition 392manifest. 393 394.. code:: shell 395 396 { 397 "tee1" : { 398 "image": "tee1.bin", 399 "pm": "tee1.dts", 400 "owner": "SiP", 401 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f" 402 }, 403 404 "tee2" : { 405 "image": "tee2.bin", 406 "pm": "tee2.dts", 407 "owner": "Plat" 408 }, 409 410 "tee3" : { 411 "image": { 412 "file": "tee3.bin", 413 "offset":"0x2000" 414 }, 415 "pm": { 416 "file": "tee3.dts", 417 "offset":"0x6000" 418 }, 419 "owner": "Plat" 420 }, 421 } 422 423SPMC manifest 424~~~~~~~~~~~~~ 425 426This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 427time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 428two different cases: 429 430- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 431 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 432 mode. 433- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 434 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 435 S-EL0. 436 437.. code:: shell 438 439 attribute { 440 spmc_id = <0x8000>; 441 maj_ver = <0x1>; 442 min_ver = <0x1>; 443 exec_state = <0x0>; 444 load_address = <0x0 0x6000000>; 445 entrypoint = <0x0 0x6000000>; 446 binary_size = <0x60000>; 447 }; 448 449- *spmc_id* defines the endpoint ID value that SPMC can query through 450 ``FFA_ID_GET``. 451- *maj_ver/min_ver*. SPMD checks provided version versus its internal 452 version and aborts if not matching. 453- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 454 Notice Hafnium used as a SPMC only supports AArch64. 455- *load_address* and *binary_size* are mostly used to verify secondary 456 entry points fit into the loaded binary image. 457- *entrypoint* defines the cold boot primary core entry point used by 458 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 459 460Other nodes in the manifest are consumed by Hafnium in the secure world. 461A sample can be found at `[7]`_: 462 463- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 464 indicates a FF-A compliant SP. The *load_address* field specifies the load 465 address at which BL2 loaded the SP package. 466- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 467 Note the primary core is declared first, then secondary cores are declared 468 in reverse order. 469- The *memory* node provides platform information on the ranges of memory 470 available to the SPMC. 471 472SPMC boot 473~~~~~~~~~ 474 475The SPMC is loaded by BL2 as the BL32 image. 476 477The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 478 479BL2 passes the SPMC manifest address to BL31 through a register. 480 481At boot time, the SPMD in BL31 runs from the primary core, initializes the core 482contexts and launches the SPMC (BL32) passing the following information through 483registers: 484 485- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 486- X1 holds the ``HW_CONFIG`` physical address. 487- X4 holds the currently running core linear id. 488 489Loading of SPs 490~~~~~~~~~~~~~~ 491 492At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 493below: 494 495.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 496 497Note this boot flow is an implementation sample on Arm's FVP platform. 498Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 499different boot flow. The flow restricts to a maximum of 8 secure partitions. 500 501Secure boot 502~~~~~~~~~~~ 503 504The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 505SPMC manifest, secure partitions and verifies them for authenticity and integrity. 506Refer to TBBR specification `[3]`_. 507 508The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 509the use of two root keys namely S-ROTPK and NS-ROTPK: 510 511- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 512- BL33 may be signed by the OEM using NS-ROTPK. 513- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 514- A maximum of 4 partitions can be signed with the S-ROTPK key and 4 partitions 515 signed with the NS-ROTPK key. 516 517Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 518 519Hafnium in the secure world 520=========================== 521 522General considerations 523---------------------- 524 525Build platform for the secure world 526~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 527 528In the Hafnium reference implementation specific code parts are only relevant to 529the secure world. Such portions are isolated in architecture specific files 530and/or enclosed by a ``SECURE_WORLD`` macro. 531 532Secure partitions scheduling 533~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 534 535The FF-A specification `[1]`_ provides two ways to relinquinsh CPU time to 536secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 537 538- the FFA_MSG_SEND_DIRECT_REQ interface. 539- the FFA_RUN interface. 540 541Additionally a secure interrupt can pre-empt the normal world execution and give 542CPU cycles by transitioning to EL3 and S-EL2. 543 544Platform topology 545~~~~~~~~~~~~~~~~~ 546 547The *execution-ctx-count* SP manifest field can take the value of one or the 548total number of PEs. The FF-A specification `[1]`_ recommends the 549following SP types: 550 551- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 552 implement the same number of ECs as the number of PEs in the platform. 553- Migratable UP SPs: a single execution context can run and be migrated on any 554 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 555 receive a direct message request originating from any physical core targeting 556 the single execution context. 557 558Parsing SP partition manifests 559------------------------------ 560 561Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 562Note the current implementation may not implement all optional fields. 563 564The SP manifest may contain memory and device regions nodes. In case of 565an S-EL2 SPMC: 566 567- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 568 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 569 specify RX/TX buffer regions in which case it is not necessary for an SP 570 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 571- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 572 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 573 additional resources (e.g. interrupts). 574 575For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 576provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 577regime. 578 579Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 580same set of page tables. It is still open whether two sets of page tables shall 581be provided per SP. The memory region node as defined in the specification 582provides a memory security attribute hinting to map either to the secure or 583non-secure EL1&0 Stage-2 table if it exists. 584 585Passing boot data to the SP 586--------------------------- 587 588In `[1]`_ , the section "Boot information protocol" defines a method for passing 589data to the SPs at boot time. It specifies the format for the boot information 590descriptor and boot information header structures, which describe the data to be 591exchanged between SPMC and SP. 592The specification also defines the types of data that can be passed. 593The aggregate of both the boot info structures and the data itself is designated 594the boot information blob, and is passed to a Partition as a contiguous memory 595region. 596 597Currently, the SPM implementation supports the FDT type which is used to pass the 598partition's DTB manifest. 599 600The region for the boot information blob is allocated through the SP package. 601 602.. image:: ../resources/diagrams/partition-package.png 603 604To adjust the space allocated for the boot information blob, the json description 605of the SP (see section `Describing secure partitions`_) shall be updated to contain 606the manifest offset. If no offset is provided the manifest offset defaults to 0x1000, 607which is the page size in the Hafnium SPMC. 608 609The configuration of the boot protocol is done in the SPs manifest. As defined by 610the specification, the manifest field 'gp-register-num' configures the GP register 611which shall be used to pass the address to the partitions boot information blob when 612booting the partition. 613In addition, the Hafnium SPMC implementation requires the boot information arguments 614to be listed in a designated DT node: 615 616.. code:: shell 617 618 boot-info { 619 compatible = "arm,ffa-manifest-boot-info"; 620 ffa_manifest; 621 }; 622 623The whole secure partition package image (see `Secure Partition packages`_) is 624mapped to the SP secure EL1&0 Stage-2 translation regime. As such, the SP can 625retrieve the address for the boot information blob in the designated GP register, 626process the boot information header and descriptors, access its own manifest 627DTB blob and extract its partition manifest properties. 628 629SP Boot order 630------------- 631 632SP manifests provide an optional boot order attribute meant to resolve 633dependencies such as an SP providing a service required to properly boot 634another SP. SPMC boots the SPs in accordance to the boot order attribute, 635lowest to the highest value. If the boot order attribute is absent from the FF-A 636manifest, the SP is treated as if it had the highest boot order value 637(i.e. lowest booting priority). 638 639It is possible for an SP to call into another SP through a direct request 640provided the latter SP has already been booted. 641 642Boot phases 643----------- 644 645Primary core boot-up 646~~~~~~~~~~~~~~~~~~~~ 647 648Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 649core. The SPMC performs its platform initializations and registers the SPMC 650secondary physical core entry point physical address by the use of the 651`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 652at secure physical FF-A instance). 653 654The SPMC then creates secure partitions based on SP packages and manifests. Each 655secure partition is launched in sequence (`SP Boot order`_) on their "primary" 656execution context. If the primary boot physical core linear id is N, an MP SP is 657started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 658UP SP, it is started using its unique EC0 on PE[N]. 659 660The SP primary EC (or the EC used when the partition is booted as described 661above): 662 663- Performs the overall SP boot time initialization, and in case of a MP SP, 664 prepares the SP environment for other execution contexts. 665- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 666 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 667 entry point for other execution contexts. 668- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 669 ``FFA_ERROR`` in case of failure. 670 671Secondary cores boot-up 672~~~~~~~~~~~~~~~~~~~~~~~ 673 674Once the system is started and NWd brought up, a secondary physical core is 675woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 676calls into the SPMD on the newly woken up physical core. Then the SPMC is 677entered at the secondary physical core entry point. 678 679In the current implementation, the first SP is resumed on the coresponding EC 680(the virtual CPU which matches the physical core). The implication is that the 681first SP must be a MP SP. 682 683In a linux based system, once secure and normal worlds are booted but prior to 684a NWd FF-A driver has been loaded: 685 686- The first SP has initialized all its ECs in response to primary core boot up 687 (at system initialization) and secondary core boot up (as a result of linux 688 invoking PSCI_CPU_ON for all secondary cores). 689- Other SPs have their first execution context initialized as a result of secure 690 world initialization on the primary boot core. Other ECs for those SPs have to 691 be run first through ffa_run to complete their initialization (which results 692 in the EC completing with FFA_MSG_WAIT). 693 694Refer to `Power management`_ for further details. 695 696Notifications 697------------- 698 699The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 700communication mechanism with non-blocking semantics. It allows for one FF-A 701endpoint to signal another for service provision, without hindering its current 702progress. 703 704Hafnium currently supports 64 notifications. The IDs of each notification define 705a position in a 64-bit bitmap. 706 707The signaling of notifications can interchangeably happen between NWd and SWd 708FF-A endpoints. 709 710The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 711VMs, and from VMs to SPs. An hypervisor component would only manage 712notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 713deployed in NWd, the Hypervisor or OS kernel must invoke the interface 714FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 715endpoint in the NWd that supports it. 716 717A sender can signal notifications once the receiver has provided it with 718permissions. Permissions are provided by invoking the interface 719FFA_NOTIFICATION_BIND. 720 721Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 722they are considered to be in a pending sate. The receiver can retrieve its 723pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 724are considered to be handled. 725 726Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 727that is in charge of donating CPU cycles for notifications handling. The 728FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 729which FF-A endpoints have pending notifications. The receiver scheduler is 730called and informed by the FF-A driver, and it should allocate CPU cycles to the 731receiver. 732 733There are two types of notifications supported: 734 735- Global, which are targeted to a FF-A endpoint and can be handled within any of 736 its execution contexts, as determined by the scheduler of the system. 737- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 738 a specific execution context, as determined by the sender. 739 740The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 741permissions to the sender. 742 743Notification signaling resorts to two interrupts: 744 745- Schedule Receiver Interrupt: non-secure physical interrupt to be handled by 746 the FF-A driver within the receiver scheduler. At initialization the SPMC 747 donates a SGI ID chosen from the secure SGI IDs range and configures it as 748 non-secure. The SPMC triggers this SGI on the currently running core when 749 there are pending notifications, and the respective receivers need CPU cycles 750 to handle them. 751- Notifications Pending Interrupt: virtual interrupt to be handled by the 752 receiver of the notification. Set when there are pending notifications for the 753 given secure partition. The NPI is pended when the NWd relinquishes CPU cycles 754 to an SP. 755 756The notifications receipt support is enabled in the partition FF-A manifest. 757 758Mandatory interfaces 759-------------------- 760 761The following interfaces are exposed to SPs: 762 763- ``FFA_VERSION`` 764- ``FFA_FEATURES`` 765- ``FFA_RX_RELEASE`` 766- ``FFA_RXTX_MAP`` 767- ``FFA_RXTX_UNMAP`` 768- ``FFA_PARTITION_INFO_GET`` 769- ``FFA_ID_GET`` 770- ``FFA_MSG_WAIT`` 771- ``FFA_MSG_SEND_DIRECT_REQ`` 772- ``FFA_MSG_SEND_DIRECT_RESP`` 773- ``FFA_MEM_DONATE`` 774- ``FFA_MEM_LEND`` 775- ``FFA_MEM_SHARE`` 776- ``FFA_MEM_RETRIEVE_REQ`` 777- ``FFA_MEM_RETRIEVE_RESP`` 778- ``FFA_MEM_RELINQUISH`` 779- ``FFA_MEM_FRAG_RX`` 780- ``FFA_MEM_FRAG_TX`` 781- ``FFA_MEM_RECLAIM`` 782- ``FFA_RUN`` 783 784As part of the FF-A v1.1 support, the following interfaces were added: 785 786 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 787 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 788 - ``FFA_NOTIFICATION_BIND`` 789 - ``FFA_NOTIFICATION_UNBIND`` 790 - ``FFA_NOTIFICATION_SET`` 791 - ``FFA_NOTIFICATION_GET`` 792 - ``FFA_NOTIFICATION_INFO_GET`` 793 - ``FFA_SPM_ID_GET`` 794 - ``FFA_SECONDARY_EP_REGISTER`` 795 - ``FFA_MEM_PERM_GET`` 796 - ``FFA_MEM_PERM_SET`` 797 798FFA_VERSION 799~~~~~~~~~~~ 800 801``FFA_VERSION`` requires a *requested_version* parameter from the caller. 802The returned value depends on the caller: 803 804- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 805 specified in the SPMC manifest. 806- SP: the SPMC returns its own implemented version. 807- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 808 809FFA_FEATURES 810~~~~~~~~~~~~ 811 812FF-A features supported by the SPMC may be discovered by secure partitions at 813boot (that is prior to NWd is booted) or run-time. 814 815The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 816FFA_SUCCESS from the SPMD. 817 818The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 819the response relayed back to the NWd. 820 821FFA_RXTX_MAP/FFA_RXTX_UNMAP 822~~~~~~~~~~~~~~~~~~~~~~~~~~~ 823 824When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 825receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 826regime as secure buffers in the MMU descriptors. 827 828When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 829SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 830descriptors. 831 832The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 833caller, either it being the Hypervisor or OS kernel, as well as a secure 834partition. 835 836FFA_PARTITION_INFO_GET 837~~~~~~~~~~~~~~~~~~~~~~ 838 839Partition info get call can originate: 840 841- from SP to SPMC 842- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 843 844FFA_ID_GET 845~~~~~~~~~~ 846 847The FF-A id space is split into a non-secure space and secure space: 848 849- FF-A ID with bit 15 clear relates to VMs. 850- FF-A ID with bit 15 set related to SPs. 851- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 852 and SPMC. 853 854The SPMD returns: 855 856- The default zero value on invocation from the Hypervisor. 857- The ``spmc_id`` value specified in the SPMC manifest on invocation from 858 the SPMC (see `SPMC manifest`_) 859 860This convention helps the SPMC to determine the origin and destination worlds in 861an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 862transactions in its world switch routine. It must not be permitted for a VM to 863use a secure FF-A ID as origin world by spoofing: 864 865- A VM-to-SP direct request/response shall set the origin world to be non-secure 866 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 867 set). 868- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 869 for both origin and destination IDs. 870 871An incoming direct message request arriving at SPMD from NWd is forwarded to 872SPMC without a specific check. The SPMC is resumed through eret and "knows" the 873message is coming from normal world in this specific code path. Thus the origin 874endpoint ID must be checked by SPMC for being a normal world ID. 875 876An SP sending a direct message request must have bit 15 set in its origin 877endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 878 879The SPMC shall reject the direct message if the claimed world in origin endpoint 880ID is not consistent: 881 882- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 883 world ID", 884- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 885 886 887FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 888~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 889 890This is a mandatory interface for secure partitions consisting in direct request 891and responses with the following rules: 892 893- An SP can send a direct request to another SP. 894- An SP can receive a direct request from another SP. 895- An SP can send a direct response to another SP. 896- An SP cannot send a direct request to an Hypervisor or OS kernel. 897- An Hypervisor or OS kernel can send a direct request to an SP. 898- An SP can send a direct response to an Hypervisor or OS kernel. 899 900FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 901~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 902 903The secure partitions notifications bitmap are statically allocated by the SPMC. 904Hence, this interface is not to be issued by secure partitions. 905 906At initialization, the SPMC is not aware of VMs/partitions deployed in the 907normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 908to be prepared to handle notifications for the provided VM ID. 909 910FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 911~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 912 913Pair of interfaces to manage permissions to signal notifications. Prior to 914handling notifications, an FF-A endpoint must allow a given sender to signal a 915bitmap of notifications. 916 917If the receiver doesn't have notification support enabled in its FF-A manifest, 918it won't be able to bind notifications, hence forbidding it to receive any 919notifications. 920 921FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 922~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 923 924FFA_NOTIFICATION_GET retrieves all pending global notifications and 925per-vCPU notifications targeted to the current vCPU. 926 927Hafnium maintains a global count of pending notifications which gets incremented 928and decremented when handling FFA_NOTIFICATION_SET and FFA_NOTIFICATION_GET 929respectively. A delayed SRI is triggered if the counter is non-zero when the 930SPMC returns to normal world. 931 932FFA_NOTIFICATION_INFO_GET 933~~~~~~~~~~~~~~~~~~~~~~~~~ 934 935Hafnium maintains a global count of pending notifications whose information 936has been retrieved by this interface. The count is incremented and decremented 937when handling FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET respectively. 938It also tracks notifications whose information has been retrieved individually, 939such that it avoids duplicating returned information for subsequent calls to 940FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 941reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 942 943FFA_SPM_ID_GET 944~~~~~~~~~~~~~~ 945 946Returns the FF-A ID allocated to an SPM component which can be one of SPMD 947or SPMC. 948 949At initialization, the SPMC queries the SPMD for the SPMC ID, using the 950FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using 951the FFA_SPM_ID_GET interface at the secure physical FF-A instance. 952 953Secure partitions call this interface at the virtual FF-A instance, to which 954the SPMC returns the priorly retrieved SPMC ID. 955 956The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the 957SPMD, which returns the SPMC ID. 958 959FFA_SECONDARY_EP_REGISTER 960~~~~~~~~~~~~~~~~~~~~~~~~~ 961 962When the SPMC boots, all secure partitions are initialized on their primary 963Execution Context. 964 965The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition 966from its first execution context, to provide the entry point address for 967secondary execution contexts. 968 969A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 970the NWd or by invocation of FFA_RUN. 971 972SPMC-SPMD direct requests/responses 973----------------------------------- 974 975Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 976Using those IDs in source/destination fields of a direct request/response 977permits SPMD to SPMC communication and either way. 978 979- SPMC to SPMD direct request/response uses SMC conduit. 980- SPMD to SPMC direct request/response uses ERET conduit. 981 982This is used in particular to convey power management messages. 983 984PE MMU configuration 985-------------------- 986 987With secure virtualization enabled (``HCR_EL2.VM = 1``) and for S-EL1 988partitions, two IPA spaces (secure and non-secure) are output from the 989secure EL1&0 Stage-1 translation. 990The EL1&0 Stage-2 translation hardware is fed by: 991 992- A secure IPA when the SP EL1&0 Stage-1 MMU is disabled. 993- One of secure or non-secure IPA when the secure EL1&0 Stage-1 MMU is enabled. 994 995``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 996NS/S IPA translations. The following controls are set up: 997``VSTCR_EL2.SW = 0`` , ``VSTCR_EL2.SA = 0``, ``VTCR_EL2.NSW = 0``, 998``VTCR_EL2.NSA = 1``: 999 1000- Stage-2 translations for the NS IPA space access the NS PA space. 1001- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 1002 1003Secure and non-secure IPA regions (rooted to by ``VTTBR_EL2`` and ``VSTTBR_EL2``) 1004use the same set of Stage-2 page tables within a SP. 1005 1006The ``VTCR_EL2/VSTCR_EL2/VTTBR_EL2/VSTTBR_EL2`` virtual address space 1007configuration is made part of a vCPU context. 1008 1009For S-EL0 partitions with VHE enabled, a single secure EL2&0 Stage-1 translation 1010regime is used for both Hafnium and the partition. 1011 1012Schedule modes and SP Call chains 1013--------------------------------- 1014 1015An SP execution context is said to be in SPMC scheduled mode if CPU cycles are 1016allocated to it by SPMC. Correspondingly, an SP execution context is said to be 1017in Normal world scheduled mode if CPU cycles are allocated by the normal world. 1018 1019A call chain represents all SPs in a sequence of invocations of a direct message 1020request. When execution on a PE is in the secure state, only a single call chain 1021that runs in the Normal World scheduled mode can exist. FF-A v1.1 spec allows 1022any number of call chains to run in the SPMC scheduled mode but the Hafnium 1023SPMC restricts the number of call chains in SPMC scheduled mode to only one for 1024keeping the implementation simple. 1025 1026Partition runtime models 1027------------------------ 1028 1029The runtime model of an endpoint describes the transitions permitted for an 1030execution context between various states. These are the four partition runtime 1031models supported (refer to `[1]`_ section 7): 1032 1033 - RTM_FFA_RUN: runtime model presented to an execution context that is 1034 allocated CPU cycles through FFA_RUN interface. 1035 - RTM_FFA_DIR_REQ: runtime model presented to an execution context that is 1036 allocated CPU cycles through FFA_MSG_SEND_DIRECT_REQ interface. 1037 - RTM_SEC_INTERRUPT: runtime model presented to an execution context that is 1038 allocated CPU cycles by SPMC to handle a secure interrupt. 1039 - RTM_SP_INIT: runtime model presented to an execution context that is 1040 allocated CPU cycles by SPMC to initialize its state. 1041 1042If an endpoint execution context attempts to make an invalid transition or a 1043valid transition that could lead to a loop in the call chain, SPMC denies the 1044transition with the help of above runtime models. 1045 1046Interrupt management 1047-------------------- 1048 1049GIC ownership 1050~~~~~~~~~~~~~ 1051 1052The SPMC owns the GIC configuration. Secure and non-secure interrupts are 1053trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 1054IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 1055virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 1056 1057Abbreviations: 1058 1059 - NS-Int: A non-secure physical interrupt. It requires a switch to the normal 1060 world to be handled if it triggers while execution is in secure world. 1061 - Other S-Int: A secure physical interrupt targeted to an SP different from 1062 the one that is currently running. 1063 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1064 running. 1065 1066Non-secure interrupt handling 1067~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1068 1069This section documents the actions supported in SPMC in response to a non-secure 1070interrupt as per the guidance provided by FF-A v1.1 EAC0 specification. 1071An SP specifies one of the following actions in its partition manifest: 1072 1073 - Non-secure interrupt is signaled. 1074 - Non-secure interrupt is signaled after a managed exit. 1075 - Non-secure interrupt is queued. 1076 1077An SP execution context in a call chain could specify a less permissive action 1078than subsequent SP execution contexts in the same call chain. The less 1079permissive action takes precedence over the more permissive actions specified 1080by the subsequent execution contexts. Please refer to FF-A v1.1 EAC0 section 10818.3.1 for further explanation. 1082 1083Secure interrupt handling 1084~~~~~~~~~~~~~~~~~~~~~~~~~ 1085 1086This section documents the support implemented for secure interrupt handling in 1087SPMC as per the guidance provided by FF-A v1.1 EAC0 specification. 1088The following assumptions are made about the system configuration: 1089 1090 - In the current implementation, S-EL1 SPs are expected to use the para 1091 virtualized ABIs for interrupt management rather than accessing the virtual 1092 GIC interface. 1093 - Unless explicitly stated otherwise, this support is applicable only for 1094 S-EL1 SPs managed by SPMC. 1095 - Secure interrupts are configured as G1S or G0 interrupts. 1096 - All physical interrupts are routed to SPMC when running a secure partition 1097 execution context. 1098 - All endpoints with multiple execution contexts have their contexts pinned 1099 to corresponding CPUs. Hence, a secure virtual interrupt cannot be signaled 1100 to a target vCPU that is currently running or blocked on a different 1101 physical CPU. 1102 1103A physical secure interrupt could trigger while CPU is executing in normal world 1104or secure world. 1105The action of SPMC for a secure interrupt depends on: the state of the target 1106execution context of the SP that is responsible for handling the interrupt; 1107whether the interrupt triggered while execution was in normal world or secure 1108world. 1109 1110Secure interrupt signaling mechanisms 1111~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1112 1113Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 1114context that it has a pending virtual interrupt and to further run the SP 1115execution context, such that it can handle the virtual interrupt. SPMC uses 1116either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 1117to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 1118the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 1119running in S-EL2. 1120 1121+-----------+---------+---------------+---------------------------------------+ 1122| SP State | Conduit | Interface and | Description | 1123| | | parameters | | 1124+-----------+---------+---------------+---------------------------------------+ 1125| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 1126| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 1127| | | | resumes execution context of SP | 1128| | | | through ERET. | 1129+-----------+---------+---------------+---------------------------------------+ 1130| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 1131| | vIRQ | | is pending. It pends vIRQ signal and | 1132| | | | resumes execution context of SP | 1133| | | | through ERET. | 1134+-----------+---------+---------------+---------------------------------------+ 1135| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 1136| | | | not resume execution context of SP. | 1137+-----------+---------+---------------+---------------------------------------+ 1138| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 1139| | vIRQ | | execution context of SP through ERET. | 1140+-----------+---------+---------------+---------------------------------------+ 1141 1142Secure interrupt completion mechanisms 1143~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1144 1145A SP signals secure interrupt handling completion to the SPMC through the 1146following mechanisms: 1147 1148 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 1149 - ``FFA_RUN`` ABI if its was in BLOCKED state. 1150 1151This is a remnant of SPMC implementation based on the FF-A v1.0 specification. 1152In the current implementation, S-EL1 SPs use the para-virtualized HVC interface 1153implemented by SPMC to perform priority drop and interrupt deactivation (SPMC 1154configures EOImode = 0, i.e. priority drop and deactivation are done together). 1155The SPMC performs checks to deny the state transition upon invocation of 1156either FFA_MSG_WAIT or FFA_RUN interface if the SP didn't perform the 1157deactivation of the secure virtual interrupt. 1158 1159If the current SP execution context was preempted by a secure interrupt to be 1160handled by execution context of target SP, SPMC resumes current SP after signal 1161completion by target SP execution context. 1162 1163Actions for a secure interrupt triggered while execution is in normal world 1164~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1165 1166+-------------------+----------+-----------------------------------------------+ 1167| State of target | Action | Description | 1168| execution context | | | 1169+-------------------+----------+-----------------------------------------------+ 1170| WAITING | Signaled | This starts a new call chain in SPMC scheduled| 1171| | | mode. | 1172+-------------------+----------+-----------------------------------------------+ 1173| PREEMPTED | Queued | The target execution must have been preempted | 1174| | | by a non-secure interrupt. SPMC queues the | 1175| | | secure virtual interrupt now. It is signaled | 1176| | | when the target execution context next enters | 1177| | | the RUNNING state. | 1178+-------------------+----------+-----------------------------------------------+ 1179| BLOCKED, RUNNING | NA | The target execution context is blocked or | 1180| | | running on a different CPU. This is not | 1181| | | supported by current SPMC implementation and | 1182| | | execution hits panic. | 1183+-------------------+----------+-----------------------------------------------+ 1184 1185If normal world execution was preempted by a secure interrupt, SPMC uses 1186FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 1187and further returns execution to normal world. 1188 1189The following figure describes interrupt handling flow when a secure interrupt 1190triggers while execution is in normal world: 1191 1192.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1193 1194A brief description of the events: 1195 1196 - 1) Secure interrupt triggers while normal world is running. 1197 - 2) FIQ gets trapped to EL3. 1198 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1199 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1200 vIRQ). 1201 - 5) Assuming SP1 vCPU is in WAITING state, SPMC signals virtual interrupt 1202 using FFA_INTERRUPT with interrupt id as an argument and resumes the SP1 1203 vCPU using ERET in SPMC scheduled mode. 1204 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual 1205 interrupt is not masked i.e., PSTATE.I = 0 1206 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized 1207 HVC call. SPMC clears the pending virtual interrupt state management 1208 and returns the pending virtual interrupt id. 1209 - 8) SP1 services the virtual interrupt and invokes the paravirtualized 1210 de-activation HVC call. SPMC de-activates the physical interrupt, 1211 clears the fields tracking the secure interrupt and resumes SP1 vCPU. 1212 - 9) SP1 performs secure interrupt completion through FFA_MSG_WAIT ABI. 1213 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1214 - 11) EL3 resumes normal world execution. 1215 1216Actions for a secure interrupt triggered while execution is in secure world 1217~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1218 1219+-------------------+----------+------------------------------------------------+ 1220| State of target | Action | Description | 1221| execution context | | | 1222+-------------------+----------+------------------------------------------------+ 1223| WAITING | Signaled | This starts a new call chain in SPMC scheduled | 1224| | | mode. | 1225+-------------------+----------+------------------------------------------------+ 1226| PREEMPTED by Self | Signaled | The target execution context reenters the | 1227| S-Int | | RUNNING state to handle the secure virtual | 1228| | | interrupt. | 1229+-------------------+----------+------------------------------------------------+ 1230| PREEMPTED by | Queued | SPMC queues the secure virtual interrupt now. | 1231| NS-Int | | It is signaled when the target execution | 1232| | | context next enters the RUNNING state. | 1233+-------------------+----------+------------------------------------------------+ 1234| BLOCKED | Signaled | Both preempted and target execution contexts | 1235| | | must have been part of the Normal world | 1236| | | scheduled call chain. Refer scenario 1 of | 1237| | | Table 8.4 in the FF-A v1.1 EAC0 spec. | 1238+-------------------+----------+------------------------------------------------+ 1239| RUNNING | NA | The target execution context is running on a | 1240| | | different CPU. This scenario is not supported | 1241| | | by current SPMC implementation and execution | 1242| | | hits panic. | 1243+-------------------+----------+------------------------------------------------+ 1244 1245The following figure describes interrupt handling flow when a secure interrupt 1246triggers while execution is in secure world. We assume OS kernel sends a direct 1247request message to SP1. Further, SP1 sends a direct request message to SP2. SP1 1248enters BLOCKED state and SPMC resumes SP2. 1249 1250.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1251 1252A brief description of the events: 1253 1254 - 1) Secure interrupt triggers while SP2 is running. 1255 - 2) SP2 gets preempted and execution traps to SPMC as IRQ. 1256 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1257 this secure interrupt. In this scenario, it is SP1. 1258 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1259 SPMC further resumes SP1 through ERET conduit. Note that SP1 remains in 1260 Normal world schedule mode. 1261 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual 1262 interrupt is not masked i.e., PSTATE.I = 0 1263 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized 1264 HVC call. SPMC clears the pending virtual interrupt state management 1265 and returns the pending virtual interrupt id. 1266 - 8) SP1 services the virtual interrupt and invokes the paravirtualized 1267 de-activation HVC call. SPMC de-activates the physical interrupt and 1268 clears the fields tracking the secure interrupt and resumes SP1 vCPU. 1269 - 9) Since SP1 direct request completed with FFA_INTERRUPT, it resumes the 1270 direct request to SP2 by invoking FFA_RUN. 1271 - 9) SPMC resumes the pre-empted vCPU of SP2. 1272 1273Power management 1274---------------- 1275 1276In platforms with or without secure virtualization: 1277 1278- The NWd owns the platform PM policy. 1279- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1280- The EL3 PSCI library is in charge of the PM coordination and control 1281 (eventually writing to platform registers). 1282- While coordinating PM events, the PSCI library calls backs into the Secure 1283 Payload Dispatcher for events the latter has statically registered to. 1284 1285When using the SPMD as a Secure Payload Dispatcher: 1286 1287- A power management event is relayed through the SPD hook to the SPMC. 1288- In the current implementation only cpu on (svc_on_finish) and cpu off 1289 (svc_off) hooks are registered. 1290- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1291 The SPMC is entered through its secondary physical core entry point. 1292- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The PM event is 1293 signaled to the SPMC through a power management framework message. 1294 It consists in a SPMD-to-SPMC direct request/response (`SPMC-SPMD direct 1295 requests/responses`_) conveying the event details and SPMC response. 1296 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1297 updates its internal state to reflect the physical core is being turned off. 1298 In the current implementation no SP is resumed as a consequence. This behavior 1299 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1300 userspace. 1301 1302Arm architecture extensions for security hardening 1303================================================== 1304 1305Hafnium supports the following architecture extensions for security hardening: 1306 1307- Pointer authentication (FEAT_PAuth): the extension permits detection of forged 1308 pointers used by ROP type of attacks through the signing of the pointer 1309 value. Hafnium is built with the compiler branch protection option to permit 1310 generation of a pointer authentication code for return addresses (pointer 1311 authentication for instructions). The APIA key is used while Hafnium runs. 1312 A random key is generated at boot time and restored upon entry into Hafnium 1313 at run-time. APIA and other keys (APIB, APDA, APDB, APGA) are saved/restored 1314 in vCPU contexts permitting to enable pointer authentication in VMs/SPs. 1315- Branch Target Identification (FEAT_BTI): the extension permits detection of 1316 unexpected indirect branches used by JOP type of attacks. Hafnium is built 1317 with the compiler branch protection option, inserting land pads at function 1318 prologues that are reached by indirect branch instructions (BR/BLR). 1319 Hafnium code pages are marked as guarded in the EL2 Stage-1 MMU descriptors 1320 such that an indirect branch must always target a landpad. A fault is 1321 triggered otherwise. VMs/SPs can (independently) mark their code pages as 1322 guarded in the EL1&0 Stage-1 translation regime. 1323- Memory Tagging Extension (FEAT_MTE): the option permits detection of out of 1324 bound memory array accesses or re-use of an already freed memory region. 1325 Hafnium enables the compiler option permitting to leverage MTE stack tagging 1326 applied to core stacks. Core stacks are marked as normal tagged memory in the 1327 EL2 Stage-1 translation regime. A synchronous data abort is generated upon tag 1328 check failure on load/stores. A random seed is generated at boot time and 1329 restored upon entry into Hafnium. MTE system registers are saved/restored in 1330 vCPU contexts permitting MTE usage from VMs/SPs. 1331 1332SMMUv3 support in Hafnium 1333========================= 1334 1335An SMMU is analogous to an MMU in a CPU. It performs address translations for 1336Direct Memory Access (DMA) requests from system I/O devices. 1337The responsibilities of an SMMU include: 1338 1339- Translation: Incoming DMA requests are translated from bus address space to 1340 system physical address space using translation tables compliant to 1341 Armv8/Armv7 VMSA descriptor format. 1342- Protection: An I/O device can be prohibited from read, write access to a 1343 memory region or allowed. 1344- Isolation: Traffic from each individial device can be independently managed. 1345 The devices are differentiated from each other using unique translation 1346 tables. 1347 1348The following diagram illustrates a typical SMMU IP integrated in a SoC with 1349several I/O devices along with Interconnect and Memory system. 1350 1351.. image:: ../resources/diagrams/MMU-600.png 1352 1353SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1354support for SMMUv3 driver in both normal and secure world. A brief introduction 1355of SMMUv3 functionality and the corresponding software support in Hafnium is 1356provided here. 1357 1358SMMUv3 features 1359--------------- 1360 1361- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1362 translation support. It can either bypass or abort incoming translations as 1363 well. 1364- Traffic (memory transactions) from each upstream I/O peripheral device, 1365 referred to as Stream, can be independently managed using a combination of 1366 several memory based configuration structures. This allows the SMMUv3 to 1367 support a large number of streams with each stream assigned to a unique 1368 translation context. 1369- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1370 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1371 are supported by SMMUv3. 1372- SMMUv3 offers non-secure stream support with secure stream support being 1373 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1374 instance for secure and non-secure stream support. 1375- It also supports sub-streams to differentiate traffic from a virtualized 1376 peripheral associated with a VM/SP. 1377- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1378 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1379 for providing Secure Stage2 translation support to upstream peripheral 1380 devices. 1381 1382SMMUv3 Programming Interfaces 1383----------------------------- 1384 1385SMMUv3 has three software interfaces that are used by the Hafnium driver to 1386configure the behaviour of SMMUv3 and manage the streams. 1387 1388- Memory based data strutures that provide unique translation context for 1389 each stream. 1390- Memory based circular buffers for command queue and event queue. 1391- A large number of SMMU configuration registers that are memory mapped during 1392 boot time by Hafnium driver. Except a few registers, all configuration 1393 registers have independent secure and non-secure versions to configure the 1394 behaviour of SMMUv3 for translation of secure and non-secure streams 1395 respectively. 1396 1397Peripheral device manifest 1398-------------------------- 1399 1400Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1401These devices are dependent on PE endpoint to initiate and receive memory 1402management transactions on their behalf. The acccess to the MMIO regions of 1403any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 1404uses the same stage 2 translations for the device as those used by partition 1405manager on behalf of the PE endpoint. This ensures that the peripheral device 1406has the same visibility of the physical address space as the endpoint. The 1407device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 1408) must specify these additional properties for each peripheral device in the 1409system : 1410 1411- smmu-id: This field helps to identify the SMMU instance that this device is 1412 upstream of. 1413- stream-ids: List of stream IDs assigned to this device. 1414 1415.. code:: shell 1416 1417 smmuv3-testengine { 1418 base-address = <0x00000000 0x2bfe0000>; 1419 pages-count = <32>; 1420 attributes = <0x3>; 1421 smmu-id = <0>; 1422 stream-ids = <0x0 0x1>; 1423 interrupts = <0x2 0x3>, <0x4 0x5>; 1424 exclusive-access; 1425 }; 1426 1427SMMUv3 driver limitations 1428------------------------- 1429 1430The primary design goal for the Hafnium SMMU driver is to support secure 1431streams. 1432 1433- Currently, the driver only supports Stage2 translations. No support for 1434 Stage1 or nested translations. 1435- Supports only AArch64 translation format. 1436- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1437 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1438- No support for independent peripheral devices. 1439 1440S-EL0 Partition support 1441======================= 1442The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1443FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1444with ARMv8.4 and FEAT_SEL2). 1445 1446S-EL0 partitions are useful for simple partitions that don't require full 1447Trusted OS functionality. It is also useful to reduce jitter and cycle 1448stealing from normal world since they are more lightweight than VMs. 1449 1450S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1451the SPMC. They are differentiated primarily by the 'exception-level' property 1452and the 'execution-ctx-count' property in the SP manifest. They are host apps 1453under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1454call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1455can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1456for memory regions. 1457 1458S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1459capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1460a S-EL0 partition to accept a direct message from secure world and normal world, 1461and generate direct responses to them. 1462 1463Memory sharing between and with S-EL0 partitions is supported. 1464Indirect messaging, Interrupt handling and Notifications are not supported with 1465S-EL0 partitions and is work in progress, planned for future releases. 1466All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not 1467supported. 1468 1469 1470References 1471========== 1472 1473.. _[1]: 1474 1475[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1476 1477.. _[2]: 1478 1479[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 1480 1481.. _[3]: 1482 1483[3] `Trusted Boot Board Requirements 1484Client <https://developer.arm.com/documentation/den0006/d/>`__ 1485 1486.. _[4]: 1487 1488[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1489 1490.. _[5]: 1491 1492[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1493 1494.. _[6]: 1495 1496[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1497 1498.. _[7]: 1499 1500[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1501 1502.. _[8]: 1503 1504[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/ 1505 1506.. _[9]: 1507 1508[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1509 1510-------------- 1511 1512*Copyright (c) 2020-2022, Arm Limited and Contributors. All rights reserved.* 1513
