1Secure Partition Manager 2************************ 3 4.. contents:: 5 6.. toctree:: 7 ffa-manifest-binding 8 9Acronyms 10======== 11 12+--------+--------------------------------------+ 13| CoT | Chain of Trust | 14+--------+--------------------------------------+ 15| DMA | Direct Memory Access | 16+--------+--------------------------------------+ 17| DTB | Device Tree Blob | 18+--------+--------------------------------------+ 19| DTS | Device Tree Source | 20+--------+--------------------------------------+ 21| EC | Execution Context | 22+--------+--------------------------------------+ 23| FIP | Firmware Image Package | 24+--------+--------------------------------------+ 25| FF-A | Firmware Framework for Arm A-profile | 26+--------+--------------------------------------+ 27| IPA | Intermediate Physical Address | 28+--------+--------------------------------------+ 29| JOP | Jump-Oriented Programming | 30+--------+--------------------------------------+ 31| NWd | Normal World | 32+--------+--------------------------------------+ 33| ODM | Original Design Manufacturer | 34+--------+--------------------------------------+ 35| OEM | Original Equipment Manufacturer | 36+--------+--------------------------------------+ 37| PA | Physical Address | 38+--------+--------------------------------------+ 39| PE | Processing Element | 40+--------+--------------------------------------+ 41| PM | Power Management | 42+--------+--------------------------------------+ 43| PVM | Primary VM | 44+--------+--------------------------------------+ 45| ROP | Return-Oriented Programming | 46+--------+--------------------------------------+ 47| SMMU | System Memory Management Unit | 48+--------+--------------------------------------+ 49| SP | Secure Partition | 50+--------+--------------------------------------+ 51| SPD | Secure Payload Dispatcher | 52+--------+--------------------------------------+ 53| SPM | Secure Partition Manager | 54+--------+--------------------------------------+ 55| SPMC | SPM Core | 56+--------+--------------------------------------+ 57| SPMD | SPM Dispatcher | 58+--------+--------------------------------------+ 59| SiP | Silicon Provider | 60+--------+--------------------------------------+ 61| SWd | Secure World | 62+--------+--------------------------------------+ 63| TLV | Tag-Length-Value | 64+--------+--------------------------------------+ 65| TOS | Trusted Operating System | 66+--------+--------------------------------------+ 67| VM | Virtual Machine | 68+--------+--------------------------------------+ 69 70Foreword 71======== 72 73Three implementations of a Secure Partition Manager co-exist in the TF-A 74codebase: 75 76#. S-EL2 SPMC based on the FF-A specification `[1]`_, enabling virtualization in 77 the secure world, managing multiple S-EL1 or S-EL0 partitions. 78#. EL3 SPMC based on the FF-A specification, managing a single S-EL1 partition 79 without virtualization in the secure world. 80#. EL3 SPM based on the MM specification, legacy implementation managing a 81 single S-EL0 partition `[2]`_. 82 83These implementations differ in their respective SW architecture and only one 84can be selected at build time. This document: 85 86- describes the implementation from bullet 1. when the SPMC resides at S-EL2. 87- is not an architecture specification and it might provide assumptions 88 on sections mandated as implementation-defined in the specification. 89- covers the implications to TF-A used as a bootloader, and Hafnium used as a 90 reference code base for an S-EL2/SPMC secure firmware on platforms 91 implementing the FEAT_SEL2 architecture extension. 92 93Terminology 94----------- 95 96- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 97 (or partitions) in the normal world. 98- The term SPMC refers to the S-EL2 component managing secure partitions in 99 the secure world when the FEAT_SEL2 architecture extension is implemented. 100- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 101 partition and implementing the FF-A ABI on platforms not implementing the 102 FEAT_SEL2 architecture extension. 103- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 104- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 105 106Support for legacy platforms 107---------------------------- 108 109The SPM is split into a dispatcher and a core component (respectively SPMD and 110SPMC) residing at different exception levels. To permit the FF-A specification 111adoption and a smooth migration, the SPMD supports an SPMC residing either at 112S-EL1 or S-EL2: 113 114- The SPMD is located at EL3 and mainly relays the FF-A protocol from NWd 115 (Hypervisor or OS kernel) to the SPMC. 116- The same SPMD component is used for both S-EL1 and S-EL2 SPMC configurations. 117- The SPMC exception level is a build time choice. 118 119TF-A supports both cases: 120 121- S-EL1 SPMC for platforms not supporting the FEAT_SEL2 architecture 122 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 123- S-EL2 SPMC for platforms implementing the FEAT_SEL2 architecture 124 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 125 126Sample reference stack 127====================== 128 129The following diagram illustrates a possible configuration when the 130FEAT_SEL2 architecture extension is implemented, showing the SPMD 131and SPMC, one or multiple secure partitions, with an optional 132Hypervisor: 133 134.. image:: ../resources/diagrams/ff-a-spm-sel2.png 135 136TF-A build options 137================== 138 139This section explains the TF-A build options involved in building with 140support for an FF-A based SPM where the SPMD is located at EL3 and the 141SPMC located at S-EL1, S-EL2 or EL3: 142 143- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 144 protocol from NWd to SWd back and forth. It is not possible to 145 enable another Secure Payload Dispatcher when this option is chosen. 146- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 147 level to being at S-EL2. It defaults to enabled (value 1) when 148 SPD=spmd is chosen. 149- **SPMC_AT_EL3**: this option adjusts the SPMC exception level to being 150 at EL3. 151- If neither ``SPMD_SPM_AT_SEL2`` or ``SPMC_AT_EL3`` are enabled the SPMC 152 exception level is set to S-EL1. 153 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 154 and exhaustive list of registers is visible at `[4]`_. 155- **SP_LAYOUT_FILE**: this option specifies a text description file 156 providing paths to SP binary images and manifests in DTS format 157 (see `Describing secure partitions`_). It 158 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 159 secure partitions are to be loaded by BL2 on behalf of the SPMC. 160 161+---------------+------------------+-------------+-------------------------+ 162| | SPMD_SPM_AT_SEL2 | SPMC_AT_EL3 | CTX_INCLUDE_EL2_REGS(*) | 163+---------------+------------------+-------------+-------------------------+ 164| SPMC at S-EL1 | 0 | 0 | 0 | 165+---------------+------------------+-------------+-------------------------+ 166| SPMC at S-EL2 | 1 (default when | 0 | 1 | 167| | SPD=spmd) | | | 168+---------------+------------------+-------------+-------------------------+ 169| SPMC at EL3 | 0 | 1 | 0 | 170+---------------+------------------+-------------+-------------------------+ 171 172Other combinations of such build options either break the build or are not 173supported. 174 175Notes: 176 177- Only Arm's FVP platform is supported to use with the TF-A reference software 178 stack. 179- When ``SPMD_SPM_AT_SEL2=1``, the reference software stack assumes enablement 180 of FEAT_PAuth, FEAT_BTI and FEAT_MTE architecture extensions. 181- ``(*) CTX_INCLUDE_EL2_REGS``, this flag is |TF-A| internal and informational 182 in this table. When set, it provides the generic support for saving/restoring 183 EL2 registers required when S-EL2 firmware is present. 184- BL32 option is re-purposed to specify the SPMC image. It can specify either 185 the Hafnium binary path (built for the secure world) or the path to a TEE 186 binary implementing FF-A interfaces. 187- BL33 option can specify the TFTF binary or a normal world loader 188 such as U-Boot or the UEFI framework payload. 189 190Sample TF-A build command line when the SPMC is located at S-EL1 191(e.g. when the FEAT_SEL2 architecture extension is not implemented): 192 193.. code:: shell 194 195 make \ 196 CROSS_COMPILE=aarch64-none-elf- \ 197 SPD=spmd \ 198 SPMD_SPM_AT_SEL2=0 \ 199 BL32=<path-to-tee-binary> \ 200 BL33=<path-to-bl33-binary> \ 201 PLAT=fvp \ 202 all fip 203 204Sample TF-A build command line when FEAT_SEL2 architecture extension is 205implemented and the SPMC is located at S-EL2: 206.. code:: shell 207 208 make \ 209 CROSS_COMPILE=aarch64-none-elf- \ 210 PLAT=fvp \ 211 SPD=spmd \ 212 ARM_ARCH_MINOR=5 \ 213 BRANCH_PROTECTION=1 \ 214 CTX_INCLUDE_PAUTH_REGS=1 \ 215 CTX_INCLUDE_MTE_REGS=1 \ 216 BL32=<path-to-hafnium-binary> \ 217 BL33=<path-to-bl33-binary> \ 218 SP_LAYOUT_FILE=sp_layout.json \ 219 all fip 220 221Sample TF-A build command line when FEAT_SEL2 architecture extension is 222implemented, the SPMC is located at S-EL2, and enabling secure boot: 223.. code:: shell 224 225 make \ 226 CROSS_COMPILE=aarch64-none-elf- \ 227 PLAT=fvp \ 228 SPD=spmd \ 229 ARM_ARCH_MINOR=5 \ 230 BRANCH_PROTECTION=1 \ 231 CTX_INCLUDE_PAUTH_REGS=1 \ 232 CTX_INCLUDE_MTE_REGS=1 \ 233 BL32=<path-to-hafnium-binary> \ 234 BL33=<path-to-bl33-binary> \ 235 SP_LAYOUT_FILE=sp_layout.json \ 236 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 237 TRUSTED_BOARD_BOOT=1 \ 238 COT=dualroot \ 239 ARM_ROTPK_LOCATION=devel_rsa \ 240 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 241 GENERATE_COT=1 \ 242 all fip 243 244Sample TF-A build command line when the SPMC is located at EL3: 245 246.. code:: shell 247 248 make \ 249 CROSS_COMPILE=aarch64-none-elf- \ 250 SPD=spmd \ 251 SPMD_SPM_AT_SEL2=0 \ 252 SPMC_AT_EL3=1 \ 253 BL32=<path-to-tee-binary> \ 254 BL33=<path-to-bl33-binary> \ 255 PLAT=fvp \ 256 all fip 257 258FVP model invocation 259==================== 260 261The FVP command line needs the following options to exercise the S-EL2 SPMC: 262 263+---------------------------------------------------+------------------------------------+ 264| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 265| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 266+---------------------------------------------------+------------------------------------+ 267| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 268| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 269| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 270| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 271| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 272| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 273| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 274| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 275+---------------------------------------------------+------------------------------------+ 276| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 277| - cluster1.has_branch_target_exception=1 | | 278+---------------------------------------------------+------------------------------------+ 279| - cluster0.has_pointer_authentication=2 | Implements FEAT_PAuth | 280| - cluster1.has_pointer_authentication=2 | | 281+---------------------------------------------------+------------------------------------+ 282| - cluster0.memory_tagging_support_level=2 | Implements FEAT_MTE2 | 283| - cluster1.memory_tagging_support_level=2 | | 284| - bp.dram_metadata.is_enabled=1 | | 285+---------------------------------------------------+------------------------------------+ 286 287Sample FVP command line invocation: 288 289.. code:: shell 290 291 <path-to-fvp-model>/FVP_Base_RevC-2xAEMvA -C pctl.startup=0.0.0.0 \ 292 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 293 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 294 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 295 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 296 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 297 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 \ 298 -C cluster0.has_pointer_authentication=2 -C cluster1.has_pointer_authentication=2 \ 299 -C cluster0.has_branch_target_exception=1 -C cluster1.has_branch_target_exception=1 \ 300 -C cluster0.memory_tagging_support_level=2 -C cluster1.memory_tagging_support_level=2 \ 301 -C bp.dram_metadata.is_enabled=1 \ 302 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B \ 303 -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 \ 304 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 \ 305 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 306 307Boot process 308============ 309 310Loading Hafnium and secure partitions in the secure world 311--------------------------------------------------------- 312 313TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 314 315SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 316Thus they are supplied as distinct signed entities within the FIP flash 317image. The FIP image itself is not signed hence this provides the ability 318to upgrade SPs in the field. 319 320Booting through TF-A 321-------------------- 322 323SP manifests 324~~~~~~~~~~~~ 325 326An SP manifest describes SP attributes as defined in `[1]`_ 327(partition manifest at virtual FF-A instance) in DTS format. It is 328represented as a single file associated with the SP. A sample is 329provided by `[5]`_. A binding document is provided by `[6]`_. 330 331Secure Partition packages 332~~~~~~~~~~~~~~~~~~~~~~~~~ 333 334Secure partitions are bundled as independent package files consisting 335of: 336 337- a header 338- a DTB 339- an image payload 340 341The header starts with a magic value and offset values to SP DTB and 342image payload. Each SP package is loaded independently by BL2 loader 343and verified for authenticity and integrity. 344 345The SP package identified by its UUID (matching FF-A uuid property) is 346inserted as a single entry into the FIP at end of the TF-A build flow 347as shown: 348 349.. code:: shell 350 351 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 352 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 353 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 354 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 355 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 356 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 357 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 358 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 359 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 360 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 361 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 362 363.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 364 365Describing secure partitions 366~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 367 368A json-formatted description file is passed to the build flow specifying paths 369to the SP binary image and associated DTS partition manifest file. The latter 370is processed by the dtc compiler to generate a DTB fed into the SP package. 371Optionally, the partition's json description can contain offsets for both 372the image and partition manifest within the SP package. Both offsets need to be 3734KB aligned, because it is the translation granule supported by Hafnium SPMC. 374These fields can be leveraged to support SPs with S1 translation granules that 375differ from 4KB, and to configure the regions allocated within the SP package, 376as well as to comply with the requirements for the implementation of the boot 377information protocol (see `Passing boot data to the SP`_ for more details). In 378case the offsets are absent in their json node, they default to 0x1000 and 3790x4000 for the manifest offset and image offset respectively. 380This file also specifies the SP owner (as an optional field) identifying the 381signing domain in case of dual root CoT. 382The SP owner can either be the silicon or the platform provider. The 383corresponding "owner" field value can either take the value of "SiP" or "Plat". 384In absence of "owner" field, it defaults to "SiP" owner. 385The UUID of the partition can be specified as a field in the description file or 386if it does not exist there the UUID is extracted from the DTS partition 387manifest. 388 389.. code:: shell 390 391 { 392 "tee1" : { 393 "image": "tee1.bin", 394 "pm": "tee1.dts", 395 "owner": "SiP", 396 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f" 397 }, 398 399 "tee2" : { 400 "image": "tee2.bin", 401 "pm": "tee2.dts", 402 "owner": "Plat" 403 }, 404 405 "tee3" : { 406 "image": { 407 "file": "tee3.bin", 408 "offset":"0x2000" 409 }, 410 "pm": { 411 "file": "tee3.dts", 412 "offset":"0x6000" 413 }, 414 "owner": "Plat" 415 }, 416 } 417 418SPMC manifest 419~~~~~~~~~~~~~ 420 421This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 422time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 423two different cases: 424 425- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 426 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 427 mode. 428- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 429 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 430 S-EL0. 431 432.. code:: shell 433 434 attribute { 435 spmc_id = <0x8000>; 436 maj_ver = <0x1>; 437 min_ver = <0x1>; 438 exec_state = <0x0>; 439 load_address = <0x0 0x6000000>; 440 entrypoint = <0x0 0x6000000>; 441 binary_size = <0x60000>; 442 }; 443 444- *spmc_id* defines the endpoint ID value that SPMC can query through 445 ``FFA_ID_GET``. 446- *maj_ver/min_ver*. SPMD checks provided version versus its internal 447 version and aborts if not matching. 448- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 449 Notice Hafnium used as a SPMC only supports AArch64. 450- *load_address* and *binary_size* are mostly used to verify secondary 451 entry points fit into the loaded binary image. 452- *entrypoint* defines the cold boot primary core entry point used by 453 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 454 455Other nodes in the manifest are consumed by Hafnium in the secure world. 456A sample can be found at `[7]`_: 457 458- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 459 indicates a FF-A compliant SP. The *load_address* field specifies the load 460 address at which BL2 loaded the SP package. 461- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 462 Note the primary core is declared first, then secondary cores are declared 463 in reverse order. 464- The *memory* node provides platform information on the ranges of memory 465 available to the SPMC. 466 467SPMC boot 468~~~~~~~~~ 469 470The SPMC is loaded by BL2 as the BL32 image. 471 472The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 473 474BL2 passes the SPMC manifest address to BL31 through a register. 475 476At boot time, the SPMD in BL31 runs from the primary core, initializes the core 477contexts and launches the SPMC (BL32) passing the following information through 478registers: 479 480- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 481- X1 holds the ``HW_CONFIG`` physical address. 482- X4 holds the currently running core linear id. 483 484Loading of SPs 485~~~~~~~~~~~~~~ 486 487At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 488below: 489 490.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 491 492Note this boot flow is an implementation sample on Arm's FVP platform. 493Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 494different boot flow. The flow restricts to a maximum of 8 secure partitions. 495 496Secure boot 497~~~~~~~~~~~ 498 499The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 500SPMC manifest, secure partitions and verifies them for authenticity and integrity. 501Refer to TBBR specification `[3]`_. 502 503The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 504the use of two root keys namely S-ROTPK and NS-ROTPK: 505 506- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 507- BL33 may be signed by the OEM using NS-ROTPK. 508- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 509- A maximum of 4 partitions can be signed with the S-ROTPK key and 4 partitions 510 signed with the NS-ROTPK key. 511 512Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 513 514Hafnium in the secure world 515=========================== 516 517General considerations 518---------------------- 519 520Build platform for the secure world 521~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 522 523In the Hafnium reference implementation specific code parts are only relevant to 524the secure world. Such portions are isolated in architecture specific files 525and/or enclosed by a ``SECURE_WORLD`` macro. 526 527Secure partitions scheduling 528~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 529 530The FF-A specification `[1]`_ provides two ways to relinquinsh CPU time to 531secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 532 533- the FFA_MSG_SEND_DIRECT_REQ interface. 534- the FFA_RUN interface. 535 536Additionally a secure interrupt can pre-empt the normal world execution and give 537CPU cycles by transitioning to EL3 and S-EL2. 538 539Platform topology 540~~~~~~~~~~~~~~~~~ 541 542The *execution-ctx-count* SP manifest field can take the value of one or the 543total number of PEs. The FF-A specification `[1]`_ recommends the 544following SP types: 545 546- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 547 implement the same number of ECs as the number of PEs in the platform. 548- Migratable UP SPs: a single execution context can run and be migrated on any 549 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 550 receive a direct message request originating from any physical core targeting 551 the single execution context. 552 553Parsing SP partition manifests 554------------------------------ 555 556Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 557Note the current implementation may not implement all optional fields. 558 559The SP manifest may contain memory and device regions nodes. In case of 560an S-EL2 SPMC: 561 562- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 563 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 564 specify RX/TX buffer regions in which case it is not necessary for an SP 565 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 566- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 567 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 568 additional resources (e.g. interrupts). 569 570For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 571provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 572regime. 573 574Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 575same set of page tables. It is still open whether two sets of page tables shall 576be provided per SP. The memory region node as defined in the specification 577provides a memory security attribute hinting to map either to the secure or 578non-secure EL1&0 Stage-2 table if it exists. 579 580Passing boot data to the SP 581--------------------------- 582 583In `[1]`_ , the section "Boot information protocol" defines a method for passing 584data to the SPs at boot time. It specifies the format for the boot information 585descriptor and boot information header structures, which describe the data to be 586exchanged between SPMC and SP. 587The specification also defines the types of data that can be passed. 588The aggregate of both the boot info structures and the data itself is designated 589the boot information blob, and is passed to a Partition as a contiguous memory 590region. 591 592Currently, the SPM implementation supports the FDT type which is used to pass the 593partition's DTB manifest. 594 595The region for the boot information blob is allocated through the SP package. 596 597.. image:: ../resources/diagrams/partition-package.png 598 599To adjust the space allocated for the boot information blob, the json description 600of the SP (see section `Describing secure partitions`_) shall be updated to contain 601the manifest offset. If no offset is provided the manifest offset defaults to 0x1000, 602which is the page size in the Hafnium SPMC. 603 604The configuration of the boot protocol is done in the SPs manifest. As defined by 605the specification, the manifest field 'gp-register-num' configures the GP register 606which shall be used to pass the address to the partitions boot information blob when 607booting the partition. 608In addition, the Hafnium SPMC implementation requires the boot information arguments 609to be listed in a designated DT node: 610 611.. code:: shell 612 613 boot-info { 614 compatible = "arm,ffa-manifest-boot-info"; 615 ffa_manifest; 616 }; 617 618The whole secure partition package image (see `Secure Partition packages`_) is 619mapped to the SP secure EL1&0 Stage-2 translation regime. As such, the SP can 620retrieve the address for the boot information blob in the designated GP register, 621process the boot information header and descriptors, access its own manifest 622DTB blob and extract its partition manifest properties. 623 624SP Boot order 625------------- 626 627SP manifests provide an optional boot order attribute meant to resolve 628dependencies such as an SP providing a service required to properly boot 629another SP. SPMC boots the SPs in accordance to the boot order attribute, 630lowest to the highest value. If the boot order attribute is absent from the FF-A 631manifest, the SP is treated as if it had the highest boot order value 632(i.e. lowest booting priority). 633 634It is possible for an SP to call into another SP through a direct request 635provided the latter SP has already been booted. 636 637Boot phases 638----------- 639 640Primary core boot-up 641~~~~~~~~~~~~~~~~~~~~ 642 643Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 644core. The SPMC performs its platform initializations and registers the SPMC 645secondary physical core entry point physical address by the use of the 646`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 647at secure physical FF-A instance). 648 649The SPMC then creates secure partitions based on SP packages and manifests. Each 650secure partition is launched in sequence (`SP Boot order`_) on their "primary" 651execution context. If the primary boot physical core linear id is N, an MP SP is 652started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 653UP SP, it is started using its unique EC0 on PE[N]. 654 655The SP primary EC (or the EC used when the partition is booted as described 656above): 657 658- Performs the overall SP boot time initialization, and in case of a MP SP, 659 prepares the SP environment for other execution contexts. 660- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 661 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 662 entry point for other execution contexts. 663- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 664 ``FFA_ERROR`` in case of failure. 665 666Secondary cores boot-up 667~~~~~~~~~~~~~~~~~~~~~~~ 668 669Once the system is started and NWd brought up, a secondary physical core is 670woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 671calls into the SPMD on the newly woken up physical core. Then the SPMC is 672entered at the secondary physical core entry point. 673 674In the current implementation, the first SP is resumed on the coresponding EC 675(the virtual CPU which matches the physical core). The implication is that the 676first SP must be a MP SP. 677 678In a linux based system, once secure and normal worlds are booted but prior to 679a NWd FF-A driver has been loaded: 680 681- The first SP has initialized all its ECs in response to primary core boot up 682 (at system initialization) and secondary core boot up (as a result of linux 683 invoking PSCI_CPU_ON for all secondary cores). 684- Other SPs have their first execution context initialized as a result of secure 685 world initialization on the primary boot core. Other ECs for those SPs have to 686 be run first through ffa_run to complete their initialization (which results 687 in the EC completing with FFA_MSG_WAIT). 688 689Refer to `Power management`_ for further details. 690 691Notifications 692------------- 693 694The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 695communication mechanism with non-blocking semantics. It allows for one FF-A 696endpoint to signal another for service provision, without hindering its current 697progress. 698 699Hafnium currently supports 64 notifications. The IDs of each notification define 700a position in a 64-bit bitmap. 701 702The signaling of notifications can interchangeably happen between NWd and SWd 703FF-A endpoints. 704 705The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 706VMs, and from VMs to SPs. An hypervisor component would only manage 707notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 708deployed in NWd, the Hypervisor or OS kernel must invoke the interface 709FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 710endpoint in the NWd that supports it. 711 712A sender can signal notifications once the receiver has provided it with 713permissions. Permissions are provided by invoking the interface 714FFA_NOTIFICATION_BIND. 715 716Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 717they are considered to be in a pending sate. The receiver can retrieve its 718pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 719are considered to be handled. 720 721Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 722that is in charge of donating CPU cycles for notifications handling. The 723FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 724which FF-A endpoints have pending notifications. The receiver scheduler is 725called and informed by the FF-A driver, and it should allocate CPU cycles to the 726receiver. 727 728There are two types of notifications supported: 729 730- Global, which are targeted to a FF-A endpoint and can be handled within any of 731 its execution contexts, as determined by the scheduler of the system. 732- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 733 a specific execution context, as determined by the sender. 734 735The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 736permissions to the sender. 737 738Notification signaling resorts to two interrupts: 739 740- Schedule Receiver Interrupt: non-secure physical interrupt to be handled by 741 the FF-A driver within the receiver scheduler. At initialization the SPMC 742 donates a SGI ID chosen from the secure SGI IDs range and configures it as 743 non-secure. The SPMC triggers this SGI on the currently running core when 744 there are pending notifications, and the respective receivers need CPU cycles 745 to handle them. 746- Notifications Pending Interrupt: virtual interrupt to be handled by the 747 receiver of the notification. Set when there are pending notifications for the 748 given secure partition. The NPI is pended when the NWd relinquishes CPU cycles 749 to an SP. 750 751The notifications receipt support is enabled in the partition FF-A manifest. 752 753Mandatory interfaces 754-------------------- 755 756The following interfaces are exposed to SPs: 757 758- ``FFA_VERSION`` 759- ``FFA_FEATURES`` 760- ``FFA_RX_RELEASE`` 761- ``FFA_RXTX_MAP`` 762- ``FFA_RXTX_UNMAP`` 763- ``FFA_PARTITION_INFO_GET`` 764- ``FFA_ID_GET`` 765- ``FFA_MSG_WAIT`` 766- ``FFA_MSG_SEND_DIRECT_REQ`` 767- ``FFA_MSG_SEND_DIRECT_RESP`` 768- ``FFA_MEM_DONATE`` 769- ``FFA_MEM_LEND`` 770- ``FFA_MEM_SHARE`` 771- ``FFA_MEM_RETRIEVE_REQ`` 772- ``FFA_MEM_RETRIEVE_RESP`` 773- ``FFA_MEM_RELINQUISH`` 774- ``FFA_MEM_FRAG_RX`` 775- ``FFA_MEM_FRAG_TX`` 776- ``FFA_MEM_RECLAIM`` 777- ``FFA_RUN`` 778 779As part of the FF-A v1.1 support, the following interfaces were added: 780 781 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 782 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 783 - ``FFA_NOTIFICATION_BIND`` 784 - ``FFA_NOTIFICATION_UNBIND`` 785 - ``FFA_NOTIFICATION_SET`` 786 - ``FFA_NOTIFICATION_GET`` 787 - ``FFA_NOTIFICATION_INFO_GET`` 788 - ``FFA_SPM_ID_GET`` 789 - ``FFA_SECONDARY_EP_REGISTER`` 790 - ``FFA_MEM_PERM_GET`` 791 - ``FFA_MEM_PERM_SET`` 792 - ``FFA_MSG_SEND2`` 793 - ``FFA_RX_ACQUIRE`` 794 795FFA_VERSION 796~~~~~~~~~~~ 797 798``FFA_VERSION`` requires a *requested_version* parameter from the caller. 799The returned value depends on the caller: 800 801- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 802 specified in the SPMC manifest. 803- SP: the SPMC returns its own implemented version. 804- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 805 806FFA_FEATURES 807~~~~~~~~~~~~ 808 809FF-A features supported by the SPMC may be discovered by secure partitions at 810boot (that is prior to NWd is booted) or run-time. 811 812The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 813FFA_SUCCESS from the SPMD. 814 815The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 816the response relayed back to the NWd. 817 818FFA_RXTX_MAP/FFA_RXTX_UNMAP 819~~~~~~~~~~~~~~~~~~~~~~~~~~~ 820 821When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 822receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 823regime as secure buffers in the MMU descriptors. 824 825When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 826SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 827descriptors. The provided addresses may be owned by a VM in the normal world, 828which is expected to receive messages from the secure world. The SPMC will in 829this case allocate internal state structures to facilitate RX buffer access 830synchronization (through FFA_RX_ACQUIRE interface), and to permit SPs to send 831messages. 832 833The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 834caller, either it being the Hypervisor or OS kernel, as well as a secure 835partition. 836 837FFA_PARTITION_INFO_GET 838~~~~~~~~~~~~~~~~~~~~~~ 839 840Partition info get call can originate: 841 842- from SP to SPMC 843- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 844 845FFA_ID_GET 846~~~~~~~~~~ 847 848The FF-A id space is split into a non-secure space and secure space: 849 850- FF-A ID with bit 15 clear relates to VMs. 851- FF-A ID with bit 15 set related to SPs. 852- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 853 and SPMC. 854 855The SPMD returns: 856 857- The default zero value on invocation from the Hypervisor. 858- The ``spmc_id`` value specified in the SPMC manifest on invocation from 859 the SPMC (see `SPMC manifest`_) 860 861This convention helps the SPMC to determine the origin and destination worlds in 862an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 863transactions in its world switch routine. It must not be permitted for a VM to 864use a secure FF-A ID as origin world by spoofing: 865 866- A VM-to-SP direct request/response shall set the origin world to be non-secure 867 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 868 set). 869- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 870 for both origin and destination IDs. 871 872An incoming direct message request arriving at SPMD from NWd is forwarded to 873SPMC without a specific check. The SPMC is resumed through eret and "knows" the 874message is coming from normal world in this specific code path. Thus the origin 875endpoint ID must be checked by SPMC for being a normal world ID. 876 877An SP sending a direct message request must have bit 15 set in its origin 878endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 879 880The SPMC shall reject the direct message if the claimed world in origin endpoint 881ID is not consistent: 882 883- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 884 world ID", 885- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 886 887 888FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 889~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 890 891This is a mandatory interface for secure partitions consisting in direct request 892and responses with the following rules: 893 894- An SP can send a direct request to another SP. 895- An SP can receive a direct request from another SP. 896- An SP can send a direct response to another SP. 897- An SP cannot send a direct request to an Hypervisor or OS kernel. 898- An Hypervisor or OS kernel can send a direct request to an SP. 899- An SP can send a direct response to an Hypervisor or OS kernel. 900 901FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 902~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 903 904The secure partitions notifications bitmap are statically allocated by the SPMC. 905Hence, this interface is not to be issued by secure partitions. 906 907At initialization, the SPMC is not aware of VMs/partitions deployed in the 908normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 909to be prepared to handle notifications for the provided VM ID. 910 911FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 912~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 913 914Pair of interfaces to manage permissions to signal notifications. Prior to 915handling notifications, an FF-A endpoint must allow a given sender to signal a 916bitmap of notifications. 917 918If the receiver doesn't have notification support enabled in its FF-A manifest, 919it won't be able to bind notifications, hence forbidding it to receive any 920notifications. 921 922FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 923~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 924 925FFA_NOTIFICATION_GET retrieves all pending global notifications and 926per-vCPU notifications targeted to the current vCPU. 927 928Hafnium maintains a global count of pending notifications which gets incremented 929and decremented when handling FFA_NOTIFICATION_SET and FFA_NOTIFICATION_GET 930respectively. A delayed SRI is triggered if the counter is non-zero when the 931SPMC returns to normal world. 932 933FFA_NOTIFICATION_INFO_GET 934~~~~~~~~~~~~~~~~~~~~~~~~~ 935 936Hafnium maintains a global count of pending notifications whose information 937has been retrieved by this interface. The count is incremented and decremented 938when handling FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET respectively. 939It also tracks notifications whose information has been retrieved individually, 940such that it avoids duplicating returned information for subsequent calls to 941FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 942reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 943 944FFA_SPM_ID_GET 945~~~~~~~~~~~~~~ 946 947Returns the FF-A ID allocated to an SPM component which can be one of SPMD 948or SPMC. 949 950At initialization, the SPMC queries the SPMD for the SPMC ID, using the 951FFA_ID_GET interface, and records it. The SPMC can also query the SPMD ID using 952the FFA_SPM_ID_GET interface at the secure physical FF-A instance. 953 954Secure partitions call this interface at the virtual FF-A instance, to which 955the SPMC returns the priorly retrieved SPMC ID. 956 957The Hypervisor or OS kernel can issue the FFA_SPM_ID_GET call handled by the 958SPMD, which returns the SPMC ID. 959 960FFA_SECONDARY_EP_REGISTER 961~~~~~~~~~~~~~~~~~~~~~~~~~ 962 963When the SPMC boots, all secure partitions are initialized on their primary 964Execution Context. 965 966The FFA_SECONDARY_EP_REGISTER interface is to be used by a secure partition 967from its first execution context, to provide the entry point address for 968secondary execution contexts. 969 970A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 971the NWd or by invocation of FFA_RUN. 972 973FFA_RX_ACQUIRE/FFA_RX_RELEASE 974~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 975 976The RX buffers can be used to pass information to an FF-A endpoint in the 977following scenarios: 978 979 - When it was targetted by a FFA_MSG_SEND2 invokation from another endpoint. 980 - Return the result of calling ``FFA_PARTITION_INFO_GET``. 981 - In a memory share operation, as part of the ``FFA_MEM_RETRIEVE_RESP``, 982 with the memory descriptor of the shared memory. 983 984If a normal world VM is expected to exchange messages with secure world, 985its RX/TX buffer addresses are forwarded to the SPMC via FFA_RXTX_MAP ABI, 986and are from this moment owned by the SPMC. 987The hypervisor must call the FFA_RX_ACQUIRE interface before attempting 988to use the RX buffer, in any of the aforementioned scenarios. A successful 989call to FFA_RX_ACQUIRE transfers ownership of RX buffer to hypervisor, such 990that it can be safely used. 991 992The FFA_RX_RELEASE interface is used after the FF-A endpoint is done with 993processing the data received in its RX buffer. If the RX buffer has been 994acquired by the hypervisor, the FFA_RX_RELEASE call must be forwarded to 995the SPMC to reestablish SPMC's RX ownership. 996 997An attempt from an SP to send a message to a normal world VM whose RX buffer 998was acquired by the hypervisor fails with error code FFA_BUSY, to preserve 999the RX buffer integrity. 1000The operation could then be conducted after FFA_RX_RELEASE. 1001 1002FFA_MSG_SEND2 1003~~~~~~~~~~~~~ 1004 1005Hafnium copies a message from the sender TX buffer into receiver's RX buffer. 1006For messages from SPs to VMs, operation is only possible if the SPMC owns 1007the receiver's RX buffer. 1008 1009Both receiver and sender need to enable support for indirect messaging, 1010in their respective partition manifest. The discovery of support 1011of such feature can be done via FFA_PARTITION_INFO_GET. 1012 1013On a successful message send, Hafnium pends an RX buffer full framework 1014notification for the receiver, to inform it about a message in the RX buffer. 1015 1016The handling of framework notifications is similar to that of 1017global notifications. Binding of these is not necessary, as these are 1018reserved to be used by the hypervisor or SPMC. 1019 1020SPMC-SPMD direct requests/responses 1021----------------------------------- 1022 1023Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 1024Using those IDs in source/destination fields of a direct request/response 1025permits SPMD to SPMC communication and either way. 1026 1027- SPMC to SPMD direct request/response uses SMC conduit. 1028- SPMD to SPMC direct request/response uses ERET conduit. 1029 1030This is used in particular to convey power management messages. 1031 1032PE MMU configuration 1033-------------------- 1034 1035With secure virtualization enabled (``HCR_EL2.VM = 1``) and for S-EL1 1036partitions, two IPA spaces (secure and non-secure) are output from the 1037secure EL1&0 Stage-1 translation. 1038The EL1&0 Stage-2 translation hardware is fed by: 1039 1040- A secure IPA when the SP EL1&0 Stage-1 MMU is disabled. 1041- One of secure or non-secure IPA when the secure EL1&0 Stage-1 MMU is enabled. 1042 1043``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 1044NS/S IPA translations. The following controls are set up: 1045``VSTCR_EL2.SW = 0`` , ``VSTCR_EL2.SA = 0``, ``VTCR_EL2.NSW = 0``, 1046``VTCR_EL2.NSA = 1``: 1047 1048- Stage-2 translations for the NS IPA space access the NS PA space. 1049- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 1050 1051Secure and non-secure IPA regions (rooted to by ``VTTBR_EL2`` and ``VSTTBR_EL2``) 1052use the same set of Stage-2 page tables within a SP. 1053 1054The ``VTCR_EL2/VSTCR_EL2/VTTBR_EL2/VSTTBR_EL2`` virtual address space 1055configuration is made part of a vCPU context. 1056 1057For S-EL0 partitions with VHE enabled, a single secure EL2&0 Stage-1 translation 1058regime is used for both Hafnium and the partition. 1059 1060Schedule modes and SP Call chains 1061--------------------------------- 1062 1063An SP execution context is said to be in SPMC scheduled mode if CPU cycles are 1064allocated to it by SPMC. Correspondingly, an SP execution context is said to be 1065in Normal world scheduled mode if CPU cycles are allocated by the normal world. 1066 1067A call chain represents all SPs in a sequence of invocations of a direct message 1068request. When execution on a PE is in the secure state, only a single call chain 1069that runs in the Normal World scheduled mode can exist. FF-A v1.1 spec allows 1070any number of call chains to run in the SPMC scheduled mode but the Hafnium 1071SPMC restricts the number of call chains in SPMC scheduled mode to only one for 1072keeping the implementation simple. 1073 1074Partition runtime models 1075------------------------ 1076 1077The runtime model of an endpoint describes the transitions permitted for an 1078execution context between various states. These are the four partition runtime 1079models supported (refer to `[1]`_ section 7): 1080 1081 - RTM_FFA_RUN: runtime model presented to an execution context that is 1082 allocated CPU cycles through FFA_RUN interface. 1083 - RTM_FFA_DIR_REQ: runtime model presented to an execution context that is 1084 allocated CPU cycles through FFA_MSG_SEND_DIRECT_REQ interface. 1085 - RTM_SEC_INTERRUPT: runtime model presented to an execution context that is 1086 allocated CPU cycles by SPMC to handle a secure interrupt. 1087 - RTM_SP_INIT: runtime model presented to an execution context that is 1088 allocated CPU cycles by SPMC to initialize its state. 1089 1090If an endpoint execution context attempts to make an invalid transition or a 1091valid transition that could lead to a loop in the call chain, SPMC denies the 1092transition with the help of above runtime models. 1093 1094Interrupt management 1095-------------------- 1096 1097GIC ownership 1098~~~~~~~~~~~~~ 1099 1100The SPMC owns the GIC configuration. Secure and non-secure interrupts are 1101trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 1102IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 1103virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 1104 1105Abbreviations: 1106 1107 - NS-Int: A non-secure physical interrupt. It requires a switch to the normal 1108 world to be handled if it triggers while execution is in secure world. 1109 - Other S-Int: A secure physical interrupt targeted to an SP different from 1110 the one that is currently running. 1111 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1112 running. 1113 1114Non-secure interrupt handling 1115~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1116 1117This section documents the actions supported in SPMC in response to a non-secure 1118interrupt as per the guidance provided by FF-A v1.1 EAC0 specification. 1119An SP specifies one of the following actions in its partition manifest: 1120 1121 - Non-secure interrupt is signaled. 1122 - Non-secure interrupt is signaled after a managed exit. 1123 - Non-secure interrupt is queued. 1124 1125An SP execution context in a call chain could specify a less permissive action 1126than subsequent SP execution contexts in the same call chain. The less 1127permissive action takes precedence over the more permissive actions specified 1128by the subsequent execution contexts. Please refer to FF-A v1.1 EAC0 section 11298.3.1 for further explanation. 1130 1131Secure interrupt handling 1132~~~~~~~~~~~~~~~~~~~~~~~~~ 1133 1134This section documents the support implemented for secure interrupt handling in 1135SPMC as per the guidance provided by FF-A v1.1 EAC0 specification. 1136The following assumptions are made about the system configuration: 1137 1138 - In the current implementation, S-EL1 SPs are expected to use the para 1139 virtualized ABIs for interrupt management rather than accessing the virtual 1140 GIC interface. 1141 - Unless explicitly stated otherwise, this support is applicable only for 1142 S-EL1 SPs managed by SPMC. 1143 - Secure interrupts are configured as G1S or G0 interrupts. 1144 - All physical interrupts are routed to SPMC when running a secure partition 1145 execution context. 1146 - All endpoints with multiple execution contexts have their contexts pinned 1147 to corresponding CPUs. Hence, a secure virtual interrupt cannot be signaled 1148 to a target vCPU that is currently running or blocked on a different 1149 physical CPU. 1150 1151A physical secure interrupt could trigger while CPU is executing in normal world 1152or secure world. 1153The action of SPMC for a secure interrupt depends on: the state of the target 1154execution context of the SP that is responsible for handling the interrupt; 1155whether the interrupt triggered while execution was in normal world or secure 1156world. 1157 1158Secure interrupt signaling mechanisms 1159~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1160 1161Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 1162context that it has a pending virtual interrupt and to further run the SP 1163execution context, such that it can handle the virtual interrupt. SPMC uses 1164either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 1165to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 1166the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 1167running in S-EL2. 1168 1169+-----------+---------+---------------+---------------------------------------+ 1170| SP State | Conduit | Interface and | Description | 1171| | | parameters | | 1172+-----------+---------+---------------+---------------------------------------+ 1173| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 1174| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 1175| | | | resumes execution context of SP | 1176| | | | through ERET. | 1177+-----------+---------+---------------+---------------------------------------+ 1178| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 1179| | vIRQ | | is pending. It pends vIRQ signal and | 1180| | | | resumes execution context of SP | 1181| | | | through ERET. | 1182+-----------+---------+---------------+---------------------------------------+ 1183| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 1184| | | | not resume execution context of SP. | 1185+-----------+---------+---------------+---------------------------------------+ 1186| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 1187| | vIRQ | | execution context of SP through ERET. | 1188+-----------+---------+---------------+---------------------------------------+ 1189 1190Secure interrupt completion mechanisms 1191~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1192 1193A SP signals secure interrupt handling completion to the SPMC through the 1194following mechanisms: 1195 1196 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 1197 - ``FFA_RUN`` ABI if its was in BLOCKED state. 1198 1199This is a remnant of SPMC implementation based on the FF-A v1.0 specification. 1200In the current implementation, S-EL1 SPs use the para-virtualized HVC interface 1201implemented by SPMC to perform priority drop and interrupt deactivation (SPMC 1202configures EOImode = 0, i.e. priority drop and deactivation are done together). 1203The SPMC performs checks to deny the state transition upon invocation of 1204either FFA_MSG_WAIT or FFA_RUN interface if the SP didn't perform the 1205deactivation of the secure virtual interrupt. 1206 1207If the current SP execution context was preempted by a secure interrupt to be 1208handled by execution context of target SP, SPMC resumes current SP after signal 1209completion by target SP execution context. 1210 1211Actions for a secure interrupt triggered while execution is in normal world 1212~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1213 1214+-------------------+----------+-----------------------------------------------+ 1215| State of target | Action | Description | 1216| execution context | | | 1217+-------------------+----------+-----------------------------------------------+ 1218| WAITING | Signaled | This starts a new call chain in SPMC scheduled| 1219| | | mode. | 1220+-------------------+----------+-----------------------------------------------+ 1221| PREEMPTED | Queued | The target execution must have been preempted | 1222| | | by a non-secure interrupt. SPMC queues the | 1223| | | secure virtual interrupt now. It is signaled | 1224| | | when the target execution context next enters | 1225| | | the RUNNING state. | 1226+-------------------+----------+-----------------------------------------------+ 1227| BLOCKED, RUNNING | NA | The target execution context is blocked or | 1228| | | running on a different CPU. This is not | 1229| | | supported by current SPMC implementation and | 1230| | | execution hits panic. | 1231+-------------------+----------+-----------------------------------------------+ 1232 1233If normal world execution was preempted by a secure interrupt, SPMC uses 1234FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 1235and further returns execution to normal world. 1236 1237The following figure describes interrupt handling flow when a secure interrupt 1238triggers while execution is in normal world: 1239 1240.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1241 1242A brief description of the events: 1243 1244 - 1) Secure interrupt triggers while normal world is running. 1245 - 2) FIQ gets trapped to EL3. 1246 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1247 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1248 vIRQ). 1249 - 5) Assuming SP1 vCPU is in WAITING state, SPMC signals virtual interrupt 1250 using FFA_INTERRUPT with interrupt id as an argument and resumes the SP1 1251 vCPU using ERET in SPMC scheduled mode. 1252 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual 1253 interrupt is not masked i.e., PSTATE.I = 0 1254 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized 1255 HVC call. SPMC clears the pending virtual interrupt state management 1256 and returns the pending virtual interrupt id. 1257 - 8) SP1 services the virtual interrupt and invokes the paravirtualized 1258 de-activation HVC call. SPMC de-activates the physical interrupt, 1259 clears the fields tracking the secure interrupt and resumes SP1 vCPU. 1260 - 9) SP1 performs secure interrupt completion through FFA_MSG_WAIT ABI. 1261 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1262 - 11) EL3 resumes normal world execution. 1263 1264Actions for a secure interrupt triggered while execution is in secure world 1265~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1266 1267+-------------------+----------+------------------------------------------------+ 1268| State of target | Action | Description | 1269| execution context | | | 1270+-------------------+----------+------------------------------------------------+ 1271| WAITING | Signaled | This starts a new call chain in SPMC scheduled | 1272| | | mode. | 1273+-------------------+----------+------------------------------------------------+ 1274| PREEMPTED by Self | Signaled | The target execution context reenters the | 1275| S-Int | | RUNNING state to handle the secure virtual | 1276| | | interrupt. | 1277+-------------------+----------+------------------------------------------------+ 1278| PREEMPTED by | Queued | SPMC queues the secure virtual interrupt now. | 1279| NS-Int | | It is signaled when the target execution | 1280| | | context next enters the RUNNING state. | 1281+-------------------+----------+------------------------------------------------+ 1282| BLOCKED | Signaled | Both preempted and target execution contexts | 1283| | | must have been part of the Normal world | 1284| | | scheduled call chain. Refer scenario 1 of | 1285| | | Table 8.4 in the FF-A v1.1 EAC0 spec. | 1286+-------------------+----------+------------------------------------------------+ 1287| RUNNING | NA | The target execution context is running on a | 1288| | | different CPU. This scenario is not supported | 1289| | | by current SPMC implementation and execution | 1290| | | hits panic. | 1291+-------------------+----------+------------------------------------------------+ 1292 1293The following figure describes interrupt handling flow when a secure interrupt 1294triggers while execution is in secure world. We assume OS kernel sends a direct 1295request message to SP1. Further, SP1 sends a direct request message to SP2. SP1 1296enters BLOCKED state and SPMC resumes SP2. 1297 1298.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1299 1300A brief description of the events: 1301 1302 - 1) Secure interrupt triggers while SP2 is running. 1303 - 2) SP2 gets preempted and execution traps to SPMC as IRQ. 1304 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1305 this secure interrupt. In this scenario, it is SP1. 1306 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1307 SPMC further resumes SP1 through ERET conduit. Note that SP1 remains in 1308 Normal world schedule mode. 1309 - 6) Execution traps to vIRQ handler in SP1 provided that the virtual 1310 interrupt is not masked i.e., PSTATE.I = 0 1311 - 7) SP1 queries for the pending virtual interrupt id using a paravirtualized 1312 HVC call. SPMC clears the pending virtual interrupt state management 1313 and returns the pending virtual interrupt id. 1314 - 8) SP1 services the virtual interrupt and invokes the paravirtualized 1315 de-activation HVC call. SPMC de-activates the physical interrupt and 1316 clears the fields tracking the secure interrupt and resumes SP1 vCPU. 1317 - 9) Since SP1 direct request completed with FFA_INTERRUPT, it resumes the 1318 direct request to SP2 by invoking FFA_RUN. 1319 - 9) SPMC resumes the pre-empted vCPU of SP2. 1320 1321EL3 interrupt handling 1322~~~~~~~~~~~~~~~~~~~~~~ 1323 1324In GICv3 based systems, EL3 interrupts are configured as Group0 secure 1325interrupts. Execution traps to SPMC when a Group0 interrupt triggers while an 1326SP is running. Further, SPMC running at S-EL2 uses FFA_EL3_INTR_HANDLE ABI to 1327request EL3 platform firmware to handle a pending Group0 interrupt. 1328Similarly, SPMD registers a handler with interrupt management framework to 1329delegate handling of Group0 interrupt to the platform if the interrupt triggers 1330in normal world. 1331 1332 - Platform hook 1333 1334 - plat_spmd_handle_group0_interrupt 1335 1336 SPMD provides platform hook to handle Group0 secure interrupts. In the 1337 current design, SPMD expects the platform not to delegate handling to the 1338 NWd (such as through SDEI) while processing Group0 interrupts. 1339 1340Power management 1341---------------- 1342 1343In platforms with or without secure virtualization: 1344 1345- The NWd owns the platform PM policy. 1346- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1347- The EL3 PSCI library is in charge of the PM coordination and control 1348 (eventually writing to platform registers). 1349- While coordinating PM events, the PSCI library calls backs into the Secure 1350 Payload Dispatcher for events the latter has statically registered to. 1351 1352When using the SPMD as a Secure Payload Dispatcher: 1353 1354- A power management event is relayed through the SPD hook to the SPMC. 1355- In the current implementation only cpu on (svc_on_finish) and cpu off 1356 (svc_off) hooks are registered. 1357- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1358 The SPMC is entered through its secondary physical core entry point. 1359- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The PM event is 1360 signaled to the SPMC through a power management framework message. 1361 It consists in a SPMD-to-SPMC direct request/response (`SPMC-SPMD direct 1362 requests/responses`_) conveying the event details and SPMC response. 1363 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1364 updates its internal state to reflect the physical core is being turned off. 1365 In the current implementation no SP is resumed as a consequence. This behavior 1366 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1367 userspace. 1368 1369Arm architecture extensions for security hardening 1370================================================== 1371 1372Hafnium supports the following architecture extensions for security hardening: 1373 1374- Pointer authentication (FEAT_PAuth): the extension permits detection of forged 1375 pointers used by ROP type of attacks through the signing of the pointer 1376 value. Hafnium is built with the compiler branch protection option to permit 1377 generation of a pointer authentication code for return addresses (pointer 1378 authentication for instructions). The APIA key is used while Hafnium runs. 1379 A random key is generated at boot time and restored upon entry into Hafnium 1380 at run-time. APIA and other keys (APIB, APDA, APDB, APGA) are saved/restored 1381 in vCPU contexts permitting to enable pointer authentication in VMs/SPs. 1382- Branch Target Identification (FEAT_BTI): the extension permits detection of 1383 unexpected indirect branches used by JOP type of attacks. Hafnium is built 1384 with the compiler branch protection option, inserting land pads at function 1385 prologues that are reached by indirect branch instructions (BR/BLR). 1386 Hafnium code pages are marked as guarded in the EL2 Stage-1 MMU descriptors 1387 such that an indirect branch must always target a landpad. A fault is 1388 triggered otherwise. VMs/SPs can (independently) mark their code pages as 1389 guarded in the EL1&0 Stage-1 translation regime. 1390- Memory Tagging Extension (FEAT_MTE): the option permits detection of out of 1391 bound memory array accesses or re-use of an already freed memory region. 1392 Hafnium enables the compiler option permitting to leverage MTE stack tagging 1393 applied to core stacks. Core stacks are marked as normal tagged memory in the 1394 EL2 Stage-1 translation regime. A synchronous data abort is generated upon tag 1395 check failure on load/stores. A random seed is generated at boot time and 1396 restored upon entry into Hafnium. MTE system registers are saved/restored in 1397 vCPU contexts permitting MTE usage from VMs/SPs. 1398 1399SMMUv3 support in Hafnium 1400========================= 1401 1402An SMMU is analogous to an MMU in a CPU. It performs address translations for 1403Direct Memory Access (DMA) requests from system I/O devices. 1404The responsibilities of an SMMU include: 1405 1406- Translation: Incoming DMA requests are translated from bus address space to 1407 system physical address space using translation tables compliant to 1408 Armv8/Armv7 VMSA descriptor format. 1409- Protection: An I/O device can be prohibited from read, write access to a 1410 memory region or allowed. 1411- Isolation: Traffic from each individial device can be independently managed. 1412 The devices are differentiated from each other using unique translation 1413 tables. 1414 1415The following diagram illustrates a typical SMMU IP integrated in a SoC with 1416several I/O devices along with Interconnect and Memory system. 1417 1418.. image:: ../resources/diagrams/MMU-600.png 1419 1420SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1421support for SMMUv3 driver in both normal and secure world. A brief introduction 1422of SMMUv3 functionality and the corresponding software support in Hafnium is 1423provided here. 1424 1425SMMUv3 features 1426--------------- 1427 1428- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1429 translation support. It can either bypass or abort incoming translations as 1430 well. 1431- Traffic (memory transactions) from each upstream I/O peripheral device, 1432 referred to as Stream, can be independently managed using a combination of 1433 several memory based configuration structures. This allows the SMMUv3 to 1434 support a large number of streams with each stream assigned to a unique 1435 translation context. 1436- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1437 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1438 are supported by SMMUv3. 1439- SMMUv3 offers non-secure stream support with secure stream support being 1440 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1441 instance for secure and non-secure stream support. 1442- It also supports sub-streams to differentiate traffic from a virtualized 1443 peripheral associated with a VM/SP. 1444- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1445 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1446 for providing Secure Stage2 translation support to upstream peripheral 1447 devices. 1448 1449SMMUv3 Programming Interfaces 1450----------------------------- 1451 1452SMMUv3 has three software interfaces that are used by the Hafnium driver to 1453configure the behaviour of SMMUv3 and manage the streams. 1454 1455- Memory based data strutures that provide unique translation context for 1456 each stream. 1457- Memory based circular buffers for command queue and event queue. 1458- A large number of SMMU configuration registers that are memory mapped during 1459 boot time by Hafnium driver. Except a few registers, all configuration 1460 registers have independent secure and non-secure versions to configure the 1461 behaviour of SMMUv3 for translation of secure and non-secure streams 1462 respectively. 1463 1464Peripheral device manifest 1465-------------------------- 1466 1467Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1468These devices are dependent on PE endpoint to initiate and receive memory 1469management transactions on their behalf. The acccess to the MMIO regions of 1470any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 1471uses the same stage 2 translations for the device as those used by partition 1472manager on behalf of the PE endpoint. This ensures that the peripheral device 1473has the same visibility of the physical address space as the endpoint. The 1474device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 1475) must specify these additional properties for each peripheral device in the 1476system : 1477 1478- smmu-id: This field helps to identify the SMMU instance that this device is 1479 upstream of. 1480- stream-ids: List of stream IDs assigned to this device. 1481 1482.. code:: shell 1483 1484 smmuv3-testengine { 1485 base-address = <0x00000000 0x2bfe0000>; 1486 pages-count = <32>; 1487 attributes = <0x3>; 1488 smmu-id = <0>; 1489 stream-ids = <0x0 0x1>; 1490 interrupts = <0x2 0x3>, <0x4 0x5>; 1491 exclusive-access; 1492 }; 1493 1494SMMUv3 driver limitations 1495------------------------- 1496 1497The primary design goal for the Hafnium SMMU driver is to support secure 1498streams. 1499 1500- Currently, the driver only supports Stage2 translations. No support for 1501 Stage1 or nested translations. 1502- Supports only AArch64 translation format. 1503- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1504 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1505- No support for independent peripheral devices. 1506 1507S-EL0 Partition support 1508======================= 1509The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1510FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1511with ARMv8.4 and FEAT_SEL2). 1512 1513S-EL0 partitions are useful for simple partitions that don't require full 1514Trusted OS functionality. It is also useful to reduce jitter and cycle 1515stealing from normal world since they are more lightweight than VMs. 1516 1517S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1518the SPMC. They are differentiated primarily by the 'exception-level' property 1519and the 'execution-ctx-count' property in the SP manifest. They are host apps 1520under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1521call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1522can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1523for memory regions. 1524 1525S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1526capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1527a S-EL0 partition to accept a direct message from secure world and normal world, 1528and generate direct responses to them. 1529All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not supported. 1530 1531Memory sharing, indirect messaging, and notifications functionality with S-EL0 1532partitions is supported. 1533 1534Interrupt handling is not supported with S-EL0 partitions and is work in 1535progress. 1536 1537References 1538========== 1539 1540.. _[1]: 1541 1542[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1543 1544.. _[2]: 1545 1546[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 1547 1548.. _[3]: 1549 1550[3] `Trusted Boot Board Requirements 1551Client <https://developer.arm.com/documentation/den0006/d/>`__ 1552 1553.. _[4]: 1554 1555[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1556 1557.. _[5]: 1558 1559[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1560 1561.. _[6]: 1562 1563[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1564 1565.. _[7]: 1566 1567[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1568 1569.. _[8]: 1570 1571[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/ 1572 1573.. _[9]: 1574 1575[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1576 1577-------------- 1578 1579*Copyright (c) 2020-2023, Arm Limited and Contributors. All rights reserved.* 1580
