1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+--------------------------------------+ 10| CoT | Chain of Trust | 11+--------+--------------------------------------+ 12| DMA | Direct Memory Access | 13+--------+--------------------------------------+ 14| DTB | Device Tree Blob | 15+--------+--------------------------------------+ 16| DTS | Device Tree Source | 17+--------+--------------------------------------+ 18| EC | Execution Context | 19+--------+--------------------------------------+ 20| FIP | Firmware Image Package | 21+--------+--------------------------------------+ 22| FF-A | Firmware Framework for Arm A-profile | 23+--------+--------------------------------------+ 24| IPA | Intermediate Physical Address | 25+--------+--------------------------------------+ 26| NWd | Normal World | 27+--------+--------------------------------------+ 28| ODM | Original Design Manufacturer | 29+--------+--------------------------------------+ 30| OEM | Original Equipment Manufacturer | 31+--------+--------------------------------------+ 32| PA | Physical Address | 33+--------+--------------------------------------+ 34| PE | Processing Element | 35+--------+--------------------------------------+ 36| PM | Power Management | 37+--------+--------------------------------------+ 38| PVM | Primary VM | 39+--------+--------------------------------------+ 40| SMMU | System Memory Management Unit | 41+--------+--------------------------------------+ 42| SP | Secure Partition | 43+--------+--------------------------------------+ 44| SPD | Secure Payload Dispatcher | 45+--------+--------------------------------------+ 46| SPM | Secure Partition Manager | 47+--------+--------------------------------------+ 48| SPMC | SPM Core | 49+--------+--------------------------------------+ 50| SPMD | SPM Dispatcher | 51+--------+--------------------------------------+ 52| SiP | Silicon Provider | 53+--------+--------------------------------------+ 54| SWd | Secure World | 55+--------+--------------------------------------+ 56| TLV | Tag-Length-Value | 57+--------+--------------------------------------+ 58| TOS | Trusted Operating System | 59+--------+--------------------------------------+ 60| VM | Virtual Machine | 61+--------+--------------------------------------+ 62 63Foreword 64======== 65 66Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 67 68- SPM based on the FF-A specification `[1]`_. 69- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 70 71Both implementations differ in their architectures and only one can be selected 72at build time. 73 74This document: 75 76- describes the FF-A implementation where the Secure Partition Manager 77 resides at EL3 and S-EL2 (or EL3 and S-EL1). 78- is not an architecture specification and it might provide assumptions 79 on sections mandated as implementation-defined in the specification. 80- covers the implications to TF-A used as a bootloader, and Hafnium 81 used as a reference code base for an S-EL2 secure firmware on 82 platforms implementing the FEAT_SEL2 (formerly Armv8.4 Secure EL2) 83 architecture extension. 84 85Terminology 86----------- 87 88- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 89 (or partitions) in the normal world. 90- The term SPMC refers to the S-EL2 component managing secure partitions in 91 the secure world when the FEAT_SEL2 architecture extension is implemented. 92- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 93 partition and implementing the FF-A ABI on platforms not implementing the 94 FEAT_SEL2 architecture extension. 95- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 96- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 97 98Support for legacy platforms 99---------------------------- 100 101In the implementation, the SPM is split into SPMD and SPMC components. 102The SPMD is located at EL3 and mainly relays FF-A messages from 103NWd (Hypervisor or OS kernel) to SPMC located either at S-EL1 or S-EL2. 104 105Hence TF-A supports both cases where the SPMC is located either at: 106 107- S-EL1 supporting platforms not implementing the FEAT_SEL2 architecture 108 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 109- or S-EL2 supporting platforms implementing the FEAT_SEL2 architecture 110 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 111 112The same TF-A SPMD component is used to support both configurations. 113The SPMC exception level is a build time choice. 114 115Sample reference stack 116====================== 117 118The following diagram illustrates a possible configuration when the 119FEAT_SEL2 architecture extension is implemented, showing the SPMD 120and SPMC, one or multiple secure partitions, with an optional 121Hypervisor: 122 123.. image:: ../resources/diagrams/ff-a-spm-sel2.png 124 125TF-A build options 126================== 127 128This section explains the TF-A build options involved in building with 129support for an FF-A based SPM where the SPMD is located at EL3 and the 130SPMC located at S-EL1, S-EL2 or EL3: 131 132- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 133 protocol from NWd to SWd back and forth. It is not possible to 134 enable another Secure Payload Dispatcher when this option is chosen. 135- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 136 level to being at S-EL2. It defaults to enabled (value 1) when 137 SPD=spmd is chosen. 138- **SPMC_AT_EL3**: this option adjusts the SPMC exception level to being 139 at EL3. 140- If neither **SPMD_SPM_AT_SEL2** or **SPMC_AT_EL3** are enabled the SPMC 141 exception level is set to S-EL1. 142- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 143 restoring) the EL2 system register context before entering (resp. 144 after leaving) the SPMC. It is mandatorily enabled when 145 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 146 and exhaustive list of registers is visible at `[4]`_. 147- **SP_LAYOUT_FILE**: this option specifies a text description file 148 providing paths to SP binary images and manifests in DTS format 149 (see `Describing secure partitions`_). It 150 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 151 secure partitions are to be loaded on behalf of the SPMC. 152 153+---------------+----------------------+------------------+-------------+ 154| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | SPMC_AT_EL3 | 155+---------------+----------------------+------------------+-------------+ 156| SPMC at S-EL1 | 0 | 0 | 0 | 157+---------------+----------------------+------------------+-------------+ 158| SPMC at S-EL2 | 1 | 1 (default when | 0 | 159| | | SPD=spmd) | | 160+---------------+----------------------+------------------+-------------+ 161| SPMC at EL3 | 0 | 0 | 1 | 162+---------------+----------------------+------------------+-------------+ 163 164Other combinations of such build options either break the build or are not 165supported. 166 167Notes: 168 169- Only Arm's FVP platform is supported to use with the TF-A reference software 170 stack. 171- The reference software stack uses FEAT_PAuth (formerly Armv8.3-PAuth) and 172 FEAT_BTI (formerly Armv8.5-BTI) architecture extensions by default at EL3 173 and S-EL2. 174- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 175 barely saving/restoring EL2 registers from an Arm arch perspective. As such 176 it is decoupled from the ``SPD=spmd`` option. 177- BL32 option is re-purposed to specify the SPMC image. It can specify either 178 the Hafnium binary path (built for the secure world) or the path to a TEE 179 binary implementing FF-A interfaces. 180- BL33 option can specify the TFTF binary or a normal world loader 181 such as U-Boot or the UEFI framework. 182 183Sample TF-A build command line when SPMC is located at S-EL1 184(e.g. when the FEAT_EL2 architecture extension is not implemented): 185 186.. code:: shell 187 188 make \ 189 CROSS_COMPILE=aarch64-none-elf- \ 190 SPD=spmd \ 191 SPMD_SPM_AT_SEL2=0 \ 192 BL32=<path-to-tee-binary> \ 193 BL33=<path-to-bl33-binary> \ 194 PLAT=fvp \ 195 all fip 196 197Sample TF-A build command line for a FEAT_SEL2 enabled system where the SPMC is 198located at S-EL2: 199 200.. code:: shell 201 202 make \ 203 CROSS_COMPILE=aarch64-none-elf- \ 204 PLAT=fvp \ 205 SPD=spmd \ 206 CTX_INCLUDE_EL2_REGS=1 \ 207 ARM_ARCH_MINOR=5 \ 208 BRANCH_PROTECTION=1 \ 209 CTX_INCLUDE_PAUTH_REGS=1 \ 210 BL32=<path-to-hafnium-binary> \ 211 BL33=<path-to-bl33-binary> \ 212 SP_LAYOUT_FILE=sp_layout.json \ 213 all fip 214 215Same as above with enabling secure boot in addition: 216 217.. code:: shell 218 219 make \ 220 CROSS_COMPILE=aarch64-none-elf- \ 221 PLAT=fvp \ 222 SPD=spmd \ 223 CTX_INCLUDE_EL2_REGS=1 \ 224 ARM_ARCH_MINOR=5 \ 225 BRANCH_PROTECTION=1 \ 226 CTX_INCLUDE_PAUTH_REGS=1 \ 227 BL32=<path-to-hafnium-binary> \ 228 BL33=<path-to-bl33-binary> \ 229 SP_LAYOUT_FILE=sp_layout.json \ 230 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 231 TRUSTED_BOARD_BOOT=1 \ 232 COT=dualroot \ 233 ARM_ROTPK_LOCATION=devel_rsa \ 234 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 235 GENERATE_COT=1 \ 236 all fip 237 238Sample TF-A build command line when SPMC is located at EL3: 239 240.. code:: shell 241 242 make \ 243 CROSS_COMPILE=aarch64-none-elf- \ 244 SPD=spmd \ 245 SPMD_SPM_AT_SEL2=0 \ 246 SPMC_AT_EL3=1 \ 247 BL32=<path-to-tee-binary> \ 248 BL33=<path-to-bl33-binary> \ 249 PLAT=fvp \ 250 all fip 251 252FVP model invocation 253==================== 254 255The FVP command line needs the following options to exercise the S-EL2 SPMC: 256 257+---------------------------------------------------+------------------------------------+ 258| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 259| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 260+---------------------------------------------------+------------------------------------+ 261| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 262| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 263| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 264| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 265| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 266| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 267| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 268| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 269+---------------------------------------------------+------------------------------------+ 270| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 271| - cluster1.has_branch_target_exception=1 | | 272+---------------------------------------------------+------------------------------------+ 273| - cluster0.restriction_on_speculative_execution=2 | Required by the EL2 context | 274| - cluster1.restriction_on_speculative_execution=2 | save/restore routine. | 275+---------------------------------------------------+------------------------------------+ 276 277Sample FVP command line invocation: 278 279.. code:: shell 280 281 <path-to-fvp-model>/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 282 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 283 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 284 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 285 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 286 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 287 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 \ 288 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 \ 289 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 \ 290 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 \ 291 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 \ 292 -C cluster0.has_branch_target_exception=1 \ 293 -C cluster1.has_branch_target_exception=1 \ 294 -C cluster0.restriction_on_speculative_execution=2 \ 295 -C cluster1.restriction_on_speculative_execution=2 296 297Boot process 298============ 299 300Loading Hafnium and secure partitions in the secure world 301--------------------------------------------------------- 302 303TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 304 305SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 306Thus they are supplied as distinct signed entities within the FIP flash 307image. The FIP image itself is not signed hence this provides the ability 308to upgrade SPs in the field. 309 310Booting through TF-A 311-------------------- 312 313SP manifests 314~~~~~~~~~~~~ 315 316An SP manifest describes SP attributes as defined in `[1]`_ 317(partition manifest at virtual FF-A instance) in DTS format. It is 318represented as a single file associated with the SP. A sample is 319provided by `[5]`_. A binding document is provided by `[6]`_. 320 321Secure Partition packages 322~~~~~~~~~~~~~~~~~~~~~~~~~ 323 324Secure partitions are bundled as independent package files consisting 325of: 326 327- a header 328- a DTB 329- an image payload 330 331The header starts with a magic value and offset values to SP DTB and 332image payload. Each SP package is loaded independently by BL2 loader 333and verified for authenticity and integrity. 334 335The SP package identified by its UUID (matching FF-A uuid property) is 336inserted as a single entry into the FIP at end of the TF-A build flow 337as shown: 338 339.. code:: shell 340 341 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 342 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 343 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 344 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 345 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 346 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 347 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 348 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 349 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 350 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 351 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 352 353.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 354 355Describing secure partitions 356~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 357 358A json-formatted description file is passed to the build flow specifying paths 359to the SP binary image and associated DTS partition manifest file. The latter 360is processed by the dtc compiler to generate a DTB fed into the SP package. 361Optionally, the partition's json description can contain offsets for both 362the image and partition manifest within the SP package. Both offsets need to be 3634KB aligned, because it is the translation granule supported by Hafnium SPMC. 364These fields can be leveraged to support SPs with S1 translation granules that 365differ from 4KB, and to configure the regions allocated within the SP package, 366as well as to comply with the requirements for the implementation of the boot 367information protocol (see `Passing boot data to the SP`_ for more details). In 368case the offsets are absent in their json node, they default to 0x1000 and 3690x4000 for the manifest offset and image offset respectively. 370This file also specifies the SP owner (as an optional field) identifying the 371signing domain in case of dual root CoT. 372The SP owner can either be the silicon or the platform provider. The 373corresponding "owner" field value can either take the value of "SiP" or "Plat". 374In absence of "owner" field, it defaults to "SiP" owner. 375The UUID of the partition can be specified as a field in the description file or 376if it does not exist there the UUID is extracted from the DTS partition 377manifest. 378 379.. code:: shell 380 381 { 382 "tee1" : { 383 "image": "tee1.bin", 384 "pm": "tee1.dts", 385 "owner": "SiP", 386 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f" 387 }, 388 389 "tee2" : { 390 "image": "tee2.bin", 391 "pm": "tee2.dts", 392 "owner": "Plat" 393 }, 394 395 "tee3" : { 396 "image": { 397 "file": "tee3.bin", 398 "offset":"0x2000" 399 }, 400 "pm": { 401 "file": "tee3.dts", 402 "offset":"0x6000" 403 }, 404 "owner": "Plat" 405 }, 406 } 407 408SPMC manifest 409~~~~~~~~~~~~~ 410 411This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 412time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 413two different cases: 414 415- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 416 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 417 mode. 418- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 419 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 420 S-EL0. 421 422.. code:: shell 423 424 attribute { 425 spmc_id = <0x8000>; 426 maj_ver = <0x1>; 427 min_ver = <0x0>; 428 exec_state = <0x0>; 429 load_address = <0x0 0x6000000>; 430 entrypoint = <0x0 0x6000000>; 431 binary_size = <0x60000>; 432 }; 433 434- *spmc_id* defines the endpoint ID value that SPMC can query through 435 ``FFA_ID_GET``. 436- *maj_ver/min_ver*. SPMD checks provided version versus its internal 437 version and aborts if not matching. 438- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 439 Notice Hafnium used as a SPMC only supports AArch64. 440- *load_address* and *binary_size* are mostly used to verify secondary 441 entry points fit into the loaded binary image. 442- *entrypoint* defines the cold boot primary core entry point used by 443 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 444 445Other nodes in the manifest are consumed by Hafnium in the secure world. 446A sample can be found at [7]: 447 448- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 449 indicates a FF-A compliant SP. The *load_address* field specifies the load 450 address at which TF-A loaded the SP package. 451- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 452 Note the primary core is declared first, then secondary core are declared 453 in reverse order. 454- The *memory* node provides platform information on the ranges of memory 455 available to the SPMC. 456 457SPMC boot 458~~~~~~~~~ 459 460The SPMC is loaded by BL2 as the BL32 image. 461 462The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 463 464BL2 passes the SPMC manifest address to BL31 through a register. 465 466At boot time, the SPMD in BL31 runs from the primary core, initializes the core 467contexts and launches the SPMC (BL32) passing the following information through 468registers: 469 470- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 471- X1 holds the ``HW_CONFIG`` physical address. 472- X4 holds the currently running core linear id. 473 474Loading of SPs 475~~~~~~~~~~~~~~ 476 477At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 478below: 479 480.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 481 482Note this boot flow is an implementation sample on Arm's FVP platform. 483Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 484different implementation. 485 486Secure boot 487~~~~~~~~~~~ 488 489The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 490SPMC manifest, secure partitions and verifies them for authenticity and integrity. 491Refer to TBBR specification `[3]`_. 492 493The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 494the use of two root keys namely S-ROTPK and NS-ROTPK: 495 496- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 497- BL33 may be signed by the OEM using NS-ROTPK. 498- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 499 500Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 501 502Hafnium in the secure world 503=========================== 504 505General considerations 506---------------------- 507 508Build platform for the secure world 509~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 510 511In the Hafnium reference implementation specific code parts are only relevant to 512the secure world. Such portions are isolated in architecture specific files 513and/or enclosed by a ``SECURE_WORLD`` macro. 514 515Secure partitions CPU scheduling 516~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 517 518The FF-A v1.0 specification `[1]`_ provides two ways to relinquinsh CPU time to 519secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 520 521- the FFA_MSG_SEND_DIRECT_REQ interface. 522- the FFA_RUN interface. 523 524Platform topology 525~~~~~~~~~~~~~~~~~ 526 527The *execution-ctx-count* SP manifest field can take the value of one or the 528total number of PEs. The FF-A v1.0 specification `[1]`_ recommends the 529following SP types: 530 531- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 532 implement the same number of ECs as the number of PEs in the platform. 533- Migratable UP SPs: a single execution context can run and be migrated on any 534 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 535 receive a direct message request originating from any physical core targeting 536 the single execution context. 537 538Parsing SP partition manifests 539------------------------------ 540 541Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 542Note the current implementation may not implement all optional fields. 543 544The SP manifest may contain memory and device regions nodes. In case of 545an S-EL2 SPMC: 546 547- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 548 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 549 specify RX/TX buffer regions in which case it is not necessary for an SP 550 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 551- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 552 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 553 additional resources (e.g. interrupts). 554 555For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 556provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 557regime. 558 559Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 560same set of page tables. It is still open whether two sets of page tables shall 561be provided per SP. The memory region node as defined in the specification 562provides a memory security attribute hinting to map either to the secure or 563non-secure EL1&0 Stage-2 table if it exists. 564 565Passing boot data to the SP 566--------------------------- 567 568In `[1]`_ , the section "Boot information protocol" defines a method for passing 569data to the SPs at boot time. It specifies the format for the boot information 570descriptor and boot information header structures, which describe the data to be 571exchanged between SPMC and SP. 572The specification also defines the types of data that can be passed. 573The aggregate of both the boot info structures and the data itself is designated 574the boot information blob, and is passed to a Partition as a contiguous memory 575region. 576 577Currently, the SPM implementation supports the FDT type which is used to pass the 578partition's DTB manifest. 579 580The region for the boot information blob is allocated through the SP package. 581 582.. image:: ../resources/diagrams/partition-package.png 583 584To adjust the space allocated for the boot information blob, the json description 585of the SP (see section `Describing secure partitions`_) shall be updated to contain 586the manifest offset. If no offset is provided the manifest offset defaults to 0x1000, 587which is the page size in the Hafnium SPMC. 588 589The configuration of the boot protocol is done in the SPs manifest. As defined by 590the specification, the manifest field 'gp-register-num' configures the GP register 591which shall be used to pass the address to the partitions boot information blob when 592booting the partition. 593In addition, the Hafnium SPMC implementation requires the boot information arguments 594to be listed in a designated DT node: 595 596.. code:: shell 597 598 boot-info { 599 compatible = "arm,ffa-manifest-boot-info"; 600 ffa_manifest; 601 }; 602 603The whole secure partition package image (see `Secure Partition packages`_) is 604mapped to the SP secure EL1&0 Stage-2 translation regime. As such, the SP can 605retrieve the address for the boot information blob in the designated GP register, 606process the boot information header and descriptors, access its own manifest 607DTB blob and extract its partition manifest properties. 608 609SP Boot order 610------------- 611 612SP manifests provide an optional boot order attribute meant to resolve 613dependencies such as an SP providing a service required to properly boot 614another SP. SPMC boots the SPs in accordance to the boot order attribute, 615lowest to the highest value. If the boot order attribute is absent from the FF-A 616manifest, the SP is treated as if it had the highest boot order value 617(i.e. lowest booting priority). 618 619It is possible for an SP to call into another SP through a direct request 620provided the latter SP has already been booted. 621 622Boot phases 623----------- 624 625Primary core boot-up 626~~~~~~~~~~~~~~~~~~~~ 627 628Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 629core. The SPMC performs its platform initializations and registers the SPMC 630secondary physical core entry point physical address by the use of the 631`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 632at secure physical FF-A instance). 633 634The SPMC then creates secure partitions based on SP packages and manifests. Each 635secure partition is launched in sequence (`SP Boot order`_) on their "primary" 636execution context. If the primary boot physical core linear id is N, an MP SP is 637started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 638UP SP, it is started using its unique EC0 on PE[N]. 639 640The SP primary EC (or the EC used when the partition is booted as described 641above): 642 643- Performs the overall SP boot time initialization, and in case of a MP SP, 644 prepares the SP environment for other execution contexts. 645- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 646 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 647 entry point for other execution contexts. 648- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 649 ``FFA_ERROR`` in case of failure. 650 651Secondary cores boot-up 652~~~~~~~~~~~~~~~~~~~~~~~ 653 654Once the system is started and NWd brought up, a secondary physical core is 655woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 656calls into the SPMD on the newly woken up physical core. Then the SPMC is 657entered at the secondary physical core entry point. 658 659In the current implementation, the first SP is resumed on the coresponding EC 660(the virtual CPU which matches the physical core). The implication is that the 661first SP must be a MP SP. 662 663In a linux based system, once secure and normal worlds are booted but prior to 664a NWd FF-A driver has been loaded: 665 666- The first SP has initialized all its ECs in response to primary core boot up 667 (at system initialization) and secondary core boot up (as a result of linux 668 invoking PSCI_CPU_ON for all secondary cores). 669- Other SPs have their first execution context initialized as a result of secure 670 world initialization on the primary boot core. Other ECs for those SPs have to 671 be run first through ffa_run to complete their initialization (which results 672 in the EC completing with FFA_MSG_WAIT). 673 674Refer to `Power management`_ for further details. 675 676Notifications 677------------- 678 679The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 680communication mechanism with non-blocking semantics. It allows for one FF-A 681endpoint to signal another for service provision, without hindering its current 682progress. 683 684Hafnium currently supports 64 notifications. The IDs of each notification define 685a position in a 64-bit bitmap. 686 687The signaling of notifications can interchangeably happen between NWd and SWd 688FF-A endpoints. 689 690The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 691VMs, and from VMs to SPs. An hypervisor component would only manage 692notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 693deployed in NWd, the Hypervisor or OS kernel must invoke the interface 694FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 695endpoint in the NWd that supports it. 696 697A sender can signal notifications once the receiver has provided it with 698permissions. Permissions are provided by invoking the interface 699FFA_NOTIFICATION_BIND. 700 701Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 702they are considered to be in a pending sate. The receiver can retrieve its 703pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 704are considered to be handled. 705 706Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 707that is in charge of donating CPU cycles for notifications handling. The 708FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 709which FF-A endpoints have pending notifications. The receiver scheduler is 710called and informed by the FF-A driver, and it should allocate CPU cycles to the 711receiver. 712 713There are two types of notifications supported: 714- Global, which are targeted to a FF-A endpoint and can be handled within any of 715its execution contexts, as determined by the scheduler of the system. 716- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 717a specific execution context, as determined by the sender. 718 719The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 720permissions to the sender. 721 722Notification signaling resorts to two interrupts: 723- Schedule Receiver Interrupt: Non-secure physical interrupt to be handled by 724the FF-A 'transport' driver within the receiver scheduler. At initialization 725the SPMC (as suggested by the spec) configures a secure SGI, as non-secure, and 726triggers it when there are pending notifications, and the respective receivers 727need CPU cycles to handle them. 728- Notifications Pending Interrupt: Virtual Interrupt to be handled by the 729receiver of the notification. Set when there are pending notifications. For 730per-vCPU the NPI is pended at the handling of FFA_NOTIFICATION_SET interface. 731 732The notifications receipt support is enabled in the partition FF-A manifest. 733 734The subsequent section provides more details about the each one of the 735FF-A interfaces for notifications support. 736 737Mandatory interfaces 738-------------------- 739 740The following interfaces are exposed to SPs: 741 742- ``FFA_VERSION`` 743- ``FFA_FEATURES`` 744- ``FFA_RX_RELEASE`` 745- ``FFA_RXTX_MAP`` 746- ``FFA_RXTX_UNMAP`` 747- ``FFA_PARTITION_INFO_GET`` 748- ``FFA_ID_GET`` 749- ``FFA_MSG_WAIT`` 750- ``FFA_MSG_SEND_DIRECT_REQ`` 751- ``FFA_MSG_SEND_DIRECT_RESP`` 752- ``FFA_MEM_DONATE`` 753- ``FFA_MEM_LEND`` 754- ``FFA_MEM_SHARE`` 755- ``FFA_MEM_RETRIEVE_REQ`` 756- ``FFA_MEM_RETRIEVE_RESP`` 757- ``FFA_MEM_RELINQUISH`` 758- ``FFA_MEM_RECLAIM`` 759 760As part of the support of FF-A v1.1, the following interfaces were added: 761 762 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 763 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 764 - ``FFA_NOTIFICATION_BIND`` 765 - ``FFA_NOTIFICATION_UNBIND`` 766 - ``FFA_NOTIFICATION_SET`` 767 - ``FFA_NOTIFICATION_GET`` 768 - ``FFA_NOTIFICATION_INFO_GET`` 769 - ``FFA_SPM_ID_GET`` 770 - ``FFA_SECONDARY_EP_REGISTER`` 771 772FFA_VERSION 773~~~~~~~~~~~ 774 775``FFA_VERSION`` requires a *requested_version* parameter from the caller. 776The returned value depends on the caller: 777 778- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 779 specified in the SPMC manifest. 780- SP: the SPMC returns its own implemented version. 781- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 782 783FFA_FEATURES 784~~~~~~~~~~~~ 785 786FF-A features supported by the SPMC may be discovered by secure partitions at 787boot (that is prior to NWd is booted) or run-time. 788 789The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 790FFA_SUCCESS from the SPMD. 791 792The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 793the response relayed back to the NWd. 794 795FFA_RXTX_MAP/FFA_RXTX_UNMAP 796~~~~~~~~~~~~~~~~~~~~~~~~~~~ 797 798When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 799receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 800regime as secure buffers in the MMU descriptors. 801 802When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 803SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 804descriptors. 805 806The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 807caller, either it being the Hypervisor or OS kernel, as well as a secure 808partition. 809 810FFA_PARTITION_INFO_GET 811~~~~~~~~~~~~~~~~~~~~~~ 812 813Partition info get call can originate: 814 815- from SP to SPMC 816- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 817 818FFA_ID_GET 819~~~~~~~~~~ 820 821The FF-A id space is split into a non-secure space and secure space: 822 823- FF-A ID with bit 15 clear relates to VMs. 824- FF-A ID with bit 15 set related to SPs. 825- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 826 and SPMC. 827 828The SPMD returns: 829 830- The default zero value on invocation from the Hypervisor. 831- The ``spmc_id`` value specified in the SPMC manifest on invocation from 832 the SPMC (see `SPMC manifest`_) 833 834This convention helps the SPMC to determine the origin and destination worlds in 835an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 836transactions in its world switch routine. It must not be permitted for a VM to 837use a secure FF-A ID as origin world by spoofing: 838 839- A VM-to-SP direct request/response shall set the origin world to be non-secure 840 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 841 set). 842- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 843 for both origin and destination IDs. 844 845An incoming direct message request arriving at SPMD from NWd is forwarded to 846SPMC without a specific check. The SPMC is resumed through eret and "knows" the 847message is coming from normal world in this specific code path. Thus the origin 848endpoint ID must be checked by SPMC for being a normal world ID. 849 850An SP sending a direct message request must have bit 15 set in its origin 851endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 852 853The SPMC shall reject the direct message if the claimed world in origin endpoint 854ID is not consistent: 855 856- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 857 world ID", 858- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 859 860 861FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 862~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 863 864This is a mandatory interface for secure partitions consisting in direct request 865and responses with the following rules: 866 867- An SP can send a direct request to another SP. 868- An SP can receive a direct request from another SP. 869- An SP can send a direct response to another SP. 870- An SP cannot send a direct request to an Hypervisor or OS kernel. 871- An Hypervisor or OS kernel can send a direct request to an SP. 872- An SP can send a direct response to an Hypervisor or OS kernel. 873 874FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 875~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 876 877The secure partitions notifications bitmap are statically allocated by the SPMC. 878Hence, this interface is not to be issued by secure partitions. 879 880At initialization, the SPMC is not aware of VMs/partitions deployed in the 881normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 882to be prepared to handle notifications for the provided VM ID. 883 884FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 885~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 886 887Pair of interfaces to manage permissions to signal notifications. Prior to 888handling notifications, an FF-A endpoint must allow a given sender to signal a 889bitmap of notifications. 890 891If the receiver doesn't have notification support enabled in its FF-A manifest, 892it won't be able to bind notifications, hence forbidding it to receive any 893notifications. 894 895FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 896~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 897 898If the notifications set are per-vCPU, the NPI interrupt is set as pending 899for a given receiver partition. 900 901The FFA_NOTIFICATION_GET will retrieve all pending global notifications and all 902pending per-vCPU notifications targeted to the current vCPU. 903 904Hafnium keeps the global counting of the pending notifications, which is 905incremented and decremented at the handling of FFA_NOTIFICATION_SET and 906FFA_NOTIFICATION_GET, respectively. If the counter reaches zero, prior to SPMC 907triggering the SRI, it won't be triggered. 908 909FFA_NOTIFICATION_INFO_GET 910~~~~~~~~~~~~~~~~~~~~~~~~~ 911 912Hafnium keeps the global counting of pending notifications whose info has been 913retrieved by this interface. The counting is incremented and decremented at the 914handling of FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET, respectively. 915It also tracks the notifications whose info has been retrieved individually, 916such that it avoids duplicating returned information for subsequent calls to 917FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 918reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 919 920FFA_SPM_ID_GET 921~~~~~~~~~~~~~~ 922 923Returns the FF-A ID allocated to the SPM component (which includes SPMC + SPMD). 924At initialization, the SPMC queries the SPMD for the SPM ID, using this 925same interface, and saves it. 926 927The call emitted at NS and secure physical FF-A instances returns the SPM ID 928specified in the SPMC manifest. 929 930Secure partitions call this interface at the virtual instance, to which the SPMC 931shall return the priorly retrieved SPM ID. 932 933The Hypervisor or OS kernel can issue an FFA_SPM_ID_GET call handled by the 934SPMD, which returns the SPM ID. 935 936FFA_SECONDARY_EP_REGISTER 937~~~~~~~~~~~~~~~~~~~~~~~~~ 938 939When the SPMC boots, all secure partitions are initialized on their primary 940Execution Context. 941 942The interface FFA_SECONDARY_EP_REGISTER is to be used by a secure partitions 943from its first execution context, to provide the entry point address for 944secondary execution contexts. 945 946A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 947the NWd or by invocation of FFA_RUN. 948 949SPMC-SPMD direct requests/responses 950----------------------------------- 951 952Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 953Using those IDs in source/destination fields of a direct request/response 954permits SPMD to SPMC communication and either way. 955 956- SPMC to SPMD direct request/response uses SMC conduit. 957- SPMD to SPMC direct request/response uses ERET conduit. 958 959PE MMU configuration 960-------------------- 961 962With secure virtualization enabled, two IPA spaces are output from the secure 963EL1&0 Stage-1 translation (secure and non-secure). The EL1&0 Stage-2 translation 964hardware is fed by: 965 966- A single secure IPA space when the SP EL1&0 Stage-1 MMU is disabled. 967- Two IPA spaces (secure and non-secure) when the SP EL1&0 Stage-1 MMU is 968 enabled. 969 970``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 971NS/S IPA translations. 972``VSTCR_EL2.SW`` = 0, ``VSTCR_EL2.SA`` = 0,``VTCR_EL2.NSW`` = 0, ``VTCR_EL2.NSA`` = 1: 973 974- Stage-2 translations for the NS IPA space access the NS PA space. 975- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 976 977Secure and non-secure IPA regions use the same set of Stage-2 page tables within 978a SP. 979 980Interrupt management 981-------------------- 982 983GIC ownership 984~~~~~~~~~~~~~ 985 986The SPMC owns the GIC configuration. Secure and non-secure interrupts are 987trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 988IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 989virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 990 991Non-secure interrupt handling 992~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 993 994The following illustrate the scenarios of non secure physical interrupts trapped 995by the SPMC: 996 997- The SP handles a managed exit operation: 998 999.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-managed-exit.png 1000 1001- The SP is pre-empted without managed exit: 1002 1003.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-sp-preemption.png 1004 1005Secure interrupt handling 1006------------------------- 1007 1008This section documents the support implemented for secure interrupt handling in 1009SPMC as per the guidance provided by FF-A v1.1 Beta0 specification. 1010The following assumptions are made about the system configuration: 1011 1012 - In the current implementation, S-EL1 SPs are expected to use the para 1013 virtualized ABIs for interrupt management rather than accessing virtual GIC 1014 interface. 1015 - Unless explicitly stated otherwise, this support is applicable only for 1016 S-EL1 SPs managed by SPMC. 1017 - Secure interrupts are configured as G1S or G0 interrupts. 1018 - All physical interrupts are routed to SPMC when running a secure partition 1019 execution context. 1020 1021A physical secure interrupt could preempt normal world execution. Moreover, when 1022the execution is in secure world, it is highly likely that the target of a 1023secure interrupt is not the currently running execution context of an SP. It 1024could be targeted to another FF-A component. Consequently, secure interrupt 1025management depends on the state of the target execution context of the SP that 1026is responsible for handling the interrupt. Hence, the spec provides guidance on 1027how to signal start and completion of secure interrupt handling as discussed in 1028further sections. 1029 1030Secure interrupt signaling mechanisms 1031~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1032 1033Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 1034context that it has a pending virtual interrupt and to further run the SP 1035execution context, such that it can handle the virtual interrupt. SPMC uses 1036either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 1037to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 1038the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 1039running in S-EL2. 1040 1041+-----------+---------+---------------+---------------------------------------+ 1042| SP State | Conduit | Interface and | Description | 1043| | | parameters | | 1044+-----------+---------+---------------+---------------------------------------+ 1045| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 1046| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 1047| | | | resumes execution context of SP | 1048| | | | through ERET. | 1049+-----------+---------+---------------+---------------------------------------+ 1050| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 1051| | vIRQ | | is pending. It pends vIRQ signal and | 1052| | | | resumes execution context of SP | 1053| | | | through ERET. | 1054+-----------+---------+---------------+---------------------------------------+ 1055| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 1056| | | | not resume execution context of SP. | 1057+-----------+---------+---------------+---------------------------------------+ 1058| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 1059| | vIRQ | | execution context of SP through ERET. | 1060+-----------+---------+---------------+---------------------------------------+ 1061 1062Secure interrupt completion mechanisms 1063~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1064 1065A SP signals secure interrupt handling completion to the SPMC through the 1066following mechanisms: 1067 1068 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 1069 - ``FFA_RUN`` ABI if its was in BLOCKED state. 1070 1071In the current implementation, S-EL1 SPs use para-virtualized HVC interface 1072implemented by SPMC to perform priority drop and interrupt deactivation (we 1073assume EOImode = 0, i.e. priority drop and deactivation are done together). 1074 1075If normal world execution was preempted by secure interrupt, SPMC uses 1076FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 1077and further return execution to normal world. If the current SP execution 1078context was preempted by a secure interrupt to be handled by execution context 1079of target SP, SPMC resumes current SP after signal completion by target SP 1080execution context. 1081 1082An action is broadly a set of steps taken by the SPMC in response to a physical 1083interrupt. In order to simplify the design, the current version of secure 1084interrupt management support in SPMC (Hafnium) does not fully implement the 1085Scheduling models and Partition runtime models. However, the current 1086implementation loosely maps to the following actions that are legally allowed 1087by the specification. Please refer to the Table 8.4 in the spec for further 1088description of actions. The action specified for a type of interrupt when the 1089SP is in the message processing running state cannot be less permissive than the 1090action specified for the same type of interrupt when the SP is in the interrupt 1091handling running state. 1092 1093+--------------------+--------------------+------------+-------------+ 1094| Runtime Model | NS-Int | Self S-Int | Other S-Int | 1095+--------------------+--------------------+------------+-------------+ 1096| Message Processing | Signalable with ME | Signalable | Signalable | 1097+--------------------+--------------------+------------+-------------+ 1098| Interrupt Handling | Queued | Queued | Queued | 1099+--------------------+--------------------+------------+-------------+ 1100 1101Abbreviations: 1102 1103 - NS-Int: A Non-secure physical interrupt. It requires a switch to the Normal 1104 world to be handled. 1105 - Other S-Int: A secure physical interrupt targeted to an SP different from 1106 the one that is currently running. 1107 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1108 running. 1109 1110The following figure describes interrupt handling flow when secure interrupt 1111triggers while in normal world: 1112 1113.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1114 1115A brief description of the events: 1116 1117 - 1) Secure interrupt triggers while normal world is running. 1118 - 2) FIQ gets trapped to EL3. 1119 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1120 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1121 vIRQ). 1122 - 5) Since SP1 vCPU is in WAITING state, SPMC signals using FFA_INTERRUPT with 1123 interrupt id as argument and resume it using ERET. 1124 - 6) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1125 masked i.e., PSTATE.I = 0 1126 - 7) SP1 services the interrupt and invokes the de-activation HVC call. 1127 - 8) SPMC does internal state management and further de-activates the physical 1128 interrupt and resumes SP vCPU. 1129 - 9) SP performs secure interrupt completion through FFA_MSG_WAIT ABI. 1130 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1131 - 11) EL3 resumes normal world execution. 1132 1133The following figure describes interrupt handling flow when secure interrupt 1134triggers while in secure world: 1135 1136.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1137 1138A brief description of the events: 1139 1140 - 1) Secure interrupt triggers while SP2 is running and SP1 is blocked. 1141 - 2) Gets trapped to SPMC as IRQ. 1142 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1143 this secure interrupt. In this scenario, it is SP1. 1144 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1145 SPMC further resumes SP1 through ERET conduit. 1146 - 5) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1147 masked i.e., PSTATE.I = 0 1148 - 6) SP1 services the secure interrupt and invokes the de-activation HVC call. 1149 - 7) SPMC does internal state management, de-activates the physical interrupt 1150 and resumes SP1 vCPU. 1151 - 8) Assuming SP1 is in BLOCKED state, SP1 performs secure interrupt completion 1152 through FFA_RUN ABI. 1153 - 9) SPMC resumes the pre-empted vCPU of SP2. 1154 1155 1156Power management 1157---------------- 1158 1159In platforms with or without secure virtualization: 1160 1161- The NWd owns the platform PM policy. 1162- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1163- The EL3 PSCI library is in charge of the PM coordination and control 1164 (eventually writing to platform registers). 1165- While coordinating PM events, the PSCI library calls backs into the Secure 1166 Payload Dispatcher for events the latter has statically registered to. 1167 1168When using the SPMD as a Secure Payload Dispatcher: 1169 1170- A power management event is relayed through the SPD hook to the SPMC. 1171- In the current implementation only cpu on (svc_on_finish) and cpu off 1172 (svc_off) hooks are registered. 1173- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1174 The SPMC is entered through its secondary physical core entry point. 1175- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The method by which 1176 the PM event is conveyed to the SPMC is implementation-defined in context of 1177 FF-A v1.0 (`SPMC-SPMD direct requests/responses`_). It consists in a SPMD-to-SPMC 1178 direct request/response conveying the PM event details and SPMC response. 1179 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1180 updates its internal state to reflect the physical core is being turned off. 1181 In the current implementation no SP is resumed as a consequence. This behavior 1182 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1183 userspace. 1184 1185SMMUv3 support in Hafnium 1186========================= 1187 1188An SMMU is analogous to an MMU in a CPU. It performs address translations for 1189Direct Memory Access (DMA) requests from system I/O devices. 1190The responsibilities of an SMMU include: 1191 1192- Translation: Incoming DMA requests are translated from bus address space to 1193 system physical address space using translation tables compliant to 1194 Armv8/Armv7 VMSA descriptor format. 1195- Protection: An I/O device can be prohibited from read, write access to a 1196 memory region or allowed. 1197- Isolation: Traffic from each individial device can be independently managed. 1198 The devices are differentiated from each other using unique translation 1199 tables. 1200 1201The following diagram illustrates a typical SMMU IP integrated in a SoC with 1202several I/O devices along with Interconnect and Memory system. 1203 1204.. image:: ../resources/diagrams/MMU-600.png 1205 1206SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1207support for SMMUv3 driver in both normal and secure world. A brief introduction 1208of SMMUv3 functionality and the corresponding software support in Hafnium is 1209provided here. 1210 1211SMMUv3 features 1212--------------- 1213 1214- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1215 translation support. It can either bypass or abort incoming translations as 1216 well. 1217- Traffic (memory transactions) from each upstream I/O peripheral device, 1218 referred to as Stream, can be independently managed using a combination of 1219 several memory based configuration structures. This allows the SMMUv3 to 1220 support a large number of streams with each stream assigned to a unique 1221 translation context. 1222- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1223 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1224 are supported by SMMUv3. 1225- SMMUv3 offers non-secure stream support with secure stream support being 1226 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1227 instance for secure and non-secure stream support. 1228- It also supports sub-streams to differentiate traffic from a virtualized 1229 peripheral associated with a VM/SP. 1230- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1231 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1232 for providing Secure Stage2 translation support to upstream peripheral 1233 devices. 1234 1235SMMUv3 Programming Interfaces 1236----------------------------- 1237 1238SMMUv3 has three software interfaces that are used by the Hafnium driver to 1239configure the behaviour of SMMUv3 and manage the streams. 1240 1241- Memory based data strutures that provide unique translation context for 1242 each stream. 1243- Memory based circular buffers for command queue and event queue. 1244- A large number of SMMU configuration registers that are memory mapped during 1245 boot time by Hafnium driver. Except a few registers, all configuration 1246 registers have independent secure and non-secure versions to configure the 1247 behaviour of SMMUv3 for translation of secure and non-secure streams 1248 respectively. 1249 1250Peripheral device manifest 1251-------------------------- 1252 1253Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1254These devices are dependent on PE endpoint to initiate and receive memory 1255management transactions on their behalf. The acccess to the MMIO regions of 1256any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 1257uses the same stage 2 translations for the device as those used by partition 1258manager on behalf of the PE endpoint. This ensures that the peripheral device 1259has the same visibility of the physical address space as the endpoint. The 1260device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 1261) must specify these additional properties for each peripheral device in the 1262system : 1263 1264- smmu-id: This field helps to identify the SMMU instance that this device is 1265 upstream of. 1266- stream-ids: List of stream IDs assigned to this device. 1267 1268.. code:: shell 1269 1270 smmuv3-testengine { 1271 base-address = <0x00000000 0x2bfe0000>; 1272 pages-count = <32>; 1273 attributes = <0x3>; 1274 smmu-id = <0>; 1275 stream-ids = <0x0 0x1>; 1276 interrupts = <0x2 0x3>, <0x4 0x5>; 1277 exclusive-access; 1278 }; 1279 1280SMMUv3 driver limitations 1281------------------------- 1282 1283The primary design goal for the Hafnium SMMU driver is to support secure 1284streams. 1285 1286- Currently, the driver only supports Stage2 translations. No support for 1287 Stage1 or nested translations. 1288- Supports only AArch64 translation format. 1289- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1290 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1291- No support for independent peripheral devices. 1292 1293S-EL0 Partition support 1294========================= 1295The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1296FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1297with ARMv8.4 and FEAT_SEL2). 1298 1299S-EL0 partitions are useful for simple partitions that don't require full 1300Trusted OS functionality. It is also useful to reduce jitter and cycle 1301stealing from normal world since they are more lightweight than VMs. 1302 1303S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1304the SPMC. They are differentiated primarily by the 'exception-level' property 1305and the 'execution-ctx-count' property in the SP manifest. They are host apps 1306under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1307call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1308can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1309for memory regions. 1310 1311S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1312capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1313a S-EL0 partition to accept a direct message from secure world and normal world, 1314and generate direct responses to them. 1315 1316Memory sharing between and with S-EL0 partitions is supported. 1317Indirect messaging, Interrupt handling and Notifications are not supported with 1318S-EL0 partitions and is work in progress, planned for future releases. 1319All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not 1320supported. 1321 1322 1323References 1324========== 1325 1326.. _[1]: 1327 1328[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1329 1330.. _[2]: 1331 1332[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 1333 1334.. _[3]: 1335 1336[3] `Trusted Boot Board Requirements 1337Client <https://developer.arm.com/documentation/den0006/d/>`__ 1338 1339.. _[4]: 1340 1341[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1342 1343.. _[5]: 1344 1345[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1346 1347.. _[6]: 1348 1349[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1350 1351.. _[7]: 1352 1353[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1354 1355.. _[8]: 1356 1357[8] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/CFQFGU6H2D5GZYMUYGTGUSXIU3OYZP6U/ 1358 1359.. _[9]: 1360 1361[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1362 1363-------------- 1364 1365*Copyright (c) 2020-2022, Arm Limited and Contributors. All rights reserved.* 1366
