1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+--------------------------------------+ 10| CoT | Chain of Trust | 11+--------+--------------------------------------+ 12| DMA | Direct Memory Access | 13+--------+--------------------------------------+ 14| DTB | Device Tree Blob | 15+--------+--------------------------------------+ 16| DTS | Device Tree Source | 17+--------+--------------------------------------+ 18| EC | Execution Context | 19+--------+--------------------------------------+ 20| FIP | Firmware Image Package | 21+--------+--------------------------------------+ 22| FF-A | Firmware Framework for Arm A-profile | 23+--------+--------------------------------------+ 24| IPA | Intermediate Physical Address | 25+--------+--------------------------------------+ 26| NWd | Normal World | 27+--------+--------------------------------------+ 28| ODM | Original Design Manufacturer | 29+--------+--------------------------------------+ 30| OEM | Original Equipment Manufacturer | 31+--------+--------------------------------------+ 32| PA | Physical Address | 33+--------+--------------------------------------+ 34| PE | Processing Element | 35+--------+--------------------------------------+ 36| PM | Power Management | 37+--------+--------------------------------------+ 38| PVM | Primary VM | 39+--------+--------------------------------------+ 40| SMMU | System Memory Management Unit | 41+--------+--------------------------------------+ 42| SP | Secure Partition | 43+--------+--------------------------------------+ 44| SPD | Secure Payload Dispatcher | 45+--------+--------------------------------------+ 46| SPM | Secure Partition Manager | 47+--------+--------------------------------------+ 48| SPMC | SPM Core | 49+--------+--------------------------------------+ 50| SPMD | SPM Dispatcher | 51+--------+--------------------------------------+ 52| SiP | Silicon Provider | 53+--------+--------------------------------------+ 54| SWd | Secure World | 55+--------+--------------------------------------+ 56| TLV | Tag-Length-Value | 57+--------+--------------------------------------+ 58| TOS | Trusted Operating System | 59+--------+--------------------------------------+ 60| VM | Virtual Machine | 61+--------+--------------------------------------+ 62 63Foreword 64======== 65 66Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 67 68- SPM based on the FF-A specification `[1]`_. 69- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 70 71Both implementations differ in their architectures and only one can be selected 72at build time. 73 74This document: 75 76- describes the FF-A implementation where the Secure Partition Manager 77 resides at EL3 and S-EL2 (or EL3 and S-EL1). 78- is not an architecture specification and it might provide assumptions 79 on sections mandated as implementation-defined in the specification. 80- covers the implications to TF-A used as a bootloader, and Hafnium 81 used as a reference code base for an S-EL2 secure firmware on 82 platforms implementing the FEAT_SEL2 (formerly Armv8.4 Secure EL2) 83 architecture extension. 84 85Terminology 86----------- 87 88- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 89 (or partitions) in the normal world. 90- The term SPMC refers to the S-EL2 component managing secure partitions in 91 the secure world when the FEAT_SEL2 architecture extension is implemented. 92- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 93 partition and implementing the FF-A ABI on platforms not implementing the 94 FEAT_SEL2 architecture extension. 95- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 96- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 97 98Support for legacy platforms 99---------------------------- 100 101In the implementation, the SPM is split into SPMD and SPMC components. 102The SPMD is located at EL3 and mainly relays FF-A messages from 103NWd (Hypervisor or OS kernel) to SPMC located either at S-EL1 or S-EL2. 104 105Hence TF-A supports both cases where the SPMC is located either at: 106 107- S-EL1 supporting platforms not implementing the FEAT_SEL2 architecture 108 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 109- or S-EL2 supporting platforms implementing the FEAT_SEL2 architecture 110 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 111 112The same TF-A SPMD component is used to support both configurations. 113The SPMC exception level is a build time choice. 114 115Sample reference stack 116====================== 117 118The following diagram illustrates a possible configuration when the 119FEAT_SEL2 architecture extension is implemented, showing the SPMD 120and SPMC, one or multiple secure partitions, with an optional 121Hypervisor: 122 123.. image:: ../resources/diagrams/ff-a-spm-sel2.png 124 125TF-A build options 126================== 127 128This section explains the TF-A build options involved in building with 129support for an FF-A based SPM where the SPMD is located at EL3 and the 130SPMC located at S-EL1, S-EL2 or EL3: 131 132- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 133 protocol from NWd to SWd back and forth. It is not possible to 134 enable another Secure Payload Dispatcher when this option is chosen. 135- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 136 level to being at S-EL2. It defaults to enabled (value 1) when 137 SPD=spmd is chosen. 138- **SPMC_AT_EL3**: this option adjusts the SPMC exception level to being 139 at EL3. 140- If neither **SPMD_SPM_AT_SEL2** or **SPMC_AT_EL3** are enabled the SPMC 141 exception level is set to S-EL1. 142- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 143 restoring) the EL2 system register context before entering (resp. 144 after leaving) the SPMC. It is mandatorily enabled when 145 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 146 and exhaustive list of registers is visible at `[4]`_. 147- **SP_LAYOUT_FILE**: this option specifies a text description file 148 providing paths to SP binary images and manifests in DTS format 149 (see `Describing secure partitions`_). It 150 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 151 secure partitions are to be loaded on behalf of the SPMC. 152 153+---------------+----------------------+------------------+-------------+ 154| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | SPMC_AT_EL3 | 155+---------------+----------------------+------------------+-------------+ 156| SPMC at S-EL1 | 0 | 0 | 0 | 157+---------------+----------------------+------------------+-------------+ 158| SPMC at S-EL2 | 1 | 1 (default when | 0 | 159| | | SPD=spmd) | | 160+---------------+----------------------+------------------+-------------+ 161| SPMC at EL3 | 0 | 0 | 1 | 162+---------------+----------------------+------------------+-------------+ 163 164Other combinations of such build options either break the build or are not 165supported. 166 167Notes: 168 169- Only Arm's FVP platform is supported to use with the TF-A reference software 170 stack. 171- The reference software stack uses FEAT_PAuth (formerly Armv8.3-PAuth) and 172 FEAT_BTI (formerly Armv8.5-BTI) architecture extensions by default at EL3 173 and S-EL2. 174- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 175 barely saving/restoring EL2 registers from an Arm arch perspective. As such 176 it is decoupled from the ``SPD=spmd`` option. 177- BL32 option is re-purposed to specify the SPMC image. It can specify either 178 the Hafnium binary path (built for the secure world) or the path to a TEE 179 binary implementing FF-A interfaces. 180- BL33 option can specify the TFTF binary or a normal world loader 181 such as U-Boot or the UEFI framework. 182 183Sample TF-A build command line when SPMC is located at S-EL1 184(e.g. when the FEAT_EL2 architecture extension is not implemented): 185 186.. code:: shell 187 188 make \ 189 CROSS_COMPILE=aarch64-none-elf- \ 190 SPD=spmd \ 191 SPMD_SPM_AT_SEL2=0 \ 192 BL32=<path-to-tee-binary> \ 193 BL33=<path-to-bl33-binary> \ 194 PLAT=fvp \ 195 all fip 196 197Sample TF-A build command line for a FEAT_SEL2 enabled system where the SPMC is 198located at S-EL2: 199 200.. code:: shell 201 202 make \ 203 CROSS_COMPILE=aarch64-none-elf- \ 204 PLAT=fvp \ 205 SPD=spmd \ 206 CTX_INCLUDE_EL2_REGS=1 \ 207 ARM_ARCH_MINOR=5 \ 208 BRANCH_PROTECTION=1 \ 209 CTX_INCLUDE_PAUTH_REGS=1 \ 210 BL32=<path-to-hafnium-binary> \ 211 BL33=<path-to-bl33-binary> \ 212 SP_LAYOUT_FILE=sp_layout.json \ 213 all fip 214 215Same as above with enabling secure boot in addition: 216 217.. code:: shell 218 219 make \ 220 CROSS_COMPILE=aarch64-none-elf- \ 221 PLAT=fvp \ 222 SPD=spmd \ 223 CTX_INCLUDE_EL2_REGS=1 \ 224 ARM_ARCH_MINOR=5 \ 225 BRANCH_PROTECTION=1 \ 226 CTX_INCLUDE_PAUTH_REGS=1 \ 227 BL32=<path-to-hafnium-binary> \ 228 BL33=<path-to-bl33-binary> \ 229 SP_LAYOUT_FILE=sp_layout.json \ 230 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 231 TRUSTED_BOARD_BOOT=1 \ 232 COT=dualroot \ 233 ARM_ROTPK_LOCATION=devel_rsa \ 234 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 235 GENERATE_COT=1 \ 236 all fip 237 238Sample TF-A build command line when SPMC is located at EL3: 239 240.. code:: shell 241 242 make \ 243 CROSS_COMPILE=aarch64-none-elf- \ 244 SPD=spmd \ 245 SPMD_SPM_AT_SEL2=0 \ 246 SPMC_AT_EL3=1 \ 247 BL32=<path-to-tee-binary> \ 248 BL33=<path-to-bl33-binary> \ 249 PLAT=fvp \ 250 all fip 251 252FVP model invocation 253==================== 254 255The FVP command line needs the following options to exercise the S-EL2 SPMC: 256 257+---------------------------------------------------+------------------------------------+ 258| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 259| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 260+---------------------------------------------------+------------------------------------+ 261| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 262| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 263| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 264| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 265| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 266| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 267| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 268| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 269+---------------------------------------------------+------------------------------------+ 270| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 271| - cluster1.has_branch_target_exception=1 | | 272+---------------------------------------------------+------------------------------------+ 273| - cluster0.restriction_on_speculative_execution=2 | Required by the EL2 context | 274| - cluster1.restriction_on_speculative_execution=2 | save/restore routine. | 275+---------------------------------------------------+------------------------------------+ 276 277Sample FVP command line invocation: 278 279.. code:: shell 280 281 <path-to-fvp-model>/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 282 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 283 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 284 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 285 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 286 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 287 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 \ 288 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 \ 289 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 \ 290 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 \ 291 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 \ 292 -C cluster0.has_branch_target_exception=1 \ 293 -C cluster1.has_branch_target_exception=1 \ 294 -C cluster0.restriction_on_speculative_execution=2 \ 295 -C cluster1.restriction_on_speculative_execution=2 296 297Boot process 298============ 299 300Loading Hafnium and secure partitions in the secure world 301--------------------------------------------------------- 302 303TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 304 305SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 306Thus they are supplied as distinct signed entities within the FIP flash 307image. The FIP image itself is not signed hence this provides the ability 308to upgrade SPs in the field. 309 310Booting through TF-A 311-------------------- 312 313SP manifests 314~~~~~~~~~~~~ 315 316An SP manifest describes SP attributes as defined in `[1]`_ 317(partition manifest at virtual FF-A instance) in DTS format. It is 318represented as a single file associated with the SP. A sample is 319provided by `[5]`_. A binding document is provided by `[6]`_. 320 321Secure Partition packages 322~~~~~~~~~~~~~~~~~~~~~~~~~ 323 324Secure partitions are bundled as independent package files consisting 325of: 326 327- a header 328- a DTB 329- an image payload 330 331The header starts with a magic value and offset values to SP DTB and 332image payload. Each SP package is loaded independently by BL2 loader 333and verified for authenticity and integrity. 334 335The SP package identified by its UUID (matching FF-A uuid property) is 336inserted as a single entry into the FIP at end of the TF-A build flow 337as shown: 338 339.. code:: shell 340 341 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 342 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 343 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 344 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 345 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 346 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 347 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 348 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 349 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 350 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 351 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 352 353.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 354 355Describing secure partitions 356~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 357 358A json-formatted description file is passed to the build flow specifying paths 359to the SP binary image and associated DTS partition manifest file. The latter 360is processed by the dtc compiler to generate a DTB fed into the SP package. 361This file also specifies the SP owner (as an optional field) identifying the 362signing domain in case of dual root CoT. 363The SP owner can either be the silicon or the platform provider. The 364corresponding "owner" field value can either take the value of "SiP" or "Plat". 365In absence of "owner" field, it defaults to "SiP" owner. 366The UUID of the partition can be specified as a field in the description file or 367if it does not exist there the UUID is extracted from the DTS partition 368manifest. 369 370.. code:: shell 371 372 { 373 "tee1" : { 374 "image": "tee1.bin", 375 "pm": "tee1.dts", 376 "owner": "SiP", 377 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f" 378 }, 379 380 "tee2" : { 381 "image": "tee2.bin", 382 "pm": "tee2.dts", 383 "owner": "Plat" 384 } 385 } 386 387SPMC manifest 388~~~~~~~~~~~~~ 389 390This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 391time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 392two different cases: 393 394- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 395 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 396 mode. 397- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 398 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 399 S-EL0. 400 401.. code:: shell 402 403 attribute { 404 spmc_id = <0x8000>; 405 maj_ver = <0x1>; 406 min_ver = <0x0>; 407 exec_state = <0x0>; 408 load_address = <0x0 0x6000000>; 409 entrypoint = <0x0 0x6000000>; 410 binary_size = <0x60000>; 411 }; 412 413- *spmc_id* defines the endpoint ID value that SPMC can query through 414 ``FFA_ID_GET``. 415- *maj_ver/min_ver*. SPMD checks provided version versus its internal 416 version and aborts if not matching. 417- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 418 Notice Hafnium used as a SPMC only supports AArch64. 419- *load_address* and *binary_size* are mostly used to verify secondary 420 entry points fit into the loaded binary image. 421- *entrypoint* defines the cold boot primary core entry point used by 422 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 423 424Other nodes in the manifest are consumed by Hafnium in the secure world. 425A sample can be found at [7]: 426 427- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 428 indicates a FF-A compliant SP. The *load_address* field specifies the load 429 address at which TF-A loaded the SP package. 430- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 431 Note the primary core is declared first, then secondary core are declared 432 in reverse order. 433- The *memory* node provides platform information on the ranges of memory 434 available to the SPMC. 435 436SPMC boot 437~~~~~~~~~ 438 439The SPMC is loaded by BL2 as the BL32 image. 440 441The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 442 443BL2 passes the SPMC manifest address to BL31 through a register. 444 445At boot time, the SPMD in BL31 runs from the primary core, initializes the core 446contexts and launches the SPMC (BL32) passing the following information through 447registers: 448 449- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 450- X1 holds the ``HW_CONFIG`` physical address. 451- X4 holds the currently running core linear id. 452 453Loading of SPs 454~~~~~~~~~~~~~~ 455 456At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 457below: 458 459.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 460 461Note this boot flow is an implementation sample on Arm's FVP platform. 462Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 463different implementation. 464 465Secure boot 466~~~~~~~~~~~ 467 468The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 469SPMC manifest, secure partitions and verifies them for authenticity and integrity. 470Refer to TBBR specification `[3]`_. 471 472The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 473the use of two root keys namely S-ROTPK and NS-ROTPK: 474 475- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 476- BL33 may be signed by the OEM using NS-ROTPK. 477- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 478 479Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 480 481Hafnium in the secure world 482=========================== 483 484General considerations 485---------------------- 486 487Build platform for the secure world 488~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 489 490In the Hafnium reference implementation specific code parts are only relevant to 491the secure world. Such portions are isolated in architecture specific files 492and/or enclosed by a ``SECURE_WORLD`` macro. 493 494Secure partitions CPU scheduling 495~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 496 497The FF-A v1.0 specification `[1]`_ provides two ways to relinquinsh CPU time to 498secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 499 500- the FFA_MSG_SEND_DIRECT_REQ interface. 501- the FFA_RUN interface. 502 503Platform topology 504~~~~~~~~~~~~~~~~~ 505 506The *execution-ctx-count* SP manifest field can take the value of one or the 507total number of PEs. The FF-A v1.0 specification `[1]`_ recommends the 508following SP types: 509 510- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 511 implement the same number of ECs as the number of PEs in the platform. 512- Migratable UP SPs: a single execution context can run and be migrated on any 513 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 514 receive a direct message request originating from any physical core targeting 515 the single execution context. 516 517Parsing SP partition manifests 518------------------------------ 519 520Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 521Note the current implementation may not implement all optional fields. 522 523The SP manifest may contain memory and device regions nodes. In case of 524an S-EL2 SPMC: 525 526- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 527 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 528 specify RX/TX buffer regions in which case it is not necessary for an SP 529 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 530- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 531 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 532 additional resources (e.g. interrupts). 533 534For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 535provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 536regime. 537 538Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 539same set of page tables. It is still open whether two sets of page tables shall 540be provided per SP. The memory region node as defined in the specification 541provides a memory security attribute hinting to map either to the secure or 542non-secure EL1&0 Stage-2 table if it exists. 543 544Passing boot data to the SP 545--------------------------- 546 547In `[1]`_ , the "Protocol for passing data" section defines a method for passing 548boot data to SPs (not currently implemented). 549 550Provided that the whole secure partition package image (see 551`Secure Partition packages`_) is mapped to the SP secure EL1&0 Stage-2 552translation regime, an SP can access its own manifest DTB blob and extract its 553partition manifest properties. 554 555SP Boot order 556------------- 557 558SP manifests provide an optional boot order attribute meant to resolve 559dependencies such as an SP providing a service required to properly boot 560another SP. SPMC boots the SPs in accordance to the boot order attribute, 561lowest to the highest value. If the boot order attribute is absent from the FF-A 562manifest, the SP is treated as if it had the highest boot order value 563(i.e. lowest booting priority). 564 565It is possible for an SP to call into another SP through a direct request 566provided the latter SP has already been booted. 567 568Boot phases 569----------- 570 571Primary core boot-up 572~~~~~~~~~~~~~~~~~~~~ 573 574Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 575core. The SPMC performs its platform initializations and registers the SPMC 576secondary physical core entry point physical address by the use of the 577`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 578at secure physical FF-A instance). 579 580The SPMC then creates secure partitions based on SP packages and manifests. Each 581secure partition is launched in sequence (`SP Boot order`_) on their "primary" 582execution context. If the primary boot physical core linear id is N, an MP SP is 583started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 584UP SP, it is started using its unique EC0 on PE[N]. 585 586The SP primary EC (or the EC used when the partition is booted as described 587above): 588 589- Performs the overall SP boot time initialization, and in case of a MP SP, 590 prepares the SP environment for other execution contexts. 591- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 592 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 593 entry point for other execution contexts. 594- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 595 ``FFA_ERROR`` in case of failure. 596 597Secondary cores boot-up 598~~~~~~~~~~~~~~~~~~~~~~~ 599 600Once the system is started and NWd brought up, a secondary physical core is 601woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 602calls into the SPMD on the newly woken up physical core. Then the SPMC is 603entered at the secondary physical core entry point. 604 605In the current implementation, the first SP is resumed on the coresponding EC 606(the virtual CPU which matches the physical core). The implication is that the 607first SP must be a MP SP. 608 609In a linux based system, once secure and normal worlds are booted but prior to 610a NWd FF-A driver has been loaded: 611 612- The first SP has initialized all its ECs in response to primary core boot up 613 (at system initialization) and secondary core boot up (as a result of linux 614 invoking PSCI_CPU_ON for all secondary cores). 615- Other SPs have their first execution context initialized as a result of secure 616 world initialization on the primary boot core. Other ECs for those SPs have to 617 be run first through ffa_run to complete their initialization (which results 618 in the EC completing with FFA_MSG_WAIT). 619 620Refer to `Power management`_ for further details. 621 622Notifications 623------------- 624 625The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 626communication mechanism with non-blocking semantics. It allows for one FF-A 627endpoint to signal another for service provision, without hindering its current 628progress. 629 630Hafnium currently supports 64 notifications. The IDs of each notification define 631a position in a 64-bit bitmap. 632 633The signaling of notifications can interchangeably happen between NWd and SWd 634FF-A endpoints. 635 636The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 637VMs, and from VMs to SPs. An hypervisor component would only manage 638notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 639deployed in NWd, the Hypervisor or OS kernel must invoke the interface 640FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 641endpoint in the NWd that supports it. 642 643A sender can signal notifications once the receiver has provided it with 644permissions. Permissions are provided by invoking the interface 645FFA_NOTIFICATION_BIND. 646 647Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 648they are considered to be in a pending sate. The receiver can retrieve its 649pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 650are considered to be handled. 651 652Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 653that is in charge of donating CPU cycles for notifications handling. The 654FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 655which FF-A endpoints have pending notifications. The receiver scheduler is 656called and informed by the FF-A driver, and it should allocate CPU cycles to the 657receiver. 658 659There are two types of notifications supported: 660- Global, which are targeted to a FF-A endpoint and can be handled within any of 661its execution contexts, as determined by the scheduler of the system. 662- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 663a specific execution context, as determined by the sender. 664 665The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 666permissions to the sender. 667 668Notification signaling resorts to two interrupts: 669- Schedule Receiver Interrupt: Non-secure physical interrupt to be handled by 670the FF-A 'transport' driver within the receiver scheduler. At initialization 671the SPMC (as suggested by the spec) configures a secure SGI, as non-secure, and 672triggers it when there are pending notifications, and the respective receivers 673need CPU cycles to handle them. 674- Notifications Pending Interrupt: Virtual Interrupt to be handled by the 675receiver of the notification. Set when there are pending notifications. For 676per-vCPU the NPI is pended at the handling of FFA_NOTIFICATION_SET interface. 677 678The notifications receipt support is enabled in the partition FF-A manifest. 679 680The subsequent section provides more details about the each one of the 681FF-A interfaces for notifications support. 682 683Mandatory interfaces 684-------------------- 685 686The following interfaces are exposed to SPs: 687 688- ``FFA_VERSION`` 689- ``FFA_FEATURES`` 690- ``FFA_RX_RELEASE`` 691- ``FFA_RXTX_MAP`` 692- ``FFA_RXTX_UNMAP`` 693- ``FFA_PARTITION_INFO_GET`` 694- ``FFA_ID_GET`` 695- ``FFA_MSG_WAIT`` 696- ``FFA_MSG_SEND_DIRECT_REQ`` 697- ``FFA_MSG_SEND_DIRECT_RESP`` 698- ``FFA_MEM_DONATE`` 699- ``FFA_MEM_LEND`` 700- ``FFA_MEM_SHARE`` 701- ``FFA_MEM_RETRIEVE_REQ`` 702- ``FFA_MEM_RETRIEVE_RESP`` 703- ``FFA_MEM_RELINQUISH`` 704- ``FFA_MEM_RECLAIM`` 705 706As part of the support of FF-A v1.1, the following interfaces were added: 707 708 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 709 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 710 - ``FFA_NOTIFICATION_BIND`` 711 - ``FFA_NOTIFICATION_UNBIND`` 712 - ``FFA_NOTIFICATION_SET`` 713 - ``FFA_NOTIFICATION_GET`` 714 - ``FFA_NOTIFICATION_INFO_GET`` 715 - ``FFA_SPM_ID_GET`` 716 - ``FFA_SECONDARY_EP_REGISTER`` 717 718FFA_VERSION 719~~~~~~~~~~~ 720 721``FFA_VERSION`` requires a *requested_version* parameter from the caller. 722The returned value depends on the caller: 723 724- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 725 specified in the SPMC manifest. 726- SP: the SPMC returns its own implemented version. 727- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 728 729FFA_FEATURES 730~~~~~~~~~~~~ 731 732FF-A features supported by the SPMC may be discovered by secure partitions at 733boot (that is prior to NWd is booted) or run-time. 734 735The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 736FFA_SUCCESS from the SPMD. 737 738The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 739the response relayed back to the NWd. 740 741FFA_RXTX_MAP/FFA_RXTX_UNMAP 742~~~~~~~~~~~~~~~~~~~~~~~~~~~ 743 744When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 745receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 746regime as secure buffers in the MMU descriptors. 747 748When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 749SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 750descriptors. 751 752The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 753caller, either it being the Hypervisor or OS kernel, as well as a secure 754partition. 755 756FFA_PARTITION_INFO_GET 757~~~~~~~~~~~~~~~~~~~~~~ 758 759Partition info get call can originate: 760 761- from SP to SPMC 762- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 763 764FFA_ID_GET 765~~~~~~~~~~ 766 767The FF-A id space is split into a non-secure space and secure space: 768 769- FF-A ID with bit 15 clear relates to VMs. 770- FF-A ID with bit 15 set related to SPs. 771- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 772 and SPMC. 773 774The SPMD returns: 775 776- The default zero value on invocation from the Hypervisor. 777- The ``spmc_id`` value specified in the SPMC manifest on invocation from 778 the SPMC (see `SPMC manifest`_) 779 780This convention helps the SPMC to determine the origin and destination worlds in 781an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 782transactions in its world switch routine. It must not be permitted for a VM to 783use a secure FF-A ID as origin world by spoofing: 784 785- A VM-to-SP direct request/response shall set the origin world to be non-secure 786 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 787 set). 788- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 789 for both origin and destination IDs. 790 791An incoming direct message request arriving at SPMD from NWd is forwarded to 792SPMC without a specific check. The SPMC is resumed through eret and "knows" the 793message is coming from normal world in this specific code path. Thus the origin 794endpoint ID must be checked by SPMC for being a normal world ID. 795 796An SP sending a direct message request must have bit 15 set in its origin 797endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 798 799The SPMC shall reject the direct message if the claimed world in origin endpoint 800ID is not consistent: 801 802- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 803 world ID", 804- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 805 806 807FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 808~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 809 810This is a mandatory interface for secure partitions consisting in direct request 811and responses with the following rules: 812 813- An SP can send a direct request to another SP. 814- An SP can receive a direct request from another SP. 815- An SP can send a direct response to another SP. 816- An SP cannot send a direct request to an Hypervisor or OS kernel. 817- An Hypervisor or OS kernel can send a direct request to an SP. 818- An SP can send a direct response to an Hypervisor or OS kernel. 819 820FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 821~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 822 823The secure partitions notifications bitmap are statically allocated by the SPMC. 824Hence, this interface is not to be issued by secure partitions. 825 826At initialization, the SPMC is not aware of VMs/partitions deployed in the 827normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 828to be prepared to handle notifications for the provided VM ID. 829 830FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 831~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 832 833Pair of interfaces to manage permissions to signal notifications. Prior to 834handling notifications, an FF-A endpoint must allow a given sender to signal a 835bitmap of notifications. 836 837If the receiver doesn't have notification support enabled in its FF-A manifest, 838it won't be able to bind notifications, hence forbidding it to receive any 839notifications. 840 841FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 842~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 843 844If the notifications set are per-vCPU, the NPI interrupt is set as pending 845for a given receiver partition. 846 847The FFA_NOTIFICATION_GET will retrieve all pending global notifications and all 848pending per-vCPU notifications targeted to the current vCPU. 849 850Hafnium keeps the global counting of the pending notifications, which is 851incremented and decremented at the handling of FFA_NOTIFICATION_SET and 852FFA_NOTIFICATION_GET, respectively. If the counter reaches zero, prior to SPMC 853triggering the SRI, it won't be triggered. 854 855FFA_NOTIFICATION_INFO_GET 856~~~~~~~~~~~~~~~~~~~~~~~~~ 857 858Hafnium keeps the global counting of pending notifications whose info has been 859retrieved by this interface. The counting is incremented and decremented at the 860handling of FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET, respectively. 861It also tracks the notifications whose info has been retrieved individually, 862such that it avoids duplicating returned information for subsequent calls to 863FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 864reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 865 866FFA_SPM_ID_GET 867~~~~~~~~~~~~~~ 868 869Returns the FF-A ID allocated to the SPM component (which includes SPMC + SPMD). 870At initialization, the SPMC queries the SPMD for the SPM ID, using this 871same interface, and saves it. 872 873The call emitted at NS and secure physical FF-A instances returns the SPM ID 874specified in the SPMC manifest. 875 876Secure partitions call this interface at the virtual instance, to which the SPMC 877shall return the priorly retrieved SPM ID. 878 879The Hypervisor or OS kernel can issue an FFA_SPM_ID_GET call handled by the 880SPMD, which returns the SPM ID. 881 882FFA_SECONDARY_EP_REGISTER 883~~~~~~~~~~~~~~~~~~~~~~~~~ 884 885When the SPMC boots, all secure partitions are initialized on their primary 886Execution Context. 887 888The interface FFA_SECONDARY_EP_REGISTER is to be used by a secure partitions 889from its first execution context, to provide the entry point address for 890secondary execution contexts. 891 892A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 893the NWd or by invocation of FFA_RUN. 894 895SPMC-SPMD direct requests/responses 896----------------------------------- 897 898Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 899Using those IDs in source/destination fields of a direct request/response 900permits SPMD to SPMC communication and either way. 901 902- SPMC to SPMD direct request/response uses SMC conduit. 903- SPMD to SPMC direct request/response uses ERET conduit. 904 905PE MMU configuration 906-------------------- 907 908With secure virtualization enabled, two IPA spaces are output from the secure 909EL1&0 Stage-1 translation (secure and non-secure). The EL1&0 Stage-2 translation 910hardware is fed by: 911 912- A single secure IPA space when the SP EL1&0 Stage-1 MMU is disabled. 913- Two IPA spaces (secure and non-secure) when the SP EL1&0 Stage-1 MMU is 914 enabled. 915 916``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 917NS/S IPA translations. 918``VSTCR_EL2.SW`` = 0, ``VSTCR_EL2.SA`` = 0,``VTCR_EL2.NSW`` = 0, ``VTCR_EL2.NSA`` = 1: 919 920- Stage-2 translations for the NS IPA space access the NS PA space. 921- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 922 923Secure and non-secure IPA regions use the same set of Stage-2 page tables within 924a SP. 925 926Interrupt management 927-------------------- 928 929GIC ownership 930~~~~~~~~~~~~~ 931 932The SPMC owns the GIC configuration. Secure and non-secure interrupts are 933trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 934IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 935virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 936 937Non-secure interrupt handling 938~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 939 940The following illustrate the scenarios of non secure physical interrupts trapped 941by the SPMC: 942 943- The SP handles a managed exit operation: 944 945.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-managed-exit.png 946 947- The SP is pre-empted without managed exit: 948 949.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-sp-preemption.png 950 951Secure interrupt handling 952------------------------- 953 954This section documents the support implemented for secure interrupt handling in 955SPMC as per the guidance provided by FF-A v1.1 Beta0 specification. 956The following assumptions are made about the system configuration: 957 958 - In the current implementation, S-EL1 SPs are expected to use the para 959 virtualized ABIs for interrupt management rather than accessing virtual GIC 960 interface. 961 - Unless explicitly stated otherwise, this support is applicable only for 962 S-EL1 SPs managed by SPMC. 963 - Secure interrupts are configured as G1S or G0 interrupts. 964 - All physical interrupts are routed to SPMC when running a secure partition 965 execution context. 966 967A physical secure interrupt could preempt normal world execution. Moreover, when 968the execution is in secure world, it is highly likely that the target of a 969secure interrupt is not the currently running execution context of an SP. It 970could be targeted to another FF-A component. Consequently, secure interrupt 971management depends on the state of the target execution context of the SP that 972is responsible for handling the interrupt. Hence, the spec provides guidance on 973how to signal start and completion of secure interrupt handling as discussed in 974further sections. 975 976Secure interrupt signaling mechanisms 977~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 978 979Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 980context that it has a pending virtual interrupt and to further run the SP 981execution context, such that it can handle the virtual interrupt. SPMC uses 982either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 983to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 984the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 985running in S-EL2. 986 987+-----------+---------+---------------+---------------------------------------+ 988| SP State | Conduit | Interface and | Description | 989| | | parameters | | 990+-----------+---------+---------------+---------------------------------------+ 991| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 992| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 993| | | | resumes execution context of SP | 994| | | | through ERET. | 995+-----------+---------+---------------+---------------------------------------+ 996| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 997| | vIRQ | | is pending. It pends vIRQ signal and | 998| | | | resumes execution context of SP | 999| | | | through ERET. | 1000+-----------+---------+---------------+---------------------------------------+ 1001| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 1002| | | | not resume execution context of SP. | 1003+-----------+---------+---------------+---------------------------------------+ 1004| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 1005| | vIRQ | | execution context of SP through ERET. | 1006+-----------+---------+---------------+---------------------------------------+ 1007 1008Secure interrupt completion mechanisms 1009~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1010 1011A SP signals secure interrupt handling completion to the SPMC through the 1012following mechanisms: 1013 1014 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 1015 - ``FFA_RUN`` ABI if its was in BLOCKED state. 1016 1017In the current implementation, S-EL1 SPs use para-virtualized HVC interface 1018implemented by SPMC to perform priority drop and interrupt deactivation (we 1019assume EOImode = 0, i.e. priority drop and deactivation are done together). 1020 1021If normal world execution was preempted by secure interrupt, SPMC uses 1022FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 1023and further return execution to normal world. If the current SP execution 1024context was preempted by a secure interrupt to be handled by execution context 1025of target SP, SPMC resumes current SP after signal completion by target SP 1026execution context. 1027 1028An action is broadly a set of steps taken by the SPMC in response to a physical 1029interrupt. In order to simplify the design, the current version of secure 1030interrupt management support in SPMC (Hafnium) does not fully implement the 1031Scheduling models and Partition runtime models. However, the current 1032implementation loosely maps to the following actions that are legally allowed 1033by the specification. Please refer to the Table 8.4 in the spec for further 1034description of actions. The action specified for a type of interrupt when the 1035SP is in the message processing running state cannot be less permissive than the 1036action specified for the same type of interrupt when the SP is in the interrupt 1037handling running state. 1038 1039+--------------------+--------------------+------------+-------------+ 1040| Runtime Model | NS-Int | Self S-Int | Other S-Int | 1041+--------------------+--------------------+------------+-------------+ 1042| Message Processing | Signalable with ME | Signalable | Signalable | 1043+--------------------+--------------------+------------+-------------+ 1044| Interrupt Handling | Queued | Queued | Queued | 1045+--------------------+--------------------+------------+-------------+ 1046 1047Abbreviations: 1048 1049 - NS-Int: A Non-secure physical interrupt. It requires a switch to the Normal 1050 world to be handled. 1051 - Other S-Int: A secure physical interrupt targeted to an SP different from 1052 the one that is currently running. 1053 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1054 running. 1055 1056The following figure describes interrupt handling flow when secure interrupt 1057triggers while in normal world: 1058 1059.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1060 1061A brief description of the events: 1062 1063 - 1) Secure interrupt triggers while normal world is running. 1064 - 2) FIQ gets trapped to EL3. 1065 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1066 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1067 vIRQ). 1068 - 5) Since SP1 vCPU is in WAITING state, SPMC signals using FFA_INTERRUPT with 1069 interrupt id as argument and resume it using ERET. 1070 - 6) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1071 masked i.e., PSTATE.I = 0 1072 - 7) SP1 services the interrupt and invokes the de-activation HVC call. 1073 - 8) SPMC does internal state management and further de-activates the physical 1074 interrupt and resumes SP vCPU. 1075 - 9) SP performs secure interrupt completion through FFA_MSG_WAIT ABI. 1076 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1077 - 11) EL3 resumes normal world execution. 1078 1079The following figure describes interrupt handling flow when secure interrupt 1080triggers while in secure world: 1081 1082.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1083 1084A brief description of the events: 1085 1086 - 1) Secure interrupt triggers while SP2 is running and SP1 is blocked. 1087 - 2) Gets trapped to SPMC as IRQ. 1088 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1089 this secure interrupt. In this scenario, it is SP1. 1090 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1091 SPMC further resumes SP1 through ERET conduit. 1092 - 5) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1093 masked i.e., PSTATE.I = 0 1094 - 6) SP1 services the secure interrupt and invokes the de-activation HVC call. 1095 - 7) SPMC does internal state management, de-activates the physical interrupt 1096 and resumes SP1 vCPU. 1097 - 8) Assuming SP1 is in BLOCKED state, SP1 performs secure interrupt completion 1098 through FFA_RUN ABI. 1099 - 9) SPMC resumes the pre-empted vCPU of SP2. 1100 1101 1102Power management 1103---------------- 1104 1105In platforms with or without secure virtualization: 1106 1107- The NWd owns the platform PM policy. 1108- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1109- The EL3 PSCI library is in charge of the PM coordination and control 1110 (eventually writing to platform registers). 1111- While coordinating PM events, the PSCI library calls backs into the Secure 1112 Payload Dispatcher for events the latter has statically registered to. 1113 1114When using the SPMD as a Secure Payload Dispatcher: 1115 1116- A power management event is relayed through the SPD hook to the SPMC. 1117- In the current implementation only cpu on (svc_on_finish) and cpu off 1118 (svc_off) hooks are registered. 1119- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1120 The SPMC is entered through its secondary physical core entry point. 1121- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The method by which 1122 the PM event is conveyed to the SPMC is implementation-defined in context of 1123 FF-A v1.0 (`SPMC-SPMD direct requests/responses`_). It consists in a SPMD-to-SPMC 1124 direct request/response conveying the PM event details and SPMC response. 1125 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1126 updates its internal state to reflect the physical core is being turned off. 1127 In the current implementation no SP is resumed as a consequence. This behavior 1128 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1129 userspace. 1130 1131SMMUv3 support in Hafnium 1132========================= 1133 1134An SMMU is analogous to an MMU in a CPU. It performs address translations for 1135Direct Memory Access (DMA) requests from system I/O devices. 1136The responsibilities of an SMMU include: 1137 1138- Translation: Incoming DMA requests are translated from bus address space to 1139 system physical address space using translation tables compliant to 1140 Armv8/Armv7 VMSA descriptor format. 1141- Protection: An I/O device can be prohibited from read, write access to a 1142 memory region or allowed. 1143- Isolation: Traffic from each individial device can be independently managed. 1144 The devices are differentiated from each other using unique translation 1145 tables. 1146 1147The following diagram illustrates a typical SMMU IP integrated in a SoC with 1148several I/O devices along with Interconnect and Memory system. 1149 1150.. image:: ../resources/diagrams/MMU-600.png 1151 1152SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1153support for SMMUv3 driver in both normal and secure world. A brief introduction 1154of SMMUv3 functionality and the corresponding software support in Hafnium is 1155provided here. 1156 1157SMMUv3 features 1158--------------- 1159 1160- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1161 translation support. It can either bypass or abort incoming translations as 1162 well. 1163- Traffic (memory transactions) from each upstream I/O peripheral device, 1164 referred to as Stream, can be independently managed using a combination of 1165 several memory based configuration structures. This allows the SMMUv3 to 1166 support a large number of streams with each stream assigned to a unique 1167 translation context. 1168- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1169 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1170 are supported by SMMUv3. 1171- SMMUv3 offers non-secure stream support with secure stream support being 1172 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1173 instance for secure and non-secure stream support. 1174- It also supports sub-streams to differentiate traffic from a virtualized 1175 peripheral associated with a VM/SP. 1176- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1177 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1178 for providing Secure Stage2 translation support to upstream peripheral 1179 devices. 1180 1181SMMUv3 Programming Interfaces 1182----------------------------- 1183 1184SMMUv3 has three software interfaces that are used by the Hafnium driver to 1185configure the behaviour of SMMUv3 and manage the streams. 1186 1187- Memory based data strutures that provide unique translation context for 1188 each stream. 1189- Memory based circular buffers for command queue and event queue. 1190- A large number of SMMU configuration registers that are memory mapped during 1191 boot time by Hafnium driver. Except a few registers, all configuration 1192 registers have independent secure and non-secure versions to configure the 1193 behaviour of SMMUv3 for translation of secure and non-secure streams 1194 respectively. 1195 1196Peripheral device manifest 1197-------------------------- 1198 1199Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1200These devices are dependent on PE endpoint to initiate and receive memory 1201management transactions on their behalf. The acccess to the MMIO regions of 1202any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 1203uses the same stage 2 translations for the device as those used by partition 1204manager on behalf of the PE endpoint. This ensures that the peripheral device 1205has the same visibility of the physical address space as the endpoint. The 1206device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 1207) must specify these additional properties for each peripheral device in the 1208system : 1209 1210- smmu-id: This field helps to identify the SMMU instance that this device is 1211 upstream of. 1212- stream-ids: List of stream IDs assigned to this device. 1213 1214.. code:: shell 1215 1216 smmuv3-testengine { 1217 base-address = <0x00000000 0x2bfe0000>; 1218 pages-count = <32>; 1219 attributes = <0x3>; 1220 smmu-id = <0>; 1221 stream-ids = <0x0 0x1>; 1222 interrupts = <0x2 0x3>, <0x4 0x5>; 1223 exclusive-access; 1224 }; 1225 1226SMMUv3 driver limitations 1227------------------------- 1228 1229The primary design goal for the Hafnium SMMU driver is to support secure 1230streams. 1231 1232- Currently, the driver only supports Stage2 translations. No support for 1233 Stage1 or nested translations. 1234- Supports only AArch64 translation format. 1235- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1236 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1237- No support for independent peripheral devices. 1238 1239S-EL0 Partition support 1240========================= 1241The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1242FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1243with ARMv8.4 and FEAT_SEL2). 1244 1245S-EL0 partitions are useful for simple partitions that don't require full 1246Trusted OS functionality. It is also useful to reduce jitter and cycle 1247stealing from normal world since they are more lightweight than VMs. 1248 1249S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1250the SPMC. They are differentiated primarily by the 'exception-level' property 1251and the 'execution-ctx-count' property in the SP manifest. They are host apps 1252under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1253call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1254can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1255for memory regions. 1256 1257S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1258capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1259a S-EL0 partition to accept a direct message from secure world and normal world, 1260and generate direct responses to them. 1261 1262Memory sharing between and with S-EL0 partitions is supported. 1263Indirect messaging, Interrupt handling and Notifications are not supported with 1264S-EL0 partitions and is work in progress, planned for future releases. 1265All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not 1266supported. 1267 1268 1269References 1270========== 1271 1272.. _[1]: 1273 1274[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1275 1276.. _[2]: 1277 1278[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 1279 1280.. _[3]: 1281 1282[3] `Trusted Boot Board Requirements 1283Client <https://developer.arm.com/documentation/den0006/d/>`__ 1284 1285.. _[4]: 1286 1287[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1288 1289.. _[5]: 1290 1291[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1292 1293.. _[6]: 1294 1295[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1296 1297.. _[7]: 1298 1299[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1300 1301.. _[8]: 1302 1303[8] https://lists.trustedfirmware.org/pipermail/tf-a/2020-February/000296.html 1304 1305.. _[9]: 1306 1307[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1308 1309-------------- 1310 1311*Copyright (c) 2020-2022, Arm Limited and Contributors. All rights reserved.* 1312
