1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+--------------------------------------+ 10| CoT | Chain of Trust | 11+--------+--------------------------------------+ 12| DMA | Direct Memory Access | 13+--------+--------------------------------------+ 14| DTB | Device Tree Blob | 15+--------+--------------------------------------+ 16| DTS | Device Tree Source | 17+--------+--------------------------------------+ 18| EC | Execution Context | 19+--------+--------------------------------------+ 20| FIP | Firmware Image Package | 21+--------+--------------------------------------+ 22| FF-A | Firmware Framework for Arm A-profile | 23+--------+--------------------------------------+ 24| IPA | Intermediate Physical Address | 25+--------+--------------------------------------+ 26| NWd | Normal World | 27+--------+--------------------------------------+ 28| ODM | Original Design Manufacturer | 29+--------+--------------------------------------+ 30| OEM | Original Equipment Manufacturer | 31+--------+--------------------------------------+ 32| PA | Physical Address | 33+--------+--------------------------------------+ 34| PE | Processing Element | 35+--------+--------------------------------------+ 36| PM | Power Management | 37+--------+--------------------------------------+ 38| PVM | Primary VM | 39+--------+--------------------------------------+ 40| SMMU | System Memory Management Unit | 41+--------+--------------------------------------+ 42| SP | Secure Partition | 43+--------+--------------------------------------+ 44| SPD | Secure Payload Dispatcher | 45+--------+--------------------------------------+ 46| SPM | Secure Partition Manager | 47+--------+--------------------------------------+ 48| SPMC | SPM Core | 49+--------+--------------------------------------+ 50| SPMD | SPM Dispatcher | 51+--------+--------------------------------------+ 52| SiP | Silicon Provider | 53+--------+--------------------------------------+ 54| SWd | Secure World | 55+--------+--------------------------------------+ 56| TLV | Tag-Length-Value | 57+--------+--------------------------------------+ 58| TOS | Trusted Operating System | 59+--------+--------------------------------------+ 60| VM | Virtual Machine | 61+--------+--------------------------------------+ 62 63Foreword 64======== 65 66Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 67 68- SPM based on the FF-A specification `[1]`_. 69- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 70 71Both implementations differ in their architectures and only one can be selected 72at build time. 73 74This document: 75 76- describes the FF-A implementation where the Secure Partition Manager 77 resides at EL3 and S-EL2 (or EL3 and S-EL1). 78- is not an architecture specification and it might provide assumptions 79 on sections mandated as implementation-defined in the specification. 80- covers the implications to TF-A used as a bootloader, and Hafnium 81 used as a reference code base for an S-EL2 secure firmware on 82 platforms implementing the FEAT_SEL2 (formerly Armv8.4 Secure EL2) 83 architecture extension. 84 85Terminology 86----------- 87 88- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 89 (or partitions) in the normal world. 90- The term SPMC refers to the S-EL2 component managing secure partitions in 91 the secure world when the FEAT_SEL2 architecture extension is implemented. 92- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 93 partition and implementing the FF-A ABI on platforms not implementing the 94 FEAT_SEL2 architecture extension. 95- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 96- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 97 98Support for legacy platforms 99---------------------------- 100 101In the implementation, the SPM is split into SPMD and SPMC components. 102The SPMD is located at EL3 and mainly relays FF-A messages from 103NWd (Hypervisor or OS kernel) to SPMC located either at S-EL1 or S-EL2. 104 105Hence TF-A supports both cases where the SPMC is located either at: 106 107- S-EL1 supporting platforms not implementing the FEAT_SEL2 architecture 108 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 109- or S-EL2 supporting platforms implementing the FEAT_SEL2 architecture 110 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 111 112The same TF-A SPMD component is used to support both configurations. 113The SPMC exception level is a build time choice. 114 115Sample reference stack 116====================== 117 118The following diagram illustrates a possible configuration when the 119FEAT_SEL2 architecture extension is implemented, showing the SPMD 120and SPMC, one or multiple secure partitions, with an optional 121Hypervisor: 122 123.. image:: ../resources/diagrams/ff-a-spm-sel2.png 124 125TF-A build options 126================== 127 128This section explains the TF-A build options involved in building with 129support for an FF-A based SPM where the SPMD is located at EL3 and the 130SPMC located at S-EL1 or S-EL2: 131 132- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 133 protocol from NWd to SWd back and forth. It is not possible to 134 enable another Secure Payload Dispatcher when this option is chosen. 135- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 136 level to being S-EL1 or S-EL2. It defaults to enabled (value 1) when 137 SPD=spmd is chosen. 138- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 139 restoring) the EL2 system register context before entering (resp. 140 after leaving) the SPMC. It is mandatorily enabled when 141 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 142 and exhaustive list of registers is visible at `[4]`_. 143- **SP_LAYOUT_FILE**: this option specifies a text description file 144 providing paths to SP binary images and manifests in DTS format 145 (see `Describing secure partitions`_). It 146 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 147 secure partitions are to be loaded on behalf of the SPMC. 148 149+---------------+----------------------+------------------+ 150| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | 151+---------------+----------------------+------------------+ 152| SPMC at S-EL1 | 0 | 0 | 153+---------------+----------------------+------------------+ 154| SPMC at S-EL2 | 1 | 1 (default when | 155| | | SPD=spmd) | 156+---------------+----------------------+------------------+ 157 158Other combinations of such build options either break the build or are not 159supported. 160 161Notes: 162 163- Only Arm's FVP platform is supported to use with the TF-A reference software 164 stack. 165- The reference software stack uses FEAT_PAuth (formerly Armv8.3-PAuth) and 166 FEAT_BTI (formerly Armv8.5-BTI) architecture extensions by default at EL3 167 and S-EL2. 168- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 169 barely saving/restoring EL2 registers from an Arm arch perspective. As such 170 it is decoupled from the ``SPD=spmd`` option. 171- BL32 option is re-purposed to specify the SPMC image. It can specify either 172 the Hafnium binary path (built for the secure world) or the path to a TEE 173 binary implementing FF-A interfaces. 174- BL33 option can specify the TFTF binary or a normal world loader 175 such as U-Boot or the UEFI framework. 176 177Sample TF-A build command line when SPMC is located at S-EL1 178(e.g. when the FEAT_EL2 architecture extension is not implemented): 179 180.. code:: shell 181 182 make \ 183 CROSS_COMPILE=aarch64-none-elf- \ 184 SPD=spmd \ 185 SPMD_SPM_AT_SEL2=0 \ 186 BL32=<path-to-tee-binary> \ 187 BL33=<path-to-bl33-binary> \ 188 PLAT=fvp \ 189 all fip 190 191Sample TF-A build command line for a FEAT_SEL2 enabled system where the SPMC is 192located at S-EL2: 193 194.. code:: shell 195 196 make \ 197 CROSS_COMPILE=aarch64-none-elf- \ 198 PLAT=fvp \ 199 SPD=spmd \ 200 CTX_INCLUDE_EL2_REGS=1 \ 201 ARM_ARCH_MINOR=5 \ 202 BRANCH_PROTECTION=1 \ 203 CTX_INCLUDE_PAUTH_REGS=1 \ 204 BL32=<path-to-hafnium-binary> \ 205 BL33=<path-to-bl33-binary> \ 206 SP_LAYOUT_FILE=sp_layout.json \ 207 all fip 208 209Same as above with enabling secure boot in addition: 210 211.. code:: shell 212 213 make \ 214 CROSS_COMPILE=aarch64-none-elf- \ 215 PLAT=fvp \ 216 SPD=spmd \ 217 CTX_INCLUDE_EL2_REGS=1 \ 218 ARM_ARCH_MINOR=5 \ 219 BRANCH_PROTECTION=1 \ 220 CTX_INCLUDE_PAUTH_REGS=1 \ 221 BL32=<path-to-hafnium-binary> \ 222 BL33=<path-to-bl33-binary> \ 223 SP_LAYOUT_FILE=sp_layout.json \ 224 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 225 TRUSTED_BOARD_BOOT=1 \ 226 COT=dualroot \ 227 ARM_ROTPK_LOCATION=devel_rsa \ 228 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 229 GENERATE_COT=1 \ 230 all fip 231 232FVP model invocation 233==================== 234 235The FVP command line needs the following options to exercise the S-EL2 SPMC: 236 237+---------------------------------------------------+------------------------------------+ 238| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 239| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 240+---------------------------------------------------+------------------------------------+ 241| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 242| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 243| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 244| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 245| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 246| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 247| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 248| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 249+---------------------------------------------------+------------------------------------+ 250| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 251| - cluster1.has_branch_target_exception=1 | | 252+---------------------------------------------------+------------------------------------+ 253| - cluster0.restriction_on_speculative_execution=2 | Required by the EL2 context | 254| - cluster1.restriction_on_speculative_execution=2 | save/restore routine. | 255+---------------------------------------------------+------------------------------------+ 256 257Sample FVP command line invocation: 258 259.. code:: shell 260 261 <path-to-fvp-model>/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 262 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 263 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 264 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 265 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 266 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 267 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 \ 268 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 \ 269 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 \ 270 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 \ 271 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 \ 272 -C cluster0.has_branch_target_exception=1 \ 273 -C cluster1.has_branch_target_exception=1 \ 274 -C cluster0.restriction_on_speculative_execution=2 \ 275 -C cluster1.restriction_on_speculative_execution=2 276 277Boot process 278============ 279 280Loading Hafnium and secure partitions in the secure world 281--------------------------------------------------------- 282 283TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 284 285SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 286Thus they are supplied as distinct signed entities within the FIP flash 287image. The FIP image itself is not signed hence this provides the ability 288to upgrade SPs in the field. 289 290Booting through TF-A 291-------------------- 292 293SP manifests 294~~~~~~~~~~~~ 295 296An SP manifest describes SP attributes as defined in `[1]`_ 297(partition manifest at virtual FF-A instance) in DTS format. It is 298represented as a single file associated with the SP. A sample is 299provided by `[5]`_. A binding document is provided by `[6]`_. 300 301Secure Partition packages 302~~~~~~~~~~~~~~~~~~~~~~~~~ 303 304Secure partitions are bundled as independent package files consisting 305of: 306 307- a header 308- a DTB 309- an image payload 310 311The header starts with a magic value and offset values to SP DTB and 312image payload. Each SP package is loaded independently by BL2 loader 313and verified for authenticity and integrity. 314 315The SP package identified by its UUID (matching FF-A uuid property) is 316inserted as a single entry into the FIP at end of the TF-A build flow 317as shown: 318 319.. code:: shell 320 321 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 322 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 323 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 324 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 325 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 326 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 327 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 328 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 329 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 330 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 331 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 332 333.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 334 335Describing secure partitions 336~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 337 338A json-formatted description file is passed to the build flow specifying paths 339to the SP binary image and associated DTS partition manifest file. The latter 340is processed by the dtc compiler to generate a DTB fed into the SP package. 341This file also specifies the SP owner (as an optional field) identifying the 342signing domain in case of dual root CoT. 343The SP owner can either be the silicon or the platform provider. The 344corresponding "owner" field value can either take the value of "SiP" or "Plat". 345In absence of "owner" field, it defaults to "SiP" owner. 346The UUID of the partition can be specified as a field in the description file or 347if it does not exist there the UUID is extracted from the DTS partition 348manifest. 349 350.. code:: shell 351 352 { 353 "tee1" : { 354 "image": "tee1.bin", 355 "pm": "tee1.dts", 356 "owner": "SiP", 357 "uuid": "1b1820fe-48f7-4175-8999-d51da00b7c9f" 358 }, 359 360 "tee2" : { 361 "image": "tee2.bin", 362 "pm": "tee2.dts", 363 "owner": "Plat" 364 } 365 } 366 367SPMC manifest 368~~~~~~~~~~~~~ 369 370This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 371time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 372two different cases: 373 374- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 375 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 376 mode. 377- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 378 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 379 S-EL0. 380 381.. code:: shell 382 383 attribute { 384 spmc_id = <0x8000>; 385 maj_ver = <0x1>; 386 min_ver = <0x0>; 387 exec_state = <0x0>; 388 load_address = <0x0 0x6000000>; 389 entrypoint = <0x0 0x6000000>; 390 binary_size = <0x60000>; 391 }; 392 393- *spmc_id* defines the endpoint ID value that SPMC can query through 394 ``FFA_ID_GET``. 395- *maj_ver/min_ver*. SPMD checks provided version versus its internal 396 version and aborts if not matching. 397- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 398 Notice Hafnium used as a SPMC only supports AArch64. 399- *load_address* and *binary_size* are mostly used to verify secondary 400 entry points fit into the loaded binary image. 401- *entrypoint* defines the cold boot primary core entry point used by 402 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 403 404Other nodes in the manifest are consumed by Hafnium in the secure world. 405A sample can be found at [7]: 406 407- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 408 indicates a FF-A compliant SP. The *load_address* field specifies the load 409 address at which TF-A loaded the SP package. 410- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 411 Note the primary core is declared first, then secondary core are declared 412 in reverse order. 413- The *memory* node provides platform information on the ranges of memory 414 available to the SPMC. 415 416SPMC boot 417~~~~~~~~~ 418 419The SPMC is loaded by BL2 as the BL32 image. 420 421The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 422 423BL2 passes the SPMC manifest address to BL31 through a register. 424 425At boot time, the SPMD in BL31 runs from the primary core, initializes the core 426contexts and launches the SPMC (BL32) passing the following information through 427registers: 428 429- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 430- X1 holds the ``HW_CONFIG`` physical address. 431- X4 holds the currently running core linear id. 432 433Loading of SPs 434~~~~~~~~~~~~~~ 435 436At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 437below: 438 439.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 440 441Note this boot flow is an implementation sample on Arm's FVP platform. 442Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 443different implementation. 444 445Secure boot 446~~~~~~~~~~~ 447 448The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 449SPMC manifest, secure partitions and verifies them for authenticity and integrity. 450Refer to TBBR specification `[3]`_. 451 452The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 453the use of two root keys namely S-ROTPK and NS-ROTPK: 454 455- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 456- BL33 may be signed by the OEM using NS-ROTPK. 457- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 458 459Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 460 461Hafnium in the secure world 462=========================== 463 464General considerations 465---------------------- 466 467Build platform for the secure world 468~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 469 470In the Hafnium reference implementation specific code parts are only relevant to 471the secure world. Such portions are isolated in architecture specific files 472and/or enclosed by a ``SECURE_WORLD`` macro. 473 474Secure partitions CPU scheduling 475~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 476 477The FF-A v1.0 specification `[1]`_ provides two ways to relinquinsh CPU time to 478secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 479 480- the FFA_MSG_SEND_DIRECT_REQ interface. 481- the FFA_RUN interface. 482 483Platform topology 484~~~~~~~~~~~~~~~~~ 485 486The *execution-ctx-count* SP manifest field can take the value of one or the 487total number of PEs. The FF-A v1.0 specification `[1]`_ recommends the 488following SP types: 489 490- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 491 implement the same number of ECs as the number of PEs in the platform. 492- Migratable UP SPs: a single execution context can run and be migrated on any 493 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 494 receive a direct message request originating from any physical core targeting 495 the single execution context. 496 497Parsing SP partition manifests 498------------------------------ 499 500Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 501Note the current implementation may not implement all optional fields. 502 503The SP manifest may contain memory and device regions nodes. In case of 504an S-EL2 SPMC: 505 506- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 507 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 508 specify RX/TX buffer regions in which case it is not necessary for an SP 509 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 510- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 511 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 512 additional resources (e.g. interrupts). 513 514For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 515provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 516regime. 517 518Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 519same set of page tables. It is still open whether two sets of page tables shall 520be provided per SP. The memory region node as defined in the specification 521provides a memory security attribute hinting to map either to the secure or 522non-secure EL1&0 Stage-2 table if it exists. 523 524Passing boot data to the SP 525--------------------------- 526 527In `[1]`_ , the "Protocol for passing data" section defines a method for passing 528boot data to SPs (not currently implemented). 529 530Provided that the whole secure partition package image (see 531`Secure Partition packages`_) is mapped to the SP secure EL1&0 Stage-2 532translation regime, an SP can access its own manifest DTB blob and extract its 533partition manifest properties. 534 535SP Boot order 536------------- 537 538SP manifests provide an optional boot order attribute meant to resolve 539dependencies such as an SP providing a service required to properly boot 540another SP. SPMC boots the SPs in accordance to the boot order attribute, 541lowest to the highest value. If the boot order attribute is absent from the FF-A 542manifest, the SP is treated as if it had the highest boot order value 543(i.e. lowest booting priority). 544 545It is possible for an SP to call into another SP through a direct request 546provided the latter SP has already been booted. 547 548Boot phases 549----------- 550 551Primary core boot-up 552~~~~~~~~~~~~~~~~~~~~ 553 554Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 555core. The SPMC performs its platform initializations and registers the SPMC 556secondary physical core entry point physical address by the use of the 557`FFA_SECONDARY_EP_REGISTER`_ interface (SMC invocation from the SPMC to the SPMD 558at secure physical FF-A instance). 559 560The SPMC then creates secure partitions based on SP packages and manifests. Each 561secure partition is launched in sequence (`SP Boot order`_) on their "primary" 562execution context. If the primary boot physical core linear id is N, an MP SP is 563started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 564UP SP, it is started using its unique EC0 on PE[N]. 565 566The SP primary EC (or the EC used when the partition is booted as described 567above): 568 569- Performs the overall SP boot time initialization, and in case of a MP SP, 570 prepares the SP environment for other execution contexts. 571- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 572 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 573 entry point for other execution contexts. 574- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 575 ``FFA_ERROR`` in case of failure. 576 577Secondary cores boot-up 578~~~~~~~~~~~~~~~~~~~~~~~ 579 580Once the system is started and NWd brought up, a secondary physical core is 581woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 582calls into the SPMD on the newly woken up physical core. Then the SPMC is 583entered at the secondary physical core entry point. 584 585In the current implementation, the first SP is resumed on the coresponding EC 586(the virtual CPU which matches the physical core). The implication is that the 587first SP must be a MP SP. 588 589In a linux based system, once secure and normal worlds are booted but prior to 590a NWd FF-A driver has been loaded: 591 592- The first SP has initialized all its ECs in response to primary core boot up 593 (at system initialization) and secondary core boot up (as a result of linux 594 invoking PSCI_CPU_ON for all secondary cores). 595- Other SPs have their first execution context initialized as a result of secure 596 world initialization on the primary boot core. Other ECs for those SPs have to 597 be run first through ffa_run to complete their initialization (which results 598 in the EC completing with FFA_MSG_WAIT). 599 600Refer to `Power management`_ for further details. 601 602Notifications 603------------- 604 605The FF-A v1.1 specification `[1]`_ defines notifications as an asynchronous 606communication mechanism with non-blocking semantics. It allows for one FF-A 607endpoint to signal another for service provision, without hindering its current 608progress. 609 610Hafnium currently supports 64 notifications. The IDs of each notification define 611a position in a 64-bit bitmap. 612 613The signaling of notifications can interchangeably happen between NWd and SWd 614FF-A endpoints. 615 616The SPMC is in charge of managing notifications from SPs to SPs, from SPs to 617VMs, and from VMs to SPs. An hypervisor component would only manage 618notifications from VMs to VMs. Given the SPMC has no visibility of the endpoints 619deployed in NWd, the Hypervisor or OS kernel must invoke the interface 620FFA_NOTIFICATION_BITMAP_CREATE to allocate the notifications bitmap per FF-A 621endpoint in the NWd that supports it. 622 623A sender can signal notifications once the receiver has provided it with 624permissions. Permissions are provided by invoking the interface 625FFA_NOTIFICATION_BIND. 626 627Notifications are signaled by invoking FFA_NOTIFICATION_SET. Henceforth 628they are considered to be in a pending sate. The receiver can retrieve its 629pending notifications invoking FFA_NOTIFICATION_GET, which, from that moment, 630are considered to be handled. 631 632Per the FF-A v1.1 spec, each FF-A endpoint must be associated with a scheduler 633that is in charge of donating CPU cycles for notifications handling. The 634FF-A driver calls FFA_NOTIFICATION_INFO_GET to retrieve the information about 635which FF-A endpoints have pending notifications. The receiver scheduler is 636called and informed by the FF-A driver, and it should allocate CPU cycles to the 637receiver. 638 639There are two types of notifications supported: 640- Global, which are targeted to a FF-A endpoint and can be handled within any of 641its execution contexts, as determined by the scheduler of the system. 642- Per-vCPU, which are targeted to a FF-A endpoint and to be handled within a 643a specific execution context, as determined by the sender. 644 645The type of a notification is set when invoking FFA_NOTIFICATION_BIND to give 646permissions to the sender. 647 648Notification signaling resorts to two interrupts: 649- Schedule Receiver Interrupt: Non-secure physical interrupt to be handled by 650the FF-A 'transport' driver within the receiver scheduler. At initialization 651the SPMC (as suggested by the spec) configures a secure SGI, as non-secure, and 652triggers it when there are pending notifications, and the respective receivers 653need CPU cycles to handle them. 654- Notifications Pending Interrupt: Virtual Interrupt to be handled by the 655receiver of the notification. Set when there are pending notifications. For 656per-vCPU the NPI is pended at the handling of FFA_NOTIFICATION_SET interface. 657 658The notifications receipt support is enabled in the partition FF-A manifest. 659 660The subsequent section provides more details about the each one of the 661FF-A interfaces for notifications support. 662 663Mandatory interfaces 664-------------------- 665 666The following interfaces are exposed to SPs: 667 668- ``FFA_VERSION`` 669- ``FFA_FEATURES`` 670- ``FFA_RX_RELEASE`` 671- ``FFA_RXTX_MAP`` 672- ``FFA_RXTX_UNMAP`` 673- ``FFA_PARTITION_INFO_GET`` 674- ``FFA_ID_GET`` 675- ``FFA_MSG_WAIT`` 676- ``FFA_MSG_SEND_DIRECT_REQ`` 677- ``FFA_MSG_SEND_DIRECT_RESP`` 678- ``FFA_MEM_DONATE`` 679- ``FFA_MEM_LEND`` 680- ``FFA_MEM_SHARE`` 681- ``FFA_MEM_RETRIEVE_REQ`` 682- ``FFA_MEM_RETRIEVE_RESP`` 683- ``FFA_MEM_RELINQUISH`` 684- ``FFA_MEM_RECLAIM`` 685 686As part of the support of FF-A v1.1, the following interfaces were added: 687 688 - ``FFA_NOTIFICATION_BITMAP_CREATE`` 689 - ``FFA_NOTIFICATION_BITMAP_DESTROY`` 690 - ``FFA_NOTIFICATION_BIND`` 691 - ``FFA_NOTIFICATION_UNBIND`` 692 - ``FFA_NOTIFICATION_SET`` 693 - ``FFA_NOTIFICATION_GET`` 694 - ``FFA_NOTIFICATION_INFO_GET`` 695 - ``FFA_SPM_ID_GET`` 696 - ``FFA_SECONDARY_EP_REGISTER`` 697 698FFA_VERSION 699~~~~~~~~~~~ 700 701``FFA_VERSION`` requires a *requested_version* parameter from the caller. 702The returned value depends on the caller: 703 704- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 705 specified in the SPMC manifest. 706- SP: the SPMC returns its own implemented version. 707- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 708 709FFA_FEATURES 710~~~~~~~~~~~~ 711 712FF-A features supported by the SPMC may be discovered by secure partitions at 713boot (that is prior to NWd is booted) or run-time. 714 715The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 716FFA_SUCCESS from the SPMD. 717 718The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 719the response relayed back to the NWd. 720 721FFA_RXTX_MAP/FFA_RXTX_UNMAP 722~~~~~~~~~~~~~~~~~~~~~~~~~~~ 723 724When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 725receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 726regime as secure buffers in the MMU descriptors. 727 728When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 729SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 730descriptors. 731 732The FFA_RXTX_UNMAP unmaps the RX/TX pair from the translation regime of the 733caller, either it being the Hypervisor or OS kernel, as well as a secure 734partition. 735 736FFA_PARTITION_INFO_GET 737~~~~~~~~~~~~~~~~~~~~~~ 738 739Partition info get call can originate: 740 741- from SP to SPMC 742- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 743 744FFA_ID_GET 745~~~~~~~~~~ 746 747The FF-A id space is split into a non-secure space and secure space: 748 749- FF-A ID with bit 15 clear relates to VMs. 750- FF-A ID with bit 15 set related to SPs. 751- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 752 and SPMC. 753 754The SPMD returns: 755 756- The default zero value on invocation from the Hypervisor. 757- The ``spmc_id`` value specified in the SPMC manifest on invocation from 758 the SPMC (see `SPMC manifest`_) 759 760This convention helps the SPMC to determine the origin and destination worlds in 761an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 762transactions in its world switch routine. It must not be permitted for a VM to 763use a secure FF-A ID as origin world by spoofing: 764 765- A VM-to-SP direct request/response shall set the origin world to be non-secure 766 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 767 set). 768- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 769 for both origin and destination IDs. 770 771An incoming direct message request arriving at SPMD from NWd is forwarded to 772SPMC without a specific check. The SPMC is resumed through eret and "knows" the 773message is coming from normal world in this specific code path. Thus the origin 774endpoint ID must be checked by SPMC for being a normal world ID. 775 776An SP sending a direct message request must have bit 15 set in its origin 777endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 778 779The SPMC shall reject the direct message if the claimed world in origin endpoint 780ID is not consistent: 781 782- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 783 world ID", 784- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 785 786 787FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 788~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 789 790This is a mandatory interface for secure partitions consisting in direct request 791and responses with the following rules: 792 793- An SP can send a direct request to another SP. 794- An SP can receive a direct request from another SP. 795- An SP can send a direct response to another SP. 796- An SP cannot send a direct request to an Hypervisor or OS kernel. 797- An Hypervisor or OS kernel can send a direct request to an SP. 798- An SP can send a direct response to an Hypervisor or OS kernel. 799 800FFA_NOTIFICATION_BITMAP_CREATE/FFA_NOTIFICATION_BITMAP_DESTROY 801~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 802 803The secure partitions notifications bitmap are statically allocated by the SPMC. 804Hence, this interface is not to be issued by secure partitions. 805 806At initialization, the SPMC is not aware of VMs/partitions deployed in the 807normal world. Hence, the Hypervisor or OS kernel must use both ABIs for SPMC 808to be prepared to handle notifications for the provided VM ID. 809 810FFA_NOTIFICATION_BIND/FFA_NOTIFICATION_UNBIND 811~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 812 813Pair of interfaces to manage permissions to signal notifications. Prior to 814handling notifications, an FF-A endpoint must allow a given sender to signal a 815bitmap of notifications. 816 817If the receiver doesn't have notification support enabled in its FF-A manifest, 818it won't be able to bind notifications, hence forbidding it to receive any 819notifications. 820 821FFA_NOTIFICATION_SET/FFA_NOTIFICATION_GET 822~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 823 824If the notifications set are per-vCPU, the NPI interrupt is set as pending 825for a given receiver partition. 826 827The FFA_NOTIFICATION_GET will retrieve all pending global notifications and all 828pending per-vCPU notifications targeted to the current vCPU. 829 830Hafnium keeps the global counting of the pending notifications, which is 831incremented and decremented at the handling of FFA_NOTIFICATION_SET and 832FFA_NOTIFICATION_GET, respectively. If the counter reaches zero, prior to SPMC 833triggering the SRI, it won't be triggered. 834 835FFA_NOTIFICATION_INFO_GET 836~~~~~~~~~~~~~~~~~~~~~~~~~ 837 838Hafnium keeps the global counting of pending notifications whose info has been 839retrieved by this interface. The counting is incremented and decremented at the 840handling of FFA_NOTIFICATION_INFO_GET and FFA_NOTIFICATION_GET, respectively. 841It also tracks the notifications whose info has been retrieved individually, 842such that it avoids duplicating returned information for subsequent calls to 843FFA_NOTIFICATION_INFO_GET. For each notification, this state information is 844reset when receiver called FFA_NOTIFICATION_GET to retrieve them. 845 846FFA_SPM_ID_GET 847~~~~~~~~~~~~~~ 848 849Returns the FF-A ID allocated to the SPM component (which includes SPMC + SPMD). 850At initialization, the SPMC queries the SPMD for the SPM ID, using this 851same interface, and saves it. 852 853The call emitted at NS and secure physical FF-A instances returns the SPM ID 854specified in the SPMC manifest. 855 856Secure partitions call this interface at the virtual instance, to which the SPMC 857shall return the priorly retrieved SPM ID. 858 859The Hypervisor or OS kernel can issue an FFA_SPM_ID_GET call handled by the 860SPMD, which returns the SPM ID. 861 862FFA_SECONDARY_EP_REGISTER 863~~~~~~~~~~~~~~~~~~~~~~~~~ 864 865When the SPMC boots, all secure partitions are initialized on their primary 866Execution Context. 867 868The interface FFA_SECONDARY_EP_REGISTER is to be used by a secure partitions 869from its first execution context, to provide the entry point address for 870secondary execution contexts. 871 872A secondary EC is first resumed either upon invocation of PSCI_CPU_ON from 873the NWd or by invocation of FFA_RUN. 874 875SPMC-SPMD direct requests/responses 876----------------------------------- 877 878Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 879Using those IDs in source/destination fields of a direct request/response 880permits SPMD to SPMC communication and either way. 881 882- SPMC to SPMD direct request/response uses SMC conduit. 883- SPMD to SPMC direct request/response uses ERET conduit. 884 885PE MMU configuration 886-------------------- 887 888With secure virtualization enabled, two IPA spaces are output from the secure 889EL1&0 Stage-1 translation (secure and non-secure). The EL1&0 Stage-2 translation 890hardware is fed by: 891 892- A single secure IPA space when the SP EL1&0 Stage-1 MMU is disabled. 893- Two IPA spaces (secure and non-secure) when the SP EL1&0 Stage-1 MMU is 894 enabled. 895 896``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 897NS/S IPA translations. 898``VSTCR_EL2.SW`` = 0, ``VSTCR_EL2.SA`` = 0,``VTCR_EL2.NSW`` = 0, ``VTCR_EL2.NSA`` = 1: 899 900- Stage-2 translations for the NS IPA space access the NS PA space. 901- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 902 903Secure and non-secure IPA regions use the same set of Stage-2 page tables within 904a SP. 905 906Interrupt management 907-------------------- 908 909GIC ownership 910~~~~~~~~~~~~~ 911 912The SPMC owns the GIC configuration. Secure and non-secure interrupts are 913trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 914IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 915virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 916 917Non-secure interrupt handling 918~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 919 920The following illustrate the scenarios of non secure physical interrupts trapped 921by the SPMC: 922 923- The SP handles a managed exit operation: 924 925.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-managed-exit.png 926 927- The SP is pre-empted without managed exit: 928 929.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-sp-preemption.png 930 931Secure interrupt handling 932------------------------- 933 934This section documents the support implemented for secure interrupt handling in 935SPMC as per the guidance provided by FF-A v1.1 Beta0 specification. 936The following assumptions are made about the system configuration: 937 938 - In the current implementation, S-EL1 SPs are expected to use the para 939 virtualized ABIs for interrupt management rather than accessing virtual GIC 940 interface. 941 - Unless explicitly stated otherwise, this support is applicable only for 942 S-EL1 SPs managed by SPMC. 943 - Secure interrupts are configured as G1S or G0 interrupts. 944 - All physical interrupts are routed to SPMC when running a secure partition 945 execution context. 946 947A physical secure interrupt could preempt normal world execution. Moreover, when 948the execution is in secure world, it is highly likely that the target of a 949secure interrupt is not the currently running execution context of an SP. It 950could be targeted to another FF-A component. Consequently, secure interrupt 951management depends on the state of the target execution context of the SP that 952is responsible for handling the interrupt. Hence, the spec provides guidance on 953how to signal start and completion of secure interrupt handling as discussed in 954further sections. 955 956Secure interrupt signaling mechanisms 957~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 958 959Signaling refers to the mechanisms used by SPMC to indicate to the SP execution 960context that it has a pending virtual interrupt and to further run the SP 961execution context, such that it can handle the virtual interrupt. SPMC uses 962either the FFA_INTERRUPT interface with ERET conduit or vIRQ signal for signaling 963to S-EL1 SPs. When normal world execution is preempted by a secure interrupt, 964the SPMD uses the FFA_INTERRUPT ABI with ERET conduit to signal interrupt to SPMC 965running in S-EL2. 966 967+-----------+---------+---------------+---------------------------------------+ 968| SP State | Conduit | Interface and | Description | 969| | | parameters | | 970+-----------+---------+---------------+---------------------------------------+ 971| WAITING | ERET, | FFA_INTERRUPT,| SPMC signals to SP the ID of pending | 972| | vIRQ | Interrupt ID | interrupt. It pends vIRQ signal and | 973| | | | resumes execution context of SP | 974| | | | through ERET. | 975+-----------+---------+---------------+---------------------------------------+ 976| BLOCKED | ERET, | FFA_INTERRUPT | SPMC signals to SP that an interrupt | 977| | vIRQ | | is pending. It pends vIRQ signal and | 978| | | | resumes execution context of SP | 979| | | | through ERET. | 980+-----------+---------+---------------+---------------------------------------+ 981| PREEMPTED | vIRQ | NA | SPMC pends the vIRQ signal but does | 982| | | | not resume execution context of SP. | 983+-----------+---------+---------------+---------------------------------------+ 984| RUNNING | ERET, | NA | SPMC pends the vIRQ signal and resumes| 985| | vIRQ | | execution context of SP through ERET. | 986+-----------+---------+---------------+---------------------------------------+ 987 988Secure interrupt completion mechanisms 989~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 990 991A SP signals secure interrupt handling completion to the SPMC through the 992following mechanisms: 993 994 - ``FFA_MSG_WAIT`` ABI if it was in WAITING state. 995 - ``FFA_RUN`` ABI if its was in BLOCKED state. 996 997In the current implementation, S-EL1 SPs use para-virtualized HVC interface 998implemented by SPMC to perform priority drop and interrupt deactivation (we 999assume EOImode = 0, i.e. priority drop and deactivation are done together). 1000 1001If normal world execution was preempted by secure interrupt, SPMC uses 1002FFA_NORMAL_WORLD_RESUME ABI to indicate completion of secure interrupt handling 1003and further return execution to normal world. If the current SP execution 1004context was preempted by a secure interrupt to be handled by execution context 1005of target SP, SPMC resumes current SP after signal completion by target SP 1006execution context. 1007 1008An action is broadly a set of steps taken by the SPMC in response to a physical 1009interrupt. In order to simplify the design, the current version of secure 1010interrupt management support in SPMC (Hafnium) does not fully implement the 1011Scheduling models and Partition runtime models. However, the current 1012implementation loosely maps to the following actions that are legally allowed 1013by the specification. Please refer to the Table 8.4 in the spec for further 1014description of actions. The action specified for a type of interrupt when the 1015SP is in the message processing running state cannot be less permissive than the 1016action specified for the same type of interrupt when the SP is in the interrupt 1017handling running state. 1018 1019+--------------------+--------------------+------------+-------------+ 1020| Runtime Model | NS-Int | Self S-Int | Other S-Int | 1021+--------------------+--------------------+------------+-------------+ 1022| Message Processing | Signalable with ME | Signalable | Signalable | 1023+--------------------+--------------------+------------+-------------+ 1024| Interrupt Handling | Queued | Queued | Queued | 1025+--------------------+--------------------+------------+-------------+ 1026 1027Abbreviations: 1028 1029 - NS-Int: A Non-secure physical interrupt. It requires a switch to the Normal 1030 world to be handled. 1031 - Other S-Int: A secure physical interrupt targeted to an SP different from 1032 the one that is currently running. 1033 - Self S-Int: A secure physical interrupt targeted to the SP that is currently 1034 running. 1035 1036The following figure describes interrupt handling flow when secure interrupt 1037triggers while in normal world: 1038 1039.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-nwd.png 1040 1041A brief description of the events: 1042 1043 - 1) Secure interrupt triggers while normal world is running. 1044 - 2) FIQ gets trapped to EL3. 1045 - 3) SPMD signals secure interrupt to SPMC at S-EL2 using FFA_INTERRUPT ABI. 1046 - 4) SPMC identifies target vCPU of SP and injects virtual interrupt (pends 1047 vIRQ). 1048 - 5) Since SP1 vCPU is in WAITING state, SPMC signals using FFA_INTERRUPT with 1049 interrupt id as argument and resume it using ERET. 1050 - 6) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1051 masked i.e., PSTATE.I = 0 1052 - 7) SP1 services the interrupt and invokes the de-activation HVC call. 1053 - 8) SPMC does internal state management and further de-activates the physical 1054 interrupt and resumes SP vCPU. 1055 - 9) SP performs secure interrupt completion through FFA_MSG_WAIT ABI. 1056 - 10) SPMC returns control to EL3 using FFA_NORMAL_WORLD_RESUME. 1057 - 11) EL3 resumes normal world execution. 1058 1059The following figure describes interrupt handling flow when secure interrupt 1060triggers while in secure world: 1061 1062.. image:: ../resources/diagrams/ffa-secure-interrupt-handling-swd.png 1063 1064A brief description of the events: 1065 1066 - 1) Secure interrupt triggers while SP2 is running and SP1 is blocked. 1067 - 2) Gets trapped to SPMC as IRQ. 1068 - 3) SPMC finds the target vCPU of secure partition responsible for handling 1069 this secure interrupt. In this scenario, it is SP1. 1070 - 4) SPMC pends vIRQ for SP1 and signals through FFA_INTERRUPT interface. 1071 SPMC further resumes SP1 through ERET conduit. 1072 - 5) Execution traps to vIRQ handler in SP1 provided that interrupt is not 1073 masked i.e., PSTATE.I = 0 1074 - 6) SP1 services the secure interrupt and invokes the de-activation HVC call. 1075 - 7) SPMC does internal state management, de-activates the physical interrupt 1076 and resumes SP1 vCPU. 1077 - 8) Assuming SP1 is in BLOCKED state, SP1 performs secure interrupt completion 1078 through FFA_RUN ABI. 1079 - 9) SPMC resumes the pre-empted vCPU of SP2. 1080 1081 1082Power management 1083---------------- 1084 1085In platforms with or without secure virtualization: 1086 1087- The NWd owns the platform PM policy. 1088- The Hypervisor or OS kernel is the component initiating PSCI service calls. 1089- The EL3 PSCI library is in charge of the PM coordination and control 1090 (eventually writing to platform registers). 1091- While coordinating PM events, the PSCI library calls backs into the Secure 1092 Payload Dispatcher for events the latter has statically registered to. 1093 1094When using the SPMD as a Secure Payload Dispatcher: 1095 1096- A power management event is relayed through the SPD hook to the SPMC. 1097- In the current implementation only cpu on (svc_on_finish) and cpu off 1098 (svc_off) hooks are registered. 1099- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 1100 The SPMC is entered through its secondary physical core entry point. 1101- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The method by which 1102 the PM event is conveyed to the SPMC is implementation-defined in context of 1103 FF-A v1.0 (`SPMC-SPMD direct requests/responses`_). It consists in a SPMD-to-SPMC 1104 direct request/response conveying the PM event details and SPMC response. 1105 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 1106 updates its internal state to reflect the physical core is being turned off. 1107 In the current implementation no SP is resumed as a consequence. This behavior 1108 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 1109 userspace. 1110 1111SMMUv3 support in Hafnium 1112========================= 1113 1114An SMMU is analogous to an MMU in a CPU. It performs address translations for 1115Direct Memory Access (DMA) requests from system I/O devices. 1116The responsibilities of an SMMU include: 1117 1118- Translation: Incoming DMA requests are translated from bus address space to 1119 system physical address space using translation tables compliant to 1120 Armv8/Armv7 VMSA descriptor format. 1121- Protection: An I/O device can be prohibited from read, write access to a 1122 memory region or allowed. 1123- Isolation: Traffic from each individial device can be independently managed. 1124 The devices are differentiated from each other using unique translation 1125 tables. 1126 1127The following diagram illustrates a typical SMMU IP integrated in a SoC with 1128several I/O devices along with Interconnect and Memory system. 1129 1130.. image:: ../resources/diagrams/MMU-600.png 1131 1132SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 1133support for SMMUv3 driver in both normal and secure world. A brief introduction 1134of SMMUv3 functionality and the corresponding software support in Hafnium is 1135provided here. 1136 1137SMMUv3 features 1138--------------- 1139 1140- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 1141 translation support. It can either bypass or abort incoming translations as 1142 well. 1143- Traffic (memory transactions) from each upstream I/O peripheral device, 1144 referred to as Stream, can be independently managed using a combination of 1145 several memory based configuration structures. This allows the SMMUv3 to 1146 support a large number of streams with each stream assigned to a unique 1147 translation context. 1148- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 1149 a Processing Element. AArch32(LPAE) and AArch64 translation table format 1150 are supported by SMMUv3. 1151- SMMUv3 offers non-secure stream support with secure stream support being 1152 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 1153 instance for secure and non-secure stream support. 1154- It also supports sub-streams to differentiate traffic from a virtualized 1155 peripheral associated with a VM/SP. 1156- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 1157 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 1158 for providing Secure Stage2 translation support to upstream peripheral 1159 devices. 1160 1161SMMUv3 Programming Interfaces 1162----------------------------- 1163 1164SMMUv3 has three software interfaces that are used by the Hafnium driver to 1165configure the behaviour of SMMUv3 and manage the streams. 1166 1167- Memory based data strutures that provide unique translation context for 1168 each stream. 1169- Memory based circular buffers for command queue and event queue. 1170- A large number of SMMU configuration registers that are memory mapped during 1171 boot time by Hafnium driver. Except a few registers, all configuration 1172 registers have independent secure and non-secure versions to configure the 1173 behaviour of SMMUv3 for translation of secure and non-secure streams 1174 respectively. 1175 1176Peripheral device manifest 1177-------------------------- 1178 1179Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 1180These devices are dependent on PE endpoint to initiate and receive memory 1181management transactions on their behalf. The acccess to the MMIO regions of 1182any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 1183uses the same stage 2 translations for the device as those used by partition 1184manager on behalf of the PE endpoint. This ensures that the peripheral device 1185has the same visibility of the physical address space as the endpoint. The 1186device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 1187) must specify these additional properties for each peripheral device in the 1188system : 1189 1190- smmu-id: This field helps to identify the SMMU instance that this device is 1191 upstream of. 1192- stream-ids: List of stream IDs assigned to this device. 1193 1194.. code:: shell 1195 1196 smmuv3-testengine { 1197 base-address = <0x00000000 0x2bfe0000>; 1198 pages-count = <32>; 1199 attributes = <0x3>; 1200 smmu-id = <0>; 1201 stream-ids = <0x0 0x1>; 1202 interrupts = <0x2 0x3>, <0x4 0x5>; 1203 exclusive-access; 1204 }; 1205 1206SMMUv3 driver limitations 1207------------------------- 1208 1209The primary design goal for the Hafnium SMMU driver is to support secure 1210streams. 1211 1212- Currently, the driver only supports Stage2 translations. No support for 1213 Stage1 or nested translations. 1214- Supports only AArch64 translation format. 1215- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 1216 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 1217- No support for independent peripheral devices. 1218 1219S-EL0 Partition support 1220========================= 1221The SPMC (Hafnium) has limited capability to run S-EL0 FF-A partitions using 1222FEAT_VHE (mandatory with ARMv8.1 in non-secure state, and in secure world 1223with ARMv8.4 and FEAT_SEL2). 1224 1225S-EL0 partitions are useful for simple partitions that don't require full 1226Trusted OS functionality. It is also useful to reduce jitter and cycle 1227stealing from normal world since they are more lightweight than VMs. 1228 1229S-EL0 partitions are presented, loaded and initialized the same as S-EL1 VMs by 1230the SPMC. They are differentiated primarily by the 'exception-level' property 1231and the 'execution-ctx-count' property in the SP manifest. They are host apps 1232under the single EL2&0 Stage-1 translation regime controlled by the SPMC and 1233call into the SPMC through SVCs as opposed to HVCs and SMCs. These partitions 1234can use FF-A defined services (FFA_MEM_PERM_*) to update or change permissions 1235for memory regions. 1236 1237S-EL0 partitions are required by the FF-A specification to be UP endpoints, 1238capable of migrating, and the SPMC enforces this requirement. The SPMC allows 1239a S-EL0 partition to accept a direct message from secure world and normal world, 1240and generate direct responses to them. 1241 1242Memory sharing between and with S-EL0 partitions is supported. 1243Indirect messaging, Interrupt handling and Notifications are not supported with 1244S-EL0 partitions and is work in progress, planned for future releases. 1245All S-EL0 partitions must use AArch64. AArch32 S-EL0 partitions are not 1246supported. 1247 1248 1249References 1250========== 1251 1252.. _[1]: 1253 1254[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 1255 1256.. _[2]: 1257 1258[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 1259 1260.. _[3]: 1261 1262[3] `Trusted Boot Board Requirements 1263Client <https://developer.arm.com/documentation/den0006/d/>`__ 1264 1265.. _[4]: 1266 1267[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 1268 1269.. _[5]: 1270 1271[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 1272 1273.. _[6]: 1274 1275[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 1276 1277.. _[7]: 1278 1279[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 1280 1281.. _[8]: 1282 1283[8] https://lists.trustedfirmware.org/pipermail/tf-a/2020-February/000296.html 1284 1285.. _[9]: 1286 1287[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 1288 1289-------------- 1290 1291*Copyright (c) 2020-2022, Arm Limited and Contributors. All rights reserved.* 1292
