1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+--------------------------------------+ 10| CoT | Chain of Trust | 11+--------+--------------------------------------+ 12| DMA | Direct Memory Access | 13+--------+--------------------------------------+ 14| DTB | Device Tree Blob | 15+--------+--------------------------------------+ 16| DTS | Device Tree Source | 17+--------+--------------------------------------+ 18| EC | Execution Context | 19+--------+--------------------------------------+ 20| FIP | Firmware Image Package | 21+--------+--------------------------------------+ 22| FF-A | Firmware Framework for Arm A-profile | 23+--------+--------------------------------------+ 24| IPA | Intermediate Physical Address | 25+--------+--------------------------------------+ 26| NWd | Normal World | 27+--------+--------------------------------------+ 28| ODM | Original Design Manufacturer | 29+--------+--------------------------------------+ 30| OEM | Original Equipment Manufacturer | 31+--------+--------------------------------------+ 32| PA | Physical Address | 33+--------+--------------------------------------+ 34| PE | Processing Element | 35+--------+--------------------------------------+ 36| PM | Power Management | 37+--------+--------------------------------------+ 38| PVM | Primary VM | 39+--------+--------------------------------------+ 40| SMMU | System Memory Management Unit | 41+--------+--------------------------------------+ 42| SP | Secure Partition | 43+--------+--------------------------------------+ 44| SPD | Secure Payload Dispatcher | 45+--------+--------------------------------------+ 46| SPM | Secure Partition Manager | 47+--------+--------------------------------------+ 48| SPMC | SPM Core | 49+--------+--------------------------------------+ 50| SPMD | SPM Dispatcher | 51+--------+--------------------------------------+ 52| SiP | Silicon Provider | 53+--------+--------------------------------------+ 54| SWd | Secure World | 55+--------+--------------------------------------+ 56| TLV | Tag-Length-Value | 57+--------+--------------------------------------+ 58| TOS | Trusted Operating System | 59+--------+--------------------------------------+ 60| VM | Virtual Machine | 61+--------+--------------------------------------+ 62 63Foreword 64======== 65 66Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 67 68- SPM based on the FF-A specification `[1]`_. 69- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 70 71Both implementations differ in their architectures and only one can be selected 72at build time. 73 74This document: 75 76- describes the FF-A implementation where the Secure Partition Manager 77 resides at EL3 and S-EL2 (or EL3 and S-EL1). 78- is not an architecture specification and it might provide assumptions 79 on sections mandated as implementation-defined in the specification. 80- covers the implications to TF-A used as a bootloader, and Hafnium 81 used as a reference code base for an S-EL2 secure firmware on 82 platforms implementing the FEAT_SEL2 (formerly Armv8.4 Secure EL2) 83 architecture extension. 84 85Terminology 86----------- 87 88- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 89 (or partitions) in the normal world. 90- The term SPMC refers to the S-EL2 component managing secure partitions in 91 the secure world when the FEAT_SEL2 architecture extension is implemented. 92- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 93 partition and implementing the FF-A ABI on platforms not implementing the 94 FEAT_SEL2 architecture extension. 95- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 96- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 97 98Support for legacy platforms 99---------------------------- 100 101In the implementation, the SPM is split into SPMD and SPMC components. 102The SPMD is located at EL3 and mainly relays FF-A messages from 103NWd (Hypervisor or OS kernel) to SPMC located either at S-EL1 or S-EL2. 104 105Hence TF-A supports both cases where the SPMC is located either at: 106 107- S-EL1 supporting platforms not implementing the FEAT_SEL2 architecture 108 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 109- or S-EL2 supporting platforms implementing the FEAT_SEL2 architecture 110 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 111 112The same TF-A SPMD component is used to support both configurations. 113The SPMC exception level is a build time choice. 114 115Sample reference stack 116====================== 117 118The following diagram illustrates a possible configuration when the 119FEAT_SEL2 architecture extension is implemented, showing the SPMD 120and SPMC, one or multiple secure partitions, with an optional 121Hypervisor: 122 123.. image:: ../resources/diagrams/ff-a-spm-sel2.png 124 125TF-A build options 126================== 127 128This section explains the TF-A build options involved in building with 129support for an FF-A based SPM where the SPMD is located at EL3 and the 130SPMC located at S-EL1 or S-EL2: 131 132- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 133 protocol from NWd to SWd back and forth. It is not possible to 134 enable another Secure Payload Dispatcher when this option is chosen. 135- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 136 level to being S-EL1 or S-EL2. It defaults to enabled (value 1) when 137 SPD=spmd is chosen. 138- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 139 restoring) the EL2 system register context before entering (resp. 140 after leaving) the SPMC. It is mandatorily enabled when 141 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 142 and exhaustive list of registers is visible at `[4]`_. 143- **SP_LAYOUT_FILE**: this option specifies a text description file 144 providing paths to SP binary images and manifests in DTS format 145 (see `Describing secure partitions`_). It 146 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 147 secure partitions are to be loaded on behalf of the SPMC. 148 149+---------------+----------------------+------------------+ 150| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | 151+---------------+----------------------+------------------+ 152| SPMC at S-EL1 | 0 | 0 | 153+---------------+----------------------+------------------+ 154| SPMC at S-EL2 | 1 | 1 (default when | 155| | | SPD=spmd) | 156+---------------+----------------------+------------------+ 157 158Other combinations of such build options either break the build or are not 159supported. 160 161Notes: 162 163- Only Arm's FVP platform is supported to use with the TF-A reference software 164 stack. 165- The reference software stack uses FEAT_PAuth (formerly Armv8.3-PAuth) and 166 FEAT_BTI (formerly Armv8.5-BTI) architecture extensions by default at EL3 167 and S-EL2. 168- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 169 barely saving/restoring EL2 registers from an Arm arch perspective. As such 170 it is decoupled from the ``SPD=spmd`` option. 171- BL32 option is re-purposed to specify the SPMC image. It can specify either 172 the Hafnium binary path (built for the secure world) or the path to a TEE 173 binary implementing FF-A interfaces. 174- BL33 option can specify the TFTF binary or a normal world loader 175 such as U-Boot or the UEFI framework. 176 177Sample TF-A build command line when SPMC is located at S-EL1 178(e.g. when the FEAT_EL2 architecture extension is not implemented): 179 180.. code:: shell 181 182 make \ 183 CROSS_COMPILE=aarch64-none-elf- \ 184 SPD=spmd \ 185 SPMD_SPM_AT_SEL2=0 \ 186 BL32=<path-to-tee-binary> \ 187 BL33=<path-to-bl33-binary> \ 188 PLAT=fvp \ 189 all fip 190 191Sample TF-A build command line for a FEAT_SEL2 enabled system where the SPMC is 192located at S-EL2: 193 194.. code:: shell 195 196 make \ 197 CROSS_COMPILE=aarch64-none-elf- \ 198 PLAT=fvp \ 199 SPD=spmd \ 200 CTX_INCLUDE_EL2_REGS=1 \ 201 ARM_ARCH_MINOR=5 \ 202 BRANCH_PROTECTION=1 \ 203 CTX_INCLUDE_PAUTH_REGS=1 \ 204 BL32=<path-to-hafnium-binary> \ 205 BL33=<path-to-bl33-binary> \ 206 SP_LAYOUT_FILE=sp_layout.json \ 207 all fip 208 209Same as above with enabling secure boot in addition: 210 211.. code:: shell 212 213 make \ 214 CROSS_COMPILE=aarch64-none-elf- \ 215 PLAT=fvp \ 216 SPD=spmd \ 217 CTX_INCLUDE_EL2_REGS=1 \ 218 ARM_ARCH_MINOR=5 \ 219 BRANCH_PROTECTION=1 \ 220 CTX_INCLUDE_PAUTH_REGS=1 \ 221 BL32=<path-to-hafnium-binary> \ 222 BL33=<path-to-bl33-binary> \ 223 SP_LAYOUT_FILE=sp_layout.json \ 224 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 225 TRUSTED_BOARD_BOOT=1 \ 226 COT=dualroot \ 227 ARM_ROTPK_LOCATION=devel_rsa \ 228 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 229 GENERATE_COT=1 \ 230 all fip 231 232FVP model invocation 233==================== 234 235The FVP command line needs the following options to exercise the S-EL2 SPMC: 236 237+---------------------------------------------------+------------------------------------+ 238| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 239| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 240+---------------------------------------------------+------------------------------------+ 241| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 242| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 243| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 244| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 245| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 246| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 247| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 248| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 249+---------------------------------------------------+------------------------------------+ 250| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 251| - cluster1.has_branch_target_exception=1 | | 252+---------------------------------------------------+------------------------------------+ 253| - cluster0.restriction_on_speculative_execution=2 | Required by the EL2 context | 254| - cluster1.restriction_on_speculative_execution=2 | save/restore routine. | 255+---------------------------------------------------+------------------------------------+ 256 257Sample FVP command line invocation: 258 259.. code:: shell 260 261 <path-to-fvp-model>/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 262 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 263 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 264 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 265 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 266 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 267 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 \ 268 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 \ 269 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 \ 270 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 \ 271 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 \ 272 -C cluster0.has_branch_target_exception=1 \ 273 -C cluster1.has_branch_target_exception=1 \ 274 -C cluster0.restriction_on_speculative_execution=2 \ 275 -C cluster1.restriction_on_speculative_execution=2 276 277Boot process 278============ 279 280Loading Hafnium and secure partitions in the secure world 281--------------------------------------------------------- 282 283TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 284 285SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 286Thus they are supplied as distinct signed entities within the FIP flash 287image. The FIP image itself is not signed hence this provides the ability 288to upgrade SPs in the field. 289 290Booting through TF-A 291-------------------- 292 293SP manifests 294~~~~~~~~~~~~ 295 296An SP manifest describes SP attributes as defined in `[1]`_ 297(partition manifest at virtual FF-A instance) in DTS format. It is 298represented as a single file associated with the SP. A sample is 299provided by `[5]`_. A binding document is provided by `[6]`_. 300 301Secure Partition packages 302~~~~~~~~~~~~~~~~~~~~~~~~~ 303 304Secure partitions are bundled as independent package files consisting 305of: 306 307- a header 308- a DTB 309- an image payload 310 311The header starts with a magic value and offset values to SP DTB and 312image payload. Each SP package is loaded independently by BL2 loader 313and verified for authenticity and integrity. 314 315The SP package identified by its UUID (matching FF-A uuid property) is 316inserted as a single entry into the FIP at end of the TF-A build flow 317as shown: 318 319.. code:: shell 320 321 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 322 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 323 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 324 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 325 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 326 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 327 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 328 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 329 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 330 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 331 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 332 333.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 334 335Describing secure partitions 336~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 337 338A json-formatted description file is passed to the build flow specifying paths 339to the SP binary image and associated DTS partition manifest file. The latter 340is processed by the dtc compiler to generate a DTB fed into the SP package. 341This file also specifies the SP owner (as an optional field) identifying the 342signing domain in case of dual root CoT. 343The SP owner can either be the silicon or the platform provider. The 344corresponding "owner" field value can either take the value of "SiP" or "Plat". 345In absence of "owner" field, it defaults to "SiP" owner. 346 347.. code:: shell 348 349 { 350 "tee1" : { 351 "image": "tee1.bin", 352 "pm": "tee1.dts", 353 "owner": "SiP" 354 }, 355 356 "tee2" : { 357 "image": "tee2.bin", 358 "pm": "tee2.dts", 359 "owner": "Plat" 360 } 361 } 362 363SPMC manifest 364~~~~~~~~~~~~~ 365 366This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 367time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 368two different cases: 369 370- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 371 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 372 mode. 373- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 374 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 375 S-EL0. 376 377.. code:: shell 378 379 attribute { 380 spmc_id = <0x8000>; 381 maj_ver = <0x1>; 382 min_ver = <0x0>; 383 exec_state = <0x0>; 384 load_address = <0x0 0x6000000>; 385 entrypoint = <0x0 0x6000000>; 386 binary_size = <0x60000>; 387 }; 388 389- *spmc_id* defines the endpoint ID value that SPMC can query through 390 ``FFA_ID_GET``. 391- *maj_ver/min_ver*. SPMD checks provided version versus its internal 392 version and aborts if not matching. 393- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 394 Notice Hafnium used as a SPMC only supports AArch64. 395- *load_address* and *binary_size* are mostly used to verify secondary 396 entry points fit into the loaded binary image. 397- *entrypoint* defines the cold boot primary core entry point used by 398 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 399 400Other nodes in the manifest are consumed by Hafnium in the secure world. 401A sample can be found at [7]: 402 403- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 404 indicates a FF-A compliant SP. The *load_address* field specifies the load 405 address at which TF-A loaded the SP package. 406- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 407 Note the primary core is declared first, then secondary core are declared 408 in reverse order. 409- The *memory* node provides platform information on the ranges of memory 410 available to the SPMC. 411 412SPMC boot 413~~~~~~~~~ 414 415The SPMC is loaded by BL2 as the BL32 image. 416 417The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image `[9]`_. 418 419BL2 passes the SPMC manifest address to BL31 through a register. 420 421At boot time, the SPMD in BL31 runs from the primary core, initializes the core 422contexts and launches the SPMC (BL32) passing the following information through 423registers: 424 425- X0 holds the ``TOS_FW_CONFIG`` physical address (or SPMC manifest blob). 426- X1 holds the ``HW_CONFIG`` physical address. 427- X4 holds the currently running core linear id. 428 429Loading of SPs 430~~~~~~~~~~~~~~ 431 432At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 433below: 434 435.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 436 437Note this boot flow is an implementation sample on Arm's FVP platform. 438Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 439different implementation. 440 441Secure boot 442~~~~~~~~~~~ 443 444The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 445SPMC manifest, secure partitions and verifies them for authenticity and integrity. 446Refer to TBBR specification `[3]`_. 447 448The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 449the use of two root keys namely S-ROTPK and NS-ROTPK: 450 451- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 452- BL33 may be signed by the OEM using NS-ROTPK. 453- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 454 455Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 456 457Hafnium in the secure world 458=========================== 459 460General considerations 461---------------------- 462 463Build platform for the secure world 464~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 465 466In the Hafnium reference implementation specific code parts are only relevant to 467the secure world. Such portions are isolated in architecture specific files 468and/or enclosed by a ``SECURE_WORLD`` macro. 469 470Secure partitions CPU scheduling 471~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 472 473The FF-A v1.0 specification `[1]`_ provides two ways to relinquinsh CPU time to 474secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 475 476- the FFA_MSG_SEND_DIRECT_REQ interface. 477- the FFA_RUN interface. 478 479Platform topology 480~~~~~~~~~~~~~~~~~ 481 482The *execution-ctx-count* SP manifest field can take the value of one or the 483total number of PEs. The FF-A v1.0 specification `[1]`_ recommends the 484following SP types: 485 486- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 487 implement the same number of ECs as the number of PEs in the platform. 488- Migratable UP SPs: a single execution context can run and be migrated on any 489 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 490 receive a direct message request originating from any physical core targeting 491 the single execution context. 492 493Parsing SP partition manifests 494------------------------------ 495 496Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 497Note the current implementation may not implement all optional fields. 498 499The SP manifest may contain memory and device regions nodes. In case of 500an S-EL2 SPMC: 501 502- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 503 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 504 specify RX/TX buffer regions in which case it is not necessary for an SP 505 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 506- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 507 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 508 additional resources (e.g. interrupts). 509 510For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 511provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 512regime. 513 514Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 515same set of page tables. It is still open whether two sets of page tables shall 516be provided per SP. The memory region node as defined in the specification 517provides a memory security attribute hinting to map either to the secure or 518non-secure EL1&0 Stage-2 table if it exists. 519 520Passing boot data to the SP 521--------------------------- 522 523In `[1]`_ , the "Protocol for passing data" section defines a method for passing 524boot data to SPs (not currently implemented). 525 526Provided that the whole secure partition package image (see 527`Secure Partition packages`_) is mapped to the SP secure EL1&0 Stage-2 528translation regime, an SP can access its own manifest DTB blob and extract its 529partition manifest properties. 530 531SP Boot order 532------------- 533 534SP manifests provide an optional boot order attribute meant to resolve 535dependencies such as an SP providing a service required to properly boot 536another SP. 537 538It is possible for an SP to call into another SP through a direct request 539provided the latter SP has already been booted. 540 541Boot phases 542----------- 543 544Primary core boot-up 545~~~~~~~~~~~~~~~~~~~~ 546 547Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 548core. The SPMC performs its platform initializations and registers the SPMC 549secondary physical core entry point physical address by the use of the 550FFA_SECONDARY_EP_REGISTER interface (SMC invocation from the SPMC to the SPMD 551at secure physical FF-A instance). This interface is implementation-defined in 552context of FF-A v1.0. 553 554The SPMC then creates secure partitions based on SP packages and manifests. Each 555secure partition is launched in sequence (`SP Boot order`_) on their "primary" 556execution context. If the primary boot physical core linear id is N, an MP SP is 557started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 558UP SP, it is started using its unique EC0 on PE[N]. 559 560The SP primary EC (or the EC used when the partition is booted as described 561above): 562 563- Performs the overall SP boot time initialization, and in case of a MP SP, 564 prepares the SP environment for other execution contexts. 565- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 566 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 567 entry point for other execution contexts. 568- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 569 ``FFA_ERROR`` in case of failure. 570 571Secondary cores boot-up 572~~~~~~~~~~~~~~~~~~~~~~~ 573 574Once the system is started and NWd brought up, a secondary physical core is 575woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 576calls into the SPMD on the newly woken up physical core. Then the SPMC is 577entered at the secondary physical core entry point. 578 579In the current implementation, the first SP is resumed on the coresponding EC 580(the virtual CPU which matches the physical core). The implication is that the 581first SP must be a MP SP. 582 583In a linux based system, once secure and normal worlds are booted but prior to 584a NWd FF-A driver has been loaded: 585 586- The first SP has initialized all its ECs in response to primary core boot up 587 (at system initialization) and secondary core boot up (as a result of linux 588 invoking PSCI_CPU_ON for all secondary cores). 589- Other SPs have their first execution context initialized as a result of secure 590 world initialization on the primary boot core. Other ECs for those SPs have to 591 be run first through ffa_run to complete their initialization (which results 592 in the EC completing with FFA_MSG_WAIT). 593 594Refer to `Power management`_ for further details. 595 596Mandatory interfaces 597-------------------- 598 599The following interfaces are exposed to SPs: 600 601- ``FFA_VERSION`` 602- ``FFA_FEATURES`` 603- ``FFA_RX_RELEASE`` 604- ``FFA_RXTX_MAP`` 605- ``FFA_RXTX_UNMAP`` (not implemented) 606- ``FFA_PARTITION_INFO_GET`` 607- ``FFA_ID_GET`` 608- ``FFA_MSG_WAIT`` 609- ``FFA_MSG_SEND_DIRECT_REQ`` 610- ``FFA_MSG_SEND_DIRECT_RESP`` 611- ``FFA_MEM_DONATE`` 612- ``FFA_MEM_LEND`` 613- ``FFA_MEM_SHARE`` 614- ``FFA_MEM_RETRIEVE_REQ`` 615- ``FFA_MEM_RETRIEVE_RESP`` 616- ``FFA_MEM_RELINQUISH`` 617- ``FFA_MEM_RECLAIM`` 618- ``FFA_SECONDARY_EP_REGISTER`` 619 620FFA_VERSION 621~~~~~~~~~~~ 622 623``FFA_VERSION`` requires a *requested_version* parameter from the caller. 624The returned value depends on the caller: 625 626- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 627 specified in the SPMC manifest. 628- SP: the SPMC returns its own implemented version. 629- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 630 631FFA_FEATURES 632~~~~~~~~~~~~ 633 634FF-A features supported by the SPMC may be discovered by secure partitions at 635boot (that is prior to NWd is booted) or run-time. 636 637The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 638FFA_SUCCESS from the SPMD. 639 640The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 641the response relayed back to the NWd. 642 643FFA_RXTX_MAP/FFA_RXTX_UNMAP 644~~~~~~~~~~~~~~~~~~~~~~~~~~~ 645 646When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 647receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 648regime as secure buffers in the MMU descriptors. 649 650When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 651SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 652descriptors. 653 654Note: 655 656- FFA_RXTX_UNMAP is not implemented. 657 658FFA_PARTITION_INFO_GET 659~~~~~~~~~~~~~~~~~~~~~~ 660 661Partition info get call can originate: 662 663- from SP to SPMC 664- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 665 666FFA_ID_GET 667~~~~~~~~~~ 668 669The FF-A id space is split into a non-secure space and secure space: 670 671- FF-A ID with bit 15 clear relates to VMs. 672- FF-A ID with bit 15 set related to SPs. 673- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 674 and SPMC. 675 676The SPMD returns: 677 678- The default zero value on invocation from the Hypervisor. 679- The ``spmc_id`` value specified in the SPMC manifest on invocation from 680 the SPMC (see `SPMC manifest`_) 681 682This convention helps the SPMC to determine the origin and destination worlds in 683an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 684transactions in its world switch routine. It must not be permitted for a VM to 685use a secure FF-A ID as origin world by spoofing: 686 687- A VM-to-SP direct request/response shall set the origin world to be non-secure 688 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 689 set). 690- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 691 for both origin and destination IDs. 692 693An incoming direct message request arriving at SPMD from NWd is forwarded to 694SPMC without a specific check. The SPMC is resumed through eret and "knows" the 695message is coming from normal world in this specific code path. Thus the origin 696endpoint ID must be checked by SPMC for being a normal world ID. 697 698An SP sending a direct message request must have bit 15 set in its origin 699endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 700 701The SPMC shall reject the direct message if the claimed world in origin endpoint 702ID is not consistent: 703 704- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 705 world ID", 706- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 707 708 709FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 710~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 711 712This is a mandatory interface for secure partitions consisting in direct request 713and responses with the following rules: 714 715- An SP can send a direct request to another SP. 716- An SP can receive a direct request from another SP. 717- An SP can send a direct response to another SP. 718- An SP cannot send a direct request to an Hypervisor or OS kernel. 719- An Hypervisor or OS kernel can send a direct request to an SP. 720- An SP can send a direct response to an Hypervisor or OS kernel. 721 722SPMC-SPMD direct requests/responses 723----------------------------------- 724 725Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 726Using those IDs in source/destination fields of a direct request/response 727permits SPMD to SPMC communication and either way. 728 729- SPMC to SPMD direct request/response uses SMC conduit. 730- SPMD to SPMC direct request/response uses ERET conduit. 731 732PE MMU configuration 733-------------------- 734 735With secure virtualization enabled, two IPA spaces are output from the secure 736EL1&0 Stage-1 translation (secure and non-secure). The EL1&0 Stage-2 translation 737hardware is fed by: 738 739- A single secure IPA space when the SP EL1&0 Stage-1 MMU is disabled. 740- Two IPA spaces (secure and non-secure) when the SP EL1&0 Stage-1 MMU is 741 enabled. 742 743``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 744NS/S IPA translations. 745``VSTCR_EL2.SW`` = 0, ``VSTCR_EL2.SA`` = 0,``VTCR_EL2.NSW`` = 0, ``VTCR_EL2.NSA`` = 1: 746 747- Stage-2 translations for the NS IPA space access the NS PA space. 748- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 749 750Secure and non-secure IPA regions use the same set of Stage-2 page tables within 751a SP. 752 753Interrupt management 754-------------------- 755 756GIC ownership 757~~~~~~~~~~~~~ 758 759The SPMC owns the GIC configuration. Secure and non-secure interrupts are 760trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 761IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 762virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 763 764Non-secure interrupt handling 765~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 766 767The following illustrate the scenarios of non secure physical interrupts trapped 768by the SPMC: 769 770- The SP handles a managed exit operation: 771 772.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-managed-exit.png 773 774- The SP is pre-empted without managed exit: 775 776.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-sp-preemption.png 777 778Secure interrupt handling 779~~~~~~~~~~~~~~~~~~~~~~~~~ 780 781The current implementation does not support handling of secure interrupts 782trapped by the SPMC at S-EL2. This is work in progress planned for future 783releases. 784 785Power management 786---------------- 787 788In platforms with or without secure virtualization: 789 790- The NWd owns the platform PM policy. 791- The Hypervisor or OS kernel is the component initiating PSCI service calls. 792- The EL3 PSCI library is in charge of the PM coordination and control 793 (eventually writing to platform registers). 794- While coordinating PM events, the PSCI library calls backs into the Secure 795 Payload Dispatcher for events the latter has statically registered to. 796 797When using the SPMD as a Secure Payload Dispatcher: 798 799- A power management event is relayed through the SPD hook to the SPMC. 800- In the current implementation only cpu on (svc_on_finish) and cpu off 801 (svc_off) hooks are registered. 802- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 803 The SPMC is entered through its secondary physical core entry point. 804- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The method by which 805 the PM event is conveyed to the SPMC is implementation-defined in context of 806 FF-A v1.0 (`SPMC-SPMD direct requests/responses`_). It consists in a SPMD-to-SPMC 807 direct request/response conveying the PM event details and SPMC response. 808 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 809 updates its internal state to reflect the physical core is being turned off. 810 In the current implementation no SP is resumed as a consequence. This behavior 811 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 812 userspace. 813 814SMMUv3 support in Hafnium 815========================= 816 817An SMMU is analogous to an MMU in a CPU. It performs address translations for 818Direct Memory Access (DMA) requests from system I/O devices. 819The responsibilities of an SMMU include: 820 821- Translation: Incoming DMA requests are translated from bus address space to 822 system physical address space using translation tables compliant to 823 Armv8/Armv7 VMSA descriptor format. 824- Protection: An I/O device can be prohibited from read, write access to a 825 memory region or allowed. 826- Isolation: Traffic from each individial device can be independently managed. 827 The devices are differentiated from each other using unique translation 828 tables. 829 830The following diagram illustrates a typical SMMU IP integrated in a SoC with 831several I/O devices along with Interconnect and Memory system. 832 833.. image:: ../resources/diagrams/MMU-600.png 834 835SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 836support for SMMUv3 driver in both normal and secure world. A brief introduction 837of SMMUv3 functionality and the corresponding software support in Hafnium is 838provided here. 839 840SMMUv3 features 841--------------- 842 843- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 844 translation support. It can either bypass or abort incoming translations as 845 well. 846- Traffic (memory transactions) from each upstream I/O peripheral device, 847 referred to as Stream, can be independently managed using a combination of 848 several memory based configuration structures. This allows the SMMUv3 to 849 support a large number of streams with each stream assigned to a unique 850 translation context. 851- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 852 a Processing Element. AArch32(LPAE) and AArch64 translation table format 853 are supported by SMMUv3. 854- SMMUv3 offers non-secure stream support with secure stream support being 855 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 856 instance for secure and non-secure stream support. 857- It also supports sub-streams to differentiate traffic from a virtualized 858 peripheral associated with a VM/SP. 859- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 860 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 861 for providing Secure Stage2 translation support to upstream peripheral 862 devices. 863 864SMMUv3 Programming Interfaces 865----------------------------- 866 867SMMUv3 has three software interfaces that are used by the Hafnium driver to 868configure the behaviour of SMMUv3 and manage the streams. 869 870- Memory based data strutures that provide unique translation context for 871 each stream. 872- Memory based circular buffers for command queue and event queue. 873- A large number of SMMU configuration registers that are memory mapped during 874 boot time by Hafnium driver. Except a few registers, all configuration 875 registers have independent secure and non-secure versions to configure the 876 behaviour of SMMUv3 for translation of secure and non-secure streams 877 respectively. 878 879Peripheral device manifest 880-------------------------- 881 882Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 883These devices are dependent on PE endpoint to initiate and receive memory 884management transactions on their behalf. The acccess to the MMIO regions of 885any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 886uses the same stage 2 translations for the device as those used by partition 887manager on behalf of the PE endpoint. This ensures that the peripheral device 888has the same visibility of the physical address space as the endpoint. The 889device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 890) must specify these additional properties for each peripheral device in the 891system : 892 893- smmu-id: This field helps to identify the SMMU instance that this device is 894 upstream of. 895- stream-ids: List of stream IDs assigned to this device. 896 897.. code:: shell 898 899 smmuv3-testengine { 900 base-address = <0x00000000 0x2bfe0000>; 901 pages-count = <32>; 902 attributes = <0x3>; 903 smmu-id = <0>; 904 stream-ids = <0x0 0x1>; 905 interrupts = <0x2 0x3>, <0x4 0x5>; 906 exclusive-access; 907 }; 908 909SMMUv3 driver limitations 910------------------------- 911 912The primary design goal for the Hafnium SMMU driver is to support secure 913streams. 914 915- Currently, the driver only supports Stage2 translations. No support for 916 Stage1 or nested translations. 917- Supports only AArch64 translation format. 918- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 919 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 920- No support for independent peripheral devices. 921 922References 923========== 924 925.. _[1]: 926 927[1] `Arm Firmware Framework for Arm A-profile <https://developer.arm.com/docs/den0077/latest>`__ 928 929.. _[2]: 930 931[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 932 933.. _[3]: 934 935[3] `Trusted Boot Board Requirements 936Client <https://developer.arm.com/documentation/den0006/d/>`__ 937 938.. _[4]: 939 940[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 941 942.. _[5]: 943 944[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 945 946.. _[6]: 947 948[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 949 950.. _[7]: 951 952[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 953 954.. _[8]: 955 956[8] https://lists.trustedfirmware.org/pipermail/tf-a/2020-February/000296.html 957 958.. _[9]: 959 960[9] https://trustedfirmware-a.readthedocs.io/en/latest/design/firmware-design.html#dynamic-configuration-during-cold-boot 961 962-------------- 963 964*Copyright (c) 2020-2021, Arm Limited and Contributors. All rights reserved.* 965
