1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+-----------------------------------+ 10| CoT | Chain of Trust | 11+--------+-----------------------------------+ 12| DMA | Direct Memory Access | 13+--------+-----------------------------------+ 14| DTB | Device Tree Blob | 15+--------+-----------------------------------+ 16| DTS | Device Tree Source | 17+--------+-----------------------------------+ 18| EC | Execution Context | 19+--------+-----------------------------------+ 20| FIP | Firmware Image Package | 21+--------+-----------------------------------+ 22| FF-A | Firmware Framework for Armv8-A | 23+--------+-----------------------------------+ 24| IPA | Intermediate Physical Address | 25+--------+-----------------------------------+ 26| NWd | Normal World | 27+--------+-----------------------------------+ 28| ODM | Original Design Manufacturer | 29+--------+-----------------------------------+ 30| OEM | Original Equipment Manufacturer | 31+--------+-----------------------------------+ 32| PA | Physical Address | 33+--------+-----------------------------------+ 34| PE | Processing Element | 35+--------+-----------------------------------+ 36| PM | Power Management | 37+--------+-----------------------------------+ 38| PVM | Primary VM | 39+--------+-----------------------------------+ 40| SMMU | System Memory Management Unit | 41+--------+-----------------------------------+ 42| SP | Secure Partition | 43+--------+-----------------------------------+ 44| SPD | Secure Payload Dispatcher | 45+--------+-----------------------------------+ 46| SPM | Secure Partition Manager | 47+--------+-----------------------------------+ 48| SPMC | SPM Core | 49+--------+-----------------------------------+ 50| SPMD | SPM Dispatcher | 51+--------+-----------------------------------+ 52| SiP | Silicon Provider | 53+--------+-----------------------------------+ 54| SWd | Secure World | 55+--------+-----------------------------------+ 56| TLV | Tag-Length-Value | 57+--------+-----------------------------------+ 58| TOS | Trusted Operating System | 59+--------+-----------------------------------+ 60| VM | Virtual Machine | 61+--------+-----------------------------------+ 62 63Foreword 64======== 65 66Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 67 68- SPM based on the FF-A specification `[1]`_. 69- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 70 71Both implementations differ in their architectures and only one can be selected 72at build time. 73 74This document: 75 76- describes the FF-A implementation where the Secure Partition Manager 77 resides at EL3 and S-EL2 (or EL3 and S-EL1). 78- is not an architecture specification and it might provide assumptions 79 on sections mandated as implementation-defined in the specification. 80- covers the implications to TF-A used as a bootloader, and Hafnium 81 used as a reference code base for an S-EL2 secure firmware on 82 platforms implementing the FEAT_SEL2 (formerly Armv8.4 Secure EL2) 83 architecture extension. 84 85Terminology 86----------- 87 88- The term Hypervisor refers to the NS-EL2 component managing Virtual Machines 89 (or partitions) in the normal world. 90- The term SPMC refers to the S-EL2 component managing secure partitions in 91 the secure world when the FEAT_SEL2 architecture extension is implemented. 92- Alternatively, SPMC can refer to an S-EL1 component, itself being a secure 93 partition and implementing the FF-A ABI on platforms not implementing the 94 FEAT_SEL2 architecture extension. 95- The term VM refers to a normal world Virtual Machine managed by an Hypervisor. 96- The term SP refers to a secure world "Virtual Machine" managed by an SPMC. 97 98Support for legacy platforms 99---------------------------- 100 101In the implementation, the SPM is split into SPMD and SPMC components. 102The SPMD is located at EL3 and mainly relays FF-A messages from 103NWd (Hypervisor or OS kernel) to SPMC located either at S-EL1 or S-EL2. 104 105Hence TF-A supports both cases where the SPMC is located either at: 106 107- S-EL1 supporting platforms not implementing the FEAT_SEL2 architecture 108 extension. The SPMD relays the FF-A protocol from EL3 to S-EL1. 109- or S-EL2 supporting platforms implementing the FEAT_SEL2 architecture 110 extension. The SPMD relays the FF-A protocol from EL3 to S-EL2. 111 112The same TF-A SPMD component is used to support both configurations. 113The SPMC exception level is a build time choice. 114 115Sample reference stack 116====================== 117 118The following diagram illustrates a possible configuration when the 119FEAT_SEL2 architecture extension is implemented, showing the SPMD 120and SPMC, one or multiple secure partitions, with an optional 121Hypervisor: 122 123.. image:: ../resources/diagrams/ff-a-spm-sel2.png 124 125TF-A build options 126================== 127 128This section explains the TF-A build options involved in building with 129support for an FF-A based SPM where the SPMD is located at EL3 and the 130SPMC located at S-EL1 or S-EL2: 131 132- **SPD=spmd**: this option selects the SPMD component to relay the FF-A 133 protocol from NWd to SWd back and forth. It is not possible to 134 enable another Secure Payload Dispatcher when this option is chosen. 135- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC exception 136 level to being S-EL1 or S-EL2. It defaults to enabled (value 1) when 137 SPD=spmd is chosen. 138- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 139 restoring) the EL2 system register context before entering (resp. 140 after leaving) the SPMC. It is mandatorily enabled when 141 ``SPMD_SPM_AT_SEL2`` is enabled. The context save/restore routine 142 and exhaustive list of registers is visible at `[4]`_. 143- **SP_LAYOUT_FILE**: this option specifies a text description file 144 providing paths to SP binary images and manifests in DTS format 145 (see `Describing secure partitions`_). It 146 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 147 secure partitions are to be loaded on behalf of the SPMC. 148 149+---------------+----------------------+------------------+ 150| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | 151+---------------+----------------------+------------------+ 152| SPMC at S-EL1 | 0 | 0 | 153+---------------+----------------------+------------------+ 154| SPMC at S-EL2 | 1 | 1 (default when | 155| | | SPD=spmd) | 156+---------------+----------------------+------------------+ 157 158Other combinations of such build options either break the build or are not 159supported. 160 161Notes: 162 163- Only Arm's FVP platform is supported to use with the TF-A reference software 164 stack. 165- The reference software stack uses FEAT_PAuth (formerly Armv8.3-PAuth) and 166 FEAT_BTI (formerly Armv8.5-BTI) architecture extensions by default at EL3 167 and S-EL2. 168- The ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 169 barely saving/restoring EL2 registers from an Arm arch perspective. As such 170 it is decoupled from the ``SPD=spmd`` option. 171- BL32 option is re-purposed to specify the SPMC image. It can specify either 172 the Hafnium binary path (built for the secure world) or the path to a TEE 173 binary implementing FF-A interfaces. 174- BL33 option can specify the TFTF binary or a normal world loader 175 such as U-Boot or the UEFI framework. 176 177Sample TF-A build command line when SPMC is located at S-EL1 178(e.g. when the FEAT_EL2 architecture extension is not implemented): 179 180.. code:: shell 181 182 make \ 183 CROSS_COMPILE=aarch64-none-elf- \ 184 SPD=spmd \ 185 SPMD_SPM_AT_SEL2=0 \ 186 BL32=<path-to-tee-binary> \ 187 BL33=<path-to-bl33-binary> \ 188 PLAT=fvp \ 189 all fip 190 191Sample TF-A build command line for a FEAT_SEL2 enabled system where the SPMC is 192located at S-EL2: 193 194.. code:: shell 195 196 make \ 197 CROSS_COMPILE=aarch64-none-elf- \ 198 PLAT=fvp \ 199 SPD=spmd \ 200 CTX_INCLUDE_EL2_REGS=1 \ 201 ARM_ARCH_MINOR=5 \ 202 BRANCH_PROTECTION=1 \ 203 CTX_INCLUDE_PAUTH_REGS=1 \ 204 BL32=<path-to-hafnium-binary> \ 205 BL33=<path-to-bl33-binary> \ 206 SP_LAYOUT_FILE=sp_layout.json \ 207 all fip 208 209Same as above with enabling secure boot in addition: 210 211.. code:: shell 212 213 make \ 214 CROSS_COMPILE=aarch64-none-elf- \ 215 PLAT=fvp \ 216 SPD=spmd \ 217 CTX_INCLUDE_EL2_REGS=1 \ 218 ARM_ARCH_MINOR=5 \ 219 BRANCH_PROTECTION=1 \ 220 CTX_INCLUDE_PAUTH_REGS=1 \ 221 BL32=<path-to-hafnium-binary> \ 222 BL33=<path-to-bl33-binary> \ 223 SP_LAYOUT_FILE=sp_layout.json \ 224 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 225 TRUSTED_BOARD_BOOT=1 \ 226 COT=dualroot \ 227 ARM_ROTPK_LOCATION=devel_rsa \ 228 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 229 GENERATE_COT=1 \ 230 all fip 231 232FVP model invocation 233==================== 234 235The FVP command line needs the following options to exercise the S-EL2 SPMC: 236 237+---------------------------------------------------+------------------------------------+ 238| - cluster0.has_arm_v8-5=1 | Implements FEAT_SEL2, FEAT_PAuth, | 239| - cluster1.has_arm_v8-5=1 | and FEAT_BTI. | 240+---------------------------------------------------+------------------------------------+ 241| - pci.pci_smmuv3.mmu.SMMU_AIDR=2 | Parameters required for the | 242| - pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B | SMMUv3.2 modeling. | 243| - pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 | | 244| - pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 | | 245| - pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 | | 246| - pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 | | 247| - pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 | | 248| - pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 | | 249+---------------------------------------------------+------------------------------------+ 250| - cluster0.has_branch_target_exception=1 | Implements FEAT_BTI. | 251| - cluster1.has_branch_target_exception=1 | | 252+---------------------------------------------------+------------------------------------+ 253| - cluster0.restriction_on_speculative_execution=2 | Required by the EL2 context | 254| - cluster1.restriction_on_speculative_execution=2 | save/restore routine. | 255+---------------------------------------------------+------------------------------------+ 256 257Sample FVP command line invocation: 258 259.. code:: shell 260 261 <path-to-fvp-model>/FVP_Base_RevC-2xAEMv8A -C pctl.startup=0.0.0.0 262 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -C bp.secure_memory=1 \ 263 -C bp.secureflashloader.fname=trusted-firmware-a/build/fvp/debug/bl1.bin \ 264 -C bp.flashloader0.fname=trusted-firmware-a/build/fvp/debug/fip.bin \ 265 -C bp.pl011_uart0.out_file=fvp-uart0.log -C bp.pl011_uart1.out_file=fvp-uart1.log \ 266 -C bp.pl011_uart2.out_file=fvp-uart2.log \ 267 -C cluster0.has_arm_v8-5=1 -C cluster1.has_arm_v8-5=1 -C pci.pci_smmuv3.mmu.SMMU_AIDR=2 \ 268 -C pci.pci_smmuv3.mmu.SMMU_IDR0=0x0046123B -C pci.pci_smmuv3.mmu.SMMU_IDR1=0x00600002 \ 269 -C pci.pci_smmuv3.mmu.SMMU_IDR3=0x1714 -C pci.pci_smmuv3.mmu.SMMU_IDR5=0xFFFF0472 \ 270 -C pci.pci_smmuv3.mmu.SMMU_S_IDR1=0xA0000002 -C pci.pci_smmuv3.mmu.SMMU_S_IDR2=0 \ 271 -C pci.pci_smmuv3.mmu.SMMU_S_IDR3=0 \ 272 -C cluster0.has_branch_target_exception=1 \ 273 -C cluster1.has_branch_target_exception=1 \ 274 -C cluster0.restriction_on_speculative_execution=2 \ 275 -C cluster1.restriction_on_speculative_execution=2 276 277Boot process 278============ 279 280Loading Hafnium and secure partitions in the secure world 281--------------------------------------------------------- 282 283TF-A BL2 is the bootlader for the SPMC and SPs in the secure world. 284 285SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 286Thus they are supplied as distinct signed entities within the FIP flash 287image. The FIP image itself is not signed hence this provides the ability 288to upgrade SPs in the field. 289 290Booting through TF-A 291-------------------- 292 293SP manifests 294~~~~~~~~~~~~ 295 296An SP manifest describes SP attributes as defined in `[1]`_ 297(partition manifest at virtual FF-A instance) in DTS format. It is 298represented as a single file associated with the SP. A sample is 299provided by `[5]`_. A binding document is provided by `[6]`_. 300 301Secure Partition packages 302~~~~~~~~~~~~~~~~~~~~~~~~~ 303 304Secure partitions are bundled as independent package files consisting 305of: 306 307- a header 308- a DTB 309- an image payload 310 311The header starts with a magic value and offset values to SP DTB and 312image payload. Each SP package is loaded independently by BL2 loader 313and verified for authenticity and integrity. 314 315The SP package identified by its UUID (matching FF-A uuid property) is 316inserted as a single entry into the FIP at end of the TF-A build flow 317as shown: 318 319.. code:: shell 320 321 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 322 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 323 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 324 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 325 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 326 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 327 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 328 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 329 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 330 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 331 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 332 333.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 334 335Describing secure partitions 336~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 337 338A json-formatted description file is passed to the build flow specifying paths 339to the SP binary image and associated DTS partition manifest file. The latter 340is processed by the dtc compiler to generate a DTB fed into the SP package. 341This file also specifies the SP owner (as an optional field) identifying the 342signing domain in case of dual root CoT. 343The SP owner can either be the silicon or the platform provider. The 344corresponding "owner" field value can either take the value of "SiP" or "Plat". 345In absence of "owner" field, it defaults to "SiP" owner. 346 347.. code:: shell 348 349 { 350 "tee1" : { 351 "image": "tee1.bin", 352 "pm": "tee1.dts", 353 "owner": "SiP" 354 }, 355 356 "tee2" : { 357 "image": "tee2.bin", 358 "pm": "tee2.dts", 359 "owner": "Plat" 360 } 361 } 362 363SPMC manifest 364~~~~~~~~~~~~~ 365 366This manifest contains the SPMC *attribute* node consumed by the SPMD at boot 367time. It implements `[1]`_ (SP manifest at physical FF-A instance) and serves 368two different cases: 369 370- The SPMC resides at S-EL1: the SPMC manifest is used by the SPMD to setup a 371 SP that co-resides with the SPMC and executes at S-EL1 or Secure Supervisor 372 mode. 373- The SPMC resides at S-EL2: the SPMC manifest is used by the SPMD to setup 374 the environment required by the SPMC to run at S-EL2. SPs run at S-EL1 or 375 S-EL0. 376 377.. code:: shell 378 379 attribute { 380 spmc_id = <0x8000>; 381 maj_ver = <0x1>; 382 min_ver = <0x0>; 383 exec_state = <0x0>; 384 load_address = <0x0 0x6000000>; 385 entrypoint = <0x0 0x6000000>; 386 binary_size = <0x60000>; 387 }; 388 389- *spmc_id* defines the endpoint ID value that SPMC can query through 390 ``FFA_ID_GET``. 391- *maj_ver/min_ver*. SPMD checks provided version versus its internal 392 version and aborts if not matching. 393- *exec_state* defines the SPMC execution state (AArch64 or AArch32). 394 Notice Hafnium used as a SPMC only supports AArch64. 395- *load_address* and *binary_size* are mostly used to verify secondary 396 entry points fit into the loaded binary image. 397- *entrypoint* defines the cold boot primary core entry point used by 398 SPMD (currently matches ``BL32_BASE``) to enter the SPMC. 399 400Other nodes in the manifest are consumed by Hafnium in the secure world. 401A sample can be found at [7]: 402 403- The *hypervisor* node describes SPs. *is_ffa_partition* boolean attribute 404 indicates a FF-A compliant SP. The *load_address* field specifies the load 405 address at which TF-A loaded the SP package. 406- *cpus* node provide the platform topology and allows MPIDR to VMPIDR mapping. 407 Note the primary core is declared first, then secondary core are declared 408 in reverse order. 409- The *memory* node provides platform information on the ranges of memory 410 available to the SPMC. 411 412SPMC boot 413~~~~~~~~~ 414 415The SPMC is loaded by BL2 as the BL32 image. 416 417The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image. 418 419BL2 passes the SPMC manifest address to BL31 through a register. 420 421At boot time, the SPMD in BL31 runs from the primary core, initializes the core 422contexts and launches the SPMC (BL32) passing the SPMC manifest address through 423a register. 424 425Loading of SPs 426~~~~~~~~~~~~~~ 427 428At boot time, BL2 loads SPs sequentially in addition to the SPMC as depicted 429below: 430 431.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 432 433Note this boot flow is an implementation sample on Arm's FVP platform. 434Platforms not using TF-A's *Firmware CONFiguration* framework would adjust to a 435different implementation. 436 437Secure boot 438~~~~~~~~~~~ 439 440The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 441SPMC manifest, secure partitions and verifies them for authenticity and integrity. 442Refer to TBBR specification `[3]`_. 443 444The multiple-signing domain feature (in current state dual signing domain `[8]`_) allows 445the use of two root keys namely S-ROTPK and NS-ROTPK: 446 447- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 448- BL33 may be signed by the OEM using NS-ROTPK. 449- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 450 451Also refer to `Describing secure partitions`_ and `TF-A build options`_ sections. 452 453Hafnium in the secure world 454=========================== 455 456General considerations 457---------------------- 458 459Build platform for the secure world 460~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 461 462In the Hafnium reference implementation specific code parts are only relevant to 463the secure world. Such portions are isolated in architecture specific files 464and/or enclosed by a ``SECURE_WORLD`` macro. 465 466Secure partitions CPU scheduling 467~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 468 469The FF-A v1.0 specification `[1]`_ provides two ways to relinquinsh CPU time to 470secure partitions. For this a VM (Hypervisor or OS kernel), or SP invokes one of: 471 472- the FFA_MSG_SEND_DIRECT_REQ interface. 473- the FFA_RUN interface. 474 475Platform topology 476~~~~~~~~~~~~~~~~~ 477 478The *execution-ctx-count* SP manifest field can take the value of one or the 479total number of PEs. The FF-A v1.0 specification `[1]`_ recommends the 480following SP types: 481 482- Pinned MP SPs: an execution context matches a physical PE. MP SPs must 483 implement the same number of ECs as the number of PEs in the platform. 484- Migratable UP SPs: a single execution context can run and be migrated on any 485 physical PE. Such SP declares a single EC in its SP manifest. An UP SP can 486 receive a direct message request originating from any physical core targeting 487 the single execution context. 488 489Parsing SP partition manifests 490------------------------------ 491 492Hafnium consumes SP manifests as defined in `[1]`_ and `SP manifests`_. 493Note the current implementation may not implement all optional fields. 494 495The SP manifest may contain memory and device regions nodes. In case of 496an S-EL2 SPMC: 497 498- Memory regions are mapped in the SP EL1&0 Stage-2 translation regime at 499 load time (or EL1&0 Stage-1 for an S-EL1 SPMC). A memory region node can 500 specify RX/TX buffer regions in which case it is not necessary for an SP 501 to explicitly invoke the ``FFA_RXTX_MAP`` interface. 502- Device regions are mapped in the SP EL1&0 Stage-2 translation regime (or 503 EL1&0 Stage-1 for an S-EL1 SPMC) as peripherals and possibly allocate 504 additional resources (e.g. interrupts). 505 506For the S-EL2 SPMC, base addresses for memory and device region nodes are IPAs 507provided the SPMC identity maps IPAs to PAs within SP EL1&0 Stage-2 translation 508regime. 509 510Note: in the current implementation both VTTBR_EL2 and VSTTBR_EL2 point to the 511same set of page tables. It is still open whether two sets of page tables shall 512be provided per SP. The memory region node as defined in the specification 513provides a memory security attribute hinting to map either to the secure or 514non-secure EL1&0 Stage-2 table if it exists. 515 516Passing boot data to the SP 517--------------------------- 518 519In `[1]`_ , the "Protocol for passing data" section defines a method for passing 520boot data to SPs (not currently implemented). 521 522Provided that the whole secure partition package image (see 523`Secure Partition packages`_) is mapped to the SP secure EL1&0 Stage-2 524translation regime, an SP can access its own manifest DTB blob and extract its 525partition manifest properties. 526 527SP Boot order 528------------- 529 530SP manifests provide an optional boot order attribute meant to resolve 531dependencies such as an SP providing a service required to properly boot 532another SP. 533 534It is possible for an SP to call into another SP through a direct request 535provided the latter SP has already been booted. 536 537Boot phases 538----------- 539 540Primary core boot-up 541~~~~~~~~~~~~~~~~~~~~ 542 543Upon boot-up, BL31 hands over to the SPMC (BL32) on the primary boot physical 544core. The SPMC performs its platform initializations and registers the SPMC 545secondary physical core entry point physical address by the use of the 546FFA_SECONDARY_EP_REGISTER interface (SMC invocation from the SPMC to the SPMD 547at secure physical FF-A instance). This interface is implementation-defined in 548context of FF-A v1.0. 549 550The SPMC then creates secure partitions based on SP packages and manifests. Each 551secure partition is launched in sequence (`SP Boot order`_) on their "primary" 552execution context. If the primary boot physical core linear id is N, an MP SP is 553started using EC[N] on PE[N] (see `Platform topology`_). If the partition is a 554UP SP, it is started using its unique EC0 on PE[N]. 555 556The SP primary EC (or the EC used when the partition is booted as described 557above): 558 559- Performs the overall SP boot time initialization, and in case of a MP SP, 560 prepares the SP environment for other execution contexts. 561- In the case of a MP SP, it invokes the FFA_SECONDARY_EP_REGISTER at secure 562 virtual FF-A instance (SMC invocation from SP to SPMC) to provide the IPA 563 entry point for other execution contexts. 564- Exits through ``FFA_MSG_WAIT`` to indicate successful initialization or 565 ``FFA_ERROR`` in case of failure. 566 567Secondary cores boot-up 568~~~~~~~~~~~~~~~~~~~~~~~ 569 570Once the system is started and NWd brought up, a secondary physical core is 571woken up by the ``PSCI_CPU_ON`` service invocation. The TF-A SPD hook mechanism 572calls into the SPMD on the newly woken up physical core. Then the SPMC is 573entered at the secondary physical core entry point. 574 575In the current implementation, the first SP is resumed on the coresponding EC 576(the virtual CPU which matches the physical core). The implication is that the 577first SP must be a MP SP. 578 579In a linux based system, once secure and normal worlds are booted but prior to 580a NWd FF-A driver has been loaded: 581 582- The first SP has initialized all its ECs in response to primary core boot up 583 (at system initialization) and secondary core boot up (as a result of linux 584 invoking PSCI_CPU_ON for all secondary cores). 585- Other SPs have their first execution context initialized as a result of secure 586 world initialization on the primary boot core. Other ECs for those SPs have to 587 be run first through ffa_run to complete their initialization (which results 588 in the EC completing with FFA_MSG_WAIT). 589 590Refer to `Power management`_ for further details. 591 592Mandatory interfaces 593-------------------- 594 595The following interfaces are exposed to SPs: 596 597- ``FFA_VERSION`` 598- ``FFA_FEATURES`` 599- ``FFA_RX_RELEASE`` 600- ``FFA_RXTX_MAP`` 601- ``FFA_RXTX_UNMAP`` (not implemented) 602- ``FFA_PARTITION_INFO_GET`` 603- ``FFA_ID_GET`` 604- ``FFA_MSG_WAIT`` 605- ``FFA_MSG_SEND_DIRECT_REQ`` 606- ``FFA_MSG_SEND_DIRECT_RESP`` 607- ``FFA_MEM_DONATE`` 608- ``FFA_MEM_LEND`` 609- ``FFA_MEM_SHARE`` 610- ``FFA_MEM_RETRIEVE_REQ`` 611- ``FFA_MEM_RETRIEVE_RESP`` 612- ``FFA_MEM_RELINQUISH`` 613- ``FFA_MEM_RECLAIM`` 614- ``FFA_SECONDARY_EP_REGISTER`` 615 616FFA_VERSION 617~~~~~~~~~~~ 618 619``FFA_VERSION`` requires a *requested_version* parameter from the caller. 620The returned value depends on the caller: 621 622- Hypervisor or OS kernel in NS-EL1/EL2: the SPMD returns the SPMC version 623 specified in the SPMC manifest. 624- SP: the SPMC returns its own implemented version. 625- SPMC at S-EL1/S-EL2: the SPMD returns its own implemented version. 626 627FFA_FEATURES 628~~~~~~~~~~~~ 629 630FF-A features supported by the SPMC may be discovered by secure partitions at 631boot (that is prior to NWd is booted) or run-time. 632 633The SPMC calling FFA_FEATURES at secure physical FF-A instance always get 634FFA_SUCCESS from the SPMD. 635 636The request made by an Hypervisor or OS kernel is forwarded to the SPMC and 637the response relayed back to the NWd. 638 639FFA_RXTX_MAP/FFA_RXTX_UNMAP 640~~~~~~~~~~~~~~~~~~~~~~~~~~~ 641 642When invoked from a secure partition FFA_RXTX_MAP maps the provided send and 643receive buffers described by their IPAs to the SP EL1&0 Stage-2 translation 644regime as secure buffers in the MMU descriptors. 645 646When invoked from the Hypervisor or OS kernel, the buffers are mapped into the 647SPMC EL2 Stage-1 translation regime and marked as NS buffers in the MMU 648descriptors. 649 650Note: 651 652- FFA_RXTX_UNMAP is not implemented. 653 654FFA_PARTITION_INFO_GET 655~~~~~~~~~~~~~~~~~~~~~~ 656 657Partition info get call can originate: 658 659- from SP to SPMC 660- from Hypervisor or OS kernel to SPMC. The request is relayed by the SPMD. 661 662FFA_ID_GET 663~~~~~~~~~~ 664 665The FF-A id space is split into a non-secure space and secure space: 666 667- FF-A ID with bit 15 clear relates to VMs. 668- FF-A ID with bit 15 set related to SPs. 669- FF-A IDs 0, 0xffff, 0x8000 are assigned respectively to the Hypervisor, SPMD 670 and SPMC. 671 672The SPMD returns: 673 674- The default zero value on invocation from the Hypervisor. 675- The ``spmc_id`` value specified in the SPMC manifest on invocation from 676 the SPMC (see `SPMC manifest`_) 677 678This convention helps the SPMC to determine the origin and destination worlds in 679an FF-A ABI invocation. In particular the SPMC shall filter unauthorized 680transactions in its world switch routine. It must not be permitted for a VM to 681use a secure FF-A ID as origin world by spoofing: 682 683- A VM-to-SP direct request/response shall set the origin world to be non-secure 684 (FF-A ID bit 15 clear) and destination world to be secure (FF-A ID bit 15 685 set). 686- Similarly, an SP-to-SP direct request/response shall set the FF-A ID bit 15 687 for both origin and destination IDs. 688 689An incoming direct message request arriving at SPMD from NWd is forwarded to 690SPMC without a specific check. The SPMC is resumed through eret and "knows" the 691message is coming from normal world in this specific code path. Thus the origin 692endpoint ID must be checked by SPMC for being a normal world ID. 693 694An SP sending a direct message request must have bit 15 set in its origin 695endpoint ID and this can be checked by the SPMC when the SP invokes the ABI. 696 697The SPMC shall reject the direct message if the claimed world in origin endpoint 698ID is not consistent: 699 700- It is either forwarded by SPMD and thus origin endpoint ID must be a "normal 701 world ID", 702- or initiated by an SP and thus origin endpoint ID must be a "secure world ID". 703 704 705FFA_MSG_SEND_DIRECT_REQ/FFA_MSG_SEND_DIRECT_RESP 706~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 707 708This is a mandatory interface for secure partitions consisting in direct request 709and responses with the following rules: 710 711- An SP can send a direct request to another SP. 712- An SP can receive a direct request from another SP. 713- An SP can send a direct response to another SP. 714- An SP cannot send a direct request to an Hypervisor or OS kernel. 715- An Hypervisor or OS kernel can send a direct request to an SP. 716- An SP can send a direct response to an Hypervisor or OS kernel. 717 718SPMC-SPMD direct requests/responses 719----------------------------------- 720 721Implementation-defined FF-A IDs are allocated to the SPMC and SPMD. 722Using those IDs in source/destination fields of a direct request/response 723permits SPMD to SPMC communication and either way. 724 725- SPMC to SPMD direct request/response uses SMC conduit. 726- SPMD to SPMC direct request/response uses ERET conduit. 727 728PE MMU configuration 729-------------------- 730 731With secure virtualization enabled, two IPA spaces are output from the secure 732EL1&0 Stage-1 translation (secure and non-secure). The EL1&0 Stage-2 translation 733hardware is fed by: 734 735- A single secure IPA space when the SP EL1&0 Stage-1 MMU is disabled. 736- Two IPA spaces (secure and non-secure) when the SP EL1&0 Stage-1 MMU is 737 enabled. 738 739``VTCR_EL2`` and ``VSTCR_EL2`` provide configuration bits for controlling the 740NS/S IPA translations. 741``VSTCR_EL2.SW`` = 0, ``VSTCR_EL2.SA`` = 0,``VTCR_EL2.NSW`` = 0, ``VTCR_EL2.NSA`` = 1: 742 743- Stage-2 translations for the NS IPA space access the NS PA space. 744- Stage-2 translation table walks for the NS IPA space are to the secure PA space. 745 746Secure and non-secure IPA regions use the same set of Stage-2 page tables within 747a SP. 748 749Interrupt management 750-------------------- 751 752GIC ownership 753~~~~~~~~~~~~~ 754 755The SPMC owns the GIC configuration. Secure and non-secure interrupts are 756trapped at S-EL2. The SPMC manages interrupt resources and allocates interrupt 757IDs based on SP manifests. The SPMC acknowledges physical interrupts and injects 758virtual interrupts by setting the use of vIRQ/vFIQ bits before resuming a SP. 759 760Non-secure interrupt handling 761~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 762 763The following illustrate the scenarios of non secure physical interrupts trapped 764by the SPMC: 765 766- The SP handles a managed exit operation: 767 768.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-managed-exit.png 769 770- The SP is pre-empted without managed exit: 771 772.. image:: ../resources/diagrams/ffa-ns-interrupt-handling-sp-preemption.png 773 774Secure interrupt handling 775~~~~~~~~~~~~~~~~~~~~~~~~~ 776 777The current implementation does not support handling of secure interrupts 778trapped by the SPMC at S-EL2. This is work in progress planned for future 779releases. 780 781Power management 782---------------- 783 784In platforms with or without secure virtualization: 785 786- The NWd owns the platform PM policy. 787- The Hypervisor or OS kernel is the component initiating PSCI service calls. 788- The EL3 PSCI library is in charge of the PM coordination and control 789 (eventually writing to platform registers). 790- While coordinating PM events, the PSCI library calls backs into the Secure 791 Payload Dispatcher for events the latter has statically registered to. 792 793When using the SPMD as a Secure Payload Dispatcher: 794 795- A power management event is relayed through the SPD hook to the SPMC. 796- In the current implementation only cpu on (svc_on_finish) and cpu off 797 (svc_off) hooks are registered. 798- The behavior for the cpu on event is described in `Secondary cores boot-up`_. 799 The SPMC is entered through its secondary physical core entry point. 800- The cpu off event occurs when the NWd calls PSCI_CPU_OFF. The method by which 801 the PM event is conveyed to the SPMC is implementation-defined in context of 802 FF-A v1.0 (`SPMC-SPMD direct requests/responses`_). It consists in a SPMD-to-SPMC 803 direct request/response conveying the PM event details and SPMC response. 804 The SPMD performs a synchronous entry into the SPMC. The SPMC is entered and 805 updates its internal state to reflect the physical core is being turned off. 806 In the current implementation no SP is resumed as a consequence. This behavior 807 ensures a minimal support for CPU hotplug e.g. when initiated by the NWd linux 808 userspace. 809 810SMMUv3 support in Hafnium 811========================= 812 813An SMMU is analogous to an MMU in a CPU. It performs address translations for 814Direct Memory Access (DMA) requests from system I/O devices. 815The responsibilities of an SMMU include: 816 817- Translation: Incoming DMA requests are translated from bus address space to 818 system physical address space using translation tables compliant to 819 Armv8/Armv7 VMSA descriptor format. 820- Protection: An I/O device can be prohibited from read, write access to a 821 memory region or allowed. 822- Isolation: Traffic from each individial device can be independently managed. 823 The devices are differentiated from each other using unique translation 824 tables. 825 826The following diagram illustrates a typical SMMU IP integrated in a SoC with 827several I/O devices along with Interconnect and Memory system. 828 829.. image:: ../resources/diagrams/MMU-600.png 830 831SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 832support for SMMUv3 driver in both normal and secure world. A brief introduction 833of SMMUv3 functionality and the corresponding software support in Hafnium is 834provided here. 835 836SMMUv3 features 837--------------- 838 839- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 840 translation support. It can either bypass or abort incoming translations as 841 well. 842- Traffic (memory transactions) from each upstream I/O peripheral device, 843 referred to as Stream, can be independently managed using a combination of 844 several memory based configuration structures. This allows the SMMUv3 to 845 support a large number of streams with each stream assigned to a unique 846 translation context. 847- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 848 a Processing Element. AArch32(LPAE) and AArch64 translation table format 849 are supported by SMMUv3. 850- SMMUv3 offers non-secure stream support with secure stream support being 851 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 852 instance for secure and non-secure stream support. 853- It also supports sub-streams to differentiate traffic from a virtualized 854 peripheral associated with a VM/SP. 855- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 856 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 857 for providing Secure Stage2 translation support to upstream peripheral 858 devices. 859 860SMMUv3 Programming Interfaces 861----------------------------- 862 863SMMUv3 has three software interfaces that are used by the Hafnium driver to 864configure the behaviour of SMMUv3 and manage the streams. 865 866- Memory based data strutures that provide unique translation context for 867 each stream. 868- Memory based circular buffers for command queue and event queue. 869- A large number of SMMU configuration registers that are memory mapped during 870 boot time by Hafnium driver. Except a few registers, all configuration 871 registers have independent secure and non-secure versions to configure the 872 behaviour of SMMUv3 for translation of secure and non-secure streams 873 respectively. 874 875Peripheral device manifest 876-------------------------- 877 878Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 879These devices are dependent on PE endpoint to initiate and receive memory 880management transactions on their behalf. The acccess to the MMIO regions of 881any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 882uses the same stage 2 translations for the device as those used by partition 883manager on behalf of the PE endpoint. This ensures that the peripheral device 884has the same visibility of the physical address space as the endpoint. The 885device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 886) must specify these additional properties for each peripheral device in the 887system : 888 889- smmu-id: This field helps to identify the SMMU instance that this device is 890 upstream of. 891- stream-ids: List of stream IDs assigned to this device. 892 893.. code:: shell 894 895 smmuv3-testengine { 896 base-address = <0x00000000 0x2bfe0000>; 897 pages-count = <32>; 898 attributes = <0x3>; 899 smmu-id = <0>; 900 stream-ids = <0x0 0x1>; 901 interrupts = <0x2 0x3>, <0x4 0x5>; 902 exclusive-access; 903 }; 904 905SMMUv3 driver limitations 906------------------------- 907 908The primary design goal for the Hafnium SMMU driver is to support secure 909streams. 910 911- Currently, the driver only supports Stage2 translations. No support for 912 Stage1 or nested translations. 913- Supports only AArch64 translation format. 914- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 915 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 916- No support for independent peripheral devices. 917 918References 919========== 920 921.. _[1]: 922 923[1] `Arm Firmware Framework for Armv8-A <https://developer.arm.com/docs/den0077/latest>`__ 924 925.. _[2]: 926 927[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 928 929.. _[3]: 930 931[3] `Trusted Boot Board Requirements 932Client <https://developer.arm.com/documentation/den0006/d/>`__ 933 934.. _[4]: 935 936[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 937 938.. _[5]: 939 940[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm/fvp/fdts/cactus.dts 941 942.. _[6]: 943 944[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 945 946.. _[7]: 947 948[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 949 950.. _[8]: 951 952[8] https://lists.trustedfirmware.org/pipermail/tf-a/2020-February/000296.html 953 954-------------- 955 956*Copyright (c) 2020-2021, Arm Limited and Contributors. All rights reserved.* 957
