1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+-----------------------------------+ 10| DMA | Direct Memory Access | 11+--------+-----------------------------------+ 12| DTB | Device Tree Blob | 13+--------+-----------------------------------+ 14| DTS | Device Tree Source | 15+--------+-----------------------------------+ 16| EC | Execution Context | 17+--------+-----------------------------------+ 18| FIP | Firmware Image Package | 19+--------+-----------------------------------+ 20| FF-A | Firmware Framework for A-class | 21+--------+-----------------------------------+ 22| IPA | Intermediate Physical Address | 23+--------+-----------------------------------+ 24| NWd | Normal World | 25+--------+-----------------------------------+ 26| ODM | Original Design Manufacturer | 27+--------+-----------------------------------+ 28| OEM | Original Equipment Manufacturer | 29+--------+-----------------------------------+ 30| PA | Physical Address | 31+--------+-----------------------------------+ 32| PE | Processing Element | 33+--------+-----------------------------------+ 34| PVM | Primary VM | 35+--------+-----------------------------------+ 36| SMMU | System Memory Management Unit | 37+--------+-----------------------------------+ 38| SP | Secure Partition | 39+--------+-----------------------------------+ 40| SPM | Secure Partition Manager | 41+--------+-----------------------------------+ 42| SPMC | SPM Core | 43+--------+-----------------------------------+ 44| SPMD | SPM Dispatcher | 45+--------+-----------------------------------+ 46| SiP | Silicon Provider | 47+--------+-----------------------------------+ 48| SWd | Secure World | 49+--------+-----------------------------------+ 50| TLV | Tag-Length-Value | 51+--------+-----------------------------------+ 52| TOS | Trusted Operating System | 53+--------+-----------------------------------+ 54| VM | Virtual Machine | 55+--------+-----------------------------------+ 56 57Foreword 58======== 59 60Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 61 62- SPM based on the FF-A specification `[1]`_. 63- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 64 65Both implementations differ in their architectures and only one can be selected 66at build time. 67 68This document: 69 70- describes the FF-A implementation where the Secure Partition Manager 71 resides at EL3 and S-EL2 (or EL3 and S-EL1). 72- is not an architecture specification and it might provide assumptions 73 on sections mandated as implementation-defined in the specification. 74- covers the implications to TF-A used as a bootloader, and Hafnium 75 used as a reference code base for an S-EL2 secure firmware on 76 platforms implementing Armv8.4-SecEL2. 77 78Terminology 79----------- 80 81- Hypervisor refers to the NS-EL2 component managing Virtual Machines (or 82 partitions) in the Normal World. 83- SPMC refers to the S-EL2 component managing Virtual Machines (or Secure 84 Partitions) in the Secure World when Armv8.4-SecEL2 extension is implemented. 85- Alternatively, SPMC can refer to an S-EL1 component, itself being a Secure 86 Partition and implementing the FF-A ABI on pre-Armv8.4 platforms. 87- VM refers to a Normal World Virtual Machine managed by an Hypervisor. 88- SP refers to a Secure World "Virtual Machine" managed by the SPMC component. 89 90Support for legacy platforms 91---------------------------- 92 93In the implementation, the SPM is split into SPMD and SPMC components 94(although not strictly mandated by the specification). SPMD is located 95at EL3 and principally relays FF-A messages from NWd (Hypervisor or OS 96kernel) to SPMC located either at S-EL1 or S-EL2. 97 98Hence TF-A must support both cases where SPMC is either located at: 99 100- S-EL1 supporting pre-Armv8.4 platforms. SPMD conveys FF-A protocol 101 from EL3 to S-EL1. 102- S-EL2 supporting platforms implementing Armv8.4-SecEL2 extension. 103 SPMD conveys FF-A protocol from EL3 to S-EL2. 104 105The same SPMD component is used to support both configurations. The SPMC 106execution level is a build time choice. 107 108Sample reference stack 109====================== 110 111The following diagram illustrates a possible configuration with SPMD and SPMC, 112one or multiple Secure Partitions, with or without an optional Hypervisor: 113 114.. image:: ../resources/diagrams/ff-a-spm-sel2.png 115 116TF-A build options 117================== 118 119The following TF-A build options are provisioned: 120 121- **SPD=spmd**: this option selects the SPMD component to relay FF-A 122 protocol from NWd to SWd back and forth. It is not possible to 123 enable another Secure Payload Dispatcher when this option is chosen. 124- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC execution 125 level to being S-EL1 or S-EL2. It defaults to enabled (value 1) when 126 SPD=spmd is chosen. 127- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 128 restoring) the EL2 system register context before entering (resp. 129 after leaving) the SPMC. It is mandatory when ``SPMD_SPM_AT_SEL2`` is 130 enabled. The context save/restore routine and exhaustive list of 131 registers is visible at `[4]`_. 132- **SP_LAYOUT_FILE**: this option provides a text description file 133 providing paths to SP binary images and DTS format manifests 134 (see `Specifying partition binary image and DT`_). It 135 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 136 secure partitions are to be loaded on behalf of SPMC. 137 138+------------------------------+----------------------+------------------+ 139| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | 140+------------------------------+----------------------+------------------+ 141| SPMC at S-EL1 (e.g. OP-TEE) | 0 | 0 | 142+------------------------------+----------------------+------------------+ 143| SPMC at S-EL2 (e.g. Hafnium) | 1 | 1 (default when | 144| | | SPD=spmd) | 145+------------------------------+----------------------+------------------+ 146 147Other combinations of such build options either break the build or are not 148supported. 149 150Note, the ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 151barely saving/restoring EL2 registers from an Arm arch perspective. As such 152it is decoupled from the ``SPD=spmd`` option. 153 154BL32 option is re-purposed to specify the SPMC image. It can specify either the 155Hafnium binary path (built for the secure world) or the path to a TEE binary 156implementing the FF-A protocol. 157 158BL33 option can specify either: 159 160- the TFTF binary or 161- the Hafnium binary path (built for the normal world) if VMs were loaded by 162 TF-A beforehand or 163- a minimal loader performing the loading of VMs and Hafnium. 164 165Sample TF-A build command line when SPMC is located at S-EL1 166(typically pre-Armv8.4): 167 168.. code:: shell 169 170 make \ 171 CROSS_COMPILE=aarch64-none-elf- \ 172 SPD=spmd \ 173 SPMD_SPM_AT_SEL2=0 \ 174 BL32=<path-to-tee-binary> \ 175 BL33=<path-to-nwd-binary> \ 176 PLAT=fvp \ 177 all fip 178 179Sample TF-A build command line for an Armv8.4-SecEL2 enabled system 180where SPMC is located at S-EL2: 181 182.. code:: shell 183 184 make \ 185 CROSS_COMPILE=aarch64-none-elf- \ 186 SPD=spmd \ 187 CTX_INCLUDE_EL2_REGS=1 \ 188 ARM_ARCH_MINOR=4 \ 189 BL32=<path-to-swd-hafnium-binary> 190 BL33=<path-to-nwd-binary> \ 191 SP_LAYOUT_FILE=sp_layout.json \ 192 PLAT=fvp \ 193 all fip 194 195Build options to enable secure boot: 196 197.. code:: shell 198 199 make \ 200 CROSS_COMPILE=aarch64-none-elf- \ 201 SPD=spmd \ 202 CTX_INCLUDE_EL2_REGS=1 \ 203 ARM_ARCH_MINOR=4 \ 204 BL32=<path-to-swd-hafnium-binary> 205 BL33=<path-to-nwd-binary> \ 206 SP_LAYOUT_FILE=../tf-a-tests/build/fvp/debug/sp_layout.json \ 207 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 208 TRUSTED_BOARD_BOOT=1 \ 209 COT=dualroot \ 210 ARM_ROTPK_LOCATION=devel_rsa \ 211 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 212 GENERATE_COT=1 \ 213 PLAT=fvp \ 214 all fip 215 216Boot process 217============ 218 219Loading Hafnium and Secure Partitions in the secure world 220--------------------------------------------------------- 221 222The Hafnium implementation in normal world requires VMs to be loaded in 223memory prior to booting. The mechanism upon which VMs are loaded and 224exposed to Hafnium are either: 225 226- by supplying a ramdisk image where VM images are concatenated (1) 227- or by providing VM load addresses within Hafnium manifest (2) 228 229TF-A is the bootlader for the Hafnium and SPs in the secure world. TF-A 230does not provide tooling or libraries manipulating ramdisks as required 231by (1). Thus BL2 loads SPs payloads independently. 232SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 233Thus they are supplied as distinct “self-contained” signed entities within 234the FIP flash image. The FIP image itself is not signed hence providing 235ability to upgrade SPs in the field. 236 237Booting through TF-A 238-------------------- 239 240SP manifests 241~~~~~~~~~~~~ 242 243An SP manifest describes SP attributes as defined in `[1]`_ 244section 3.1 (partition manifest at virtual FF-A instance) in DTS text format. It 245is represented as a single file associated with the SP. A sample is 246provided by `[5]`_. A binding document is provided by `[6]`_. 247 248Secure Partition packages 249~~~~~~~~~~~~~~~~~~~~~~~~~ 250 251Secure Partitions are bundled as independent package files consisting 252of: 253 254- a header 255- a DTB 256- an image payload 257 258The header starts with a magic value and offset values to SP DTB and 259image payload. Each SP package is loaded independently by BL2 loader 260and verified for authenticity and integrity. 261 262The SP package identified by its UUID (matching FF-A uuid) is inserted 263as a single entry into the FIP at end of the TF-A build flow as shown: 264 265.. code:: shell 266 267 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 268 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 269 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 270 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 271 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 272 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 273 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 274 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 275 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 276 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 277 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 278 279.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 280 281Specifying partition binary image and DT 282~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 283 284A description file (json format) is passed to the build flow specifying 285paths to the SP binary image and associated DTS partition manifest file. 286The latter is going through the dtc compiler to generate the dtb fed into 287the SP package. 288This file also specifies the owner of the SP, which is an optional field and 289identifies the signing domain in case of dualroot CoT. 290The possible owner of an SP could either be Silicon Provider or Platform, and 291the corresponding "owner" field value could either be "SiP" or "Plat". 292In absence of "owner" field, it defaults to "SiP". 293 294.. code:: shell 295 296 { 297 "tee1" : { 298 "image": "tee1.bin", 299 "pm": "tee1.dts", 300 "owner": "SiP" 301 }, 302 303 "tee2" : { 304 "image": "tee2.bin", 305 "pm": "tee2.dts", 306 "owner": "Plat" 307 } 308 } 309 310SPMC manifest 311~~~~~~~~~~~~~ 312 313This manifest contains an SPMC attributes node consumed by SPMD at boot time. It 314is implementing the description from `[1]`_ section 3.2 (SP manifest at physical 315FF-A instance). The SP manifest at physical FF-A instance is used by the SPMD to 316setup a SP that co-resides with the SPMC and executes at S-EL1 or Secure 317Supervisor mode. 318 319In this implementation its usage is extended to the secure physical FF-A 320instance where SPMC executes at S-EL2. 321 322.. code:: shell 323 324 attribute { 325 spmc_id = <0x8000>; 326 maj_ver = <0x1>; 327 min_ver = <0x0>; 328 exec_state = <0x0>; 329 load_address = <0x0 0x6000000>; 330 entrypoint = <0x0 0x6000000>; 331 binary_size = <0x60000>; 332 }; 333 334- *spmc_id* defines the endpoint ID value that SPMC can query through 335 ``FFA_ID_GET``. 336- *maj_ver/min_ver*. SPMD checks provided version versus its internal 337 version and aborts if not matching. 338- *exec_state* defines SPMC execution state (can be AArch64 for 339 Hafnium, or AArch64/AArch32 for OP-TEE at S-EL1). 340- *load_address* and *binary_size* are mostly used to verify secondary 341 entry points fit into the loaded binary image. 342- *entrypoint* defines the cold boot primary core entry point used by 343 SPMD (currently matches ``BL32_BASE``) 344 345Other nodes in the manifest are consumed by Hafnium in the secure world. 346A sample can be found at [7]: 347 348- The *chosen* node is currently unused in SWd. It is meant for NWd to 349 specify the init ramdisk image. 350- The *hypervisor* node describes SPs. *is_ffa_partition* boolean 351 attribute indicates an SP. Load-addr field specifies the load address 352 at which TF-A loaded the SP package. 353- *cpus* node provide the platform topology and allows MPIDR to VMPIDR 354 mapping. Notice with current implementation primary cpu is declared 355 first, then secondary cpus must be declared in reverse order. 356 357SPMC boot 358~~~~~~~~~ 359 360The SPMC is loaded by BL2 as the BL32 image. 361 362The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image. 363 364BL2 passes the SPMC manifest address to BL31 through a register. 365 366BL31(SPMD) runs from primary core, initializes the core contexts and 367launches BL32 passing the SPMC manifest address through a register. 368 369Loading of SPs 370~~~~~~~~~~~~~~ 371 372.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 373 374 375Notice this boot flow is an implementation sample on Arm's FVP platform. Platforms 376not using FW_CONFIG would adjust to a different implementation. 377 378Secure boot 379~~~~~~~~~~~ 380 381The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 382SPMC manifest and Secure Partitions and verifies them for authenticity and integrity. 383Refer to TBBR specification `[3]`_. 384 385The multiple-signing domain feature (in current state dual signing domain) allows 386the use of two root keys namely S-ROTPK and NS-ROTPK (see `[8]`_): 387 388- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 389- BL33 may be signed by the OEM using NS-ROTPK. 390- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 391 392Longer term multiple signing domain will allow additional signing keys, e.g. 393if SPs originate from different parties. 394 395See `TF-A build options`_ for a sample build command line. 396 397Hafnium in the secure world 398=========================== 399 400**NOTE: this section is work in progress. Descriptions and implementation choices 401are subject to evolve.** 402 403General considerations 404---------------------- 405 406Build platform for the secure world 407~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 408 409The implementation might add specific code parts only relevant to the 410secure world. Such code parts might be isolated into different files 411and/or conditional code enclosed by a ``SECURE_WORLD`` macro. 412 413Secure Partitions CPU scheduling 414~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 415 416In the normal world, VMs are scheduled by the FFA_RUN ABI invoked from the 417primary scheduler (in the primary VM), or by a direct message request or 418response. 419 420With the FF-A EAC specification, Secure Partitions are scheduled by direct 421message invocations from a NWd VM or another SP. 422 423Platform topology 424~~~~~~~~~~~~~~~~~ 425 426As stated in `[1]`_ section 4.4.1 the SPMC implementation assumes the 427following SP types: 428 429- Pinned MP SPs: an Execution Context id matches a physical PE id. MP 430 SPs must implement the same number of ECs as the number of PEs in the 431 platform. Hence the *execution-ctx-count* as defined by 432 `[1]`_ (or NWd-Hafnium *vcpu_count*) can only take the 433 value of one or the number of physical PEs. 434- Migratable UP SPs: a single execution context can run and be migrated 435 on any physical PE. It declares a single EC in its SP manifest. An UP 436 SP can receive a direct message request on any physical core. 437 438Usage of PSCI services in the secure world 439~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 440 441- The normal world Hypervisor (optional) or OS kernel issues PSCI service 442 invocations e.g. to request PSCI version, wake-up a secondary core, or request 443 core suspend. This happens at the non-secure physical FF-A instance. In the 444 example case of Hafnium in the normal world, it boots on the primary core and 445 one of the first initialization step is to request the PSCI version. It then 446 launches the primary VM. The primary VM upon initializing performs PSCI service 447 calls (at non-secure virtual FF-A instance) which are trapped by the 448 Hypervisor. Invocation from OS Kernel ends straight at EL3. The PVM issues 449 ``PSCI_CPU_ON`` service calls to wake-up secondary cores by passing an 450 ``MPIDR``, entry point address and a CPU context address. The EL3 PSCI layer 451 then performs an exception return to the secondary core entry point on the 452 targeted core. Other PSCI calls can happen at run-time from the PVM e.g. to 453 request core suspend. 454- In the existing TF-A PSCI standard library, PSCI service calls are filtered at 455 EL3 to only originate from the NWd. Thus concerning the SPMC (at secure 456 physical FF-A instance) the PSCI service invocations cannot happen as in the 457 normal world. For example, a ``PSCI_CPU_ON`` service invocation from the SPMC 458 does not reach the PSCI layer. 459 460Parsing SP partition manifests 461------------------------------ 462 463Hafnium must be able to consume SP manifests as defined in 464`[1]`_ section 3.1, at least for the mandatory fields. 465 466The SP manifest may contain memory and device regions nodes. 467 468- Memory regions shall be mapped in the SP Stage-2 translation regime at 469 load time. A memory region node can specify RX/TX buffer regions in which 470 case it is not necessary for an SP to explicitly call the ``FFA_RXTX_MAP`` 471 service. 472- Device regions shall be mapped in SP Stage-2 translation regime as 473 peripherals and possibly allocate additional resources (e.g. interrupts) 474 475Base addresses for memory and device region nodes are IPAs provided SPMC 476identity maps IPAs to PAs within SP Stage-2 translation regime. 477 478Note: currently both VTTBR_EL2 and VSTTBR_EL2 resolve to the same set of page 479tables. It is still open whether two sets of page tables shall be provided per 480SP. The memory region node as defined in the spec (section 3.1 Table 10) 481provides a memory security attribute hinting to map either to the secure or 482non-secure stage-2 table. 483 484Passing boot data to the SP 485--------------------------- 486 487`[1]`_ Section 3.4.2 “Protocol for passing data” defines a 488method to passing boot data to SPs (not currently implemented). 489 490Provided that the whole Secure Partition package image (see `Secure 491Partition packages`_) is mapped to the SP's secure Stage-2 translation 492regime, an SP can access its own manifest DTB blob and extract its partition 493manifest properties. 494 495SP Boot order 496------------- 497 498SP manifests provide an optional boot order attribute meant to resolve 499dependencies such as an SP providing a service required to properly boot 500another SP. 501 502Boot phases 503----------- 504 505Primary core boot-up 506~~~~~~~~~~~~~~~~~~~~ 507 508The SPMC performs its platform initializations then loads and creates 509secure partitions based on SP packages and manifests. Then each secure 510partition is launched in sequence (see `SP Boot order`_) on their primary 511Execution Context. 512 513Notice the primary physical core may not be core 0. Hence if the primary 514core linear id is N, the 1:1 mapping requires MP SPs are launched using 515EC[N] on PE[N] (see `Platform topology`_). 516 517The SP's primary Execution Context (or the EC used when the partition is booted) 518exits through ``FFA_MSG_WAIT`` to indicate successful initialization. 519 520Secondary physical core boot-up 521~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 522 523Upon boot-up, the SPMC running on the primary core performs 524implementation-defined SPMD service calls at secure physical FF-A instance 525to register the secondary physical cores entry points and context information: 526 527- This is done through a direct message request invocation to the SPMD 528 (``SET_ENTRY_POINT``). This service call does not wake-up the targeted 529 core immediately. The secondary core is woken up later by a NWd 530 ``PSCI_CPU_ON`` service invocation. A notification is passed from EL3 531 PSCI layer to the SPMD, and then to SPMC through an implementation-defined 532 interface. 533- The SPMC/SPMD interface can consist of FF-A direct message requests/responses 534 transporting PM events. 535 536If there is no Hypervisor in the normal world, the OS Kernel issues 537``PSCI_CPU_ON`` calls that are directly trapped to EL3. 538 539When a secondary physical core wakes-up the SPMD notifies the SPMC which updates 540its internal states reflecting current physical core is being turned on. 541It might then return straight to the SPMD and then to the NWd. 542 543*(under discussion)* There may be possibility that an SP registers "PM events" 544(during primary EC boot stage) through an ad-hoc interface. Such events would 545be relayed by SPMC to one or more registered SPs on need basis 546(see `Power management`_). 547 548Secondary virtual core boot-up 549~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 550 551In the example case where Hafnium exists in the normal world, secondary VMs 552issue a ``PSCI_CPU_ON`` service call which is trapped to the Hypervisor. The 553latter then enables the vCPU context for the targeted core, and switches to 554the PVM down to the kernel driver with an ``HF_WAKE_UP`` message. The NWd 555driver in PVM can then schedule the newly woken up vCPU context. 556 557In the secure world the primary EC of a given SP passes the secondary EC entry 558point and context. The SMC service call is trapped into the SPMC. This can be 559either *(under discussion)*: 560 561- a specific interface registering the secondary EC entry point, 562 similarly to above ``SET_ENTRY_POINT`` service. 563- Re-purposing the ``PSCI_CPU_ON`` function id. It is 564 assumed that even if the input arguments are the same as the ones defined in 565 the PSCI standard, the usage deviates by the fact the secondary EC is not 566 woken up immediately. At least for the FF-A EAC where only 567 direct messaging is allowed, it is only after the first direct 568 message invocation that the secondary EC is entered. This option 569 might be preferred when the same code base is re-used for a VM or 570 an SP. The ABI to wake-up a secondary EC can remain similar. 571 572SPs are always scheduled from the NWd, this paradigm did not change from legacy 573TEEs. There must always be some logic (or driver) in the NWd to relinquish CPU 574cycles to the SWd. If primary core is 0, an SP EC[x>0] entry point is supplied 575by the SP EC[0] when the system boots in SWd. But this EC[x] is not immediately 576entered at boot. Later in the boot process when NWd is up, a direct message 577request issued from physical core 1 ends up in SP EC[1], and only at this stage 578this context is effectively scheduled. 579 580It should be possible for an SP to call into another SP through direct message 581provided the latter SP has been booted already. The "boot-order" field in 582partition manifests (`SP Boot order`_) fulfills the dependency towards availability 583of a service within an SP offered to another SP. 584 585Mandatory interfaces 586-------------------- 587 588The following interfaces must be exposed to any VM or SP: 589 590- ``FFA_STATUS`` 591- ``FFA_ERROR`` 592- ``FFA_INTERRUPT`` 593- ``FFA_VERSION`` 594- ``FFA_FEATURES`` 595- ``FFA_RX_RELEASE`` 596- ``FFA_RXTX_MAP`` 597- ``FFA_RXTX_UNMAP`` 598- ``FFA_PARTITION_INFO_GET`` 599- ``FFA_ID_GET`` 600 601FFA_VERSION 602~~~~~~~~~~~ 603 604Per `[1]`_ section 8.1 ``FFA_VERSION`` requires a 605*requested_version* parameter from the caller. 606 607In the current implementation when ``FFA_VERSION`` is invoked from: 608 609- Hypervisor in NS-EL2: the SPMD returns the SPMC version specified 610 in the SPMC manifest. 611- OS kernel in NS-EL1 when NS-EL2 is not present: the SPMD returns the 612 SPMC version specified in the SPMC manifest. 613- VM in NWd: the Hypervisor returns its implemented version. 614- SP in SWd: the SPMC returns its implemented version. 615- SPMC at S-EL1/S-EL2: the SPMD returns its implemented version. 616 617FFA_FEATURES 618~~~~~~~~~~~~ 619 620FF-A features may be discovered by Secure Partitions while booting 621through the SPMC. However, SPMC cannot get features from Hypervisor 622early at boot time as NS world is not setup yet. 623 624The Hypervisor may decide to gather FF-A features from SPMC through SPMD 625once at boot time and store the result. Later when a VM requests FF-A 626features, the Hypervisor can adjust its own set of features with what 627SPMC advertised, if necessary. Another approach is to always forward FF-A 628features to the SPMC when a VM requests it to the Hypervisor. Although 629the result is not supposed to change over time so there may not be added 630value doing the systematic forwarding. 631 632FFA_RXTX_MAP/FFA_RXTX_UNMAP 633~~~~~~~~~~~~~~~~~~~~~~~~~~~ 634 635VM mailboxes are re-purposed to serve as SP RX/TX buffers. The RX/TX 636map API maps the send and receive buffer IPAs to the SP Stage-2 translation regime. 637 638Hafnium in the normal world defines VMs and their attributes as logical structures, 639including a mailbox used for FF-A indirect messaging, memory sharing, or the 640`FFA_PARTITION_INFO_GET`_ ABI. 641This same mailbox structure is re-used in the SPMC. `[1]`_ states only direct 642messaging is allowed to SPs. Thus mailbox usage is restricted to implementing 643`FFA_PARTITION_INFO_GET`_ and memory sharing ABIs. 644 645FFA_PARTITION_INFO_GET 646~~~~~~~~~~~~~~~~~~~~~~ 647 648Partition info get service call can originate: 649 650- from SP to SPM 651- from VM to Hypervisor 652- from Hypervisor to SPM 653 654For the latter case, the service call must be forwarded through the SPMD. 655 656FFA_ID_GET 657~~~~~~~~~~ 658 659The SPMD returns: 660 661- a default zero value on invocation from the Hypervisor. 662- The ``spmc_id`` value specified in the SPMC manifest on invocation from 663 the SPMC (see `SPMC manifest`_) 664 665The FF-A id space is split into a non-secure space and secure space: 666 667- FF-A id with bit 15 clear refer to normal world VMs. 668- FF-A id with bit 15 set refer to secure world SPs 669 670Such convention helps the SPMC discriminating the origin and destination worlds 671in an FF-A service invocation. In particular the SPMC shall filter unauthorized 672transactions in its world switch routine. It must not be permitted for a VM to 673use a secure FF-A id as origin world through spoofing: 674 675- A VM-to-SP messaging passing shall have an origin world being non-secure 676 (FF-A id bit 15 clear) and destination world being secure (FF-A id bit 15 677 set). 678- Similarly, an SP-to-SP message shall have FF-A id bit 15 set for both origin 679 and destination ids. 680 681An incoming direct message request arriving at SPMD from NWd is forwarded to 682SPMC without a specific check. The SPMC is resumed through eret and "knows" the 683message is coming from normal world in this specific code path. Thus the origin 684endpoint id must be checked by SPMC for being a normal world id. 685 686An SP sending a direct message request must have bit 15 set in its origin 687endpoint id and this can be checked by the SPMC when the SP invokes the ABI. 688 689The SPMC shall reject the direct message if the claimed world in origin endpoint 690id is not consistent: 691 692- It is either forwarded by SPMD and thus origin endpoint id must be a "normal 693 world id", 694- or initiated by an SP and thus origin endpoint id must be a "secure world id". 695 696Direct messaging 697---------------- 698 699This is a mandatory interface for Secure Partitions consisting in direct 700message request and responses. 701 702The ``ffa_handler`` Hafnium function may: 703 704- trigger a world change e.g. when an SP invokes the direct message 705 response ABI to a VM. 706- handle multiple requests from the NWd without resuming an SP. 707 708SP-to-SP 709~~~~~~~~ 710 711- An SP can send a direct message request to another SP 712- An SP can receive a direct message response from another SP. 713 714VM-to-SP 715~~~~~~~~ 716 717- A VM can send a direct message request to an SP 718- An SP can send a direct message response to a VM 719 720SPMC-SPMD messaging 721~~~~~~~~~~~~~~~~~~~ 722 723Specific implementation-defined endpoint IDs are allocated to the SPMC and SPMD. 724Referring those IDs in source/destination fields of a direct message 725request/response permits SPMD to SPMC messaging back and forth. 726 727Per `[1]`_ Table 114 Config No. 1 (physical FF-A instance): 728 729- SPMC=>SPMD direct message request uses SMC conduit 730- SPMD=>SPMC direct message request uses ERET conduit 731 732Per `[1]`_ Table 118 Config No. 1 (physical FF-A instance): 733 734- SPMC=>SPMD direct message response uses SMC conduit 735- SPMD=>SPMC direct message response uses ERET conduit 736 737Memory management 738----------------- 739 740This section only deals with the PE MMU configuration. 741 742Hafnium in the normal world deals with NS buffers only and provisions 743a single root page table directory to VMs. In context of S-EL2 enabled 744firmware, two IPA spaces are output from Stage-1 translation (secure 745and non-secure). The Stage-2 translation handles: 746 747- A single secure IPA space when an SP Stage-1 MMU is disabled. 748- Two IPA spaces (secure and non-secure) when Stage-1 MMU is enabled. 749 750``VTCR_EL2`` and ``VSTCR_EL2`` provide additional bits for controlling the 751NS/S IPA translations (``VSTCR_EL2.SW``, ``VSTCR_EL2.SA``, ``VTCR_EL2.NSW``, 752``VTCR_EL2.NSA``). There may be two approaches: 753 754- secure and non-secure mappings are rooted as two separate root page 755 tables 756- secure and non-secure mappings use the same root page table. Access 757 from S-EL1 to an NS region translates to a secure physical address 758 space access. 759 760Interrupt management 761-------------------- 762 763Road to a para-virtualized interface 764~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 765 766Current Hafnium implementation uses an ad-hoc mechanism for a VM to get 767a pending interrupt number through an hypercall. The PVM injects 768interrupts to VMs by delegation from the Hypervisor. The PVM probes a 769pending interrupt directly from the GIC distributor. 770 771The short-term plan is to have Hafnium/SPMC in the secure world owner 772of the GIC configuration. 773 774The SPMC fully owns the GIC configuration at S-EL2. The SPMC manages 775interrupt resources and allocates interrupt ID based on SP manifests. 776The SPMC acknowledges physical interrupts and injects virtual interrupts 777by setting the vIRQ bit when resuming an SP. A Secure Partition gathers 778the interrupt number through an hypercall. 779 780Notice the SPMC/SPMD has to handle Group0 secure interrupts in addition 781to Group1 S/NS interrupts. 782 783Power management 784---------------- 785 786Assumption on the Nwd: 787 788- NWd is the best candidate to own the platform Power Management 789 policy. It is master to invoking PSCI service calls from physical 790 CPUs. 791- EL3 monitor is in charge of the PM control part (its PSCI layer 792 actually writing to platform registers). 793- It is fine for the Hypervisor to trap PSCI calls and relay to EL3, or 794 OS kernel driver to emit PSCI service calls. 795 796PSCI notification are relayed through the SPMD/SPD PM hooks to the SPMC. 797This can either be through re-use of PSCI FIDs or an FF-A direct message 798from SPMD to SPMC. 799 800The SPMD performs an exception return to the SPMC which is resumed to 801its ``eret_handler`` routine. It is then either consuming a PSCI FID or 802an FF-A FID. Depending on the servicing, the SPMC may return directly to 803the SPMD (and then NWd) without resuming an SP at this stage. An example 804of this is invocation of ``FFA_PARTITION_INFO_GET`` from NWd relayed by 805the SPMD to the SPMC. The SPMC returns the needed partition information 806to the SPMD (then NWd) without actually resuming a partition in secure world. 807 808*(under discussion)* 809About using PSCI FIDs from SPMD to SPMC to notify of PM events, it is still 810questioned what to use as the return code from the SPMC. 811If the function ID used by the SPMC is not an FF-A ID when doing SMC, then the 812EL3 std svc handler won't route the response to the SPMD. That's where comes the 813idea to embed the notification into an FF-A message. The SPMC can discriminate 814this message as being a PSCI event, process it, and reply with an FF-A return 815message that the SPMD receives as an acknowledgement. 816 817SP notification 818--------------- 819 820Power management notifications are conveyed from PSCI library to the 821SPMD / SPD hooks. A range of events can be relayed to SPMC. 822 823SPs may need to be notified about specific PM events. 824 825- SPs might register PM events to the SPMC 826- On SPMD to SPMC notification, a limited range of SPs may be notified 827 through a direct message. 828- This assumes the mentioned SPs supports managed exit. 829 830The SPMC is the first to be notified about PM events from the SPMD. It is up 831to the SPMC to arbitrate to which SP it needs to send PM events. 832An SP explicitly registers to receive notifications to specific PM events. 833The register operation can either be an implementation-defined service call 834to the SPMC when the primary SP EC boots, or be supplied through the SP 835manifest. 836 837Support for SMMUv3 in Hafnium 838============================= 839 840An SMMU is analogous to an MMU in a CPU. It performs address translations for 841Direct Memory Access (DMA) requests from system I/O devices. 842The responsibilities of an SMMU include: 843 844- Translation: Incoming DMA requests are translated from bus address space to 845 system physical address space using translation tables compliant to 846 Armv8/Armv7 VMSA descriptor format. 847- Protection: An I/O device can be prohibited from read, write access to a 848 memory region or allowed. 849- Isolation: Traffic from each individial device can be independently managed. 850 The devices are differentiated from each other using unique translation 851 tables. 852 853The following diagram illustrates a typical SMMU IP integrated in a SoC with 854several I/O devices along with Interconnect and Memory system. 855 856.. image:: ../resources/diagrams/MMU-600.png 857 858SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 859support for SMMUv3 driver in both Normal and Secure World. A brief introduction 860of SMMUv3 functionality and the corresponding software support in Hafnium is 861provided here. 862 863SMMUv3 features 864--------------- 865 866- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 867 translation support. It can either bypass or abort incoming translations as 868 well. 869- Traffic (memory transactions) from each upstream I/O peripheral device, 870 referred to as Stream, can be independently managed using a combination of 871 several memory based configuration structures. This allows the SMMUv3 to 872 support a large number of streams with each stream assigned to a unique 873 translation context. 874- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 875 a Processing Element. AArch32(LPAE) and AArch64 translation table format 876 are supported by SMMUv3. 877- SMMUv3 offers non-secure stream support with secure stream support being 878 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 879 instance for secure and non-secure stream support. 880- It also supports sub-streams to differentiate traffic from a virtualized 881 peripheral associated with a VM/SP. 882- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 883 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 884 for providing Secure Stage2 translation support to upstream peripheral 885 devices. 886 887SMMUv3 Programming Interfaces 888----------------------------- 889 890SMMUv3 has three software interfaces that are used by the Hafnium driver to 891configure the behaviour of SMMUv3 and manage the streams. 892 893- Memory based data strutures that provide unique translation context for 894 each stream. 895- Memory based circular buffers for command queue and event queue. 896- A large number of SMMU configuration registers that are memory mapped during 897 boot time by Hafnium driver. Except a few registers, all configuration 898 registers have independent secure and non-secure versions to configure the 899 behaviour of SMMUv3 for translation of secure and non-secure streams 900 respectively. 901 902Peripheral device manifest 903-------------------------- 904 905Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 906These devices are dependent on PE endpoint to initiate and receive memory 907management transactions on their behalf. The acccess to the MMIO regions of 908any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 909uses the same stage 2 translations for the device as those used by partition 910manager on behalf of the PE endpoint. This ensures that the peripheral device 911has the same visibility of the physical address space as the endpoint. The 912device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 913) must specify these additional properties for each peripheral device in the 914system : 915 916- smmu-id: This field helps to identify the SMMU instance that this device is 917 upstream of. 918- stream-ids: List of stream IDs assigned to this device. 919 920.. code:: shell 921 922 smmuv3-testengine { 923 base-address = <0x00000000 0x2bfe0000>; 924 pages-count = <32>; 925 attributes = <0x3>; 926 smmu-id = <0>; 927 stream-ids = <0x0 0x1>; 928 interrupts = <0x2 0x3>, <0x4 0x5>; 929 exclusive-access; 930 }; 931 932SMMUv3 driver limitations 933------------------------- 934 935The primary design goal for the Hafnium SMMU driver is to support secure 936streams. 937 938- Currently, the driver only supports Stage2 translations. No support for 939 Stage1 or nested translations. 940- Supports only AArch64 translation format. 941- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 942 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 943- No support for independent peripheral devices. 944 945References 946========== 947 948.. _[1]: 949 950[1] `Arm Firmware Framework for Armv8-A <https://developer.arm.com/docs/den0077/latest>`__ 951 952.. _[2]: 953 954[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 955 956.. _[3]: 957 958[3] `Trusted Boot Board Requirements 959Client <https://developer.arm.com/docs/den0006/latest/trusted-board-boot-requirements-client-tbbr-client-armv8-a>`__ 960 961.. _[4]: 962 963[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 964 965.. _[5]: 966 967[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/cactus.dts 968 969.. _[6]: 970 971[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/ffa-manifest-binding.html 972 973.. _[7]: 974 975[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 976 977.. _[8]: 978 979[8] https://developer.trustedfirmware.org/w/tf_a/poc-multiple-signing-domains/ 980 981-------------- 982 983*Copyright (c) 2020-2021, Arm Limited and Contributors. All rights reserved.* 984
