1Secure Partition Manager 2************************ 3 4.. contents:: 5 6Acronyms 7======== 8 9+--------+-----------------------------------+ 10| DMA | Direct Memory Access | 11+--------+-----------------------------------+ 12| DTB | Device Tree Blob | 13+--------+-----------------------------------+ 14| DTS | Device Tree Source | 15+--------+-----------------------------------+ 16| EC | Execution Context | 17+--------+-----------------------------------+ 18| FIP | Firmware Image Package | 19+--------+-----------------------------------+ 20| FF-A | Firmware Framework for A-class | 21+--------+-----------------------------------+ 22| IPA | Intermediate Physical Address | 23+--------+-----------------------------------+ 24| NWd | Normal World | 25+--------+-----------------------------------+ 26| ODM | Original Design Manufacturer | 27+--------+-----------------------------------+ 28| OEM | Original Equipment Manufacturer | 29+--------+-----------------------------------+ 30| PA | Physical Address | 31+--------+-----------------------------------+ 32| PE | Processing Element | 33+--------+-----------------------------------+ 34| PVM | Primary VM | 35+--------+-----------------------------------+ 36| PSA | Platform Security Architecture | 37+--------+-----------------------------------+ 38| SMMU | System Memory Management Unit | 39+--------+-----------------------------------+ 40| SP | Secure Partition | 41+--------+-----------------------------------+ 42| SPM | Secure Partition Manager | 43+--------+-----------------------------------+ 44| SPMC | SPM Core | 45+--------+-----------------------------------+ 46| SPMD | SPM Dispatcher | 47+--------+-----------------------------------+ 48| SiP | Silicon Provider | 49+--------+-----------------------------------+ 50| SWd | Secure World | 51+--------+-----------------------------------+ 52| TLV | Tag-Length-Value | 53+--------+-----------------------------------+ 54| TOS | Trusted Operating System | 55+--------+-----------------------------------+ 56| VM | Virtual Machine | 57+--------+-----------------------------------+ 58 59Foreword 60======== 61 62Two implementations of a Secure Partition Manager co-exist in the TF-A codebase: 63 64- SPM based on the PSA FF-A specification `[1]`_. 65- SPM based on the MM interface to communicate with an S-EL0 partition `[2]`_. 66 67Both implementations differ in their architectures and only one can be selected 68at build time. 69 70This document: 71 72- describes the PSA FF-A implementation where the Secure Partition Manager 73 resides at EL3 and S-EL2 (or EL3 and S-EL1). 74- is not an architecture specification and it might provide assumptions 75 on sections mandated as implementation-defined in the specification. 76- covers the implications to TF-A used as a bootloader, and Hafnium 77 used as a reference code base for an S-EL2 secure firmware on 78 platforms implementing Armv8.4-SecEL2. 79 80Terminology 81----------- 82 83- Hypervisor refers to the NS-EL2 component managing Virtual Machines (or 84 partitions) in the Normal World. 85- SPMC refers to the S-EL2 component managing Virtual Machines (or Secure 86 Partitions) in the Secure World when Armv8.4-SecEL2 extension is implemented. 87- Alternatively, SPMC can refer to an S-EL1 component, itself being a Secure 88 Partition and implementing the FF-A ABI on pre-Armv8.4 platforms. 89- VM refers to a Normal World Virtual Machine managed by an Hypervisor. 90- SP refers to a Secure World "Virtual Machine" managed by the SPMC component. 91 92Support for legacy platforms 93---------------------------- 94 95In the implementation, the SPM is split into SPMD and SPMC components 96(although not strictly mandated by the specification). SPMD is located 97at EL3 and principally relays FF-A messages from NWd (Hypervisor or OS 98kernel) to SPMC located either at S-EL1 or S-EL2. 99 100Hence TF-A must support both cases where SPMC is either located at: 101 102- S-EL1 supporting pre-Armv8.4 platforms. SPMD conveys FF-A protocol 103 from EL3 to S-EL1. 104- S-EL2 supporting platforms implementing Armv8.4-SecEL2 extension. 105 SPMD conveys FF-A protocol from EL3 to S-EL2. 106 107The same SPMD component is used to support both configurations. The SPMC 108execution level is a build time choice. 109 110Sample reference stack 111====================== 112 113The following diagram illustrates a possible configuration with SPMD and SPMC, 114one or multiple Secure Partitions, with or without an optional Hypervisor: 115 116.. image:: ../resources/diagrams/ff-a-spm-sel2.png 117 118TF-A build options 119================== 120 121The following TF-A build options are provisioned: 122 123- **SPD=spmd**: this option selects the SPMD component to relay FF-A 124 protocol from NWd to SWd back and forth. It is not possible to 125 enable another Secure Payload Dispatcher when this option is chosen. 126- **SPMD_SPM_AT_SEL2**: this option adjusts the SPMC execution 127 level to being S-EL1 or S-EL2. It defaults to enabled (value 1) when 128 SPD=spmd is chosen. 129- **CTX_INCLUDE_EL2_REGS**: this option permits saving (resp. 130 restoring) the EL2 system register context before entering (resp. 131 after leaving) the SPMC. It is mandatory when ``SPMD_SPM_AT_SEL2`` is 132 enabled. The context save/restore routine and exhaustive list of 133 registers is visible at `[4]`_. 134- **SP_LAYOUT_FILE**: this option provides a text description file 135 providing paths to SP binary images and DTS format manifests 136 (see `Specifying partition binary image and DT`_). It 137 is required when ``SPMD_SPM_AT_SEL2`` is enabled hence when multiple 138 secure partitions are to be loaded on behalf of SPMC. 139 140+------------------------------+----------------------+------------------+ 141| | CTX_INCLUDE_EL2_REGS | SPMD_SPM_AT_SEL2 | 142+------------------------------+----------------------+------------------+ 143| SPMC at S-EL1 (e.g. OP-TEE) | 0 | 0 | 144+------------------------------+----------------------+------------------+ 145| SPMC at S-EL2 (e.g. Hafnium) | 1 | 1 (default when | 146| | | SPD=spmd) | 147+------------------------------+----------------------+------------------+ 148 149Other combinations of such build options either break the build or are not 150supported. 151 152Note, the ``CTX_INCLUDE_EL2_REGS`` option provides the generic support for 153barely saving/restoring EL2 registers from an Arm arch perspective. As such 154it is decoupled from the ``SPD=spmd`` option. 155 156BL32 option is re-purposed to specify the SPMC image. It can specify either the 157Hafnium binary path (built for the secure world) or the path to a TEE binary 158implementing the FF-A protocol. 159 160BL33 option can specify either: 161 162- the TFTF binary or 163- the Hafnium binary path (built for the normal world) if VMs were loaded by 164 TF-A beforehand or 165- a minimal loader performing the loading of VMs and Hafnium. 166 167Sample TF-A build command line when SPMC is located at S-EL1 168(typically pre-Armv8.4): 169 170.. code:: shell 171 172 make \ 173 CROSS_COMPILE=aarch64-none-elf- \ 174 SPD=spmd \ 175 SPMD_SPM_AT_SEL2=0 \ 176 BL32=<path-to-tee-binary> \ 177 BL33=<path-to-nwd-binary> \ 178 PLAT=fvp \ 179 all fip 180 181Sample TF-A build command line for an Armv8.4-SecEL2 enabled system 182where SPMC is located at S-EL2: 183 184.. code:: shell 185 186 make \ 187 CROSS_COMPILE=aarch64-none-elf- \ 188 SPD=spmd \ 189 CTX_INCLUDE_EL2_REGS=1 \ 190 ARM_ARCH_MINOR=4 \ 191 BL32=<path-to-swd-hafnium-binary> 192 BL33=<path-to-nwd-binary> \ 193 SP_LAYOUT_FILE=sp_layout.json \ 194 PLAT=fvp \ 195 all fip 196 197Build options to enable secure boot: 198 199.. code:: shell 200 201 make \ 202 CROSS_COMPILE=aarch64-none-elf- \ 203 SPD=spmd \ 204 CTX_INCLUDE_EL2_REGS=1 \ 205 ARM_ARCH_MINOR=4 \ 206 BL32=<path-to-swd-hafnium-binary> 207 BL33=<path-to-nwd-binary> \ 208 SP_LAYOUT_FILE=../tf-a-tests/build/fvp/debug/sp_layout.json \ 209 MBEDTLS_DIR=<path-to-mbedtls-lib> \ 210 TRUSTED_BOARD_BOOT=1 \ 211 COT=dualroot \ 212 ARM_ROTPK_LOCATION=devel_rsa \ 213 ROT_KEY=plat/arm/board/common/rotpk/arm_rotprivk_rsa.pem \ 214 GENERATE_COT=1 \ 215 PLAT=fvp \ 216 all fip 217 218Boot process 219============ 220 221Loading Hafnium and Secure Partitions in the secure world 222--------------------------------------------------------- 223 224The Hafnium implementation in normal world requires VMs to be loaded in 225memory prior to booting. The mechanism upon which VMs are loaded and 226exposed to Hafnium are either: 227 228- by supplying a ramdisk image where VM images are concatenated (1) 229- or by providing VM load addresses within Hafnium manifest (2) 230 231TF-A is the bootlader for the Hafnium and SPs in the secure world. TF-A 232does not provide tooling or libraries manipulating ramdisks as required 233by (1). Thus BL2 loads SPs payloads independently. 234SPs may be signed by different parties (SiP, OEM/ODM, TOS vendor, etc.). 235Thus they are supplied as distinct “self-contained” signed entities within 236the FIP flash image. The FIP image itself is not signed hence providing 237ability to upgrade SPs in the field. 238 239Booting through TF-A 240-------------------- 241 242SP manifests 243~~~~~~~~~~~~ 244 245An SP manifest describes SP attributes as defined in `[1]`_ 246section 3.1 (partition manifest at virtual FF-A instance) in DTS text format. It 247is represented as a single file associated with the SP. A sample is 248provided by `[5]`_. A binding document is provided by `[6]`_. 249 250Secure Partition packages 251~~~~~~~~~~~~~~~~~~~~~~~~~ 252 253Secure Partitions are bundled as independent package files consisting 254of: 255 256- a header 257- a DTB 258- an image payload 259 260The header starts with a magic value and offset values to SP DTB and 261image payload. Each SP package is loaded independently by BL2 loader 262and verified for authenticity and integrity. 263 264The SP package identified by its UUID (matching FF-A uuid) is inserted 265as a single entry into the FIP at end of the TF-A build flow as shown: 266 267.. code:: shell 268 269 Trusted Boot Firmware BL2: offset=0x1F0, size=0x8AE1, cmdline="--tb-fw" 270 EL3 Runtime Firmware BL31: offset=0x8CD1, size=0x13000, cmdline="--soc-fw" 271 Secure Payload BL32 (Trusted OS): offset=0x1BCD1, size=0x15270, cmdline="--tos-fw" 272 Non-Trusted Firmware BL33: offset=0x30F41, size=0x92E0, cmdline="--nt-fw" 273 HW_CONFIG: offset=0x3A221, size=0x2348, cmdline="--hw-config" 274 TB_FW_CONFIG: offset=0x3C569, size=0x37A, cmdline="--tb-fw-config" 275 SOC_FW_CONFIG: offset=0x3C8E3, size=0x48, cmdline="--soc-fw-config" 276 TOS_FW_CONFIG: offset=0x3C92B, size=0x427, cmdline="--tos-fw-config" 277 NT_FW_CONFIG: offset=0x3CD52, size=0x48, cmdline="--nt-fw-config" 278 B4B5671E-4A90-4FE1-B81F-FB13DAE1DACB: offset=0x3CD9A, size=0xC168, cmdline="--blob" 279 D1582309-F023-47B9-827C-4464F5578FC8: offset=0x48F02, size=0xC168, cmdline="--blob" 280 281.. uml:: ../resources/diagrams/plantuml/fip-secure-partitions.puml 282 283Specifying partition binary image and DT 284~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 285 286A description file (json format) is passed to the build flow specifying 287paths to the SP binary image and associated DTS partition manifest file. 288The latter is going through the dtc compiler to generate the dtb fed into 289the SP package. 290This file also specifies the owner of the SP, which is an optional field and 291identifies the signing domain in case of dualroot CoT. 292The possible owner of an SP could either be Silicon Provider or Platform, and 293the corresponding "owner" field value could either be "SiP" or "Plat". 294In absence of "owner" field, it defaults to "SiP". 295 296.. code:: shell 297 298 { 299 "tee1" : { 300 "image": "tee1.bin", 301 "pm": "tee1.dts", 302 "owner": "SiP" 303 }, 304 305 "tee2" : { 306 "image": "tee2.bin", 307 "pm": "tee2.dts", 308 "owner": "Plat" 309 } 310 } 311 312SPMC manifest 313~~~~~~~~~~~~~ 314 315This manifest contains an SPMC attributes node consumed by SPMD at boot time. It 316is implementing the description from `[1]`_ section 3.2 (SP manifest at physical 317FF-A instance). The SP manifest at physical FF-A instance is used by the SPMD to 318setup a SP that co-resides with the SPMC and executes at S-EL1 or Secure 319Supervisor mode. 320 321In this implementation its usage is extended to the secure physical FF-A 322instance where SPMC executes at S-EL2. 323 324.. code:: shell 325 326 attribute { 327 spmc_id = <0x8000>; 328 maj_ver = <0x1>; 329 min_ver = <0x0>; 330 exec_state = <0x0>; 331 load_address = <0x0 0x6000000>; 332 entrypoint = <0x0 0x6000000>; 333 binary_size = <0x60000>; 334 }; 335 336- *spmc_id* defines the endpoint ID value that SPMC can query through 337 ``FFA_ID_GET``. 338- *maj_ver/min_ver*. SPMD checks provided version versus its internal 339 version and aborts if not matching. 340- *exec_state* defines SPMC execution state (can be AArch64 for 341 Hafnium, or AArch64/AArch32 for OP-TEE at S-EL1). 342- *load_address* and *binary_size* are mostly used to verify secondary 343 entry points fit into the loaded binary image. 344- *entrypoint* defines the cold boot primary core entry point used by 345 SPMD (currently matches ``BL32_BASE``) 346 347Other nodes in the manifest are consumed by Hafnium in the secure world. 348A sample can be found at [7]: 349 350- The *chosen* node is currently unused in SWd. It is meant for NWd to 351 specify the init ramdisk image. 352- The *hypervisor* node describes SPs. *is_ffa_partition* boolean 353 attribute indicates an SP. Load-addr field specifies the load address 354 at which TF-A loaded the SP package. 355- *cpus* node provide the platform topology and allows MPIDR to VMPIDR 356 mapping. Notice with current implementation primary cpu is declared 357 first, then secondary cpus must be declared in reverse order. 358 359SPMC boot 360~~~~~~~~~ 361 362The SPMC is loaded by BL2 as the BL32 image. 363 364The SPMC manifest is loaded by BL2 as the ``TOS_FW_CONFIG`` image. 365 366BL2 passes the SPMC manifest address to BL31 through a register. 367 368BL31(SPMD) runs from primary core, initializes the core contexts and 369launches BL32 passing the SPMC manifest address through a register. 370 371Loading of SPs 372~~~~~~~~~~~~~~ 373 374.. uml:: ../resources/diagrams/plantuml/bl2-loading-sp.puml 375 376 377Notice this boot flow is an implementation sample on Arm's FVP platform. Platforms 378not using FW_CONFIG would adjust to a different implementation. 379 380Secure boot 381~~~~~~~~~~~ 382 383The SP content certificate is inserted as a separate FIP item so that BL2 loads SPMC, 384SPMC manifest and Secure Partitions and verifies them for authenticity and integrity. 385Refer to TBBR specification `[3]`_. 386 387The multiple-signing domain feature (in current state dual signing domain) allows 388the use of two root keys namely S-ROTPK and NS-ROTPK (see `[8]`_): 389 390- SPMC (BL32) and SPMC manifest are signed by the SiP using the S-ROTPK. 391- BL33 may be signed by the OEM using NS-ROTPK. 392- An SP may be signed either by SiP (using S-ROTPK) or by OEM (using NS-ROTPK). 393 394Longer term multiple signing domain will allow additional signing keys, e.g. 395if SPs originate from different parties. 396 397See `TF-A build options`_ for a sample build command line. 398 399Hafnium in the secure world 400=========================== 401 402**NOTE: this section is work in progress. Descriptions and implementation choices 403are subject to evolve.** 404 405General considerations 406---------------------- 407 408Build platform for the secure world 409~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 410 411The implementation might add specific code parts only relevant to the 412secure world. Such code parts might be isolated into different files 413and/or conditional code enclosed by a ``SECURE_WORLD`` macro. 414 415Secure Partitions CPU scheduling 416~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 417 418In the normal world, VMs are scheduled by the FFA_RUN ABI invoked from the 419primary scheduler (in the primary VM), or by a direct message request or 420response. 421 422With the FF-A EAC specification, Secure Partitions are scheduled by direct 423message invocations from a NWd VM or another SP. 424 425Platform topology 426~~~~~~~~~~~~~~~~~ 427 428As stated in `[1]`_ section 4.4.1 the SPMC implementation assumes the 429following SP types: 430 431- Pinned MP SPs: an Execution Context id matches a physical PE id. MP 432 SPs must implement the same number of ECs as the number of PEs in the 433 platform. Hence the *execution-ctx-count* as defined by 434 `[1]`_ (or NWd-Hafnium *vcpu_count*) can only take the 435 value of one or the number of physical PEs. 436- Migratable UP SPs: a single execution context can run and be migrated 437 on any physical PE. It declares a single EC in its SP manifest. An UP 438 SP can receive a direct message request on any physical core. 439 440Usage of PSCI services in the secure world 441~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 442 443- The normal world Hypervisor (optional) or OS kernel issues PSCI service 444 invocations e.g. to request PSCI version, wake-up a secondary core, or request 445 core suspend. This happens at the non-secure physical FF-A instance. In the 446 example case of Hafnium in the normal world, it boots on the primary core and 447 one of the first initialization step is to request the PSCI version. It then 448 launches the primary VM. The primary VM upon initializing performs PSCI service 449 calls (at non-secure virtual FF-A instance) which are trapped by the 450 Hypervisor. Invocation from OS Kernel ends straight at EL3. The PVM issues 451 ``PSCI_CPU_ON`` service calls to wake-up secondary cores by passing an 452 ``MPIDR``, entry point address and a CPU context address. The EL3 PSCI layer 453 then performs an exception return to the secondary core entry point on the 454 targeted core. Other PSCI calls can happen at run-time from the PVM e.g. to 455 request core suspend. 456- In the existing TF-A PSCI standard library, PSCI service calls are filtered at 457 EL3 to only originate from the NWd. Thus concerning the SPMC (at secure 458 physical FF-A instance) the PSCI service invocations cannot happen as in the 459 normal world. For example, a ``PSCI_CPU_ON`` service invocation from the SPMC 460 does not reach the PSCI layer. 461 462Parsing SP partition manifests 463------------------------------ 464 465Hafnium must be able to consume SP manifests as defined in 466`[1]`_ section 3.1, at least for the mandatory fields. 467 468The SP manifest may contain memory and device regions nodes. 469 470- Memory regions shall be mapped in the SP Stage-2 translation regime at 471 load time. A memory region node can specify RX/TX buffer regions in which 472 case it is not necessary for an SP to explicitly call the ``FFA_RXTX_MAP`` 473 service. 474- Device regions shall be mapped in SP Stage-2 translation regime as 475 peripherals and possibly allocate additional resources (e.g. interrupts) 476 477Base addresses for memory and device region nodes are IPAs provided SPMC 478identity maps IPAs to PAs within SP Stage-2 translation regime. 479 480Note: currently both VTTBR_EL2 and VSTTBR_EL2 resolve to the same set of page 481tables. It is still open whether two sets of page tables shall be provided per 482SP. The memory region node as defined in the spec (section 3.1 Table 10) 483provides a memory security attribute hinting to map either to the secure or 484non-secure stage-2 table. 485 486Passing boot data to the SP 487--------------------------- 488 489`[1]`_ Section 3.4.2 “Protocol for passing data” defines a 490method to passing boot data to SPs (not currently implemented). 491 492Provided that the whole Secure Partition package image (see `Secure 493Partition packages`_) is mapped to the SP's secure Stage-2 translation 494regime, an SP can access its own manifest DTB blob and extract its partition 495manifest properties. 496 497SP Boot order 498------------- 499 500SP manifests provide an optional boot order attribute meant to resolve 501dependencies such as an SP providing a service required to properly boot 502another SP. 503 504Boot phases 505----------- 506 507Primary core boot-up 508~~~~~~~~~~~~~~~~~~~~ 509 510The SPMC performs its platform initializations then loads and creates 511secure partitions based on SP packages and manifests. Then each secure 512partition is launched in sequence (see `SP Boot order`_) on their primary 513Execution Context. 514 515Notice the primary physical core may not be core 0. Hence if the primary 516core linear id is N, the 1:1 mapping requires MP SPs are launched using 517EC[N] on PE[N] (see `Platform topology`_). 518 519The SP's primary Execution Context (or the EC used when the partition is booted) 520exits through ``FFA_MSG_WAIT`` to indicate successful initialization. 521 522Secondary physical core boot-up 523~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 524 525Upon boot-up, the SPMC running on the primary core performs 526implementation-defined SPMD service calls at secure physical FF-A instance 527to register the secondary physical cores entry points and context information: 528 529- This is done through a direct message request invocation to the SPMD 530 (``SET_ENTRY_POINT``). This service call does not wake-up the targeted 531 core immediately. The secondary core is woken up later by a NWd 532 ``PSCI_CPU_ON`` service invocation. A notification is passed from EL3 533 PSCI layer to the SPMD, and then to SPMC through an implementation-defined 534 interface. 535- The SPMC/SPMD interface can consist of FF-A direct message requests/responses 536 transporting PM events. 537 538If there is no Hypervisor in the normal world, the OS Kernel issues 539``PSCI_CPU_ON`` calls that are directly trapped to EL3. 540 541When a secondary physical core wakes-up the SPMD notifies the SPMC which updates 542its internal states reflecting current physical core is being turned on. 543It might then return straight to the SPMD and then to the NWd. 544 545*(under discussion)* There may be possibility that an SP registers "PM events" 546(during primary EC boot stage) through an ad-hoc interface. Such events would 547be relayed by SPMC to one or more registered SPs on need basis 548(see `Power management`_). 549 550Secondary virtual core boot-up 551~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 552 553In the example case where Hafnium exists in the normal world, secondary VMs 554issue a ``PSCI_CPU_ON`` service call which is trapped to the Hypervisor. The 555latter then enables the vCPU context for the targeted core, and switches to 556the PVM down to the kernel driver with an ``HF_WAKE_UP`` message. The NWd 557driver in PVM can then schedule the newly woken up vCPU context. 558 559In the secure world the primary EC of a given SP passes the secondary EC entry 560point and context. The SMC service call is trapped into the SPMC. This can be 561either *(under discussion)*: 562 563- a specific interface registering the secondary EC entry point, 564 similarly to above ``SET_ENTRY_POINT`` service. 565- Re-purposing the ``PSCI_CPU_ON`` function id. It is 566 assumed that even if the input arguments are the same as the ones defined in 567 the PSCI standard, the usage deviates by the fact the secondary EC is not 568 woken up immediately. At least for the PSA-FF-A EAC where only 569 direct messaging is allowed, it is only after the first direct 570 message invocation that the secondary EC is entered. This option 571 might be preferred when the same code base is re-used for a VM or 572 an SP. The ABI to wake-up a secondary EC can remain similar. 573 574SPs are always scheduled from the NWd, this paradigm did not change from legacy 575TEEs. There must always be some logic (or driver) in the NWd to relinquish CPU 576cycles to the SWd. If primary core is 0, an SP EC[x>0] entry point is supplied 577by the SP EC[0] when the system boots in SWd. But this EC[x] is not immediately 578entered at boot. Later in the boot process when NWd is up, a direct message 579request issued from physical core 1 ends up in SP EC[1], and only at this stage 580this context is effectively scheduled. 581 582It should be possible for an SP to call into another SP through direct message 583provided the latter SP has been booted already. The "boot-order" field in 584partition manifests (`SP Boot order`_) fulfills the dependency towards availability 585of a service within an SP offered to another SP. 586 587Mandatory interfaces 588-------------------- 589 590The following interfaces must be exposed to any VM or SP: 591 592- ``FFA_STATUS`` 593- ``FFA_ERROR`` 594- ``FFA_INTERRUPT`` 595- ``FFA_VERSION`` 596- ``FFA_FEATURES`` 597- ``FFA_RX_RELEASE`` 598- ``FFA_RXTX_MAP`` 599- ``FFA_RXTX_UNMAP`` 600- ``FFA_PARTITION_INFO_GET`` 601- ``FFA_ID_GET`` 602 603FFA_VERSION 604~~~~~~~~~~~ 605 606Per `[1]`_ section 8.1 ``FFA_VERSION`` requires a 607*requested_version* parameter from the caller. 608 609In the current implementation when ``FFA_VERSION`` is invoked from: 610 611- Hypervisor in NS-EL2: the SPMD returns the SPMC version specified 612 in the SPMC manifest. 613- OS kernel in NS-EL1 when NS-EL2 is not present: the SPMD returns the 614 SPMC version specified in the SPMC manifest. 615- VM in NWd: the Hypervisor returns its implemented version. 616- SP in SWd: the SPMC returns its implemented version. 617- SPMC at S-EL1/S-EL2: the SPMD returns its implemented version. 618 619FFA_FEATURES 620~~~~~~~~~~~~ 621 622FF-A features may be discovered by Secure Partitions while booting 623through the SPMC. However, SPMC cannot get features from Hypervisor 624early at boot time as NS world is not setup yet. 625 626The Hypervisor may decide to gather FF-A features from SPMC through SPMD 627once at boot time and store the result. Later when a VM requests FF-A 628features, the Hypervisor can adjust its own set of features with what 629SPMC advertised, if necessary. Another approach is to always forward FF-A 630features to the SPMC when a VM requests it to the Hypervisor. Although 631the result is not supposed to change over time so there may not be added 632value doing the systematic forwarding. 633 634FFA_RXTX_MAP/FFA_RXTX_UNMAP 635~~~~~~~~~~~~~~~~~~~~~~~~~~~ 636 637VM mailboxes are re-purposed to serve as SP RX/TX buffers. The RX/TX 638map API maps the send and receive buffer IPAs to the SP Stage-2 translation regime. 639 640Hafnium in the normal world defines VMs and their attributes as logical structures, 641including a mailbox used for FF-A indirect messaging, memory sharing, or the 642`FFA_PARTITION_INFO_GET`_ ABI. 643This same mailbox structure is re-used in the SPMC. `[1]`_ states only direct 644messaging is allowed to SPs. Thus mailbox usage is restricted to implementing 645`FFA_PARTITION_INFO_GET`_ and memory sharing ABIs. 646 647FFA_PARTITION_INFO_GET 648~~~~~~~~~~~~~~~~~~~~~~ 649 650Partition info get service call can originate: 651 652- from SP to SPM 653- from VM to Hypervisor 654- from Hypervisor to SPM 655 656For the latter case, the service call must be forwarded through the SPMD. 657 658FFA_ID_GET 659~~~~~~~~~~ 660 661The SPMD returns: 662 663- a default zero value on invocation from the Hypervisor. 664- The ``spmc_id`` value specified in the SPMC manifest on invocation from 665 the SPMC (see `SPMC manifest`_) 666 667The FF-A id space is split into a non-secure space and secure space: 668 669- FF-A id with bit 15 clear refer to normal world VMs. 670- FF-A id with bit 15 set refer to secure world SPs 671 672Such convention helps the SPMC discriminating the origin and destination worlds 673in an FF-A service invocation. In particular the SPMC shall filter unauthorized 674transactions in its world switch routine. It must not be permitted for a VM to 675use a secure FF-A id as origin world through spoofing: 676 677- A VM-to-SP messaging passing shall have an origin world being non-secure 678 (FF-A id bit 15 clear) and destination world being secure (FF-A id bit 15 679 set). 680- Similarly, an SP-to-SP message shall have FF-A id bit 15 set for both origin 681 and destination ids. 682 683An incoming direct message request arriving at SPMD from NWd is forwarded to 684SPMC without a specific check. The SPMC is resumed through eret and "knows" the 685message is coming from normal world in this specific code path. Thus the origin 686endpoint id must be checked by SPMC for being a normal world id. 687 688An SP sending a direct message request must have bit 15 set in its origin 689endpoint id and this can be checked by the SPMC when the SP invokes the ABI. 690 691The SPMC shall reject the direct message if the claimed world in origin endpoint 692id is not consistent: 693 694- It is either forwarded by SPMD and thus origin endpoint id must be a "normal 695 world id", 696- or initiated by an SP and thus origin endpoint id must be a "secure world id". 697 698Direct messaging 699---------------- 700 701This is a mandatory interface for Secure Partitions consisting in direct 702message request and responses. 703 704The ``ffa_handler`` Hafnium function may: 705 706- trigger a world change e.g. when an SP invokes the direct message 707 response ABI to a VM. 708- handle multiple requests from the NWd without resuming an SP. 709 710SP-to-SP 711~~~~~~~~ 712 713- An SP can send a direct message request to another SP 714- An SP can receive a direct message response from another SP. 715 716VM-to-SP 717~~~~~~~~ 718 719- A VM can send a direct message request to an SP 720- An SP can send a direct message response to a VM 721 722SPMC-SPMD messaging 723~~~~~~~~~~~~~~~~~~~ 724 725Specific implementation-defined endpoint IDs are allocated to the SPMC and SPMD. 726Referring those IDs in source/destination fields of a direct message 727request/response permits SPMD to SPMC messaging back and forth. 728 729Per `[1]`_ Table 114 Config No. 1 (physical FF-A instance): 730 731- SPMC=>SPMD direct message request uses SMC conduit 732- SPMD=>SPMC direct message request uses ERET conduit 733 734Per `[1]`_ Table 118 Config No. 1 (physical FF-A instance): 735 736- SPMC=>SPMD direct message response uses SMC conduit 737- SPMD=>SPMC direct message response uses ERET conduit 738 739Memory management 740----------------- 741 742This section only deals with the PE MMU configuration. 743 744Hafnium in the normal world deals with NS buffers only and provisions 745a single root page table directory to VMs. In context of S-EL2 enabled 746firmware, two IPA spaces are output from Stage-1 translation (secure 747and non-secure). The Stage-2 translation handles: 748 749- A single secure IPA space when an SP Stage-1 MMU is disabled. 750- Two IPA spaces (secure and non-secure) when Stage-1 MMU is enabled. 751 752``VTCR_EL2`` and ``VSTCR_EL2`` provide additional bits for controlling the 753NS/S IPA translations (``VSTCR_EL2.SW``, ``VSTCR_EL2.SA``, ``VTCR_EL2.NSW``, 754``VTCR_EL2.NSA``). There may be two approaches: 755 756- secure and non-secure mappings are rooted as two separate root page 757 tables 758- secure and non-secure mappings use the same root page table. Access 759 from S-EL1 to an NS region translates to a secure physical address 760 space access. 761 762Interrupt management 763-------------------- 764 765Road to a para-virtualized interface 766~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 767 768Current Hafnium implementation uses an ad-hoc mechanism for a VM to get 769a pending interrupt number through an hypercall. The PVM injects 770interrupts to VMs by delegation from the Hypervisor. The PVM probes a 771pending interrupt directly from the GIC distributor. 772 773The short-term plan is to have Hafnium/SPMC in the secure world owner 774of the GIC configuration. 775 776The SPMC fully owns the GIC configuration at S-EL2. The SPMC manages 777interrupt resources and allocates interrupt ID based on SP manifests. 778The SPMC acknowledges physical interrupts and injects virtual interrupts 779by setting the vIRQ bit when resuming an SP. A Secure Partition gathers 780the interrupt number through an hypercall. 781 782Notice the SPMC/SPMD has to handle Group0 secure interrupts in addition 783to Group1 S/NS interrupts. 784 785Power management 786---------------- 787 788Assumption on the Nwd: 789 790- NWd is the best candidate to own the platform Power Management 791 policy. It is master to invoking PSCI service calls from physical 792 CPUs. 793- EL3 monitor is in charge of the PM control part (its PSCI layer 794 actually writing to platform registers). 795- It is fine for the Hypervisor to trap PSCI calls and relay to EL3, or 796 OS kernel driver to emit PSCI service calls. 797 798PSCI notification are relayed through the SPMD/SPD PM hooks to the SPMC. 799This can either be through re-use of PSCI FIDs or an FF-A direct message 800from SPMD to SPMC. 801 802The SPMD performs an exception return to the SPMC which is resumed to 803its ``eret_handler`` routine. It is then either consuming a PSCI FID or 804an FF-A FID. Depending on the servicing, the SPMC may return directly to 805the SPMD (and then NWd) without resuming an SP at this stage. An example 806of this is invocation of ``FFA_PARTITION_INFO_GET`` from NWd relayed by 807the SPMD to the SPMC. The SPMC returns the needed partition information 808to the SPMD (then NWd) without actually resuming a partition in secure world. 809 810*(under discussion)* 811About using PSCI FIDs from SPMD to SPMC to notify of PM events, it is still 812questioned what to use as the return code from the SPMC. 813If the function ID used by the SPMC is not an FF-A ID when doing SMC, then the 814EL3 std svc handler won't route the response to the SPMD. That's where comes the 815idea to embed the notification into an FF-A message. The SPMC can discriminate 816this message as being a PSCI event, process it, and reply with an FF-A return 817message that the SPMD receives as an acknowledgement. 818 819SP notification 820--------------- 821 822Power management notifications are conveyed from PSCI library to the 823SPMD / SPD hooks. A range of events can be relayed to SPMC. 824 825SPs may need to be notified about specific PM events. 826 827- SPs might register PM events to the SPMC 828- On SPMD to SPMC notification, a limited range of SPs may be notified 829 through a direct message. 830- This assumes the mentioned SPs supports managed exit. 831 832The SPMC is the first to be notified about PM events from the SPMD. It is up 833to the SPMC to arbitrate to which SP it needs to send PM events. 834An SP explicitly registers to receive notifications to specific PM events. 835The register operation can either be an implementation-defined service call 836to the SPMC when the primary SP EC boots, or be supplied through the SP 837manifest. 838 839Support for SMMUv3 in Hafnium 840============================= 841 842An SMMU is analogous to an MMU in a CPU. It performs address translations for 843Direct Memory Access (DMA) requests from system I/O devices. 844The responsibilities of an SMMU include: 845 846- Translation: Incoming DMA requests are translated from bus address space to 847 system physical address space using translation tables compliant to 848 Armv8/Armv7 VMSA descriptor format. 849- Protection: An I/O device can be prohibited from read, write access to a 850 memory region or allowed. 851- Isolation: Traffic from each individial device can be independently managed. 852 The devices are differentiated from each other using unique translation 853 tables. 854 855The following diagram illustrates a typical SMMU IP integrated in a SoC with 856several I/O devices along with Interconnect and Memory system. 857 858.. image:: ../resources/diagrams/MMU-600.png 859 860SMMU has several versions including SMMUv1, SMMUv2 and SMMUv3. Hafnium provides 861support for SMMUv3 driver in both Normal and Secure World. A brief introduction 862of SMMUv3 functionality and the corresponding software support in Hafnium is 863provided here. 864 865SMMUv3 features 866--------------- 867 868- SMMUv3 provides Stage1, Stage2 translation as well as nested (Stage1 + Stage2) 869 translation support. It can either bypass or abort incoming translations as 870 well. 871- Traffic (memory transactions) from each upstream I/O peripheral device, 872 referred to as Stream, can be independently managed using a combination of 873 several memory based configuration structures. This allows the SMMUv3 to 874 support a large number of streams with each stream assigned to a unique 875 translation context. 876- Support for Armv8.1 VMSA where the SMMU shares the translation tables with 877 a Processing Element. AArch32(LPAE) and AArch64 translation table format 878 are supported by SMMUv3. 879- SMMUv3 offers non-secure stream support with secure stream support being 880 optional. Logically, SMMUv3 behaves as if there is an indepdendent SMMU 881 instance for secure and non-secure stream support. 882- It also supports sub-streams to differentiate traffic from a virtualized 883 peripheral associated with a VM/SP. 884- Additionally, SMMUv3.2 provides support for PEs implementing Armv8.4-A 885 extensions. Consequently, SPM depends on Secure EL2 support in SMMUv3.2 886 for providing Secure Stage2 translation support to upstream peripheral 887 devices. 888 889SMMUv3 Programming Interfaces 890----------------------------- 891 892SMMUv3 has three software interfaces that are used by the Hafnium driver to 893configure the behaviour of SMMUv3 and manage the streams. 894 895- Memory based data strutures that provide unique translation context for 896 each stream. 897- Memory based circular buffers for command queue and event queue. 898- A large number of SMMU configuration registers that are memory mapped during 899 boot time by Hafnium driver. Except a few registers, all configuration 900 registers have independent secure and non-secure versions to configure the 901 behaviour of SMMUv3 for translation of secure and non-secure streams 902 respectively. 903 904Peripheral device manifest 905-------------------------- 906 907Currently, SMMUv3 driver in Hafnium only supports dependent peripheral devices. 908These devices are dependent on PE endpoint to initiate and receive memory 909management transactions on their behalf. The acccess to the MMIO regions of 910any such device is assigned to the endpoint during boot. Moreover, SMMUv3 driver 911uses the same stage 2 translations for the device as those used by partition 912manager on behalf of the PE endpoint. This ensures that the peripheral device 913has the same visibility of the physical address space as the endpoint. The 914device node of the corresponding partition manifest (refer to `[1]`_ section 3.2 915) must specify these additional properties for each peripheral device in the 916system : 917 918- smmu-id: This field helps to identify the SMMU instance that this device is 919 upstream of. 920- stream-ids: List of stream IDs assigned to this device. 921 922.. code:: shell 923 924 smmuv3-testengine { 925 base-address = <0x00000000 0x2bfe0000>; 926 pages-count = <32>; 927 attributes = <0x3>; 928 smmu-id = <0>; 929 stream-ids = <0x0 0x1>; 930 interrupts = <0x2 0x3>, <0x4 0x5>; 931 exclusive-access; 932 }; 933 934SMMUv3 driver limitations 935------------------------- 936 937The primary design goal for the Hafnium SMMU driver is to support secure 938streams. 939 940- Currently, the driver only supports Stage2 translations. No support for 941 Stage1 or nested translations. 942- Supports only AArch64 translation format. 943- No support for features such as PCI Express (PASIDs, ATS, PRI), MSI, RAS, 944 Fault handling, Performance Monitor Extensions, Event Handling, MPAM. 945- No support for independent peripheral devices. 946 947References 948========== 949 950.. _[1]: 951 952[1] `Platform Security Architecture Firmware Framework for Arm® v8-A 1.0 Platform Design Document <https://developer.arm.com/docs/den0077/latest>`__ 953 954.. _[2]: 955 956[2] :ref:`Secure Partition Manager using MM interface<Secure Partition Manager (MM)>` 957 958.. _[3]: 959 960[3] `Trusted Boot Board Requirements 961Client <https://developer.arm.com/docs/den0006/latest/trusted-board-boot-requirements-client-tbbr-client-armv8-a>`__ 962 963.. _[4]: 964 965[4] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/lib/el3_runtime/aarch64/context.S#n45 966 967.. _[5]: 968 969[5] https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/cactus.dts 970 971.. _[6]: 972 973[6] https://trustedfirmware-a.readthedocs.io/en/latest/components/psa-ffa-manifest-binding.html 974 975.. _[7]: 976 977[7] https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/plat/arm/board/fvp/fdts/fvp_spmc_manifest.dts 978 979.. _[8]: 980 981[8] https://developer.trustedfirmware.org/w/tf_a/poc-multiple-signing-domains/ 982 983-------------- 984 985*Copyright (c) 2020, Arm Limited and Contributors. All rights reserved.* 986
