1*4882a593Smuzhiyun.. SPDX-License-Identifier: GPL-2.0 2*4882a593Smuzhiyun 3*4882a593Smuzhiyun========================================= 4*4882a593Smuzhiyuns390 (IBM Z) Ultravisor and Protected VMs 5*4882a593Smuzhiyun========================================= 6*4882a593Smuzhiyun 7*4882a593SmuzhiyunSummary 8*4882a593Smuzhiyun------- 9*4882a593SmuzhiyunProtected virtual machines (PVM) are KVM VMs that do not allow KVM to 10*4882a593Smuzhiyunaccess VM state like guest memory or guest registers. Instead, the 11*4882a593SmuzhiyunPVMs are mostly managed by a new entity called Ultravisor (UV). The UV 12*4882a593Smuzhiyunprovides an API that can be used by PVMs and KVM to request management 13*4882a593Smuzhiyunactions. 14*4882a593Smuzhiyun 15*4882a593SmuzhiyunEach guest starts in non-protected mode and then may make a request to 16*4882a593Smuzhiyuntransition into protected mode. On transition, KVM registers the guest 17*4882a593Smuzhiyunand its VCPUs with the Ultravisor and prepares everything for running 18*4882a593Smuzhiyunit. 19*4882a593Smuzhiyun 20*4882a593SmuzhiyunThe Ultravisor will secure and decrypt the guest's boot memory 21*4882a593Smuzhiyun(i.e. kernel/initrd). It will safeguard state changes like VCPU 22*4882a593Smuzhiyunstarts/stops and injected interrupts while the guest is running. 23*4882a593Smuzhiyun 24*4882a593SmuzhiyunAs access to the guest's state, such as the SIE state description, is 25*4882a593Smuzhiyunnormally needed to be able to run a VM, some changes have been made in 26*4882a593Smuzhiyunthe behavior of the SIE instruction. A new format 4 state description 27*4882a593Smuzhiyunhas been introduced, where some fields have different meanings for a 28*4882a593SmuzhiyunPVM. SIE exits are minimized as much as possible to improve speed and 29*4882a593Smuzhiyunreduce exposed guest state. 30*4882a593Smuzhiyun 31*4882a593Smuzhiyun 32*4882a593SmuzhiyunInterrupt injection 33*4882a593Smuzhiyun------------------- 34*4882a593SmuzhiyunInterrupt injection is safeguarded by the Ultravisor. As KVM doesn't 35*4882a593Smuzhiyunhave access to the VCPUs' lowcores, injection is handled via the 36*4882a593Smuzhiyunformat 4 state description. 37*4882a593Smuzhiyun 38*4882a593SmuzhiyunMachine check, external, IO and restart interruptions each can be 39*4882a593Smuzhiyuninjected on SIE entry via a bit in the interrupt injection control 40*4882a593Smuzhiyunfield (offset 0x54). If the guest cpu is not enabled for the interrupt 41*4882a593Smuzhiyunat the time of injection, a validity interception is recognized. The 42*4882a593Smuzhiyunformat 4 state description contains fields in the interception data 43*4882a593Smuzhiyunblock where data associated with the interrupt can be transported. 44*4882a593Smuzhiyun 45*4882a593SmuzhiyunProgram and Service Call exceptions have another layer of 46*4882a593Smuzhiyunsafeguarding; they can only be injected for instructions that have 47*4882a593Smuzhiyunbeen intercepted into KVM. The exceptions need to be a valid outcome 48*4882a593Smuzhiyunof an instruction emulation by KVM, e.g. we can never inject a 49*4882a593Smuzhiyunaddressing exception as they are reported by SIE since KVM has no 50*4882a593Smuzhiyunaccess to the guest memory. 51*4882a593Smuzhiyun 52*4882a593Smuzhiyun 53*4882a593SmuzhiyunMask notification interceptions 54*4882a593Smuzhiyun------------------------------- 55*4882a593SmuzhiyunKVM cannot intercept lctl(g) and lpsw(e) anymore in order to be 56*4882a593Smuzhiyunnotified when a PVM enables a certain class of interrupt. As a 57*4882a593Smuzhiyunreplacement, two new interception codes have been introduced: One 58*4882a593Smuzhiyunindicating that the contents of CRs 0, 6, or 14 have been changed, 59*4882a593Smuzhiyunindicating different interruption subclasses; and one indicating that 60*4882a593SmuzhiyunPSW bit 13 has been changed, indicating that a machine check 61*4882a593Smuzhiyunintervention was requested and those are now enabled. 62*4882a593Smuzhiyun 63*4882a593SmuzhiyunInstruction emulation 64*4882a593Smuzhiyun--------------------- 65*4882a593SmuzhiyunWith the format 4 state description for PVMs, the SIE instruction already 66*4882a593Smuzhiyuninterprets more instructions than it does with format 2. It is not able 67*4882a593Smuzhiyunto interpret every instruction, but needs to hand some tasks to KVM; 68*4882a593Smuzhiyuntherefore, the SIE and the ultravisor safeguard emulation inputs and outputs. 69*4882a593Smuzhiyun 70*4882a593SmuzhiyunThe control structures associated with SIE provide the Secure 71*4882a593SmuzhiyunInstruction Data Area (SIDA), the Interception Parameters (IP) and the 72*4882a593SmuzhiyunSecure Interception General Register Save Area. Guest GRs and most of 73*4882a593Smuzhiyunthe instruction data, such as I/O data structures, are filtered. 74*4882a593SmuzhiyunInstruction data is copied to and from the SIDA when needed. Guest 75*4882a593SmuzhiyunGRs are put into / retrieved from the Secure Interception General 76*4882a593SmuzhiyunRegister Save Area. 77*4882a593Smuzhiyun 78*4882a593SmuzhiyunOnly GR values needed to emulate an instruction will be copied into this 79*4882a593Smuzhiyunsave area and the real register numbers will be hidden. 80*4882a593Smuzhiyun 81*4882a593SmuzhiyunThe Interception Parameters state description field still contains 82*4882a593Smuzhiyunthe bytes of the instruction text, but with pre-set register values 83*4882a593Smuzhiyuninstead of the actual ones. I.e. each instruction always uses the same 84*4882a593Smuzhiyuninstruction text, in order not to leak guest instruction text. 85*4882a593SmuzhiyunThis also implies that the register content that a guest had in r<n> 86*4882a593Smuzhiyunmay be in r<m> from the hypervisor's point of view. 87*4882a593Smuzhiyun 88*4882a593SmuzhiyunThe Secure Instruction Data Area contains instruction storage 89*4882a593Smuzhiyundata. Instruction data, i.e. data being referenced by an instruction 90*4882a593Smuzhiyunlike the SCCB for sclp, is moved via the SIDA. When an instruction is 91*4882a593Smuzhiyunintercepted, the SIE will only allow data and program interrupts for 92*4882a593Smuzhiyunthis instruction to be moved to the guest via the two data areas 93*4882a593Smuzhiyundiscussed before. Other data is either ignored or results in validity 94*4882a593Smuzhiyuninterceptions. 95*4882a593Smuzhiyun 96*4882a593Smuzhiyun 97*4882a593SmuzhiyunInstruction emulation interceptions 98*4882a593Smuzhiyun----------------------------------- 99*4882a593SmuzhiyunThere are two types of SIE secure instruction intercepts: the normal 100*4882a593Smuzhiyunand the notification type. Normal secure instruction intercepts will 101*4882a593Smuzhiyunmake the guest pending for instruction completion of the intercepted 102*4882a593Smuzhiyuninstruction type, i.e. on SIE entry it is attempted to complete 103*4882a593Smuzhiyunemulation of the instruction with the data provided by KVM. That might 104*4882a593Smuzhiyunbe a program exception or instruction completion. 105*4882a593Smuzhiyun 106*4882a593SmuzhiyunThe notification type intercepts inform KVM about guest environment 107*4882a593Smuzhiyunchanges due to guest instruction interpretation. Such an interception 108*4882a593Smuzhiyunis recognized, for example, for the store prefix instruction to provide 109*4882a593Smuzhiyunthe new lowcore location. On SIE reentry, any KVM data in the data areas 110*4882a593Smuzhiyunis ignored and execution continues as if the guest instruction had 111*4882a593Smuzhiyuncompleted. For that reason KVM is not allowed to inject a program 112*4882a593Smuzhiyuninterrupt. 113*4882a593Smuzhiyun 114*4882a593SmuzhiyunLinks 115*4882a593Smuzhiyun----- 116*4882a593Smuzhiyun`KVM Forum 2019 presentation <https://static.sched.com/hosted_files/kvmforum2019/3b/ibm_protected_vms_s390x.pdf>`_ 117