1*4882a593Smuzhiyun=================== 2*4882a593SmuzhiyunLinux IOMMU Support 3*4882a593Smuzhiyun=================== 4*4882a593Smuzhiyun 5*4882a593SmuzhiyunThe architecture spec can be obtained from the below location. 6*4882a593Smuzhiyun 7*4882a593Smuzhiyunhttp://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf 8*4882a593Smuzhiyun 9*4882a593SmuzhiyunThis guide gives a quick cheat sheet for some basic understanding. 10*4882a593Smuzhiyun 11*4882a593SmuzhiyunSome Keywords 12*4882a593Smuzhiyun 13*4882a593Smuzhiyun- DMAR - DMA remapping 14*4882a593Smuzhiyun- DRHD - DMA Remapping Hardware Unit Definition 15*4882a593Smuzhiyun- RMRR - Reserved memory Region Reporting Structure 16*4882a593Smuzhiyun- ZLR - Zero length reads from PCI devices 17*4882a593Smuzhiyun- IOVA - IO Virtual address. 18*4882a593Smuzhiyun 19*4882a593SmuzhiyunBasic stuff 20*4882a593Smuzhiyun----------- 21*4882a593Smuzhiyun 22*4882a593SmuzhiyunACPI enumerates and lists the different DMA engines in the platform, and 23*4882a593Smuzhiyundevice scope relationships between PCI devices and which DMA engine controls 24*4882a593Smuzhiyunthem. 25*4882a593Smuzhiyun 26*4882a593SmuzhiyunWhat is RMRR? 27*4882a593Smuzhiyun------------- 28*4882a593Smuzhiyun 29*4882a593SmuzhiyunThere are some devices the BIOS controls, for e.g USB devices to perform 30*4882a593SmuzhiyunPS2 emulation. The regions of memory used for these devices are marked 31*4882a593Smuzhiyunreserved in the e820 map. When we turn on DMA translation, DMA to those 32*4882a593Smuzhiyunregions will fail. Hence BIOS uses RMRR to specify these regions along with 33*4882a593Smuzhiyundevices that need to access these regions. OS is expected to setup 34*4882a593Smuzhiyununity mappings for these regions for these devices to access these regions. 35*4882a593Smuzhiyun 36*4882a593SmuzhiyunHow is IOVA generated? 37*4882a593Smuzhiyun---------------------- 38*4882a593Smuzhiyun 39*4882a593SmuzhiyunWell behaved drivers call pci_map_*() calls before sending command to device 40*4882a593Smuzhiyunthat needs to perform DMA. Once DMA is completed and mapping is no longer 41*4882a593Smuzhiyunrequired, device performs a pci_unmap_*() calls to unmap the region. 42*4882a593Smuzhiyun 43*4882a593SmuzhiyunThe Intel IOMMU driver allocates a virtual address per domain. Each PCIE 44*4882a593Smuzhiyundevice has its own domain (hence protection). Devices under p2p bridges 45*4882a593Smuzhiyunshare the virtual address with all devices under the p2p bridge due to 46*4882a593Smuzhiyuntransaction id aliasing for p2p bridges. 47*4882a593Smuzhiyun 48*4882a593SmuzhiyunIOVA generation is pretty generic. We used the same technique as vmalloc() 49*4882a593Smuzhiyunbut these are not global address spaces, but separate for each domain. 50*4882a593SmuzhiyunDifferent DMA engines may support different number of domains. 51*4882a593Smuzhiyun 52*4882a593SmuzhiyunWe also allocate guard pages with each mapping, so we can attempt to catch 53*4882a593Smuzhiyunany overflow that might happen. 54*4882a593Smuzhiyun 55*4882a593Smuzhiyun 56*4882a593SmuzhiyunGraphics Problems? 57*4882a593Smuzhiyun------------------ 58*4882a593SmuzhiyunIf you encounter issues with graphics devices, you can try adding 59*4882a593Smuzhiyunoption intel_iommu=igfx_off to turn off the integrated graphics engine. 60*4882a593SmuzhiyunIf this fixes anything, please ensure you file a bug reporting the problem. 61*4882a593Smuzhiyun 62*4882a593SmuzhiyunSome exceptions to IOVA 63*4882a593Smuzhiyun----------------------- 64*4882a593SmuzhiyunInterrupt ranges are not address translated, (0xfee00000 - 0xfeefffff). 65*4882a593SmuzhiyunThe same is true for peer to peer transactions. Hence we reserve the 66*4882a593Smuzhiyunaddress from PCI MMIO ranges so they are not allocated for IOVA addresses. 67*4882a593Smuzhiyun 68*4882a593Smuzhiyun 69*4882a593SmuzhiyunFault reporting 70*4882a593Smuzhiyun--------------- 71*4882a593SmuzhiyunWhen errors are reported, the DMA engine signals via an interrupt. The fault 72*4882a593Smuzhiyunreason and device that caused it with fault reason is printed on console. 73*4882a593Smuzhiyun 74*4882a593SmuzhiyunSee below for sample. 75*4882a593Smuzhiyun 76*4882a593Smuzhiyun 77*4882a593SmuzhiyunBoot Message Sample 78*4882a593Smuzhiyun------------------- 79*4882a593Smuzhiyun 80*4882a593SmuzhiyunSomething like this gets printed indicating presence of DMAR tables 81*4882a593Smuzhiyunin ACPI. 82*4882a593Smuzhiyun 83*4882a593SmuzhiyunACPI: DMAR (v001 A M I OEMDMAR 0x00000001 MSFT 0x00000097) @ 0x000000007f5b5ef0 84*4882a593Smuzhiyun 85*4882a593SmuzhiyunWhen DMAR is being processed and initialized by ACPI, prints DMAR locations 86*4882a593Smuzhiyunand any RMRR's processed:: 87*4882a593Smuzhiyun 88*4882a593Smuzhiyun ACPI DMAR:Host address width 36 89*4882a593Smuzhiyun ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000 90*4882a593Smuzhiyun ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000 91*4882a593Smuzhiyun ACPI DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000 92*4882a593Smuzhiyun ACPI DMAR:RMRR base: 0x00000000000ed000 end: 0x00000000000effff 93*4882a593Smuzhiyun ACPI DMAR:RMRR base: 0x000000007f600000 end: 0x000000007fffffff 94*4882a593Smuzhiyun 95*4882a593SmuzhiyunWhen DMAR is enabled for use, you will notice.. 96*4882a593Smuzhiyun 97*4882a593SmuzhiyunPCI-DMA: Using DMAR IOMMU 98*4882a593Smuzhiyun------------------------- 99*4882a593Smuzhiyun 100*4882a593SmuzhiyunFault reporting 101*4882a593Smuzhiyun^^^^^^^^^^^^^^^ 102*4882a593Smuzhiyun 103*4882a593Smuzhiyun:: 104*4882a593Smuzhiyun 105*4882a593Smuzhiyun DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000 106*4882a593Smuzhiyun DMAR:[fault reason 05] PTE Write access is not set 107*4882a593Smuzhiyun DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000 108*4882a593Smuzhiyun DMAR:[fault reason 05] PTE Write access is not set 109*4882a593Smuzhiyun 110*4882a593SmuzhiyunTBD 111*4882a593Smuzhiyun---- 112*4882a593Smuzhiyun 113*4882a593Smuzhiyun- For compatibility testing, could use unity map domain for all devices, just 114*4882a593Smuzhiyun provide a 1-1 for all useful memory under a single domain for all devices. 115*4882a593Smuzhiyun- API for paravirt ops for abstracting functionality for VMM folks. 116