1*4882a593Smuzhiyun.. _embargoed_hardware_issues: 2*4882a593Smuzhiyun 3*4882a593SmuzhiyunEmbargoed hardware issues 4*4882a593Smuzhiyun========================= 5*4882a593Smuzhiyun 6*4882a593SmuzhiyunScope 7*4882a593Smuzhiyun----- 8*4882a593Smuzhiyun 9*4882a593SmuzhiyunHardware issues which result in security problems are a different category 10*4882a593Smuzhiyunof security bugs than pure software bugs which only affect the Linux 11*4882a593Smuzhiyunkernel. 12*4882a593Smuzhiyun 13*4882a593SmuzhiyunHardware issues like Meltdown, Spectre, L1TF etc. must be treated 14*4882a593Smuzhiyundifferently because they usually affect all Operating Systems ("OS") and 15*4882a593Smuzhiyuntherefore need coordination across different OS vendors, distributions, 16*4882a593Smuzhiyunhardware vendors and other parties. For some of the issues, software 17*4882a593Smuzhiyunmitigations can depend on microcode or firmware updates, which need further 18*4882a593Smuzhiyuncoordination. 19*4882a593Smuzhiyun 20*4882a593Smuzhiyun.. _Contact: 21*4882a593Smuzhiyun 22*4882a593SmuzhiyunContact 23*4882a593Smuzhiyun------- 24*4882a593Smuzhiyun 25*4882a593SmuzhiyunThe Linux kernel hardware security team is separate from the regular Linux 26*4882a593Smuzhiyunkernel security team. 27*4882a593Smuzhiyun 28*4882a593SmuzhiyunThe team only handles the coordination of embargoed hardware security 29*4882a593Smuzhiyunissues. Reports of pure software security bugs in the Linux kernel are not 30*4882a593Smuzhiyunhandled by this team and the reporter will be guided to contact the regular 31*4882a593SmuzhiyunLinux kernel security team (:ref:`Documentation/admin-guide/ 32*4882a593Smuzhiyun<securitybugs>`) instead. 33*4882a593Smuzhiyun 34*4882a593SmuzhiyunThe team can be contacted by email at <hardware-security@kernel.org>. This 35*4882a593Smuzhiyunis a private list of security officers who will help you to coordinate an 36*4882a593Smuzhiyunissue according to our documented process. 37*4882a593Smuzhiyun 38*4882a593SmuzhiyunThe list is encrypted and email to the list can be sent by either PGP or 39*4882a593SmuzhiyunS/MIME encrypted and must be signed with the reporter's PGP key or S/MIME 40*4882a593Smuzhiyuncertificate. The list's PGP key and S/MIME certificate are available from 41*4882a593Smuzhiyunthe following URLs: 42*4882a593Smuzhiyun 43*4882a593Smuzhiyun - PGP: https://www.kernel.org/static/files/hardware-security.asc 44*4882a593Smuzhiyun - S/MIME: https://www.kernel.org/static/files/hardware-security.crt 45*4882a593Smuzhiyun 46*4882a593SmuzhiyunWhile hardware security issues are often handled by the affected hardware 47*4882a593Smuzhiyunvendor, we welcome contact from researchers or individuals who have 48*4882a593Smuzhiyunidentified a potential hardware flaw. 49*4882a593Smuzhiyun 50*4882a593SmuzhiyunHardware security officers 51*4882a593Smuzhiyun^^^^^^^^^^^^^^^^^^^^^^^^^^ 52*4882a593Smuzhiyun 53*4882a593SmuzhiyunThe current team of hardware security officers: 54*4882a593Smuzhiyun 55*4882a593Smuzhiyun - Linus Torvalds (Linux Foundation Fellow) 56*4882a593Smuzhiyun - Greg Kroah-Hartman (Linux Foundation Fellow) 57*4882a593Smuzhiyun - Thomas Gleixner (Linux Foundation Fellow) 58*4882a593Smuzhiyun 59*4882a593SmuzhiyunOperation of mailing-lists 60*4882a593Smuzhiyun^^^^^^^^^^^^^^^^^^^^^^^^^^ 61*4882a593Smuzhiyun 62*4882a593SmuzhiyunThe encrypted mailing-lists which are used in our process are hosted on 63*4882a593SmuzhiyunLinux Foundation's IT infrastructure. By providing this service, members 64*4882a593Smuzhiyunof Linux Foundation's IT operations personnel technically have the 65*4882a593Smuzhiyunability to access the embargoed information, but are obliged to 66*4882a593Smuzhiyunconfidentiality by their employment contract. Linux Foundation IT 67*4882a593Smuzhiyunpersonnel are also responsible for operating and managing the rest of 68*4882a593Smuzhiyunkernel.org infrastructure. 69*4882a593Smuzhiyun 70*4882a593SmuzhiyunThe Linux Foundation's current director of IT Project infrastructure is 71*4882a593SmuzhiyunKonstantin Ryabitsev. 72*4882a593Smuzhiyun 73*4882a593Smuzhiyun 74*4882a593SmuzhiyunNon-disclosure agreements 75*4882a593Smuzhiyun------------------------- 76*4882a593Smuzhiyun 77*4882a593SmuzhiyunThe Linux kernel hardware security team is not a formal body and therefore 78*4882a593Smuzhiyununable to enter into any non-disclosure agreements. The kernel community 79*4882a593Smuzhiyunis aware of the sensitive nature of such issues and offers a Memorandum of 80*4882a593SmuzhiyunUnderstanding instead. 81*4882a593Smuzhiyun 82*4882a593Smuzhiyun 83*4882a593SmuzhiyunMemorandum of Understanding 84*4882a593Smuzhiyun--------------------------- 85*4882a593Smuzhiyun 86*4882a593SmuzhiyunThe Linux kernel community has a deep understanding of the requirement to 87*4882a593Smuzhiyunkeep hardware security issues under embargo for coordination between 88*4882a593Smuzhiyundifferent OS vendors, distributors, hardware vendors and other parties. 89*4882a593Smuzhiyun 90*4882a593SmuzhiyunThe Linux kernel community has successfully handled hardware security 91*4882a593Smuzhiyunissues in the past and has the necessary mechanisms in place to allow 92*4882a593Smuzhiyuncommunity compliant development under embargo restrictions. 93*4882a593Smuzhiyun 94*4882a593SmuzhiyunThe Linux kernel community has a dedicated hardware security team for 95*4882a593Smuzhiyuninitial contact, which oversees the process of handling such issues under 96*4882a593Smuzhiyunembargo rules. 97*4882a593Smuzhiyun 98*4882a593SmuzhiyunThe hardware security team identifies the developers (domain experts) who 99*4882a593Smuzhiyunwill form the initial response team for a particular issue. The initial 100*4882a593Smuzhiyunresponse team can bring in further developers (domain experts) to address 101*4882a593Smuzhiyunthe issue in the best technical way. 102*4882a593Smuzhiyun 103*4882a593SmuzhiyunAll involved developers pledge to adhere to the embargo rules and to keep 104*4882a593Smuzhiyunthe received information confidential. Violation of the pledge will lead to 105*4882a593Smuzhiyunimmediate exclusion from the current issue and removal from all related 106*4882a593Smuzhiyunmailing-lists. In addition, the hardware security team will also exclude 107*4882a593Smuzhiyunthe offender from future issues. The impact of this consequence is a highly 108*4882a593Smuzhiyuneffective deterrent in our community. In case a violation happens the 109*4882a593Smuzhiyunhardware security team will inform the involved parties immediately. If you 110*4882a593Smuzhiyunor anyone becomes aware of a potential violation, please report it 111*4882a593Smuzhiyunimmediately to the Hardware security officers. 112*4882a593Smuzhiyun 113*4882a593Smuzhiyun 114*4882a593SmuzhiyunProcess 115*4882a593Smuzhiyun^^^^^^^ 116*4882a593Smuzhiyun 117*4882a593SmuzhiyunDue to the globally distributed nature of Linux kernel development, 118*4882a593Smuzhiyunface-to-face meetings are almost impossible to address hardware security 119*4882a593Smuzhiyunissues. Phone conferences are hard to coordinate due to time zones and 120*4882a593Smuzhiyunother factors and should be only used when absolutely necessary. Encrypted 121*4882a593Smuzhiyunemail has been proven to be the most effective and secure communication 122*4882a593Smuzhiyunmethod for these types of issues. 123*4882a593Smuzhiyun 124*4882a593SmuzhiyunStart of Disclosure 125*4882a593Smuzhiyun""""""""""""""""""" 126*4882a593Smuzhiyun 127*4882a593SmuzhiyunDisclosure starts by contacting the Linux kernel hardware security team by 128*4882a593Smuzhiyunemail. This initial contact should contain a description of the problem and 129*4882a593Smuzhiyuna list of any known affected hardware. If your organization builds or 130*4882a593Smuzhiyundistributes the affected hardware, we encourage you to also consider what 131*4882a593Smuzhiyunother hardware could be affected. 132*4882a593Smuzhiyun 133*4882a593SmuzhiyunThe hardware security team will provide an incident-specific encrypted 134*4882a593Smuzhiyunmailing-list which will be used for initial discussion with the reporter, 135*4882a593Smuzhiyunfurther disclosure and coordination. 136*4882a593Smuzhiyun 137*4882a593SmuzhiyunThe hardware security team will provide the disclosing party a list of 138*4882a593Smuzhiyundevelopers (domain experts) who should be informed initially about the 139*4882a593Smuzhiyunissue after confirming with the developers that they will adhere to this 140*4882a593SmuzhiyunMemorandum of Understanding and the documented process. These developers 141*4882a593Smuzhiyunform the initial response team and will be responsible for handling the 142*4882a593Smuzhiyunissue after initial contact. The hardware security team is supporting the 143*4882a593Smuzhiyunresponse team, but is not necessarily involved in the mitigation 144*4882a593Smuzhiyundevelopment process. 145*4882a593Smuzhiyun 146*4882a593SmuzhiyunWhile individual developers might be covered by a non-disclosure agreement 147*4882a593Smuzhiyunvia their employer, they cannot enter individual non-disclosure agreements 148*4882a593Smuzhiyunin their role as Linux kernel developers. They will, however, agree to 149*4882a593Smuzhiyunadhere to this documented process and the Memorandum of Understanding. 150*4882a593Smuzhiyun 151*4882a593SmuzhiyunThe disclosing party should provide a list of contacts for all other 152*4882a593Smuzhiyunentities who have already been, or should be, informed about the issue. 153*4882a593SmuzhiyunThis serves several purposes: 154*4882a593Smuzhiyun 155*4882a593Smuzhiyun - The list of disclosed entities allows communication accross the 156*4882a593Smuzhiyun industry, e.g. other OS vendors, HW vendors, etc. 157*4882a593Smuzhiyun 158*4882a593Smuzhiyun - The disclosed entities can be contacted to name experts who should 159*4882a593Smuzhiyun participate in the mitigation development. 160*4882a593Smuzhiyun 161*4882a593Smuzhiyun - If an expert which is required to handle an issue is employed by an 162*4882a593Smuzhiyun listed entity or member of an listed entity, then the response teams can 163*4882a593Smuzhiyun request the disclosure of that expert from that entity. This ensures 164*4882a593Smuzhiyun that the expert is also part of the entity's response team. 165*4882a593Smuzhiyun 166*4882a593SmuzhiyunDisclosure 167*4882a593Smuzhiyun"""""""""" 168*4882a593Smuzhiyun 169*4882a593SmuzhiyunThe disclosing party provides detailed information to the initial response 170*4882a593Smuzhiyunteam via the specific encrypted mailing-list. 171*4882a593Smuzhiyun 172*4882a593SmuzhiyunFrom our experience the technical documentation of these issues is usually 173*4882a593Smuzhiyuna sufficient starting point and further technical clarification is best 174*4882a593Smuzhiyundone via email. 175*4882a593Smuzhiyun 176*4882a593SmuzhiyunMitigation development 177*4882a593Smuzhiyun"""""""""""""""""""""" 178*4882a593Smuzhiyun 179*4882a593SmuzhiyunThe initial response team sets up an encrypted mailing-list or repurposes 180*4882a593Smuzhiyunan existing one if appropriate. 181*4882a593Smuzhiyun 182*4882a593SmuzhiyunUsing a mailing-list is close to the normal Linux development process and 183*4882a593Smuzhiyunhas been successfully used in developing mitigations for various hardware 184*4882a593Smuzhiyunsecurity issues in the past. 185*4882a593Smuzhiyun 186*4882a593SmuzhiyunThe mailing-list operates in the same way as normal Linux development. 187*4882a593SmuzhiyunPatches are posted, discussed and reviewed and if agreed on applied to a 188*4882a593Smuzhiyunnon-public git repository which is only accessible to the participating 189*4882a593Smuzhiyundevelopers via a secure connection. The repository contains the main 190*4882a593Smuzhiyundevelopment branch against the mainline kernel and backport branches for 191*4882a593Smuzhiyunstable kernel versions as necessary. 192*4882a593Smuzhiyun 193*4882a593SmuzhiyunThe initial response team will identify further experts from the Linux 194*4882a593Smuzhiyunkernel developer community as needed. Bringing in experts can happen at any 195*4882a593Smuzhiyuntime of the development process and needs to be handled in a timely manner. 196*4882a593Smuzhiyun 197*4882a593SmuzhiyunIf an expert is employed by or member of an entity on the disclosure list 198*4882a593Smuzhiyunprovided by the disclosing party, then participation will be requested from 199*4882a593Smuzhiyunthe relevant entity. 200*4882a593Smuzhiyun 201*4882a593SmuzhiyunIf not, then the disclosing party will be informed about the experts 202*4882a593Smuzhiyunparticipation. The experts are covered by the Memorandum of Understanding 203*4882a593Smuzhiyunand the disclosing party is requested to acknowledge the participation. In 204*4882a593Smuzhiyuncase that the disclosing party has a compelling reason to object, then this 205*4882a593Smuzhiyunobjection has to be raised within five work days and resolved with the 206*4882a593Smuzhiyunincident team immediately. If the disclosing party does not react within 207*4882a593Smuzhiyunfive work days this is taken as silent acknowledgement. 208*4882a593Smuzhiyun 209*4882a593SmuzhiyunAfter acknowledgement or resolution of an objection the expert is disclosed 210*4882a593Smuzhiyunby the incident team and brought into the development process. 211*4882a593Smuzhiyun 212*4882a593Smuzhiyun 213*4882a593SmuzhiyunCoordinated release 214*4882a593Smuzhiyun""""""""""""""""""" 215*4882a593Smuzhiyun 216*4882a593SmuzhiyunThe involved parties will negotiate the date and time where the embargo 217*4882a593Smuzhiyunends. At that point the prepared mitigations are integrated into the 218*4882a593Smuzhiyunrelevant kernel trees and published. 219*4882a593Smuzhiyun 220*4882a593SmuzhiyunWhile we understand that hardware security issues need coordinated embargo 221*4882a593Smuzhiyuntime, the embargo time should be constrained to the minimum time which is 222*4882a593Smuzhiyunrequired for all involved parties to develop, test and prepare the 223*4882a593Smuzhiyunmitigations. Extending embargo time artificially to meet conference talk 224*4882a593Smuzhiyundates or other non-technical reasons is creating more work and burden for 225*4882a593Smuzhiyunthe involved developers and response teams as the patches need to be kept 226*4882a593Smuzhiyunup to date in order to follow the ongoing upstream kernel development, 227*4882a593Smuzhiyunwhich might create conflicting changes. 228*4882a593Smuzhiyun 229*4882a593SmuzhiyunCVE assignment 230*4882a593Smuzhiyun"""""""""""""" 231*4882a593Smuzhiyun 232*4882a593SmuzhiyunNeither the hardware security team nor the initial response team assign 233*4882a593SmuzhiyunCVEs, nor are CVEs required for the development process. If CVEs are 234*4882a593Smuzhiyunprovided by the disclosing party they can be used for documentation 235*4882a593Smuzhiyunpurposes. 236*4882a593Smuzhiyun 237*4882a593SmuzhiyunProcess ambassadors 238*4882a593Smuzhiyun------------------- 239*4882a593Smuzhiyun 240*4882a593SmuzhiyunFor assistance with this process we have established ambassadors in various 241*4882a593Smuzhiyunorganizations, who can answer questions about or provide guidance on the 242*4882a593Smuzhiyunreporting process and further handling. Ambassadors are not involved in the 243*4882a593Smuzhiyundisclosure of a particular issue, unless requested by a response team or by 244*4882a593Smuzhiyunan involved disclosed party. The current ambassadors list: 245*4882a593Smuzhiyun 246*4882a593Smuzhiyun ============= ======================================================== 247*4882a593Smuzhiyun ARM Grant Likely <grant.likely@arm.com> 248*4882a593Smuzhiyun AMD Tom Lendacky <tom.lendacky@amd.com> 249*4882a593Smuzhiyun IBM Z Christian Borntraeger <borntraeger@de.ibm.com> 250*4882a593Smuzhiyun IBM Power Anton Blanchard <anton@linux.ibm.com> 251*4882a593Smuzhiyun Intel Tony Luck <tony.luck@intel.com> 252*4882a593Smuzhiyun Qualcomm Trilok Soni <tsoni@codeaurora.org> 253*4882a593Smuzhiyun 254*4882a593Smuzhiyun Microsoft James Morris <jamorris@linux.microsoft.com> 255*4882a593Smuzhiyun VMware 256*4882a593Smuzhiyun Xen Andrew Cooper <andrew.cooper3@citrix.com> 257*4882a593Smuzhiyun 258*4882a593Smuzhiyun Canonical John Johansen <john.johansen@canonical.com> 259*4882a593Smuzhiyun Debian Ben Hutchings <ben@decadent.org.uk> 260*4882a593Smuzhiyun Oracle Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> 261*4882a593Smuzhiyun Red Hat Josh Poimboeuf <jpoimboe@redhat.com> 262*4882a593Smuzhiyun SUSE Jiri Kosina <jkosina@suse.cz> 263*4882a593Smuzhiyun 264*4882a593Smuzhiyun Amazon 265*4882a593Smuzhiyun Google Kees Cook <keescook@chromium.org> 266*4882a593Smuzhiyun ============= ======================================================== 267*4882a593Smuzhiyun 268*4882a593SmuzhiyunIf you want your organization to be added to the ambassadors list, please 269*4882a593Smuzhiyuncontact the hardware security team. The nominated ambassador has to 270*4882a593Smuzhiyununderstand and support our process fully and is ideally well connected in 271*4882a593Smuzhiyunthe Linux kernel community. 272*4882a593Smuzhiyun 273*4882a593SmuzhiyunEncrypted mailing-lists 274*4882a593Smuzhiyun----------------------- 275*4882a593Smuzhiyun 276*4882a593SmuzhiyunWe use encrypted mailing-lists for communication. The operating principle 277*4882a593Smuzhiyunof these lists is that email sent to the list is encrypted either with the 278*4882a593Smuzhiyunlist's PGP key or with the list's S/MIME certificate. The mailing-list 279*4882a593Smuzhiyunsoftware decrypts the email and re-encrypts it individually for each 280*4882a593Smuzhiyunsubscriber with the subscriber's PGP key or S/MIME certificate. Details 281*4882a593Smuzhiyunabout the mailing-list software and the setup which is used to ensure the 282*4882a593Smuzhiyunsecurity of the lists and protection of the data can be found here: 283*4882a593Smuzhiyunhttps://korg.wiki.kernel.org/userdoc/remail. 284*4882a593Smuzhiyun 285*4882a593SmuzhiyunList keys 286*4882a593Smuzhiyun^^^^^^^^^ 287*4882a593Smuzhiyun 288*4882a593SmuzhiyunFor initial contact see :ref:`Contact`. For incident specific mailing-lists 289*4882a593Smuzhiyunthe key and S/MIME certificate are conveyed to the subscribers by email 290*4882a593Smuzhiyunsent from the specific list. 291*4882a593Smuzhiyun 292*4882a593SmuzhiyunSubscription to incident specific lists 293*4882a593Smuzhiyun^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 294*4882a593Smuzhiyun 295*4882a593SmuzhiyunSubscription is handled by the response teams. Disclosed parties who want 296*4882a593Smuzhiyunto participate in the communication send a list of potential subscribers to 297*4882a593Smuzhiyunthe response team so the response team can validate subscription requests. 298*4882a593Smuzhiyun 299*4882a593SmuzhiyunEach subscriber needs to send a subscription request to the response team 300*4882a593Smuzhiyunby email. The email must be signed with the subscriber's PGP key or S/MIME 301*4882a593Smuzhiyuncertificate. If a PGP key is used, it must be available from a public key 302*4882a593Smuzhiyunserver and is ideally connected to the Linux kernel's PGP web of trust. See 303*4882a593Smuzhiyunalso: https://www.kernel.org/signature.html. 304*4882a593Smuzhiyun 305*4882a593SmuzhiyunThe response team verifies that the subscriber request is valid and adds 306*4882a593Smuzhiyunthe subscriber to the list. After subscription the subscriber will receive 307*4882a593Smuzhiyunemail from the mailing-list which is signed either with the list's PGP key 308*4882a593Smuzhiyunor the list's S/MIME certificate. The subscriber's email client can extract 309*4882a593Smuzhiyunthe PGP key or the S/MIME certificate from the signature so the subscriber 310*4882a593Smuzhiyuncan send encrypted email to the list. 311*4882a593Smuzhiyun 312