1// -*- mode:doc; -*- 2// vim: set syntax=asciidoc: 3 4== Contributing to Buildroot 5 6There are many ways in which you can contribute to Buildroot: analyzing 7and fixing bugs, analyzing and fixing package build failures detected by 8the autobuilders, testing and reviewing patches sent by other 9developers, working on the items in our TODO list and sending your own 10improvements to Buildroot or its manual. The following sections give a 11little more detail on each of these items. 12 13If you are interested in contributing to Buildroot, the first thing you 14should do is to subscribe to the Buildroot mailing list. This list is 15the main way of interacting with other Buildroot developers and to send 16contributions to. If you aren't subscribed yet, then refer to 17xref:community-resources[] for the subscription link. 18 19If you are going to touch the code, it is highly recommended to use a 20git repository of Buildroot, rather than starting from an extracted 21source code tarball. Git is the easiest way to develop from and directly 22send your patches to the mailing list. Refer to xref:getting-buildroot[] 23for more information on obtaining a Buildroot git tree. 24 25=== Reproducing, analyzing and fixing bugs 26 27A first way of contributing is to have a look at the open bug reports in 28the https://bugs.buildroot.org/buglist.cgi?product=buildroot[Buildroot bug 29tracker]. As we strive to keep the bug count as small as possible, all 30help in reproducing, analyzing and fixing reported bugs is more than 31welcome. Don't hesitate to add a comment to bug reports reporting your 32findings, even if you don't yet see the full picture. 33 34=== Analyzing and fixing autobuild failures 35 36The Buildroot autobuilders are a set of build machines that continuously 37run Buildroot builds based on random configurations. This is done for 38all architectures supported by Buildroot, with various toolchains, and 39with a random selection of packages. With the large commit activity on 40Buildroot, these autobuilders are a great help in detecting problems 41very early after commit. 42 43All build results are available at http://autobuild.buildroot.org[], 44statistics are at http://autobuild.buildroot.org/stats.php[]. Every day, 45an overview of all failed packages is sent to the mailing list. 46 47Detecting problems is great, but obviously these problems have to be 48fixed as well. Your contribution is very welcome here! There are 49basically two things that can be done: 50 51- Analyzing the problems. The daily summary mails do not contain details 52 about the actual failures: in order to see what's going on you have to 53 open the build log and check the last output. Having someone doing 54 this for all packages in the mail is very useful for other developers, 55 as they can make a quick initial analysis based on this output alone. 56 57- Fixing a problem. When fixing autobuild failures, you should follow 58 these steps: 59 . Check if you can reproduce the problem by building with the same 60 configuration. You can do this manually, or use the 61 http://git.buildroot.org/buildroot-test/tree/utils/br-reproduce-build[br-reproduce-build] 62 script that will automatically clone a Buildroot git repository, 63 checkout the correct revision, download and set the right 64 configuration, and start the build. 65 . Analyze the problem and create a fix. 66 . Verify that the problem is really fixed by starting from a clean 67 Buildroot tree and only applying your fix. 68 . Send the fix to the Buildroot mailing list (see 69 xref:submitting-patches[]). In case you created a patch against the 70 package sources, you should also send the patch upstream so that the 71 problem will be fixed in a later release, and the patch in Buildroot 72 can be removed. 73 In the commit message of a patch fixing an autobuild failure, add a 74 reference to the build result directory, as follows: 75--------------------- 76Fixes: http://autobuild.buildroot.org/results/51000a9d4656afe9e0ea6f07b9f8ed374c2e4069 77--------------------- 78 79=== Reviewing and testing patches 80 81With the amount of patches sent to the mailing list each day, the 82maintainer has a very hard job to judge which patches are ready to apply 83and which ones aren't. Contributors can greatly help here by reviewing 84and testing these patches. 85 86In the review process, do not hesitate to respond to patch submissions 87for remarks, suggestions or anything that will help everyone to 88understand the patches and make them better. Please use internet 89style replies in plain text emails when responding to patch 90submissions. 91 92To indicate approval of a patch, there are three formal tags that keep 93track of this approval. To add your tag to a patch, reply to it with the 94approval tag below the original author's Signed-off-by line. These tags 95will be picked up automatically by patchwork (see 96xref:apply-patches-patchwork[]) and will be part of the commit log when 97the patch is accepted. 98 99Tested-by:: Indicates that the patch has been tested successfully. 100 You are encouraged to specify what kind of testing you performed 101 (compile-test on architecture X and Y, runtime test on target A, 102 ...). This additional information helps other testers and the 103 maintainer. 104 105Reviewed-by:: Indicates that you code-reviewed the patch and did your 106 best in spotting problems, but you are not sufficiently familiar with 107 the area touched to provide an Acked-by tag. This means that there 108 may be remaining problems in the patch that would be spotted by 109 someone with more experience in that area. Should such problems be 110 detected, your Reviewed-by tag remains appropriate and you cannot 111 be blamed. 112 113Acked-by:: Indicates that you code-reviewed the patch and you are 114 familiar enough with the area touched to feel that the patch can be 115 committed as-is (no additional changes required). In case it later 116 turns out that something is wrong with the patch, your Acked-by could 117 be considered inappropriate. The difference between Acked-by and 118 Reviewed-by is thus mainly that you are prepared to take the blame on 119 Acked patches, but not on Reviewed ones. 120 121If you reviewed a patch and have comments on it, you should simply reply 122to the patch stating these comments, without providing a Reviewed-by or 123Acked-by tag. These tags should only be provided if you judge the patch 124to be good as it is. 125 126It is important to note that neither Reviewed-by nor Acked-by imply 127that testing has been performed. To indicate that you both reviewed and 128tested the patch, provide two separate tags (Reviewed/Acked-by and 129Tested-by). 130 131Note also that _any developer_ can provide Tested/Reviewed/Acked-by 132tags, without exception, and we encourage everyone to do this. Buildroot 133does not have a defined group of _core_ developers, it just so happens 134that some developers are more active than others. The maintainer will 135value tags according to the track record of their submitter. Tags 136provided by a regular contributor will naturally be trusted more than 137tags provided by a newcomer. As you provide tags more regularly, your 138'trustworthiness' (in the eyes of the maintainer) will go up, but _any_ 139tag provided is valuable. 140 141Buildroot's Patchwork website can be used to pull in patches for testing 142purposes. Please see xref:apply-patches-patchwork[] for more 143information on using Buildroot's Patchwork website to apply patches. 144 145[[apply-patches-patchwork]] 146==== Applying Patches from Patchwork 147 148The main use of Buildroot's Patchwork website for a developer is for 149pulling in patches into their local git repository for testing 150purposes. 151 152When browsing patches in the patchwork management interface, an +mbox+ 153link is provided at the top of the page. Copy this link address and 154run the following commands: 155 156--------------------- 157$ git checkout -b <test-branch-name> 158$ wget -O - <mbox-url> | git am 159--------------------- 160 161Another option for applying patches is to create a bundle. A bundle is 162a set of patches that you can group together using the patchwork 163interface. Once the bundle is created and the bundle is made public, 164you can copy the +mbox+ link for the bundle and apply the bundle 165using the above commands. 166 167 168=== Work on items from the TODO list 169 170If you want to contribute to Buildroot but don't know where to start, 171and you don't like any of the above topics, you can always work on items 172from the http://elinux.org/Buildroot#Todo_list[Buildroot TODO list]. 173Don't hesitate to discuss an item first on the mailing list or on IRC. 174Do edit the wiki to indicate when you start working on an item, so we 175avoid duplicate efforts. 176 177[[submitting-patches]] 178=== Submitting patches 179 180[NOTE] 181_Please, do not attach patches to bugs, send them to the mailing list 182instead_. 183 184If you made some changes to Buildroot and you would like to contribute 185them to the Buildroot project, proceed as follows. 186 187==== The formatting of a patch 188 189We expect patches to be formatted in a specific way. This is necessary 190to make it easy to review patches, to be able to apply them easily to 191the git repository, to make it easy to find back in the history how 192and why things have changed, and to make it possible to use +git 193bisect+ to locate the origin of a problem. 194 195First of all, it is essential that the patch has a good commit 196message. The commit message should start with a separate line with a 197brief summary of the change, prefixed by the area touched by the 198patch. A few examples of good commit titles: 199 200* +package/linuxptp: bump version to 2.0+ 201 202* +configs/imx23evk: bump Linux version to 4.19+ 203 204* +package/pkg-generic: postpone evaluation of dependency conditions+ 205 206* +boot/uboot: needs host-{flex,bison}+ 207 208* +support/testing: add python-ubjson tests+ 209 210The description that follows the prefix should start with a lower case 211letter (i.e "bump", "needs", "postpone", "add" in the above examples). 212 213Second, the body of the commit message should describe _why_ this 214change is needed, and if necessary also give details about _how_ it 215was done. When writing the commit message, think of how the reviewers 216will read it, but also think about how you will read it when you look 217at this change again a few years down the line. 218 219Third, the patch itself should do only one change, but do it 220completely. Two unrelated or weakly related changes should usually be 221done in two separate patches. This usually means that a patch affects 222only a single package. If several changes are related, it is often 223still possible to split them up in small patches and apply them in a 224specific order. Small patches make it easier to review, and often 225make it easier to understand afterwards why a change was done. 226However, each patch must be complete. It is not allowed that the 227build is broken when only the first but not the second patch is 228applied. This is necessary to be able to use +git bisect+ afterwards. 229 230Of course, while you're doing your development, you're probably going 231back and forth between packages, and certainly not committing things 232immediately in a way that is clean enough for submission. So most 233developers rewrite the history of commits to produce a clean set of 234commits that is appropriate for submission. To do this, you need to 235use _interactive rebasing_. You can learn about it 236https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History[in the Pro 237Git book]. Sometimes, it is even easier to discard you history with 238+git reset --soft origin/master+ and select individual changes with 239+git add -i+ or +git add -p+. 240 241Finally, the patch should be signed off. This is done by adding 242+Signed-off-by: Your Real Name <your@email.address>+ at the end of the 243commit message. +git commit -s+ does that for you, if configured 244properly. The +Signed-off-by+ tag means that you publish the patch 245under the Buildroot license (i.e. GPL-2.0+, except for package patches, 246which have the upstream license), and that you are allowed to do so. 247See http://developercertificate.org/[the Developer Certificate of 248Origin] for details. 249 250When adding new packages, you should submit every package in a 251separate patch. This patch should have the update to 252+package/Config.in+, the package +Config.in+ file, the +.mk+ file, the 253+.hash+ file, any init script, and all package patches. If the package 254has many sub-options, these are sometimes better added as separate 255follow-up patches. The summary line should be something like 256+<packagename>: new package+. The body of the commit message can be 257empty for simple packages, or it can contain the description of the 258package (like the Config.in help text). If anything special has to be 259done to build the package, this should also be explained explicitly in 260the commit message body. 261 262When you bump a package to a new version, you should also submit a 263separate patch for each package. Don't forget to update the +.hash+ 264file, or add it if it doesn't exist yet. Also don't forget to check if 265the +_LICENSE+ and +_LICENSE_FILES+ are still valid. The summary line 266should be something like +<packagename>: bump to version <new 267version>+. If the new version only contains security updates compared 268to the existing one, the summary should be +<packagename>: security 269bump to version <new version>+ and the commit message body should show 270the CVE numbers that are fixed. If some package patches can be removed 271in the new version, it should be explained explicitly why they can be 272removed, preferably with the upstream commit ID. Also any other 273required changes should be explained explicitly, like configure 274options that no longer exist or are no longer needed. 275 276If you are interested in getting notified of build failures and of 277further changes in the packages you added or modified, please add 278yourself to the DEVELOPERS file. This should be done in the same patch 279creating or modifying the package. See xref:DEVELOPERS[the DEVELOPERS file] 280for more information. 281 282Buildroot provides a handy tool to check for common coding style 283mistakes on files you created or modified, called +check-package+ (see 284xref:check-package[] for more information). 285 286==== Preparing a patch series 287 288Starting from the changes committed in your local git view, _rebase_ 289your development branch on top of the upstream tree before generating 290a patch set. To do so, run: 291 292--------------------- 293$ git fetch --all --tags 294$ git rebase origin/master 295--------------------- 296 297Now, you are ready to generate then submit your patch set. 298 299To generate it, run: 300 301--------------------- 302$ git format-patch -M -n -s -o outgoing origin/master 303--------------------- 304 305This will generate patch files in the +outgoing+ subdirectory, 306automatically adding the +Signed-off-by+ line. 307 308Once patch files are generated, you can review/edit the commit message 309before submitting them, using your favorite text editor. 310 311Buildroot provides a handy tool to know to whom your patches should be 312sent, called +get-developers+ (see xref:DEVELOPERS[] for more 313information). This tool reads your patches and outputs the appropriate 314+git send-email+ command to use: 315 316--------------------- 317$ ./utils/get-developers outgoing/* 318--------------------- 319 320Use the output of +get-developers+ to send your patches: 321 322--------------------- 323$ git send-email --to buildroot@buildroot.org --cc bob --cc alice outgoing/* 324--------------------- 325 326Alternatively, +get-developers -e+ can be used directly with the 327+--cc-cmd+ argument to +git send-email+ to automatically CC the 328affected developers: 329 330--------------------- 331$ git send-email --to buildroot@buildroot.org \ 332 --cc-cmd './utils/get-developers -e' origin/master 333--------------------- 334 335+git+ can be configured to automatically do this out of the box with: 336 337--------------------- 338$ git config sendemail.to buildroot@buildroot.org 339$ git config sendemail.ccCmd "$(pwd)/utils/get-developers -e" 340--------------------- 341 342And then just do: 343 344--------------------- 345$ git send-email origin/master 346--------------------- 347 348Note that +git+ should be configured to use your mail account. 349To configure +git+, see +man git-send-email+ or google it. 350 351If you do not use +git send-email+, make sure posted *patches are not 352line-wrapped*, otherwise they cannot easily be applied. In such a case, 353fix your e-mail client, or better yet, learn to use +git send-email+. 354 355==== Cover letter 356 357If you want to present the whole patch set in a separate mail, add 358+--cover-letter+ to the +git format-patch+ command (see +man 359git-format-patch+ for further information). This will generate a 360template for an introduction e-mail to your patch series. 361 362A 'cover letter' may be useful to introduce the changes you propose 363in the following cases: 364 365* large number of commits in the series; 366 367* deep impact of the changes in the rest of the project; 368 369* RFC footnote:[RFC: (Request for comments) change proposal]; 370 371* whenever you feel it will help presenting your work, your choices, 372 the review process, etc. 373 374==== Patches for maintenance branches 375 376When fixing bugs on a maintenance branch, bugs should be fixed on the 377master branch first. The commit log for such a patch may then contain a 378post-commit note specifying what branches are affected: 379 380---- 381package/foo: fix stuff 382 383Signed-off-by: Your Real Name <your@email.address> 384--- 385Backport to: 2020.02.x, 2020.05.x 386(2020.08.x not affected as the version was bumped) 387---- 388 389Those changes will then be backported by a maintainer to the affected 390branches. 391 392However, some bugs may apply only to a specific release, for example 393because it is using an older version of a package. In that case, patches 394should be based off the maintenance branch, and the patch subject prefix 395must include the maintenance branch name (for example "[PATCH 2020.02.x]"). 396This can be done with the +git format-patch+ flag +--subject-prefix+: 397 398--------------------- 399$ git format-patch --subject-prefix "PATCH 2020.02.x" \ 400 -M -s -o outgoing origin/2020.02.x 401--------------------- 402 403Then send the patches with +git send-email+, as described above. 404 405==== Patch revision changelog 406 407When improvements are requested, the new revision of each commit 408should include a changelog of the modifications between each 409submission. Note that when your patch series is introduced by a cover 410letter, an overall changelog may be added to the cover letter in 411addition to the changelog in the individual commits. 412The best thing to rework a patch series is by interactive rebasing: 413+git rebase -i origin/master+. Consult the git manual for more 414information. 415 416When added to the individual commits, this changelog is added when 417editing the commit message. Below the +Signed-off-by+ section, add 418+---+ and your changelog. 419 420Although the changelog will be visible for the reviewers in the mail 421thread, as well as in http://patchwork.buildroot.org[patchwork], +git+ 422will automatically ignores lines below +---+ when the patch will be 423merged. This is the intended behavior: the changelog is not meant to 424be preserved forever in the +git+ history of the project. 425 426Hereafter the recommended layout: 427 428--------------- 429Patch title: short explanation, max 72 chars 430 431A paragraph that explains the problem, and how it manifests itself. If 432the problem is complex, it is OK to add more paragraphs. All paragraphs 433should be wrapped at 72 characters. 434 435A paragraph that explains the root cause of the problem. Again, more 436than one paragraph is OK. 437 438Finally, one or more paragraphs that explain how the problem is solved. 439Don't hesitate to explain complex solutions in detail. 440 441Signed-off-by: John DOE <john.doe@example.net> 442 443--- 444Changes v2 -> v3: 445 - foo bar (suggested by Jane) 446 - bar buz 447 448Changes v1 -> v2: 449 - alpha bravo (suggested by John) 450 - charly delta 451--------------- 452 453Any patch revision should include the version number. The version number 454is simply composed of the letter +v+ followed by an +integer+ greater or 455equal to two (i.e. "PATCH v2", "PATCH v3" ...). 456 457This can be easily handled with +git format-patch+ by using the option 458+--subject-prefix+: 459 460--------------------- 461$ git format-patch --subject-prefix "PATCH v4" \ 462 -M -s -o outgoing origin/master 463--------------------- 464 465Since git version 1.8.1, you can also use +-v <n>+ (where <n> is the 466version number): 467 468--------------------- 469$ git format-patch -v4 -M -s -o outgoing origin/master 470--------------------- 471 472When you provide a new version of a patch, please mark the old one as 473superseded in http://patchwork.buildroot.org[patchwork]. You need to 474create an account on http://patchwork.buildroot.org[patchwork] to be 475able to modify the status of your patches. Note that you can only change 476the status of patches you submitted yourself, which means the email 477address you register in http://patchwork.buildroot.org[patchwork] should 478match the one you use for sending patches to the mailing list. 479 480You can also add the +--in-reply-to <message-id>+ option when 481submitting a patch to the mailing list. The id of the mail to reply to 482can be found under the "Message Id" tag on 483http://patchwork.buildroot.org[patchwork]. The advantage of 484*in-reply-to* is that patchwork will automatically mark the previous 485version of the patch as superseded. 486 487[[reporting-bugs]] 488=== Reporting issues/bugs or getting help 489 490Before reporting any issue, please check in 491xref:community-resources[the mailing list archive] whether someone has 492already reported and/or fixed a similar problem. 493 494However you choose to report bugs or get help, either by 495opening a bug in the xref:community-resources[bug tracker] or by 496xref:community-resources[sending a mail to the mailing list], there are 497a number of details to provide in order to help people reproduce and 498find a solution to the issue. 499 500Try to think as if you were trying to help someone else; in 501that case, what would you need? 502 503Here is a short list of details to provide in such case: 504 505* host machine (OS/release) 506* version of Buildroot 507* target for which the build fails 508* package(s) for which the build fails 509* the command that fails and its output 510* any information you think that may be relevant 511 512Additionally, you should add the +.config+ file (or if you know how, a 513+defconfig+; see xref:customize-store-buildroot-config[]). 514 515If some of these details are too large, do not hesitate to use a 516pastebin service. Note that not all available pastebin services will 517preserve Unix-style line terminators when downloading raw pastes. 518Following pastebin services are known to work correctly: 519- https://gist.github.com/ 520- http://code.bulix.org/ 521 522=== Using the runtime tests framework 523 524Buildroot includes a run-time testing framework built upon Python 525scripting and QEMU runtime execution. The goals of the framework are 526the following: 527 528* build a well defined Buildroot configuration 529* optionally, verify some properties of the build output 530* optionally, boot the build results under Qemu, and verify that a 531 given feature is working as expected 532 533The entry point to use the runtime tests framework is the 534+support/testing/run-tests+ tool, which has a series of options 535documented in the tool's help '-h' description. Some common options 536include setting the download folder, the output folder, keeping build 537output, and for multiple test cases, you can set the JLEVEL for each. 538 539Here is an example walk through of running a test case. 540 541* For a first step, let us see what all the test case options are. The test 542cases can be listed by executing +support/testing/run-tests -l+. These tests 543can all be run individually during test development from the console. Both 544one at a time and selectively as a group of a subset of tests. 545 546--------------------- 547$ support/testing/run-tests -l 548List of tests 549test_run (tests.utils.test_check_package.TestCheckPackage) 550test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootMusl) ... ok 551test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc) ... ok 552test_run (tests.toolchain.test_external.TestExternalToolchainCCache) ... ok 553test_run (tests.toolchain.test_external.TestExternalToolchainCtngMusl) ... ok 554test_run (tests.toolchain.test_external.TestExternalToolchainLinaroArm) ... ok 555test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv4) ... ok 556test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv5) ... ok 557test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv7) ... ok 558[snip] 559test_run (tests.init.test_systemd.TestInitSystemSystemdRoFull) ... ok 560test_run (tests.init.test_systemd.TestInitSystemSystemdRoIfupdown) ... ok 561test_run (tests.init.test_systemd.TestInitSystemSystemdRoNetworkd) ... ok 562test_run (tests.init.test_systemd.TestInitSystemSystemdRwFull) ... ok 563test_run (tests.init.test_systemd.TestInitSystemSystemdRwIfupdown) ... ok 564test_run (tests.init.test_systemd.TestInitSystemSystemdRwNetworkd) ... ok 565test_run (tests.init.test_busybox.TestInitSystemBusyboxRo) ... ok 566test_run (tests.init.test_busybox.TestInitSystemBusyboxRoNet) ... ok 567test_run (tests.init.test_busybox.TestInitSystemBusyboxRw) ... ok 568test_run (tests.init.test_busybox.TestInitSystemBusyboxRwNet) ... ok 569 570Ran 157 tests in 0.021s 571 572OK 573--------------------- 574 575* Then, to run one test case: 576 577--------------------- 578$ support/testing/run-tests -d dl -o output_folder -k tests.init.test_busybox.TestInitSystemBusyboxRw 57915:03:26 TestInitSystemBusyboxRw Starting 58015:03:28 TestInitSystemBusyboxRw Building 58115:08:18 TestInitSystemBusyboxRw Building done 58215:08:27 TestInitSystemBusyboxRw Cleaning up 583. 584Ran 1 test in 301.140s 585 586OK 587--------------------- 588 589The standard output indicates if the test is successful or not. By 590default, the output folder for the test is deleted automatically 591unless the option +-k+ is passed to *keep* the output directory. 592 593==== Creating a test case 594 595Within the Buildroot repository, the testing framework is organized at the 596top level in +support/testing/+ by folders of +conf+, +infra+ and +tests+. 597All the test cases live under the +tests+ folder and are organized in various 598folders representing the category of test. 599 600The best way to get familiar with how to create a test case is to look 601at a few of the basic file system +support/testing/tests/fs/+ and init 602+support/testing/tests/init/+ test scripts. Those tests give good 603examples of a basic tests that include both checking the build 604results, and doing runtime tests. There are other more advanced cases 605that use things like nested +br2-external+ folders to provide 606skeletons and additional packages. 607 608Creating a basic test case involves: 609 610* Defining a test class that inherits from +infra.basetest.BRTest+ 611 612* Defining the +config+ member of the test class, to the Buildroot 613 configuration to build for this test case. It can optionally rely on 614 configuration snippets provided by the runtime test infrastructure: 615 +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ to get a basic 616 architecture/toolchain configuration, and 617 +infra.basetest.MINIMAL_CONFIG+ to not build any filesystem. The 618 advantage of using +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ is that a 619 matching Linux kernel image is provided, which allows to boot the 620 resulting image in Qemu without having to build a Linux kernel image 621 as part of the test case, therefore significant decreasing the build 622 time required for the test case. 623 624* Implementing a +def test_run(self):+ function to implement the 625 actual tests to run after the build has completed. They may be tests 626 that verify the build output, by running command on the host using 627 the +run_cmd_on_host()+ helper function. Or they may boot the 628 generated system in Qemu using the +Emulator+ object available as 629 +self.emulator+ in the test case. For example +self.emulator.boot()+ 630 allows to boot the system in Qemu, +self.emulator.login()+ allows to 631 login, +self.emulator.run()+ allows to run shell commands inside 632 Qemu. 633 634After creating the test script, add yourself to the +DEVELOPERS+ file to 635be the maintainer of that test case. 636 637==== Debugging a test case 638 639When a test case runs, the +output_folder+ will contain the following: 640 641--------------------- 642$ ls output_folder/ 643TestInitSystemBusyboxRw/ 644TestInitSystemBusyboxRw-build.log 645TestInitSystemBusyboxRw-run.log 646--------------------- 647 648+TestInitSystemBusyboxRw/+ is the Buildroot output directory, and it 649is preserved only if the +-k+ option is passed. 650 651+TestInitSystemBusyboxRw-build.log+ is the log of the Buildroot build. 652 653+TestInitSystemBusyboxRw-run.log+ is the log of the Qemu boot and 654test. This file will only exist if the build was successful and the 655test case involves booting under Qemu. 656 657If you want to manually run Qemu to do manual tests of the build 658result, the first few lines of +TestInitSystemBusyboxRw-run.log+ 659contain the Qemu command line to use. 660 661You can also make modifications to the current sources inside the 662+output_folder+ (e.g. for debug purposes) and rerun the standard 663Buildroot make targets (in order to regenerate the complete image with 664the new modifications) and then rerun the test. 665 666==== Runtime tests and Gitlab CI 667 668All runtime tests are regularly executed by Buildroot Gitlab CI 669infrastructure, see .gitlab.yml and 670https://gitlab.com/buildroot.org/buildroot/-/jobs. 671 672You can also use Gitlab CI to test your new test cases, or verify that 673existing tests continue to work after making changes in Buildroot. 674 675In order to achieve this, you need to create a fork of the Buildroot 676project on Gitlab, and be able to push branches to your Buildroot fork 677on Gitlab. 678 679The name of the branch that you push will determine if a Gitlab CI 680pipeline will be triggered or not, and for which test cases. 681 682In the examples below, the <name> component of the branch name is an 683arbitrary string you choose. 684 685* To trigger all run-test test case jobs, push a branch that ends with 686 +-runtime-tests+: 687 688--------------------- 689 $ git push gitlab HEAD:<name>-runtime-tests 690--------------------- 691 692* To trigger one or several test case jobs, push a branch that ends 693 with the complete test case name 694 (+tests.init.test_busybox.TestInitSystemBusyboxRo+) or with the name 695 of a category of tests (+tests.init.test_busybox+): 696 697--------------------- 698 $ git push gitlab HEAD:<name>-<test case name> 699--------------------- 700 701Example to run one test: 702 703--------------------- 704 $ git push gitlab HEAD:foo-tests.init.test_busybox.TestInitSystemBusyboxRo 705--------------------- 706 707Examples to run several tests part of the same group: 708 709--------------------- 710 $ git push gitlab HEAD:foo-tests.init.test_busybox 711 $ git push gitlab HEAD:foo-tests.init 712--------------------- 713