xref: /OK3568_Linux_fs/buildroot/docs/manual/contribute.txt (revision 4882a59341e53eb6f0b4789bf948001014eff981)
1// -*- mode:doc; -*-
2// vim: set syntax=asciidoc:
3
4== Contributing to Buildroot
5
6There are many ways in which you can contribute to Buildroot: analyzing
7and fixing bugs, analyzing and fixing package build failures detected by
8the autobuilders, testing and reviewing patches sent by other
9developers, working on the items in our TODO list and sending your own
10improvements to Buildroot or its manual. The following sections give a
11little more detail on each of these items.
12
13If you are interested in contributing to Buildroot, the first thing you
14should do is to subscribe to the Buildroot mailing list. This list is
15the main way of interacting with other Buildroot developers and to send
16contributions to. If you aren't subscribed yet, then refer to
17xref:community-resources[] for the subscription link.
18
19If you are going to touch the code, it is highly recommended to use a
20git repository of Buildroot, rather than starting from an extracted
21source code tarball. Git is the easiest way to develop from and directly
22send your patches to the mailing list. Refer to xref:getting-buildroot[]
23for more information on obtaining a Buildroot git tree.
24
25=== Reproducing, analyzing and fixing bugs
26
27A first way of contributing is to have a look at the open bug reports in
28the https://bugs.buildroot.org/buglist.cgi?product=buildroot[Buildroot bug
29tracker]. As we strive to keep the bug count as small as possible, all
30help in reproducing, analyzing and fixing reported bugs is more than
31welcome. Don't hesitate to add a comment to bug reports reporting your
32findings, even if you don't yet see the full picture.
33
34=== Analyzing and fixing autobuild failures
35
36The Buildroot autobuilders are a set of build machines that continuously
37run Buildroot builds based on random configurations. This is done for
38all architectures supported by Buildroot, with various toolchains, and
39with a random selection of packages. With the large commit activity on
40Buildroot, these autobuilders are a great help in detecting problems
41very early after commit.
42
43All build results are available at http://autobuild.buildroot.org[],
44statistics are at http://autobuild.buildroot.org/stats.php[]. Every day,
45an overview of all failed packages is sent to the mailing list.
46
47Detecting problems is great, but obviously these problems have to be
48fixed as well. Your contribution is very welcome here! There are
49basically two things that can be done:
50
51- Analyzing the problems. The daily summary mails do not contain details
52  about the actual failures: in order to see what's going on you have to
53  open the build log and check the last output. Having someone doing
54  this for all packages in the mail is very useful for other developers,
55  as they can make a quick initial analysis based on this output alone.
56
57- Fixing a problem. When fixing autobuild failures, you should follow
58  these steps:
59  . Check if you can reproduce the problem by building with the same
60    configuration. You can do this manually, or use the
61    http://git.buildroot.org/buildroot-test/tree/utils/br-reproduce-build[br-reproduce-build]
62    script that will automatically clone a Buildroot git repository,
63    checkout the correct revision, download and set the right
64    configuration, and start the build.
65  . Analyze the problem and create a fix.
66  . Verify that the problem is really fixed by starting from a clean
67    Buildroot tree and only applying your fix.
68  . Send the fix to the Buildroot mailing list (see
69    xref:submitting-patches[]). In case you created a patch against the
70    package sources, you should also send the patch upstream so that the
71    problem will be fixed in a later release, and the patch in Buildroot
72    can be removed.
73    In the commit message of a patch fixing an autobuild failure, add a
74    reference to the build result directory, as follows:
75---------------------
76Fixes: http://autobuild.buildroot.org/results/51000a9d4656afe9e0ea6f07b9f8ed374c2e4069
77---------------------
78
79=== Reviewing and testing patches
80
81With the amount of patches sent to the mailing list each day, the
82maintainer has a very hard job to judge which patches are ready to apply
83and which ones aren't. Contributors can greatly help here by reviewing
84and testing these patches.
85
86In the review process, do not hesitate to respond to patch submissions
87for remarks, suggestions or anything that will help everyone to
88understand the patches and make them better. Please use internet
89style replies in plain text emails when responding to patch
90submissions.
91
92To indicate approval of a patch, there are three formal tags that keep
93track of this approval. To add your tag to a patch, reply to it with the
94approval tag below the original author's Signed-off-by line. These tags
95will be picked up automatically by patchwork (see
96xref:apply-patches-patchwork[]) and will be part of the commit log when
97the patch is accepted.
98
99Tested-by:: Indicates that the patch has been tested successfully.
100  You are encouraged to specify what kind of testing you performed
101  (compile-test on architecture X and Y, runtime test on target A,
102  ...). This additional information helps other testers and the
103  maintainer.
104
105Reviewed-by:: Indicates that you code-reviewed the patch and did your
106  best in spotting problems, but you are not sufficiently familiar with
107  the area touched to provide an Acked-by tag. This means that there
108  may be remaining problems in the patch that would be spotted by
109  someone with more experience in that area. Should such problems be
110  detected, your Reviewed-by tag remains appropriate and you cannot
111  be blamed.
112
113Acked-by:: Indicates that you code-reviewed the patch and you are
114  familiar enough with the area touched to feel that the patch can be
115  committed as-is (no additional changes required). In case it later
116  turns out that something is wrong with the patch, your Acked-by could
117  be considered inappropriate. The difference between Acked-by and
118  Reviewed-by is thus mainly that you are prepared to take the blame on
119  Acked patches, but not on Reviewed ones.
120
121If you reviewed a patch and have comments on it, you should simply reply
122to the patch stating these comments, without providing a Reviewed-by or
123Acked-by tag. These tags should only be provided if you judge the patch
124to be good as it is.
125
126It is important to note that neither Reviewed-by nor Acked-by imply
127that testing has been performed. To indicate that you both reviewed and
128tested the patch, provide two separate tags (Reviewed/Acked-by and
129Tested-by).
130
131Note also that _any developer_ can provide Tested/Reviewed/Acked-by
132tags, without exception, and we encourage everyone to do this. Buildroot
133does not have a defined group of _core_ developers, it just so happens
134that some developers are more active than others. The maintainer will
135value tags according to the track record of their submitter. Tags
136provided by a regular contributor will naturally be trusted more than
137tags provided by a newcomer. As you provide tags more regularly, your
138'trustworthiness' (in the eyes of the maintainer) will go up, but _any_
139tag provided is valuable.
140
141Buildroot's Patchwork website can be used to pull in patches for testing
142purposes. Please see xref:apply-patches-patchwork[] for more
143information on using Buildroot's Patchwork website to apply patches.
144
145[[apply-patches-patchwork]]
146==== Applying Patches from Patchwork
147
148The main use of Buildroot's Patchwork website for a developer is for
149pulling in patches into their local git repository for testing
150purposes.
151
152When browsing patches in the patchwork management interface, an +mbox+
153link is provided at the top of the page. Copy this link address and
154run the following commands:
155
156---------------------
157$ git checkout -b <test-branch-name>
158$ wget -O - <mbox-url> | git am
159---------------------
160
161Another option for applying patches is to create a bundle. A bundle is
162a set of patches that you can group together using the patchwork
163interface. Once the bundle is created and the bundle is made public,
164you can copy the +mbox+ link for the bundle and apply the bundle
165using the above commands.
166
167
168=== Work on items from the TODO list
169
170If you want to contribute to Buildroot but don't know where to start,
171and you don't like any of the above topics, you can always work on items
172from the http://elinux.org/Buildroot#Todo_list[Buildroot TODO list].
173Don't hesitate to discuss an item first on the mailing list or on IRC.
174Do edit the wiki to indicate when you start working on an item, so we
175avoid duplicate efforts.
176
177[[submitting-patches]]
178=== Submitting patches
179
180[NOTE]
181_Please, do not attach patches to bugs, send them to the mailing list
182instead_.
183
184If you made some changes to Buildroot and you would like to contribute
185them to the Buildroot project, proceed as follows.
186
187==== The formatting of a patch
188
189We expect patches to be formatted in a specific way. This is necessary
190to make it easy to review patches, to be able to apply them easily to
191the git repository, to make it easy to find back in the history how
192and why things have changed, and to make it possible to use +git
193bisect+ to locate the origin of a problem.
194
195First of all, it is essential that the patch has a good commit
196message. The commit message should start with a separate line with a
197brief summary of the change, prefixed by the area touched by the
198patch. A few examples of good commit titles:
199
200* +package/linuxptp: bump version to 2.0+
201
202* +configs/imx23evk: bump Linux version to 4.19+
203
204* +package/pkg-generic: postpone evaluation of dependency conditions+
205
206* +boot/uboot: needs host-{flex,bison}+
207
208* +support/testing: add python-ubjson tests+
209
210The description that follows the prefix should start with a lower case
211letter (i.e "bump", "needs", "postpone", "add" in the above examples).
212
213Second, the body of the commit message should describe _why_ this
214change is needed, and if necessary also give details about _how_ it
215was done. When writing the commit message, think of how the reviewers
216will read it, but also think about how you will read it when you look
217at this change again a few years down the line.
218
219Third, the patch itself should do only one change, but do it
220completely. Two unrelated or weakly related changes should usually be
221done in two separate patches. This usually means that a patch affects
222only a single package. If several changes are related, it is often
223still possible to split them up in small patches and apply them in a
224specific order. Small patches make it easier to review, and often
225make it easier to understand afterwards why a change was done.
226However, each patch must be complete. It is not allowed that the
227build is broken when only the first but not the second patch is
228applied. This is necessary to be able to use +git bisect+ afterwards.
229
230Of course, while you're doing your development, you're probably going
231back and forth between packages, and certainly not committing things
232immediately in a way that is clean enough for submission. So most
233developers rewrite the history of commits to produce a clean set of
234commits that is appropriate for submission. To do this, you need to
235use _interactive rebasing_. You can learn about it
236https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History[in the Pro
237Git book]. Sometimes, it is even easier to discard you history with
238+git reset --soft origin/master+ and select individual changes with
239+git add -i+ or +git add -p+.
240
241Finally, the patch should be signed off. This is done by adding
242+Signed-off-by: Your Real Name <your@email.address>+ at the end of the
243commit message. +git commit -s+ does that for you, if configured
244properly. The +Signed-off-by+ tag means that you publish the patch
245under the Buildroot license (i.e. GPL-2.0+, except for package patches,
246which have the upstream license), and that you are allowed to do so.
247See http://developercertificate.org/[the Developer Certificate of
248Origin] for details.
249
250When adding new packages, you should submit every package in a
251separate patch. This patch should have the update to
252+package/Config.in+, the package +Config.in+ file, the +.mk+ file, the
253+.hash+ file, any init script, and all package patches. If the package
254has many sub-options, these are sometimes better added as separate
255follow-up patches. The summary line should be something like
256+<packagename>: new package+. The body of the commit message can be
257empty for simple packages, or it can contain the description of the
258package (like the Config.in help text). If anything special has to be
259done to build the package, this should also be explained explicitly in
260the commit message body.
261
262When you bump a package to a new version, you should also submit a
263separate patch for each package. Don't forget to update the +.hash+
264file, or add it if it doesn't exist yet. Also don't forget to check if
265the +_LICENSE+ and +_LICENSE_FILES+ are still valid. The summary line
266should be something like +<packagename>: bump to version <new
267version>+. If the new version only contains security updates compared
268to the existing one, the summary should be +<packagename>: security
269bump to version <new version>+ and the commit message body should show
270the CVE numbers that are fixed. If some package patches can be removed
271in the new version, it should be explained explicitly why they can be
272removed, preferably with the upstream commit ID. Also any other
273required changes should be explained explicitly, like configure
274options that no longer exist or are no longer needed.
275
276If you are interested in getting notified of build failures and of
277further changes in the packages you added or modified, please add
278yourself to the DEVELOPERS file. This should be done in the same patch
279creating or modifying the package. See xref:DEVELOPERS[the DEVELOPERS file]
280for more information.
281
282Buildroot provides a handy tool to check for common coding style
283mistakes on files you created or modified, called +check-package+ (see
284xref:check-package[] for more information).
285
286==== Preparing a patch series
287
288Starting from the changes committed in your local git view, _rebase_
289your development branch on top of the upstream tree before generating
290a patch set. To do so, run:
291
292---------------------
293$ git fetch --all --tags
294$ git rebase origin/master
295---------------------
296
297Now, you are ready to generate then submit your patch set.
298
299To generate it, run:
300
301---------------------
302$ git format-patch -M -n -s -o outgoing origin/master
303---------------------
304
305This will generate patch files in the +outgoing+ subdirectory,
306automatically adding the +Signed-off-by+ line.
307
308Once patch files are generated, you can review/edit the commit message
309before submitting them, using your favorite text editor.
310
311Buildroot provides a handy tool to know to whom your patches should be
312sent, called +get-developers+ (see xref:DEVELOPERS[] for more
313information). This tool reads your patches and outputs the appropriate
314+git send-email+ command to use:
315
316---------------------
317$ ./utils/get-developers outgoing/*
318---------------------
319
320Use the output of +get-developers+ to send your patches:
321
322---------------------
323$ git send-email --to buildroot@buildroot.org --cc bob --cc alice outgoing/*
324---------------------
325
326Alternatively, +get-developers -e+ can be used directly with the
327+--cc-cmd+ argument to +git send-email+ to automatically CC the
328affected developers:
329
330---------------------
331$ git send-email --to buildroot@buildroot.org \
332      --cc-cmd './utils/get-developers -e' origin/master
333---------------------
334
335+git+ can be configured to automatically do this out of the box with:
336
337---------------------
338$ git config sendemail.to buildroot@buildroot.org
339$ git config sendemail.ccCmd "$(pwd)/utils/get-developers -e"
340---------------------
341
342And then just do:
343
344---------------------
345$ git send-email origin/master
346---------------------
347
348Note that +git+ should be configured to use your mail account.
349To configure +git+, see +man git-send-email+ or google it.
350
351If you do not use +git send-email+, make sure posted *patches are not
352line-wrapped*, otherwise they cannot easily be applied. In such a case,
353fix your e-mail client, or better yet, learn to use +git send-email+.
354
355==== Cover letter
356
357If you want to present the whole patch set in a separate mail, add
358+--cover-letter+ to the +git format-patch+ command (see +man
359git-format-patch+ for further information). This will generate a
360template for an introduction e-mail to your patch series.
361
362A 'cover letter' may be useful to introduce the changes you propose
363in the following cases:
364
365* large number of commits in the series;
366
367* deep impact of the changes in the rest of the project;
368
369* RFC footnote:[RFC: (Request for comments) change proposal];
370
371* whenever you feel it will help presenting your work, your choices,
372  the review process, etc.
373
374==== Patches for maintenance branches
375
376When fixing bugs on a maintenance branch, bugs should be fixed on the
377master branch first. The commit log for such a patch may then contain a
378post-commit note specifying what branches are affected:
379
380----
381package/foo: fix stuff
382
383Signed-off-by: Your Real Name <your@email.address>
384---
385Backport to: 2020.02.x, 2020.05.x
386(2020.08.x not affected as the version was bumped)
387----
388
389Those changes will then be backported by a maintainer to the affected
390branches.
391
392However, some bugs may apply only to a specific release, for example
393because it is using an older version of a package. In that case, patches
394should be based off the maintenance branch, and the patch subject prefix
395must include the maintenance branch name (for example "[PATCH 2020.02.x]").
396This can be done with the +git format-patch+ flag +--subject-prefix+:
397
398---------------------
399$ git format-patch --subject-prefix "PATCH 2020.02.x" \
400    -M -s -o outgoing origin/2020.02.x
401---------------------
402
403Then send the patches with +git send-email+, as described above.
404
405==== Patch revision changelog
406
407When improvements are requested, the new revision of each commit
408should include a changelog of the modifications between each
409submission. Note that when your patch series is introduced by a cover
410letter, an overall changelog may be added to the cover letter in
411addition to the changelog in the individual commits.
412The best thing to rework a patch series is by interactive rebasing:
413+git rebase -i origin/master+. Consult the git manual for more
414information.
415
416When added to the individual commits, this changelog is added when
417editing the commit message. Below the +Signed-off-by+ section, add
418+---+ and your changelog.
419
420Although the changelog will be visible for the reviewers in the mail
421thread, as well as in http://patchwork.buildroot.org[patchwork], +git+
422will automatically ignores lines below +---+ when the patch will be
423merged. This is the intended behavior: the changelog is not meant to
424be preserved forever in the +git+ history of the project.
425
426Hereafter the recommended layout:
427
428---------------
429Patch title: short explanation, max 72 chars
430
431A paragraph that explains the problem, and how it manifests itself. If
432the problem is complex, it is OK to add more paragraphs. All paragraphs
433should be wrapped at 72 characters.
434
435A paragraph that explains the root cause of the problem. Again, more
436than one paragraph is OK.
437
438Finally, one or more paragraphs that explain how the problem is solved.
439Don't hesitate to explain complex solutions in detail.
440
441Signed-off-by: John DOE <john.doe@example.net>
442
443---
444Changes v2 -> v3:
445  - foo bar  (suggested by Jane)
446  - bar buz
447
448Changes v1 -> v2:
449  - alpha bravo  (suggested by John)
450  - charly delta
451---------------
452
453Any patch revision should include the version number. The version number
454is simply composed of the letter +v+ followed by an +integer+ greater or
455equal to two (i.e. "PATCH v2", "PATCH v3" ...).
456
457This can be easily handled with +git format-patch+ by using the option
458+--subject-prefix+:
459
460---------------------
461$ git format-patch --subject-prefix "PATCH v4" \
462    -M -s -o outgoing origin/master
463---------------------
464
465Since git version 1.8.1, you can also use +-v <n>+ (where <n> is the
466version number):
467
468---------------------
469$ git format-patch -v4 -M -s -o outgoing origin/master
470---------------------
471
472When you provide a new version of a patch, please mark the old one as
473superseded in http://patchwork.buildroot.org[patchwork]. You need to
474create an account on http://patchwork.buildroot.org[patchwork] to be
475able to modify the status of your patches. Note that you can only change
476the status of patches you submitted yourself, which means the email
477address you register in http://patchwork.buildroot.org[patchwork] should
478match the one you use for sending patches to the mailing list.
479
480You can also add the +--in-reply-to <message-id>+ option when
481submitting a patch to the mailing list. The id of the mail to reply to
482can be found under the "Message Id" tag on
483http://patchwork.buildroot.org[patchwork].  The advantage of
484*in-reply-to* is that patchwork will automatically mark the previous
485version of the patch as superseded.
486
487[[reporting-bugs]]
488=== Reporting issues/bugs or getting help
489
490Before reporting any issue, please check in
491xref:community-resources[the mailing list archive] whether someone has
492already reported and/or fixed a similar problem.
493
494However you choose to report bugs or get help, either by
495opening a bug in the xref:community-resources[bug tracker] or by
496xref:community-resources[sending a mail to the mailing list], there are
497a number of details to provide in order to help people reproduce and
498find a solution to the issue.
499
500Try to think as if you were trying to help someone else; in
501that case, what would you need?
502
503Here is a short list of details to provide in such case:
504
505* host machine (OS/release)
506* version of Buildroot
507* target for which the build fails
508* package(s) for which the build fails
509* the command that fails and its output
510* any information you think that may be relevant
511
512Additionally, you should add the +.config+ file (or if you know how, a
513+defconfig+; see xref:customize-store-buildroot-config[]).
514
515If some of these details are too large, do not hesitate to use a
516pastebin service. Note that not all available pastebin services will
517preserve Unix-style line terminators when downloading raw pastes.
518Following pastebin services are known to work correctly:
519- https://gist.github.com/
520- http://code.bulix.org/
521
522=== Using the runtime tests framework
523
524Buildroot includes a run-time testing framework built upon Python
525scripting and QEMU runtime execution. The goals of the framework are
526the following:
527
528* build a well defined Buildroot configuration
529* optionally, verify some properties of the build output
530* optionally, boot the build results under Qemu, and verify that a
531  given feature is working as expected
532
533The entry point to use the runtime tests framework is the
534+support/testing/run-tests+ tool, which has a series of options
535documented in the tool's help '-h' description. Some common options
536include setting the download folder, the output folder, keeping build
537output, and for multiple test cases, you can set the JLEVEL for each.
538
539Here is an example walk through of running a test case.
540
541* For a first step, let us see what all the test case options are. The test
542cases can be listed by executing +support/testing/run-tests -l+. These tests
543can all be run individually during test development from the console. Both
544one at a time and selectively as a group of a subset of tests.
545
546---------------------
547$ support/testing/run-tests -l
548List of tests
549test_run (tests.utils.test_check_package.TestCheckPackage)
550test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootMusl) ... ok
551test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc) ... ok
552test_run (tests.toolchain.test_external.TestExternalToolchainCCache) ... ok
553test_run (tests.toolchain.test_external.TestExternalToolchainCtngMusl) ... ok
554test_run (tests.toolchain.test_external.TestExternalToolchainLinaroArm) ... ok
555test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv4) ... ok
556test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv5) ... ok
557test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv7) ... ok
558[snip]
559test_run (tests.init.test_systemd.TestInitSystemSystemdRoFull) ... ok
560test_run (tests.init.test_systemd.TestInitSystemSystemdRoIfupdown) ... ok
561test_run (tests.init.test_systemd.TestInitSystemSystemdRoNetworkd) ... ok
562test_run (tests.init.test_systemd.TestInitSystemSystemdRwFull) ... ok
563test_run (tests.init.test_systemd.TestInitSystemSystemdRwIfupdown) ... ok
564test_run (tests.init.test_systemd.TestInitSystemSystemdRwNetworkd) ... ok
565test_run (tests.init.test_busybox.TestInitSystemBusyboxRo) ... ok
566test_run (tests.init.test_busybox.TestInitSystemBusyboxRoNet) ... ok
567test_run (tests.init.test_busybox.TestInitSystemBusyboxRw) ... ok
568test_run (tests.init.test_busybox.TestInitSystemBusyboxRwNet) ... ok
569
570Ran 157 tests in 0.021s
571
572OK
573---------------------
574
575* Then, to run one test case:
576
577---------------------
578$ support/testing/run-tests -d dl -o output_folder -k tests.init.test_busybox.TestInitSystemBusyboxRw
57915:03:26 TestInitSystemBusyboxRw                  Starting
58015:03:28 TestInitSystemBusyboxRw                  Building
58115:08:18 TestInitSystemBusyboxRw                  Building done
58215:08:27 TestInitSystemBusyboxRw                  Cleaning up
583.
584Ran 1 test in 301.140s
585
586OK
587---------------------
588
589The standard output indicates if the test is successful or not. By
590default, the output folder for the test is deleted automatically
591unless the option +-k+ is passed to *keep* the output directory.
592
593==== Creating a test case
594
595Within the Buildroot repository, the testing framework is organized at the
596top level in +support/testing/+ by folders of +conf+, +infra+ and +tests+.
597All the test cases live under the +tests+ folder and are organized in various
598folders representing the category of test.
599
600The best way to get familiar with how to create a test case is to look
601at a few of the basic file system +support/testing/tests/fs/+ and init
602+support/testing/tests/init/+ test scripts. Those tests give good
603examples of a basic tests that include both checking the build
604results, and doing runtime tests. There are other more advanced cases
605that use things like nested +br2-external+ folders to provide
606skeletons and additional packages.
607
608Creating a basic test case involves:
609
610* Defining a test class that inherits from +infra.basetest.BRTest+
611
612* Defining the +config+ member of the test class, to the Buildroot
613  configuration to build for this test case. It can optionally rely on
614  configuration snippets provided by the runtime test infrastructure:
615  +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ to get a basic
616  architecture/toolchain configuration, and
617  +infra.basetest.MINIMAL_CONFIG+ to not build any filesystem. The
618  advantage of using +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ is that a
619  matching Linux kernel image is provided, which allows to boot the
620  resulting image in Qemu without having to build a Linux kernel image
621  as part of the test case, therefore significant decreasing the build
622  time required for the test case.
623
624* Implementing a +def test_run(self):+ function to implement the
625  actual tests to run after the build has completed. They may be tests
626  that verify the build output, by running command on the host using
627  the +run_cmd_on_host()+ helper function. Or they may boot the
628  generated system in Qemu using the +Emulator+ object available as
629  +self.emulator+ in the test case. For example +self.emulator.boot()+
630  allows to boot the system in Qemu, +self.emulator.login()+ allows to
631  login, +self.emulator.run()+ allows to run shell commands inside
632  Qemu.
633
634After creating the test script, add yourself to the +DEVELOPERS+ file to
635be the maintainer of that test case.
636
637==== Debugging a test case
638
639When a test case runs, the +output_folder+ will contain the following:
640
641---------------------
642$ ls output_folder/
643TestInitSystemBusyboxRw/
644TestInitSystemBusyboxRw-build.log
645TestInitSystemBusyboxRw-run.log
646---------------------
647
648+TestInitSystemBusyboxRw/+ is the Buildroot output directory, and it
649is preserved only if the +-k+ option is passed.
650
651+TestInitSystemBusyboxRw-build.log+ is the log of the Buildroot build.
652
653+TestInitSystemBusyboxRw-run.log+ is the log of the Qemu boot and
654test. This file will only exist if the build was successful and the
655test case involves booting under Qemu.
656
657If you want to manually run Qemu to do manual tests of the build
658result, the first few lines of +TestInitSystemBusyboxRw-run.log+
659contain the Qemu command line to use.
660
661You can also make modifications to the current sources inside the
662+output_folder+ (e.g. for debug purposes) and rerun the standard
663Buildroot make targets (in order to regenerate the complete image with
664the new modifications) and then rerun the test.
665
666==== Runtime tests and Gitlab CI
667
668All runtime tests are regularly executed by Buildroot Gitlab CI
669infrastructure, see .gitlab.yml and
670https://gitlab.com/buildroot.org/buildroot/-/jobs.
671
672You can also use Gitlab CI to test your new test cases, or verify that
673existing tests continue to work after making changes in Buildroot.
674
675In order to achieve this, you need to create a fork of the Buildroot
676project on Gitlab, and be able to push branches to your Buildroot fork
677on Gitlab.
678
679The name of the branch that you push will determine if a Gitlab CI
680pipeline will be triggered or not, and for which test cases.
681
682In the examples below, the <name> component of the branch name is an
683arbitrary string you choose.
684
685* To trigger all run-test test case jobs, push a branch that ends with
686  +-runtime-tests+:
687
688---------------------
689 $ git push gitlab HEAD:<name>-runtime-tests
690---------------------
691
692* To trigger one or several test case jobs, push a branch that ends
693  with the complete test case name
694  (+tests.init.test_busybox.TestInitSystemBusyboxRo+) or with the name
695  of a category of tests (+tests.init.test_busybox+):
696
697---------------------
698 $ git push gitlab HEAD:<name>-<test case name>
699---------------------
700
701Example to run one test:
702
703---------------------
704 $ git push gitlab HEAD:foo-tests.init.test_busybox.TestInitSystemBusyboxRo
705---------------------
706
707Examples to run several tests part of the same group:
708
709---------------------
710 $ git push gitlab HEAD:foo-tests.init.test_busybox
711 $ git push gitlab HEAD:foo-tests.init
712---------------------
713