Lines Matching +full:hardware +full:- +full:docs
1 .. SPDX-License-Identifier: CC-BY-SA-2.0-UK
23 release date. This manual is a work-in-progress and is being initially
27 - *yocto-autobuilder2:* This
28 :yocto_git:`README.md </yocto-autobuilder2/tree/README.md>`
30 Autobuilder. The ``yocto-autobuilder2`` repository represents the
35 - *yocto-autobuilder-helper:* This :yocto_git:`README </yocto-autobuilder-helper/tree/README/>`
37 configuration. The ``yocto-autobuilder-helper`` repository contains
42 …independent of any CI system, which means the code can work `Buildbot <https://docs.buildbot.net/0…
51 released software across supported hardware in an automated and regular
57 `Buildbot <https://docs.buildbot.net/0.9.15.post1/>`__ (version 9) to
58 drive its integration and testing. Buildbot Nine has a plug-in interface
60 ``yocto-autobuilder2`` repository, adding its own console UI plugin. The
61 resulting UI plug-in allows you to visualize builds in a way suited to
65 scripts found in the ``yocto-autobuilder-helper`` repository. The
67 information and is release-specific, which makes it highly customizable
68 on a per-project basis. The layer is CI system-agnostic and contains a
84 .. image:: figures/ab-test-cluster.png
87 Yocto Project Tests - Types of Testing Overview
93 - *Build Testing:* Tests whether specific configurations build by
103 - *Sanity Checks During the Build Process:* Tests initiated through
104 the :ref:`insane <ref-classes-insane>`
110 - *Build Performance Testing:* Tests whether or not commonly used steps
112 commonly used usage scenarios are run through ``oe-build-perf-test``.
120 - *eSDK Testing:* Image tests initiated through the following command::
122 $ bitbake image -c testsdkext
124 …The tests utilize the :ref:`testsdkext <ref-classes-testsdk>` class and the ``do_testsdkext`` task.
126 - *Feature Testing:* Various scenario-based tests are run through the
127 …:ref:`OpenEmbedded Self test (oe-selftest) <ref-manual/release-process:Testing and Quality Assuran…
130 - *Image Testing:* Image tests initiated through the following command::
132 $ bitbake image -c testimage
134 The tests utilize the :ref:`testimage* <ref-classes-testimage*>`
135 classes and the :ref:`ref-tasks-testimage` task.
137 - *Layer Testing:* The Autobuilder has the possibility to test whether
142 - *Package Testing:* A Package Test (ptest) runs tests against packages
145 ptest <dev-manual/common-tasks:Testing Packages With ptest>` section
150 - *SDK Testing:* Image tests initiated through the following command::
152 $ bitbake image -c testsdk
154 The tests utilize the :ref:`testsdk <ref-classes-testsdk>` class and
157 - *Unit Testing:* Unit tests on various components of the system run
158 through :ref:`bitbake-selftest <ref-manual/release-process:Testing and Quality Assurance>` and
159 :ref:`oe-selftest <ref-manual/release-process:Testing and Quality Assurance>`.
161 - *Automatic Upgrade Helper:* This target tests whether new versions of
171 - *bitbake-selftest:*
173 These tests are self-contained and test BitBake as well as its APIs,
179 $ bitbake-selftest
182 variable when running "bitbake-selftest" as follows::
184 $ BB_SKIP_NETTESTS=yes bitbake-selftest
189 $ bitbake-selftest -v
194 "bitbake-selftest" command. For example, to specify the tests for the
197 $ bitbake-selftest bb.test.data.module
202 $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override
205 unittest <https://docs.python.org/3/library/unittest.html>`__.
207 - *oe-selftest:*
209 - These tests use OE to test the workflows, which include testing
212 - The tests can take advantage of parallelism through the "-j"
218 - The tests are based on Python unittest.
220 - The code for the tests resides in
223 - To run all the tests, enter the following command::
225 $ oe-selftest -a
227 - To run a specific test, use the following command form where
230 $ oe-selftest -r <testname>
235 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar
241 $ oe-selftest -r tinfoil
243 - *testimage:*
245 - These tests build an image, boot it, and run tests against the
248 - The code for these tests resides in ``meta/lib/oeqa/runtime/cases/``.
250 - You need to set the :term:`IMAGE_CLASSES` variable as follows::
254 - Run the tests using the following command form::
256 $ bitbake image -c testimage
258 - *testsdk:*
260 - These tests build an SDK, install it, and then run tests against
263 - The code for these tests resides in ``meta/lib/oeqa/sdk/cases/``.
265 - Run the test using the following command form::
267 $ bitbake image -c testsdk
269 - *testsdk_ext:*
271 - These tests build an extended SDK (eSDK), install that eSDK, and
274 - The code for these tests resides in ``meta/lib/oeqa/esdk``.
276 - To run the tests, use the following command form::
278 $ bitbake image -c testsdkext
280 - *oe-build-perf-test:*
282 - These tests run through commonly used usage scenarios and measure
285 - The code for these tests resides in ``meta/lib/oeqa/buildperf``.
287 - To run the tests, use the following command form::
289 $ oe-build-perf-test <options>
293 Scripts include the ``build-perf-test-wrapper`` script with
294 examples of how to use the oe-build-perf-test from the command
297 Use the ``oe-git-archive`` command to store test results into a
300 Use the ``oe-build-perf-report`` command to generate text reports
303 …:yocto_dl:`/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_w…
305 …:yocto_dl:`/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_w…
307 - The tests are contained in ``lib/oeqa/buildperf/test_basic.py``.
313 :ref:`test-manual/intro:How Tests Map to Areas of Code` section.
318 For oe-selftest. bitbake testcases reside in the ``lib/bb/tests/``
321 ``bitbake-selftest``
322 --------------------
345 these tests at: https://docs.python.org/3/library/unittest.html.
347 ``oe-selftest``
348 ---------------
358 result = runCmd('bitbake-layers show-cross-depends')
359 …spell' in result.output, msg = "No dependencies were shown. bitbake-layers show-cross-depends outp…
363 from ``unittest.TestCase``, which runs the ``bitbake-layers`` command
370 - *Obtaining the value of a bitbake variable:* Use
374 - *Running a bitbake invocation for a build:* Use
377 - *Running a command:* Use ``oeqa.utils.commandsrunCmd()``
391 -------------
394 hardware or under QEMU. As a result, they are assumed to be running in a
401 @OEHasPackage(['python3-core'])
403 cmd = "python3 -c \\"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
415 in this example would only make sense if python3-core is installed in
419 ---------------
432 subprocess.check_output(['git', 'commit', '-m', "'test commit'"], cwd=cls.myapp_dst)
451 -----------
460 if not (self.tc.hasHostPackage("nativesdk-python3-core") or
461 self.tc.hasHostPackage("python3-core-native")):
465 cmd = "python3 -c \\"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
469 In this example, if nativesdk-python3-core has been installed into the SDK, the code runs
473 ``oe-build-perf-test``
474 ----------------------
482 """Bitbake parsing (bitbake -p)"""
486 self.measure_cmd_resources(['bitbake', '-p'], 'parse_1',
487 'bitbake -p (no caches)')
490 self.measure_cmd_resources(['bitbake', '-p'], 'parse_2',
491 'bitbake -p (no tmp/cache)')
493 self.measure_cmd_resources(['bitbake', '-p'], 'parse_3',
494 'bitbake -p (cached)')
517 be set to an isolated directory. Alternatively, you can use the "-f"
523 This is particularly true for oe-selftests since these can run in