1*4882a593SmuzhiyunTesting in U-Boot 2*4882a593Smuzhiyun================= 3*4882a593Smuzhiyun 4*4882a593SmuzhiyunU-Boot has a large amount of code. This file describes how this code is 5*4882a593Smuzhiyuntested and what tests you should write when adding a new feature. 6*4882a593Smuzhiyun 7*4882a593Smuzhiyun 8*4882a593SmuzhiyunRunning tests 9*4882a593Smuzhiyun------------- 10*4882a593Smuzhiyun 11*4882a593SmuzhiyunTo run most tests on sandbox, type this: 12*4882a593Smuzhiyun 13*4882a593Smuzhiyun test/run 14*4882a593Smuzhiyun 15*4882a593Smuzhiyunin the U-Boot directory. Note that only the pytest suite is run using this 16*4882a593Smuzhiyuncommand. 17*4882a593Smuzhiyun 18*4882a593Smuzhiyun 19*4882a593SmuzhiyunSandbox 20*4882a593Smuzhiyun------- 21*4882a593SmuzhiyunU-Boot can be built as a user-space application (e.g. for Linux). This 22*4882a593Smuzhiyunallows test to be executed without needing target hardware. The 'sandbox' 23*4882a593Smuzhiyuntarget provides this feature and it is widely used in tests. 24*4882a593Smuzhiyun 25*4882a593Smuzhiyun 26*4882a593SmuzhiyunPytest Suite 27*4882a593Smuzhiyun------------ 28*4882a593Smuzhiyun 29*4882a593SmuzhiyunMany tests are available using the pytest suite, in test/py. This can run 30*4882a593Smuzhiyuneither on sandbox or on real hardware. It relies on the U-Boot console to 31*4882a593Smuzhiyuninject test commands and check the result. It is slower to run than C code, 32*4882a593Smuzhiyunbut provides the ability to unify lots of tests and summarise their results. 33*4882a593Smuzhiyun 34*4882a593SmuzhiyunYou can run the tests on sandbox with: 35*4882a593Smuzhiyun 36*4882a593Smuzhiyun ./test/py/test.py --bd sandbox --build 37*4882a593Smuzhiyun 38*4882a593SmuzhiyunThis will produce HTML output in build-sandbox/test-log.html 39*4882a593Smuzhiyun 40*4882a593SmuzhiyunSee test/py/README.md for more information about the pytest suite. 41*4882a593Smuzhiyun 42*4882a593Smuzhiyun 43*4882a593Smuzhiyuntbot 44*4882a593Smuzhiyun---- 45*4882a593Smuzhiyun 46*4882a593SmuzhiyunTbot provides a way to execute tests on target hardware. It is intended for 47*4882a593Smuzhiyuntrying out both U-Boot and Linux (and potentially other software) on a 48*4882a593Smuzhiyunnumber of boards automatically. It can be used to create a continuous test 49*4882a593Smuzhiyunenvironment. See http://www.tbot.tools for more information. 50*4882a593Smuzhiyun 51*4882a593Smuzhiyun 52*4882a593SmuzhiyunAd-hoc tests 53*4882a593Smuzhiyun------------ 54*4882a593Smuzhiyun 55*4882a593SmuzhiyunThere are several ad-hoc tests which run outside the pytest environment: 56*4882a593Smuzhiyun 57*4882a593Smuzhiyun test/fs - File system test (shell script) 58*4882a593Smuzhiyun test/image - FIT and legacy image tests (shell script and Python) 59*4882a593Smuzhiyun test/stdint - A test that stdint.h can be used in U-Boot (shell script) 60*4882a593Smuzhiyun trace - Test for the tracing feature (shell script) 61*4882a593Smuzhiyun 62*4882a593SmuzhiyunTODO: Move these into pytest. 63*4882a593Smuzhiyun 64*4882a593Smuzhiyun 65*4882a593SmuzhiyunWhen to write tests 66*4882a593Smuzhiyun------------------- 67*4882a593Smuzhiyun 68*4882a593SmuzhiyunIf you add code to U-Boot without a test you are taking a risk. Even if you 69*4882a593Smuzhiyunperform thorough manual testing at the time of submission, it may break when 70*4882a593Smuzhiyunfuture changes are made to U-Boot. It may even break when applied to mainline, 71*4882a593Smuzhiyunif other changes interact with it. A good mindset is that untested code 72*4882a593Smuzhiyunprobably doesn't work and should be deleted. 73*4882a593Smuzhiyun 74*4882a593SmuzhiyunYou can assume that the Pytest suite will be run before patches are accepted 75*4882a593Smuzhiyunto mainline, so this provides protection against future breakage. 76*4882a593Smuzhiyun 77*4882a593SmuzhiyunOn the other hand there is quite a bit of code that is not covered with tests, 78*4882a593Smuzhiyunor is covered sparingly. So here are some suggestions: 79*4882a593Smuzhiyun 80*4882a593Smuzhiyun- If you are adding a new uclass, add a sandbox driver and a test that uses it 81*4882a593Smuzhiyun- If you are modifying code covered by an existing test, add a new test case 82*4882a593Smuzhiyun to cover your changes 83*4882a593Smuzhiyun- If the code you are modifying has not tests, consider writing one. Even a 84*4882a593Smuzhiyun very basic test is useful, and may be picked up and enhanced by others. It 85*4882a593Smuzhiyun is much easier to add onto a test - writing a new large test can seem 86*4882a593Smuzhiyun daunting to most contributors. 87*4882a593Smuzhiyun 88*4882a593Smuzhiyun 89*4882a593SmuzhiyunFuture work 90*4882a593Smuzhiyun----------- 91*4882a593Smuzhiyun 92*4882a593SmuzhiyunConverting existing shell scripts into pytest tests. 93