test/py: add DFU test

Add a test of DFU functionality to the Python test suite. The test
starts DFU in U-Boot, waits for USB device enumeration on the host,
executes dfu-util multiple times to test

test/py: add DFU test

Add a test of DFU functionality to the Python test suite. The test
starts DFU in U-Boot, waits for USB device enumeration on the host,
executes dfu-util multiple times to test various transfer sizes, many
of which trigger USB driver edge cases, and finally aborts the DFU
command in U-Boot.

This test mirrors the functionality previously available via the shell
scripts in test/dfu, and hence those are removed too.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: ums: add filesystem-based testing

Enhance the UMS test to optionally mount a partition and read/write a file
to it, validating that the content written and read back are identical.

This en

test/py: ums: add filesystem-based testing

Enhance the UMS test to optionally mount a partition and read/write a file
to it, validating that the content written and read back are identical.

This enhancement is backwards-compatible; old boardenv contents that don't
define the new configuration data will cause the test code to perform as
before.

test/ums/ is deleted since the Python test now performs the same testing
that it did.

The code is also re-written to make use of the recently added utility
module, and split it up into nested functions so the overall logic of
the test process can be followed more easily without the details
cluttering the code.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: move U-Boot respawn trigger to the test core

Prior to this change, U-Boot was lazilly (re-)spawned if/when a test
attempted to interact with it, and no active connection existed. This
appro

test/py: move U-Boot respawn trigger to the test core

Prior to this change, U-Boot was lazilly (re-)spawned if/when a test
attempted to interact with it, and no active connection existed. This
approach was simple, yet had the disadvantage that U-Boot might be
spawned in the middle of a test function, e.g. after the test had already
performed actions such as creating data files, etc. In that case, this
could cause the log to contain the sequence (1) some test logs, (2)
U-Boot's boot process, (3) the rest of that test's logs. This isn't
optimally readable. This issue will affect the upcoming DFU and enhanced
UMS tests.

This change converts u_boot_console to be a function-scoped fixture, so
that pytest attempts to re-create the object for each test invocation.
This allows the fixture factory function to ensure that U-Boot is spawned
prior to every test. In practice, the same object is returned each time
so there is essentially no additional overhead due to this change.

This allows us to remove:

- The explicit ensure_spawned() call from test_sleep, since the core now
ensures that the spawn happens before the test code is executed.

- The laxy calls to ensure_spawned() in the u_boot_console_*
implementations.

The one downside is that test_env's "state_ttest_env" fixture must be
converted to a function-scoped fixture too, since a module-scoped fixture
cannot use a function-scoped fixture. To avoid overhead, we use the same
trick of returning the same object each time.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: add a test for the sleep command

Execute "sleep", and validate that it sleeps for approximately the correct
amount of time.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Sim

test/py: add a test for the sleep command

Execute "sleep", and validate that it sleeps for approximately the correct
amount of time.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: test the ums command

This test invokes the "ums" command in U-Boot, and validates that a USB
storage device is enumerated on the test host system, and can be read
from.

Signed-off-by: Step

test/py: test the ums command

This test invokes the "ums" command in U-Boot, and validates that a USB
storage device is enumerated on the test host system, and can be read
from.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: test the shell if command

Migrate all most tests from command_ut.c into the Python test system.
This allows the tests to be run against any U-Boot binary that supports
the if command (i.e.

test/py: test the shell if command

Migrate all most tests from command_ut.c into the Python test system.
This allows the tests to be run against any U-Boot binary that supports
the if command (i.e. where hush is enabled) without requiring that
binary to be permanently bloated with the code from command_ut.

Some tests in command_ut.c can only be executed from C code, since they
test internal (more unit-level) features of various U-Boot APIs. The
migrated tests can all operate directly from the U-Boot console.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: add test of basic shell functionality

This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting

test/py: add test of basic shell functionality

This tests whether the following features of the U-Boot shell:
- Execution of a directly entered command.
- Compound commands (; delimiter).
- Quoting of arguments containing spaces.
- Executing commands from environment variables.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: test the md/mw commands

This tests whether md/mw work, and affect each-other.

Command repeat is also tested.

test/cmd_repeat.sh is removed, since the new Python-based test does
everything

test/py: test the md/mw commands

This tests whether md/mw work, and affect each-other.

Command repeat is also tested.

test/cmd_repeat.sh is removed, since the new Python-based test does
everything it used to.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: add test of setenv/printenv/echo

This tests basic environment variable functionality.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.co

test/py: add test of setenv/printenv/echo

This tests basic environment variable functionality.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: test that sandbox exits when asked

Test the sandbox port's implementation of the reset command and SIGHUP
handling. These should both cause the U-Boot process to exit gracefully.

Signed-of

test/py: test that sandbox exits when asked

Test the sandbox port's implementation of the reset command and SIGHUP
handling. These should both cause the U-Boot process to exit gracefully.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>

show more ...

test/py: Implement pytest infrastructure

This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach
to the U-

test/py: Implement pytest infrastructure

This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach
to the U-Boot console, run the entire script of tests against it, and
summarize the results. Advantages of this approach are:

- Testing is performed in the same way a user or script would interact
with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and
more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.

A few simple tests are provided as examples. Soon, we should convert as
many as possible of the other tests in test/* and test/cmd_ut.c too.

The hook scripts, relay control utilities, and udev rules I use for my
own HW setup are published at https://github.com/swarren/uboot-test-hooks.

See README.md for more details!

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Michal Simek <michal.simek@xilinx.com>
Tested-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org> #v3

show more ...