1*4882a593Smuzhiyun# 2*4882a593Smuzhiyun# This example shows the bisect tests (git bisect and config bisect) 3*4882a593Smuzhiyun# 4*4882a593Smuzhiyun 5*4882a593Smuzhiyun 6*4882a593Smuzhiyun# The config that includes this file may define a RUN_TEST 7*4882a593Smuzhiyun# variable that will tell this config what test to run. 8*4882a593Smuzhiyun# (what to set the TEST option to). 9*4882a593Smuzhiyun# 10*4882a593SmuzhiyunDEFAULTS IF NOT DEFINED RUN_TEST 11*4882a593Smuzhiyun# Requires that hackbench is in the PATH 12*4882a593SmuzhiyunRUN_TEST := ${SSH} hackbench 50 13*4882a593Smuzhiyun 14*4882a593Smuzhiyun 15*4882a593Smuzhiyun# Set TEST to 'bisect' to do a normal git bisect. You need 16*4882a593Smuzhiyun# to modify the options below to make it bisect the exact 17*4882a593Smuzhiyun# commits you are interested in. 18*4882a593Smuzhiyun# 19*4882a593SmuzhiyunTEST_START IF ${TEST} == bisect 20*4882a593SmuzhiyunTEST_TYPE = bisect 21*4882a593Smuzhiyun# You must set the commit that was considered good (git bisect good) 22*4882a593SmuzhiyunBISECT_GOOD = v3.3 23*4882a593Smuzhiyun# You must set the commit that was considered bad (git bisect bad) 24*4882a593SmuzhiyunBISECT_BAD = HEAD 25*4882a593Smuzhiyun# It's best to specify the branch to checkout before starting the bisect. 26*4882a593SmuzhiyunCHECKOUT = origin/master 27*4882a593Smuzhiyun# This can be build, boot, or test. Here we are doing a bisect 28*4882a593Smuzhiyun# that requires to run a test to know if the bisect was good or bad. 29*4882a593Smuzhiyun# The test should exit with 0 on good, non-zero for bad. But see 30*4882a593Smuzhiyun# the BISECT_RET_* options in samples.conf to override this. 31*4882a593SmuzhiyunBISECT_TYPE = test 32*4882a593SmuzhiyunTEST = ${RUN_TEST} 33*4882a593Smuzhiyun# It is usually a good idea to confirm that the GOOD and the BAD 34*4882a593Smuzhiyun# commits are truly good and bad respectively. Having BISECT_CHECK 35*4882a593Smuzhiyun# set to 1 will check both that the good commit works and the bad 36*4882a593Smuzhiyun# commit fails. If you only want to check one or the other, 37*4882a593Smuzhiyun# set BISECT_CHECK to 'good' or to 'bad'. 38*4882a593SmuzhiyunBISECT_CHECK = 1 39*4882a593Smuzhiyun#BISECT_CHECK = good 40*4882a593Smuzhiyun#BISECT_CHECK = bad 41*4882a593Smuzhiyun 42*4882a593Smuzhiyun# Usually it's a good idea to specify the exact config you 43*4882a593Smuzhiyun# want to use throughout the entire bisect. Here we placed 44*4882a593Smuzhiyun# it in the directory we called ktest.pl from and named it 45*4882a593Smuzhiyun# 'config-bisect'. 46*4882a593SmuzhiyunMIN_CONFIG = ${THIS_DIR}/config-bisect 47*4882a593Smuzhiyun# By default, if we are doing a BISECT_TYPE = test run but the 48*4882a593Smuzhiyun# build or boot fails, ktest.pl will do a 'git bisect skip'. 49*4882a593Smuzhiyun# Uncomment the below option to make ktest stop testing on such 50*4882a593Smuzhiyun# an error. 51*4882a593Smuzhiyun#BISECT_SKIP = 0 52*4882a593Smuzhiyun# Now if you had BISECT_SKIP = 0 and the test fails, you can 53*4882a593Smuzhiyun# examine what happened and then do 'git bisect log > /tmp/replay' 54*4882a593Smuzhiyun# Set BISECT_REPLAY to /tmp/replay and ktest.pl will run the 55*4882a593Smuzhiyun# 'git bisect replay /tmp/replay' before continuing the bisect test. 56*4882a593Smuzhiyun#BISECT_REPLAY = /tmp/replay 57*4882a593Smuzhiyun# If you used BISECT_REPLAY after the bisect test failed, you may 58*4882a593Smuzhiyun# not want to continue the bisect on that commit that failed. 59*4882a593Smuzhiyun# By setting BISECT_START to a new commit. ktest.pl will checkout 60*4882a593Smuzhiyun# that commit after it has performed the 'git bisect replay' but 61*4882a593Smuzhiyun# before it continues running the bisect test. 62*4882a593Smuzhiyun#BISECT_START = 2545eb6198e7e1ec50daa0cfc64a4cdfecf24ec9 63*4882a593Smuzhiyun 64*4882a593Smuzhiyun# Now if you don't trust ktest.pl to make the decisions for you, then 65*4882a593Smuzhiyun# set BISECT_MANUAL to 1. This will cause ktest.pl not to decide 66*4882a593Smuzhiyun# if the commit was good or bad. Instead, it will ask you to tell 67*4882a593Smuzhiyun# it if the current commit was good. In the mean time, you could 68*4882a593Smuzhiyun# take the result, load it on any machine you want. Run several tests, 69*4882a593Smuzhiyun# or whatever you feel like. Then, when you are happy, you can tell 70*4882a593Smuzhiyun# ktest if you think it was good or not and ktest.pl will continue 71*4882a593Smuzhiyun# the git bisect. You can even change what commit it is currently at. 72*4882a593Smuzhiyun#BISECT_MANUAL = 1 73*4882a593Smuzhiyun 74*4882a593Smuzhiyun 75*4882a593Smuzhiyun# One of the unique tests that ktest does is the config bisect. 76*4882a593Smuzhiyun# Currently (which hopefully will be fixed soon), the bad config 77*4882a593Smuzhiyun# must be a superset of the good config. This is because it only 78*4882a593Smuzhiyun# searches for a config that causes the target to fail. If the 79*4882a593Smuzhiyun# good config is not a subset of the bad config, or if the target 80*4882a593Smuzhiyun# fails because of a lack of a config, then it will not find 81*4882a593Smuzhiyun# the config for you. 82*4882a593SmuzhiyunTEST_START IF ${TEST} == config-bisect 83*4882a593SmuzhiyunTEST_TYPE = config_bisect 84*4882a593Smuzhiyun# set to build, boot, test 85*4882a593SmuzhiyunCONFIG_BISECT_TYPE = boot 86*4882a593Smuzhiyun# Set the config that is considered bad. 87*4882a593SmuzhiyunCONFIG_BISECT = ${THIS_DIR}/config-bad 88*4882a593Smuzhiyun# This config is optional. By default it uses the 89*4882a593Smuzhiyun# MIN_CONFIG as the good config. 90*4882a593SmuzhiyunCONFIG_BISECT_GOOD = ${THIS_DIR}/config-good 91