1*4882a593Smuzhiyun================================== 2*4882a593SmuzhiyunKernel Lock Torture Test Operation 3*4882a593Smuzhiyun================================== 4*4882a593Smuzhiyun 5*4882a593SmuzhiyunCONFIG_LOCK_TORTURE_TEST 6*4882a593Smuzhiyun======================== 7*4882a593Smuzhiyun 8*4882a593SmuzhiyunThe CONFIG LOCK_TORTURE_TEST config option provides a kernel module 9*4882a593Smuzhiyunthat runs torture tests on core kernel locking primitives. The kernel 10*4882a593Smuzhiyunmodule, 'locktorture', may be built after the fact on the running 11*4882a593Smuzhiyunkernel to be tested, if desired. The tests periodically output status 12*4882a593Smuzhiyunmessages via printk(), which can be examined via the dmesg (perhaps 13*4882a593Smuzhiyungrepping for "torture"). The test is started when the module is loaded, 14*4882a593Smuzhiyunand stops when the module is unloaded. This program is based on how RCU 15*4882a593Smuzhiyunis tortured, via rcutorture. 16*4882a593Smuzhiyun 17*4882a593SmuzhiyunThis torture test consists of creating a number of kernel threads which 18*4882a593Smuzhiyunacquire the lock and hold it for specific amount of time, thus simulating 19*4882a593Smuzhiyundifferent critical region behaviors. The amount of contention on the lock 20*4882a593Smuzhiyuncan be simulated by either enlarging this critical region hold time and/or 21*4882a593Smuzhiyuncreating more kthreads. 22*4882a593Smuzhiyun 23*4882a593Smuzhiyun 24*4882a593SmuzhiyunModule Parameters 25*4882a593Smuzhiyun================= 26*4882a593Smuzhiyun 27*4882a593SmuzhiyunThis module has the following parameters: 28*4882a593Smuzhiyun 29*4882a593Smuzhiyun 30*4882a593SmuzhiyunLocktorture-specific 31*4882a593Smuzhiyun-------------------- 32*4882a593Smuzhiyun 33*4882a593Smuzhiyunnwriters_stress 34*4882a593Smuzhiyun Number of kernel threads that will stress exclusive lock 35*4882a593Smuzhiyun ownership (writers). The default value is twice the number 36*4882a593Smuzhiyun of online CPUs. 37*4882a593Smuzhiyun 38*4882a593Smuzhiyunnreaders_stress 39*4882a593Smuzhiyun Number of kernel threads that will stress shared lock 40*4882a593Smuzhiyun ownership (readers). The default is the same amount of writer 41*4882a593Smuzhiyun locks. If the user did not specify nwriters_stress, then 42*4882a593Smuzhiyun both readers and writers be the amount of online CPUs. 43*4882a593Smuzhiyun 44*4882a593Smuzhiyuntorture_type 45*4882a593Smuzhiyun Type of lock to torture. By default, only spinlocks will 46*4882a593Smuzhiyun be tortured. This module can torture the following locks, 47*4882a593Smuzhiyun with string values as follows: 48*4882a593Smuzhiyun 49*4882a593Smuzhiyun - "lock_busted": 50*4882a593Smuzhiyun Simulates a buggy lock implementation. 51*4882a593Smuzhiyun 52*4882a593Smuzhiyun - "spin_lock": 53*4882a593Smuzhiyun spin_lock() and spin_unlock() pairs. 54*4882a593Smuzhiyun 55*4882a593Smuzhiyun - "spin_lock_irq": 56*4882a593Smuzhiyun spin_lock_irq() and spin_unlock_irq() pairs. 57*4882a593Smuzhiyun 58*4882a593Smuzhiyun - "rw_lock": 59*4882a593Smuzhiyun read/write lock() and unlock() rwlock pairs. 60*4882a593Smuzhiyun 61*4882a593Smuzhiyun - "rw_lock_irq": 62*4882a593Smuzhiyun read/write lock_irq() and unlock_irq() 63*4882a593Smuzhiyun rwlock pairs. 64*4882a593Smuzhiyun 65*4882a593Smuzhiyun - "mutex_lock": 66*4882a593Smuzhiyun mutex_lock() and mutex_unlock() pairs. 67*4882a593Smuzhiyun 68*4882a593Smuzhiyun - "rtmutex_lock": 69*4882a593Smuzhiyun rtmutex_lock() and rtmutex_unlock() pairs. 70*4882a593Smuzhiyun Kernel must have CONFIG_RT_MUTEX=y. 71*4882a593Smuzhiyun 72*4882a593Smuzhiyun - "rwsem_lock": 73*4882a593Smuzhiyun read/write down() and up() semaphore pairs. 74*4882a593Smuzhiyun 75*4882a593Smuzhiyun 76*4882a593SmuzhiyunTorture-framework (RCU + locking) 77*4882a593Smuzhiyun--------------------------------- 78*4882a593Smuzhiyun 79*4882a593Smuzhiyunshutdown_secs 80*4882a593Smuzhiyun The number of seconds to run the test before terminating 81*4882a593Smuzhiyun the test and powering off the system. The default is 82*4882a593Smuzhiyun zero, which disables test termination and system shutdown. 83*4882a593Smuzhiyun This capability is useful for automated testing. 84*4882a593Smuzhiyun 85*4882a593Smuzhiyunonoff_interval 86*4882a593Smuzhiyun The number of seconds between each attempt to execute a 87*4882a593Smuzhiyun randomly selected CPU-hotplug operation. Defaults 88*4882a593Smuzhiyun to zero, which disables CPU hotplugging. In 89*4882a593Smuzhiyun CONFIG_HOTPLUG_CPU=n kernels, locktorture will silently 90*4882a593Smuzhiyun refuse to do any CPU-hotplug operations regardless of 91*4882a593Smuzhiyun what value is specified for onoff_interval. 92*4882a593Smuzhiyun 93*4882a593Smuzhiyunonoff_holdoff 94*4882a593Smuzhiyun The number of seconds to wait until starting CPU-hotplug 95*4882a593Smuzhiyun operations. This would normally only be used when 96*4882a593Smuzhiyun locktorture was built into the kernel and started 97*4882a593Smuzhiyun automatically at boot time, in which case it is useful 98*4882a593Smuzhiyun in order to avoid confusing boot-time code with CPUs 99*4882a593Smuzhiyun coming and going. This parameter is only useful if 100*4882a593Smuzhiyun CONFIG_HOTPLUG_CPU is enabled. 101*4882a593Smuzhiyun 102*4882a593Smuzhiyunstat_interval 103*4882a593Smuzhiyun Number of seconds between statistics-related printk()s. 104*4882a593Smuzhiyun By default, locktorture will report stats every 60 seconds. 105*4882a593Smuzhiyun Setting the interval to zero causes the statistics to 106*4882a593Smuzhiyun be printed -only- when the module is unloaded. 107*4882a593Smuzhiyun 108*4882a593Smuzhiyunstutter 109*4882a593Smuzhiyun The length of time to run the test before pausing for this 110*4882a593Smuzhiyun same period of time. Defaults to "stutter=5", so as 111*4882a593Smuzhiyun to run and pause for (roughly) five-second intervals. 112*4882a593Smuzhiyun Specifying "stutter=0" causes the test to run continuously 113*4882a593Smuzhiyun without pausing. 114*4882a593Smuzhiyun 115*4882a593Smuzhiyunshuffle_interval 116*4882a593Smuzhiyun The number of seconds to keep the test threads affinitied 117*4882a593Smuzhiyun to a particular subset of the CPUs, defaults to 3 seconds. 118*4882a593Smuzhiyun Used in conjunction with test_no_idle_hz. 119*4882a593Smuzhiyun 120*4882a593Smuzhiyunverbose 121*4882a593Smuzhiyun Enable verbose debugging printing, via printk(). Enabled 122*4882a593Smuzhiyun by default. This extra information is mostly related to 123*4882a593Smuzhiyun high-level errors and reports from the main 'torture' 124*4882a593Smuzhiyun framework. 125*4882a593Smuzhiyun 126*4882a593Smuzhiyun 127*4882a593SmuzhiyunStatistics 128*4882a593Smuzhiyun========== 129*4882a593Smuzhiyun 130*4882a593SmuzhiyunStatistics are printed in the following format:: 131*4882a593Smuzhiyun 132*4882a593Smuzhiyun spin_lock-torture: Writes: Total: 93746064 Max/Min: 0/0 Fail: 0 133*4882a593Smuzhiyun (A) (B) (C) (D) (E) 134*4882a593Smuzhiyun 135*4882a593Smuzhiyun (A): Lock type that is being tortured -- torture_type parameter. 136*4882a593Smuzhiyun 137*4882a593Smuzhiyun (B): Number of writer lock acquisitions. If dealing with a read/write 138*4882a593Smuzhiyun primitive a second "Reads" statistics line is printed. 139*4882a593Smuzhiyun 140*4882a593Smuzhiyun (C): Number of times the lock was acquired. 141*4882a593Smuzhiyun 142*4882a593Smuzhiyun (D): Min and max number of times threads failed to acquire the lock. 143*4882a593Smuzhiyun 144*4882a593Smuzhiyun (E): true/false values if there were errors acquiring the lock. This should 145*4882a593Smuzhiyun -only- be positive if there is a bug in the locking primitive's 146*4882a593Smuzhiyun implementation. Otherwise a lock should never fail (i.e., spin_lock()). 147*4882a593Smuzhiyun Of course, the same applies for (C), above. A dummy example of this is 148*4882a593Smuzhiyun the "lock_busted" type. 149*4882a593Smuzhiyun 150*4882a593SmuzhiyunUsage 151*4882a593Smuzhiyun===== 152*4882a593Smuzhiyun 153*4882a593SmuzhiyunThe following script may be used to torture locks:: 154*4882a593Smuzhiyun 155*4882a593Smuzhiyun #!/bin/sh 156*4882a593Smuzhiyun 157*4882a593Smuzhiyun modprobe locktorture 158*4882a593Smuzhiyun sleep 3600 159*4882a593Smuzhiyun rmmod locktorture 160*4882a593Smuzhiyun dmesg | grep torture: 161*4882a593Smuzhiyun 162*4882a593SmuzhiyunThe output can be manually inspected for the error flag of "!!!". 163*4882a593SmuzhiyunOne could of course create a more elaborate script that automatically 164*4882a593Smuzhiyunchecked for such errors. The "rmmod" command forces a "SUCCESS", 165*4882a593Smuzhiyun"FAILURE", or "RCU_HOTPLUG" indication to be printk()ed. The first 166*4882a593Smuzhiyuntwo are self-explanatory, while the last indicates that while there 167*4882a593Smuzhiyunwere no locking failures, CPU-hotplug problems were detected. 168*4882a593Smuzhiyun 169*4882a593SmuzhiyunAlso see: Documentation/RCU/torture.rst 170