1*4882a593Smuzhiyun.. SPDX-License-Identifier: GPL-2.0 2*4882a593Smuzhiyun 3*4882a593Smuzhiyun======================== 4*4882a593SmuzhiyunRCU and lockdep checking 5*4882a593Smuzhiyun======================== 6*4882a593Smuzhiyun 7*4882a593SmuzhiyunAll flavors of RCU have lockdep checking available, so that lockdep is 8*4882a593Smuzhiyunaware of when each task enters and leaves any flavor of RCU read-side 9*4882a593Smuzhiyuncritical section. Each flavor of RCU is tracked separately (but note 10*4882a593Smuzhiyunthat this is not the case in 2.6.32 and earlier). This allows lockdep's 11*4882a593Smuzhiyuntracking to include RCU state, which can sometimes help when debugging 12*4882a593Smuzhiyundeadlocks and the like. 13*4882a593Smuzhiyun 14*4882a593SmuzhiyunIn addition, RCU provides the following primitives that check lockdep's 15*4882a593Smuzhiyunstate:: 16*4882a593Smuzhiyun 17*4882a593Smuzhiyun rcu_read_lock_held() for normal RCU. 18*4882a593Smuzhiyun rcu_read_lock_bh_held() for RCU-bh. 19*4882a593Smuzhiyun rcu_read_lock_sched_held() for RCU-sched. 20*4882a593Smuzhiyun srcu_read_lock_held() for SRCU. 21*4882a593Smuzhiyun 22*4882a593SmuzhiyunThese functions are conservative, and will therefore return 1 if they 23*4882a593Smuzhiyunaren't certain (for example, if CONFIG_DEBUG_LOCK_ALLOC is not set). 24*4882a593SmuzhiyunThis prevents things like WARN_ON(!rcu_read_lock_held()) from giving false 25*4882a593Smuzhiyunpositives when lockdep is disabled. 26*4882a593Smuzhiyun 27*4882a593SmuzhiyunIn addition, a separate kernel config parameter CONFIG_PROVE_RCU enables 28*4882a593Smuzhiyunchecking of rcu_dereference() primitives: 29*4882a593Smuzhiyun 30*4882a593Smuzhiyun rcu_dereference(p): 31*4882a593Smuzhiyun Check for RCU read-side critical section. 32*4882a593Smuzhiyun rcu_dereference_bh(p): 33*4882a593Smuzhiyun Check for RCU-bh read-side critical section. 34*4882a593Smuzhiyun rcu_dereference_sched(p): 35*4882a593Smuzhiyun Check for RCU-sched read-side critical section. 36*4882a593Smuzhiyun srcu_dereference(p, sp): 37*4882a593Smuzhiyun Check for SRCU read-side critical section. 38*4882a593Smuzhiyun rcu_dereference_check(p, c): 39*4882a593Smuzhiyun Use explicit check expression "c" along with 40*4882a593Smuzhiyun rcu_read_lock_held(). This is useful in code that is 41*4882a593Smuzhiyun invoked by both RCU readers and updaters. 42*4882a593Smuzhiyun rcu_dereference_bh_check(p, c): 43*4882a593Smuzhiyun Use explicit check expression "c" along with 44*4882a593Smuzhiyun rcu_read_lock_bh_held(). This is useful in code that 45*4882a593Smuzhiyun is invoked by both RCU-bh readers and updaters. 46*4882a593Smuzhiyun rcu_dereference_sched_check(p, c): 47*4882a593Smuzhiyun Use explicit check expression "c" along with 48*4882a593Smuzhiyun rcu_read_lock_sched_held(). This is useful in code that 49*4882a593Smuzhiyun is invoked by both RCU-sched readers and updaters. 50*4882a593Smuzhiyun srcu_dereference_check(p, c): 51*4882a593Smuzhiyun Use explicit check expression "c" along with 52*4882a593Smuzhiyun srcu_read_lock_held(). This is useful in code that 53*4882a593Smuzhiyun is invoked by both SRCU readers and updaters. 54*4882a593Smuzhiyun rcu_dereference_raw(p): 55*4882a593Smuzhiyun Don't check. (Use sparingly, if at all.) 56*4882a593Smuzhiyun rcu_dereference_protected(p, c): 57*4882a593Smuzhiyun Use explicit check expression "c", and omit all barriers 58*4882a593Smuzhiyun and compiler constraints. This is useful when the data 59*4882a593Smuzhiyun structure cannot change, for example, in code that is 60*4882a593Smuzhiyun invoked only by updaters. 61*4882a593Smuzhiyun rcu_access_pointer(p): 62*4882a593Smuzhiyun Return the value of the pointer and omit all barriers, 63*4882a593Smuzhiyun but retain the compiler constraints that prevent duplicating 64*4882a593Smuzhiyun or coalescsing. This is useful when when testing the 65*4882a593Smuzhiyun value of the pointer itself, for example, against NULL. 66*4882a593Smuzhiyun 67*4882a593SmuzhiyunThe rcu_dereference_check() check expression can be any boolean 68*4882a593Smuzhiyunexpression, but would normally include a lockdep expression. However, 69*4882a593Smuzhiyunany boolean expression can be used. For a moderately ornate example, 70*4882a593Smuzhiyunconsider the following:: 71*4882a593Smuzhiyun 72*4882a593Smuzhiyun file = rcu_dereference_check(fdt->fd[fd], 73*4882a593Smuzhiyun lockdep_is_held(&files->file_lock) || 74*4882a593Smuzhiyun atomic_read(&files->count) == 1); 75*4882a593Smuzhiyun 76*4882a593SmuzhiyunThis expression picks up the pointer "fdt->fd[fd]" in an RCU-safe manner, 77*4882a593Smuzhiyunand, if CONFIG_PROVE_RCU is configured, verifies that this expression 78*4882a593Smuzhiyunis used in: 79*4882a593Smuzhiyun 80*4882a593Smuzhiyun1. An RCU read-side critical section (implicit), or 81*4882a593Smuzhiyun2. with files->file_lock held, or 82*4882a593Smuzhiyun3. on an unshared files_struct. 83*4882a593Smuzhiyun 84*4882a593SmuzhiyunIn case (1), the pointer is picked up in an RCU-safe manner for vanilla 85*4882a593SmuzhiyunRCU read-side critical sections, in case (2) the ->file_lock prevents 86*4882a593Smuzhiyunany change from taking place, and finally, in case (3) the current task 87*4882a593Smuzhiyunis the only task accessing the file_struct, again preventing any change 88*4882a593Smuzhiyunfrom taking place. If the above statement was invoked only from updater 89*4882a593Smuzhiyuncode, it could instead be written as follows:: 90*4882a593Smuzhiyun 91*4882a593Smuzhiyun file = rcu_dereference_protected(fdt->fd[fd], 92*4882a593Smuzhiyun lockdep_is_held(&files->file_lock) || 93*4882a593Smuzhiyun atomic_read(&files->count) == 1); 94*4882a593Smuzhiyun 95*4882a593SmuzhiyunThis would verify cases #2 and #3 above, and furthermore lockdep would 96*4882a593Smuzhiyuncomplain if this was used in an RCU read-side critical section unless one 97*4882a593Smuzhiyunof these two cases held. Because rcu_dereference_protected() omits all 98*4882a593Smuzhiyunbarriers and compiler constraints, it generates better code than do the 99*4882a593Smuzhiyunother flavors of rcu_dereference(). On the other hand, it is illegal 100*4882a593Smuzhiyunto use rcu_dereference_protected() if either the RCU-protected pointer 101*4882a593Smuzhiyunor the RCU-protected data that it points to can change concurrently. 102*4882a593Smuzhiyun 103*4882a593SmuzhiyunLike rcu_dereference(), when lockdep is enabled, RCU list and hlist 104*4882a593Smuzhiyuntraversal primitives check for being called from within an RCU read-side 105*4882a593Smuzhiyuncritical section. However, a lockdep expression can be passed to them 106*4882a593Smuzhiyunas a additional optional argument. With this lockdep expression, these 107*4882a593Smuzhiyuntraversal primitives will complain only if the lockdep expression is 108*4882a593Smuzhiyunfalse and they are called from outside any RCU read-side critical section. 109*4882a593Smuzhiyun 110*4882a593SmuzhiyunFor example, the workqueue for_each_pwq() macro is intended to be used 111*4882a593Smuzhiyuneither within an RCU read-side critical section or with wq->mutex held. 112*4882a593SmuzhiyunIt is thus implemented as follows:: 113*4882a593Smuzhiyun 114*4882a593Smuzhiyun #define for_each_pwq(pwq, wq) 115*4882a593Smuzhiyun list_for_each_entry_rcu((pwq), &(wq)->pwqs, pwqs_node, 116*4882a593Smuzhiyun lock_is_held(&(wq->mutex).dep_map)) 117