1*4882a593Smuzhiyunperf-probe(1) 2*4882a593Smuzhiyun============= 3*4882a593Smuzhiyun 4*4882a593SmuzhiyunNAME 5*4882a593Smuzhiyun---- 6*4882a593Smuzhiyunperf-probe - Define new dynamic tracepoints 7*4882a593Smuzhiyun 8*4882a593SmuzhiyunSYNOPSIS 9*4882a593Smuzhiyun-------- 10*4882a593Smuzhiyun[verse] 11*4882a593Smuzhiyun'perf probe' [options] --add='PROBE' [...] 12*4882a593Smuzhiyunor 13*4882a593Smuzhiyun'perf probe' [options] PROBE 14*4882a593Smuzhiyunor 15*4882a593Smuzhiyun'perf probe' [options] --del='[GROUP:]EVENT' [...] 16*4882a593Smuzhiyunor 17*4882a593Smuzhiyun'perf probe' --list[=[GROUP:]EVENT] 18*4882a593Smuzhiyunor 19*4882a593Smuzhiyun'perf probe' [options] --line='LINE' 20*4882a593Smuzhiyunor 21*4882a593Smuzhiyun'perf probe' [options] --vars='PROBEPOINT' 22*4882a593Smuzhiyunor 23*4882a593Smuzhiyun'perf probe' [options] --funcs 24*4882a593Smuzhiyunor 25*4882a593Smuzhiyun'perf probe' [options] --definition='PROBE' [...] 26*4882a593Smuzhiyun 27*4882a593SmuzhiyunDESCRIPTION 28*4882a593Smuzhiyun----------- 29*4882a593SmuzhiyunThis command defines dynamic tracepoint events, by symbol and registers 30*4882a593Smuzhiyunwithout debuginfo, or by C expressions (C line numbers, C function names, 31*4882a593Smuzhiyunand C local variables) with debuginfo. 32*4882a593Smuzhiyun 33*4882a593Smuzhiyun 34*4882a593SmuzhiyunOPTIONS 35*4882a593Smuzhiyun------- 36*4882a593Smuzhiyun-k:: 37*4882a593Smuzhiyun--vmlinux=PATH:: 38*4882a593Smuzhiyun Specify vmlinux path which has debuginfo (Dwarf binary). 39*4882a593Smuzhiyun Only when using this with --definition, you can give an offline 40*4882a593Smuzhiyun vmlinux file. 41*4882a593Smuzhiyun 42*4882a593Smuzhiyun-m:: 43*4882a593Smuzhiyun--module=MODNAME|PATH:: 44*4882a593Smuzhiyun Specify module name in which perf-probe searches probe points 45*4882a593Smuzhiyun or lines. If a path of module file is passed, perf-probe 46*4882a593Smuzhiyun treat it as an offline module (this means you can add a probe on 47*4882a593Smuzhiyun a module which has not been loaded yet). 48*4882a593Smuzhiyun 49*4882a593Smuzhiyun-s:: 50*4882a593Smuzhiyun--source=PATH:: 51*4882a593Smuzhiyun Specify path to kernel source. 52*4882a593Smuzhiyun 53*4882a593Smuzhiyun-v:: 54*4882a593Smuzhiyun--verbose:: 55*4882a593Smuzhiyun Be more verbose (show parsed arguments, etc). 56*4882a593Smuzhiyun Can not use with -q. 57*4882a593Smuzhiyun 58*4882a593Smuzhiyun-q:: 59*4882a593Smuzhiyun--quiet:: 60*4882a593Smuzhiyun Be quiet (do not show any messages including errors). 61*4882a593Smuzhiyun Can not use with -v. 62*4882a593Smuzhiyun 63*4882a593Smuzhiyun-a:: 64*4882a593Smuzhiyun--add=:: 65*4882a593Smuzhiyun Define a probe event (see PROBE SYNTAX for detail). 66*4882a593Smuzhiyun 67*4882a593Smuzhiyun-d:: 68*4882a593Smuzhiyun--del=:: 69*4882a593Smuzhiyun Delete probe events. This accepts glob wildcards('*', '?') and character 70*4882a593Smuzhiyun classes(e.g. [a-z], [!A-Z]). 71*4882a593Smuzhiyun 72*4882a593Smuzhiyun-l:: 73*4882a593Smuzhiyun--list[=[GROUP:]EVENT]:: 74*4882a593Smuzhiyun List up current probe events. This can also accept filtering patterns of 75*4882a593Smuzhiyun event names. 76*4882a593Smuzhiyun When this is used with --cache, perf shows all cached probes instead of 77*4882a593Smuzhiyun the live probes. 78*4882a593Smuzhiyun 79*4882a593Smuzhiyun-L:: 80*4882a593Smuzhiyun--line=:: 81*4882a593Smuzhiyun Show source code lines which can be probed. This needs an argument 82*4882a593Smuzhiyun which specifies a range of the source code. (see LINE SYNTAX for detail) 83*4882a593Smuzhiyun 84*4882a593Smuzhiyun-V:: 85*4882a593Smuzhiyun--vars=:: 86*4882a593Smuzhiyun Show available local variables at given probe point. The argument 87*4882a593Smuzhiyun syntax is same as PROBE SYNTAX, but NO ARGs. 88*4882a593Smuzhiyun 89*4882a593Smuzhiyun--externs:: 90*4882a593Smuzhiyun (Only for --vars) Show external defined variables in addition to local 91*4882a593Smuzhiyun variables. 92*4882a593Smuzhiyun 93*4882a593Smuzhiyun--no-inlines:: 94*4882a593Smuzhiyun (Only for --add) Search only for non-inlined functions. The functions 95*4882a593Smuzhiyun which do not have instances are ignored. 96*4882a593Smuzhiyun 97*4882a593Smuzhiyun-F:: 98*4882a593Smuzhiyun--funcs[=FILTER]:: 99*4882a593Smuzhiyun Show available functions in given module or kernel. With -x/--exec, 100*4882a593Smuzhiyun can also list functions in a user space executable / shared library. 101*4882a593Smuzhiyun This also can accept a FILTER rule argument. 102*4882a593Smuzhiyun 103*4882a593Smuzhiyun-D:: 104*4882a593Smuzhiyun--definition=:: 105*4882a593Smuzhiyun Show trace-event definition converted from given probe-event instead 106*4882a593Smuzhiyun of write it into tracing/[k,u]probe_events. 107*4882a593Smuzhiyun 108*4882a593Smuzhiyun--filter=FILTER:: 109*4882a593Smuzhiyun (Only for --vars and --funcs) Set filter. FILTER is a combination of glob 110*4882a593Smuzhiyun pattern, see FILTER PATTERN for detail. 111*4882a593Smuzhiyun Default FILTER is "!__k???tab_* & !__crc_*" for --vars, and "!_*" 112*4882a593Smuzhiyun for --funcs. 113*4882a593Smuzhiyun If several filters are specified, only the last filter is used. 114*4882a593Smuzhiyun 115*4882a593Smuzhiyun-f:: 116*4882a593Smuzhiyun--force:: 117*4882a593Smuzhiyun Forcibly add events with existing name. 118*4882a593Smuzhiyun 119*4882a593Smuzhiyun-n:: 120*4882a593Smuzhiyun--dry-run:: 121*4882a593Smuzhiyun Dry run. With this option, --add and --del doesn't execute actual 122*4882a593Smuzhiyun adding and removal operations. 123*4882a593Smuzhiyun 124*4882a593Smuzhiyun--cache:: 125*4882a593Smuzhiyun (With --add) Cache the probes. Any events which successfully added 126*4882a593Smuzhiyun are also stored in the cache file. 127*4882a593Smuzhiyun (With --list) Show cached probes. 128*4882a593Smuzhiyun (With --del) Remove cached probes. 129*4882a593Smuzhiyun 130*4882a593Smuzhiyun--max-probes=NUM:: 131*4882a593Smuzhiyun Set the maximum number of probe points for an event. Default is 128. 132*4882a593Smuzhiyun 133*4882a593Smuzhiyun--target-ns=PID: 134*4882a593Smuzhiyun Obtain mount namespace information from the target pid. This is 135*4882a593Smuzhiyun used when creating a uprobe for a process that resides in a 136*4882a593Smuzhiyun different mount namespace from the perf(1) utility. 137*4882a593Smuzhiyun 138*4882a593Smuzhiyun-x:: 139*4882a593Smuzhiyun--exec=PATH:: 140*4882a593Smuzhiyun Specify path to the executable or shared library file for user 141*4882a593Smuzhiyun space tracing. Can also be used with --funcs option. 142*4882a593Smuzhiyun 143*4882a593Smuzhiyun--demangle:: 144*4882a593Smuzhiyun Demangle application symbols. --no-demangle is also available 145*4882a593Smuzhiyun for disabling demangling. 146*4882a593Smuzhiyun 147*4882a593Smuzhiyun--demangle-kernel:: 148*4882a593Smuzhiyun Demangle kernel symbols. --no-demangle-kernel is also available 149*4882a593Smuzhiyun for disabling kernel demangling. 150*4882a593Smuzhiyun 151*4882a593SmuzhiyunIn absence of -m/-x options, perf probe checks if the first argument after 152*4882a593Smuzhiyunthe options is an absolute path name. If its an absolute path, perf probe 153*4882a593Smuzhiyunuses it as a target module/target user space binary to probe. 154*4882a593Smuzhiyun 155*4882a593SmuzhiyunPROBE SYNTAX 156*4882a593Smuzhiyun------------ 157*4882a593SmuzhiyunProbe points are defined by following syntax. 158*4882a593Smuzhiyun 159*4882a593Smuzhiyun 1) Define event based on function name 160*4882a593Smuzhiyun [[GROUP:]EVENT=]FUNC[@SRC][:RLN|+OFFS|%return|;PTN] [ARG ...] 161*4882a593Smuzhiyun 162*4882a593Smuzhiyun 2) Define event based on source file with line number 163*4882a593Smuzhiyun [[GROUP:]EVENT=]SRC:ALN [ARG ...] 164*4882a593Smuzhiyun 165*4882a593Smuzhiyun 3) Define event based on source file with lazy pattern 166*4882a593Smuzhiyun [[GROUP:]EVENT=]SRC;PTN [ARG ...] 167*4882a593Smuzhiyun 168*4882a593Smuzhiyun 4) Pre-defined SDT events or cached event with name 169*4882a593Smuzhiyun %[sdt_PROVIDER:]SDTEVENT 170*4882a593Smuzhiyun or, 171*4882a593Smuzhiyun sdt_PROVIDER:SDTEVENT 172*4882a593Smuzhiyun 173*4882a593Smuzhiyun'EVENT' specifies the name of new event, if omitted, it will be set the name of the probed function, and for return probes, a "\_\_return" suffix is automatically added to the function name. You can also specify a group name by 'GROUP', if omitted, set 'probe' is used for kprobe and 'probe_<bin>' is used for uprobe. 174*4882a593SmuzhiyunNote that using existing group name can conflict with other events. Especially, using the group name reserved for kernel modules can hide embedded events in the 175*4882a593Smuzhiyunmodules. 176*4882a593Smuzhiyun'FUNC' specifies a probed function name, and it may have one of the following options; '+OFFS' is the offset from function entry address in bytes, ':RLN' is the relative-line number from function entry line, and '%return' means that it probes function return. And ';PTN' means lazy matching pattern (see LAZY MATCHING). Note that ';PTN' must be the end of the probe point definition. In addition, '@SRC' specifies a source file which has that function. 177*4882a593SmuzhiyunIt is also possible to specify a probe point by the source line number or lazy matching by using 'SRC:ALN' or 'SRC;PTN' syntax, where 'SRC' is the source file path, ':ALN' is the line number and ';PTN' is the lazy matching pattern. 178*4882a593Smuzhiyun'ARG' specifies the arguments of this probe point, (see PROBE ARGUMENT). 179*4882a593Smuzhiyun'SDTEVENT' and 'PROVIDER' is the pre-defined event name which is defined by user SDT (Statically Defined Tracing) or the pre-cached probes with event name. 180*4882a593SmuzhiyunNote that before using the SDT event, the target binary (on which SDT events are defined) must be scanned by linkperf:perf-buildid-cache[1] to make SDT events as cached events. 181*4882a593Smuzhiyun 182*4882a593SmuzhiyunFor details of the SDT, see below. 183*4882a593Smuzhiyunhttps://sourceware.org/gdb/onlinedocs/gdb/Static-Probe-Points.html 184*4882a593Smuzhiyun 185*4882a593SmuzhiyunESCAPED CHARACTER 186*4882a593Smuzhiyun----------------- 187*4882a593Smuzhiyun 188*4882a593SmuzhiyunIn the probe syntax, '=', '@', '+', ':' and ';' are treated as a special character. You can use a backslash ('\') to escape the special characters. 189*4882a593SmuzhiyunThis is useful if you need to probe on a specific versioned symbols, like @GLIBC_... suffixes, or also you need to specify a source file which includes the special characters. 190*4882a593SmuzhiyunNote that usually single backslash is consumed by shell, so you might need to pass double backslash (\\) or wrapping with single quotes (\'AAA\@BBB'). 191*4882a593SmuzhiyunSee EXAMPLES how it is used. 192*4882a593Smuzhiyun 193*4882a593SmuzhiyunPROBE ARGUMENT 194*4882a593Smuzhiyun-------------- 195*4882a593SmuzhiyunEach probe argument follows below syntax. 196*4882a593Smuzhiyun 197*4882a593Smuzhiyun [NAME=]LOCALVAR|$retval|%REG|@SYMBOL[:TYPE][@user] 198*4882a593Smuzhiyun 199*4882a593Smuzhiyun'NAME' specifies the name of this argument (optional). You can use the name of local variable, local data structure member (e.g. var->field, var.field2), local array with fixed index (e.g. array[1], var->array[0], var->pointer[2]), or kprobe-tracer argument format (e.g. $retval, %ax, etc). Note that the name of this argument will be set as the last member name if you specify a local data structure member (e.g. field2 for 'var->field1.field2'.) 200*4882a593Smuzhiyun'$vars' and '$params' special arguments are also available for NAME, '$vars' is expanded to the local variables (including function parameters) which can access at given probe point. '$params' is expanded to only the function parameters. 201*4882a593Smuzhiyun'TYPE' casts the type of this argument (optional). If omitted, perf probe automatically set the type based on debuginfo (*). Currently, basic types (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal integers (x/x8/x16/x32/x64), signedness casting (u/s), "string" and bitfield are supported. (see TYPES for detail) 202*4882a593SmuzhiyunOn x86 systems %REG is always the short form of the register: for example %AX. %RAX or %EAX is not valid. 203*4882a593Smuzhiyun"@user" is a special attribute which means the LOCALVAR will be treated as a user-space memory. This is only valid for kprobe event. 204*4882a593Smuzhiyun 205*4882a593SmuzhiyunTYPES 206*4882a593Smuzhiyun----- 207*4882a593SmuzhiyunBasic types (u8/u16/u32/u64/s8/s16/s32/s64) and hexadecimal integers (x8/x16/x32/x64) are integer types. Prefix 's' and 'u' means those types are signed and unsigned respectively, and 'x' means that is shown in hexadecimal format. Traced arguments are shown in decimal (sNN/uNN) or hex (xNN). You can also use 's' or 'u' to specify only signedness and leave its size auto-detected by perf probe. Moreover, you can use 'x' to explicitly specify to be shown in hexadecimal (the size is also auto-detected). 208*4882a593SmuzhiyunString type is a special type, which fetches a "null-terminated" string from kernel space. This means it will fail and store NULL if the string container has been paged out. You can specify 'string' type only for the local variable or structure member which is an array of or a pointer to 'char' or 'unsigned char' type. 209*4882a593SmuzhiyunBitfield is another special type, which takes 3 parameters, bit-width, bit-offset, and container-size (usually 32). The syntax is; 210*4882a593Smuzhiyun 211*4882a593Smuzhiyun b<bit-width>@<bit-offset>/<container-size> 212*4882a593Smuzhiyun 213*4882a593SmuzhiyunLINE SYNTAX 214*4882a593Smuzhiyun----------- 215*4882a593SmuzhiyunLine range is described by following syntax. 216*4882a593Smuzhiyun 217*4882a593Smuzhiyun "FUNC[@SRC][:RLN[+NUM|-RLN2]]|SRC[:ALN[+NUM|-ALN2]]" 218*4882a593Smuzhiyun 219*4882a593SmuzhiyunFUNC specifies the function name of showing lines. 'RLN' is the start line 220*4882a593Smuzhiyunnumber from function entry line, and 'RLN2' is the end line number. As same as 221*4882a593Smuzhiyunprobe syntax, 'SRC' means the source file path, 'ALN' is start line number, 222*4882a593Smuzhiyunand 'ALN2' is end line number in the file. It is also possible to specify how 223*4882a593Smuzhiyunmany lines to show by using 'NUM'. Moreover, 'FUNC@SRC' combination is good 224*4882a593Smuzhiyunfor searching a specific function when several functions share same name. 225*4882a593SmuzhiyunSo, "source.c:100-120" shows lines between 100th to l20th in source.c file. And "func:10+20" shows 20 lines from 10th line of func function. 226*4882a593Smuzhiyun 227*4882a593SmuzhiyunLAZY MATCHING 228*4882a593Smuzhiyun------------- 229*4882a593Smuzhiyun The lazy line matching is similar to glob matching but ignoring spaces in both of pattern and target. So this accepts wildcards('*', '?') and character classes(e.g. [a-z], [!A-Z]). 230*4882a593Smuzhiyun 231*4882a593Smuzhiyune.g. 232*4882a593Smuzhiyun 'a=*' can matches 'a=b', 'a = b', 'a == b' and so on. 233*4882a593Smuzhiyun 234*4882a593SmuzhiyunThis provides some sort of flexibility and robustness to probe point definitions against minor code changes. For example, actual 10th line of schedule() can be moved easily by modifying schedule(), but the same line matching 'rq=cpu_rq*' may still exist in the function.) 235*4882a593Smuzhiyun 236*4882a593SmuzhiyunFILTER PATTERN 237*4882a593Smuzhiyun-------------- 238*4882a593Smuzhiyun The filter pattern is a glob matching pattern(s) to filter variables. 239*4882a593Smuzhiyun In addition, you can use "!" for specifying filter-out rule. You also can give several rules combined with "&" or "|", and fold those rules as one rule by using "(" ")". 240*4882a593Smuzhiyun 241*4882a593Smuzhiyune.g. 242*4882a593Smuzhiyun With --filter "foo* | bar*", perf probe -V shows variables which start with "foo" or "bar". 243*4882a593Smuzhiyun With --filter "!foo* & *bar", perf probe -V shows variables which don't start with "foo" and end with "bar", like "fizzbar". But "foobar" is filtered out. 244*4882a593Smuzhiyun 245*4882a593SmuzhiyunEXAMPLES 246*4882a593Smuzhiyun-------- 247*4882a593SmuzhiyunDisplay which lines in schedule() can be probed: 248*4882a593Smuzhiyun 249*4882a593Smuzhiyun ./perf probe --line schedule 250*4882a593Smuzhiyun 251*4882a593SmuzhiyunAdd a probe on schedule() function 12th line with recording cpu local variable: 252*4882a593Smuzhiyun 253*4882a593Smuzhiyun ./perf probe schedule:12 cpu 254*4882a593Smuzhiyun or 255*4882a593Smuzhiyun ./perf probe --add='schedule:12 cpu' 256*4882a593Smuzhiyun 257*4882a593SmuzhiyunAdd one or more probes which has the name start with "schedule". 258*4882a593Smuzhiyun 259*4882a593Smuzhiyun ./perf probe schedule* 260*4882a593Smuzhiyun or 261*4882a593Smuzhiyun ./perf probe --add='schedule*' 262*4882a593Smuzhiyun 263*4882a593SmuzhiyunAdd probes on lines in schedule() function which calls update_rq_clock(). 264*4882a593Smuzhiyun 265*4882a593Smuzhiyun ./perf probe 'schedule;update_rq_clock*' 266*4882a593Smuzhiyun or 267*4882a593Smuzhiyun ./perf probe --add='schedule;update_rq_clock*' 268*4882a593Smuzhiyun 269*4882a593SmuzhiyunDelete all probes on schedule(). 270*4882a593Smuzhiyun 271*4882a593Smuzhiyun ./perf probe --del='schedule*' 272*4882a593Smuzhiyun 273*4882a593SmuzhiyunAdd probes at zfree() function on /bin/zsh 274*4882a593Smuzhiyun 275*4882a593Smuzhiyun ./perf probe -x /bin/zsh zfree or ./perf probe /bin/zsh zfree 276*4882a593Smuzhiyun 277*4882a593SmuzhiyunAdd probes at malloc() function on libc 278*4882a593Smuzhiyun 279*4882a593Smuzhiyun ./perf probe -x /lib/libc.so.6 malloc or ./perf probe /lib/libc.so.6 malloc 280*4882a593Smuzhiyun 281*4882a593SmuzhiyunAdd a uprobe to a target process running in a different mount namespace 282*4882a593Smuzhiyun 283*4882a593Smuzhiyun ./perf probe --target-ns <target pid> -x /lib64/libc.so.6 malloc 284*4882a593Smuzhiyun 285*4882a593SmuzhiyunAdd a USDT probe to a target process running in a different mount namespace 286*4882a593Smuzhiyun 287*4882a593Smuzhiyun ./perf probe --target-ns <target pid> -x /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/lib/amd64/server/libjvm.so %sdt_hotspot:thread__sleep__end 288*4882a593Smuzhiyun 289*4882a593SmuzhiyunAdd a probe on specific versioned symbol by backslash escape 290*4882a593Smuzhiyun 291*4882a593Smuzhiyun ./perf probe -x /lib64/libc-2.25.so 'malloc_get_state\@GLIBC_2.2.5' 292*4882a593Smuzhiyun 293*4882a593SmuzhiyunAdd a probe in a source file using special characters by backslash escape 294*4882a593Smuzhiyun 295*4882a593Smuzhiyun ./perf probe -x /opt/test/a.out 'foo\+bar.c:4' 296*4882a593Smuzhiyun 297*4882a593Smuzhiyun 298*4882a593SmuzhiyunSEE ALSO 299*4882a593Smuzhiyun-------- 300*4882a593Smuzhiyunlinkperf:perf-trace[1], linkperf:perf-record[1], linkperf:perf-buildid-cache[1] 301