1*4882a593Smuzhiyun.. SPDX-License-Identifier: GPL-2.0 2*4882a593Smuzhiyun 3*4882a593Smuzhiyun========== 4*4882a593SmuzhiyunNetconsole 5*4882a593Smuzhiyun========== 6*4882a593Smuzhiyun 7*4882a593Smuzhiyun 8*4882a593Smuzhiyunstarted by Ingo Molnar <mingo@redhat.com>, 2001.09.17 9*4882a593Smuzhiyun 10*4882a593Smuzhiyun2.6 port and netpoll api by Matt Mackall <mpm@selenic.com>, Sep 9 2003 11*4882a593Smuzhiyun 12*4882a593SmuzhiyunIPv6 support by Cong Wang <xiyou.wangcong@gmail.com>, Jan 1 2013 13*4882a593Smuzhiyun 14*4882a593SmuzhiyunExtended console support by Tejun Heo <tj@kernel.org>, May 1 2015 15*4882a593Smuzhiyun 16*4882a593SmuzhiyunPlease send bug reports to Matt Mackall <mpm@selenic.com> 17*4882a593SmuzhiyunSatyam Sharma <satyam.sharma@gmail.com>, and Cong Wang <xiyou.wangcong@gmail.com> 18*4882a593Smuzhiyun 19*4882a593SmuzhiyunIntroduction: 20*4882a593Smuzhiyun============= 21*4882a593Smuzhiyun 22*4882a593SmuzhiyunThis module logs kernel printk messages over UDP allowing debugging of 23*4882a593Smuzhiyunproblem where disk logging fails and serial consoles are impractical. 24*4882a593Smuzhiyun 25*4882a593SmuzhiyunIt can be used either built-in or as a module. As a built-in, 26*4882a593Smuzhiyunnetconsole initializes immediately after NIC cards and will bring up 27*4882a593Smuzhiyunthe specified interface as soon as possible. While this doesn't allow 28*4882a593Smuzhiyuncapture of early kernel panics, it does capture most of the boot 29*4882a593Smuzhiyunprocess. 30*4882a593Smuzhiyun 31*4882a593SmuzhiyunSender and receiver configuration: 32*4882a593Smuzhiyun================================== 33*4882a593Smuzhiyun 34*4882a593SmuzhiyunIt takes a string configuration parameter "netconsole" in the 35*4882a593Smuzhiyunfollowing format:: 36*4882a593Smuzhiyun 37*4882a593Smuzhiyun netconsole=[+][src-port]@[src-ip]/[<dev>],[tgt-port]@<tgt-ip>/[tgt-macaddr] 38*4882a593Smuzhiyun 39*4882a593Smuzhiyun where 40*4882a593Smuzhiyun + if present, enable extended console support 41*4882a593Smuzhiyun src-port source for UDP packets (defaults to 6665) 42*4882a593Smuzhiyun src-ip source IP to use (interface address) 43*4882a593Smuzhiyun dev network interface (eth0) 44*4882a593Smuzhiyun tgt-port port for logging agent (6666) 45*4882a593Smuzhiyun tgt-ip IP address for logging agent 46*4882a593Smuzhiyun tgt-macaddr ethernet MAC address for logging agent (broadcast) 47*4882a593Smuzhiyun 48*4882a593SmuzhiyunExamples:: 49*4882a593Smuzhiyun 50*4882a593Smuzhiyun linux netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc 51*4882a593Smuzhiyun 52*4882a593Smuzhiyunor:: 53*4882a593Smuzhiyun 54*4882a593Smuzhiyun insmod netconsole netconsole=@/,@10.0.0.2/ 55*4882a593Smuzhiyun 56*4882a593Smuzhiyunor using IPv6:: 57*4882a593Smuzhiyun 58*4882a593Smuzhiyun insmod netconsole netconsole=@/,@fd00:1:2:3::1/ 59*4882a593Smuzhiyun 60*4882a593SmuzhiyunIt also supports logging to multiple remote agents by specifying 61*4882a593Smuzhiyunparameters for the multiple agents separated by semicolons and the 62*4882a593Smuzhiyuncomplete string enclosed in "quotes", thusly:: 63*4882a593Smuzhiyun 64*4882a593Smuzhiyun modprobe netconsole netconsole="@/,@10.0.0.2/;@/eth1,6892@10.0.0.3/" 65*4882a593Smuzhiyun 66*4882a593SmuzhiyunBuilt-in netconsole starts immediately after the TCP stack is 67*4882a593Smuzhiyuninitialized and attempts to bring up the supplied dev at the supplied 68*4882a593Smuzhiyunaddress. 69*4882a593Smuzhiyun 70*4882a593SmuzhiyunThe remote host has several options to receive the kernel messages, 71*4882a593Smuzhiyunfor example: 72*4882a593Smuzhiyun 73*4882a593Smuzhiyun1) syslogd 74*4882a593Smuzhiyun 75*4882a593Smuzhiyun2) netcat 76*4882a593Smuzhiyun 77*4882a593Smuzhiyun On distributions using a BSD-based netcat version (e.g. Fedora, 78*4882a593Smuzhiyun openSUSE and Ubuntu) the listening port must be specified without 79*4882a593Smuzhiyun the -p switch:: 80*4882a593Smuzhiyun 81*4882a593Smuzhiyun nc -u -l -p <port>' / 'nc -u -l <port> 82*4882a593Smuzhiyun 83*4882a593Smuzhiyun or:: 84*4882a593Smuzhiyun 85*4882a593Smuzhiyun netcat -u -l -p <port>' / 'netcat -u -l <port> 86*4882a593Smuzhiyun 87*4882a593Smuzhiyun3) socat 88*4882a593Smuzhiyun 89*4882a593Smuzhiyun:: 90*4882a593Smuzhiyun 91*4882a593Smuzhiyun socat udp-recv:<port> - 92*4882a593Smuzhiyun 93*4882a593SmuzhiyunDynamic reconfiguration: 94*4882a593Smuzhiyun======================== 95*4882a593Smuzhiyun 96*4882a593SmuzhiyunDynamic reconfigurability is a useful addition to netconsole that enables 97*4882a593Smuzhiyunremote logging targets to be dynamically added, removed, or have their 98*4882a593Smuzhiyunparameters reconfigured at runtime from a configfs-based userspace interface. 99*4882a593Smuzhiyun[ Note that the parameters of netconsole targets that were specified/created 100*4882a593Smuzhiyunfrom the boot/module option are not exposed via this interface, and hence 101*4882a593Smuzhiyuncannot be modified dynamically. ] 102*4882a593Smuzhiyun 103*4882a593SmuzhiyunTo include this feature, select CONFIG_NETCONSOLE_DYNAMIC when building the 104*4882a593Smuzhiyunnetconsole module (or kernel, if netconsole is built-in). 105*4882a593Smuzhiyun 106*4882a593SmuzhiyunSome examples follow (where configfs is mounted at the /sys/kernel/config 107*4882a593Smuzhiyunmountpoint). 108*4882a593Smuzhiyun 109*4882a593SmuzhiyunTo add a remote logging target (target names can be arbitrary):: 110*4882a593Smuzhiyun 111*4882a593Smuzhiyun cd /sys/kernel/config/netconsole/ 112*4882a593Smuzhiyun mkdir target1 113*4882a593Smuzhiyun 114*4882a593SmuzhiyunNote that newly created targets have default parameter values (as mentioned 115*4882a593Smuzhiyunabove) and are disabled by default -- they must first be enabled by writing 116*4882a593Smuzhiyun"1" to the "enabled" attribute (usually after setting parameters accordingly) 117*4882a593Smuzhiyunas described below. 118*4882a593Smuzhiyun 119*4882a593SmuzhiyunTo remove a target:: 120*4882a593Smuzhiyun 121*4882a593Smuzhiyun rmdir /sys/kernel/config/netconsole/othertarget/ 122*4882a593Smuzhiyun 123*4882a593SmuzhiyunThe interface exposes these parameters of a netconsole target to userspace: 124*4882a593Smuzhiyun 125*4882a593Smuzhiyun ============== ================================= ============ 126*4882a593Smuzhiyun enabled Is this target currently enabled? (read-write) 127*4882a593Smuzhiyun extended Extended mode enabled (read-write) 128*4882a593Smuzhiyun dev_name Local network interface name (read-write) 129*4882a593Smuzhiyun local_port Source UDP port to use (read-write) 130*4882a593Smuzhiyun remote_port Remote agent's UDP port (read-write) 131*4882a593Smuzhiyun local_ip Source IP address to use (read-write) 132*4882a593Smuzhiyun remote_ip Remote agent's IP address (read-write) 133*4882a593Smuzhiyun local_mac Local interface's MAC address (read-only) 134*4882a593Smuzhiyun remote_mac Remote agent's MAC address (read-write) 135*4882a593Smuzhiyun ============== ================================= ============ 136*4882a593Smuzhiyun 137*4882a593SmuzhiyunThe "enabled" attribute is also used to control whether the parameters of 138*4882a593Smuzhiyuna target can be updated or not -- you can modify the parameters of only 139*4882a593Smuzhiyundisabled targets (i.e. if "enabled" is 0). 140*4882a593Smuzhiyun 141*4882a593SmuzhiyunTo update a target's parameters:: 142*4882a593Smuzhiyun 143*4882a593Smuzhiyun cat enabled # check if enabled is 1 144*4882a593Smuzhiyun echo 0 > enabled # disable the target (if required) 145*4882a593Smuzhiyun echo eth2 > dev_name # set local interface 146*4882a593Smuzhiyun echo 10.0.0.4 > remote_ip # update some parameter 147*4882a593Smuzhiyun echo cb:a9:87:65:43:21 > remote_mac # update more parameters 148*4882a593Smuzhiyun echo 1 > enabled # enable target again 149*4882a593Smuzhiyun 150*4882a593SmuzhiyunYou can also update the local interface dynamically. This is especially 151*4882a593Smuzhiyunuseful if you want to use interfaces that have newly come up (and may not 152*4882a593Smuzhiyunhave existed when netconsole was loaded / initialized). 153*4882a593Smuzhiyun 154*4882a593SmuzhiyunExtended console: 155*4882a593Smuzhiyun================= 156*4882a593Smuzhiyun 157*4882a593SmuzhiyunIf '+' is prefixed to the configuration line or "extended" config file 158*4882a593Smuzhiyunis set to 1, extended console support is enabled. An example boot 159*4882a593Smuzhiyunparam follows:: 160*4882a593Smuzhiyun 161*4882a593Smuzhiyun linux netconsole=+4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc 162*4882a593Smuzhiyun 163*4882a593SmuzhiyunLog messages are transmitted with extended metadata header in the 164*4882a593Smuzhiyunfollowing format which is the same as /dev/kmsg:: 165*4882a593Smuzhiyun 166*4882a593Smuzhiyun <level>,<sequnum>,<timestamp>,<contflag>;<message text> 167*4882a593Smuzhiyun 168*4882a593SmuzhiyunNon printable characters in <message text> are escaped using "\xff" 169*4882a593Smuzhiyunnotation. If the message contains optional dictionary, verbatim 170*4882a593Smuzhiyunnewline is used as the delimeter. 171*4882a593Smuzhiyun 172*4882a593SmuzhiyunIf a message doesn't fit in certain number of bytes (currently 1000), 173*4882a593Smuzhiyunthe message is split into multiple fragments by netconsole. These 174*4882a593Smuzhiyunfragments are transmitted with "ncfrag" header field added:: 175*4882a593Smuzhiyun 176*4882a593Smuzhiyun ncfrag=<byte-offset>/<total-bytes> 177*4882a593Smuzhiyun 178*4882a593SmuzhiyunFor example, assuming a lot smaller chunk size, a message "the first 179*4882a593Smuzhiyunchunk, the 2nd chunk." may be split as follows:: 180*4882a593Smuzhiyun 181*4882a593Smuzhiyun 6,416,1758426,-,ncfrag=0/31;the first chunk, 182*4882a593Smuzhiyun 6,416,1758426,-,ncfrag=16/31; the 2nd chunk. 183*4882a593Smuzhiyun 184*4882a593SmuzhiyunMiscellaneous notes: 185*4882a593Smuzhiyun==================== 186*4882a593Smuzhiyun 187*4882a593Smuzhiyun.. Warning:: 188*4882a593Smuzhiyun 189*4882a593Smuzhiyun the default target ethernet setting uses the broadcast 190*4882a593Smuzhiyun ethernet address to send packets, which can cause increased load on 191*4882a593Smuzhiyun other systems on the same ethernet segment. 192*4882a593Smuzhiyun 193*4882a593Smuzhiyun.. Tip:: 194*4882a593Smuzhiyun 195*4882a593Smuzhiyun some LAN switches may be configured to suppress ethernet broadcasts 196*4882a593Smuzhiyun so it is advised to explicitly specify the remote agents' MAC addresses 197*4882a593Smuzhiyun from the config parameters passed to netconsole. 198*4882a593Smuzhiyun 199*4882a593Smuzhiyun.. Tip:: 200*4882a593Smuzhiyun 201*4882a593Smuzhiyun to find out the MAC address of, say, 10.0.0.2, you may try using:: 202*4882a593Smuzhiyun 203*4882a593Smuzhiyun ping -c 1 10.0.0.2 ; /sbin/arp -n | grep 10.0.0.2 204*4882a593Smuzhiyun 205*4882a593Smuzhiyun.. Tip:: 206*4882a593Smuzhiyun 207*4882a593Smuzhiyun in case the remote logging agent is on a separate LAN subnet than 208*4882a593Smuzhiyun the sender, it is suggested to try specifying the MAC address of the 209*4882a593Smuzhiyun default gateway (you may use /sbin/route -n to find it out) as the 210*4882a593Smuzhiyun remote MAC address instead. 211*4882a593Smuzhiyun 212*4882a593Smuzhiyun.. note:: 213*4882a593Smuzhiyun 214*4882a593Smuzhiyun the network device (eth1 in the above case) can run any kind 215*4882a593Smuzhiyun of other network traffic, netconsole is not intrusive. Netconsole 216*4882a593Smuzhiyun might cause slight delays in other traffic if the volume of kernel 217*4882a593Smuzhiyun messages is high, but should have no other impact. 218*4882a593Smuzhiyun 219*4882a593Smuzhiyun.. note:: 220*4882a593Smuzhiyun 221*4882a593Smuzhiyun if you find that the remote logging agent is not receiving or 222*4882a593Smuzhiyun printing all messages from the sender, it is likely that you have set 223*4882a593Smuzhiyun the "console_loglevel" parameter (on the sender) to only send high 224*4882a593Smuzhiyun priority messages to the console. You can change this at runtime using:: 225*4882a593Smuzhiyun 226*4882a593Smuzhiyun dmesg -n 8 227*4882a593Smuzhiyun 228*4882a593Smuzhiyun or by specifying "debug" on the kernel command line at boot, to send 229*4882a593Smuzhiyun all kernel messages to the console. A specific value for this parameter 230*4882a593Smuzhiyun can also be set using the "loglevel" kernel boot option. See the 231*4882a593Smuzhiyun dmesg(8) man page and Documentation/admin-guide/kernel-parameters.rst 232*4882a593Smuzhiyun for details. 233*4882a593Smuzhiyun 234*4882a593SmuzhiyunNetconsole was designed to be as instantaneous as possible, to 235*4882a593Smuzhiyunenable the logging of even the most critical kernel bugs. It works 236*4882a593Smuzhiyunfrom IRQ contexts as well, and does not enable interrupts while 237*4882a593Smuzhiyunsending packets. Due to these unique needs, configuration cannot 238*4882a593Smuzhiyunbe more automatic, and some fundamental limitations will remain: 239*4882a593Smuzhiyunonly IP networks, UDP packets and ethernet devices are supported. 240