Android procfs demystified – sysrq-trigger

31 minute read

The magic SysRq key is a key combination(Alt+SysRq+<cmd>) implemented by linux kernel, it is usually used to get more information for further investigation when system goes unresponsive, it will respond unless the kernel is completely locked up.

sysrq is system request for short, by the way.

To use this feature, kernel must be configured with CONFIG_MAGIC_SYSRQ, use below command to check this out:

# cat /proc/config.gz | gunzip | grep SYSRQ

Here is the example output of above command issued on raspberry pi board running LineageOS 16:

CONFIG_MAGIC_SYSRQ=y
CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1
CONFIG_MAGIC_SYSRQ_SERIAL=y

Before using it, it must be enabled, confirm the status with sysctl:

# sysctl kernel.sysrq

If disabled, then do the following to enable it:

# sysctl kernel.sysrq=1

Supported commands

The following commands are excerpted from kernel documentation Linux Magic System Request Key Hacks, I only put those commands confirmed to be working on rpi3b board:

Command Function
b Will immediately reboot the system without syncing or unmounting your disks.
c Will perform a system crash by a NULL pointer dereference. A crashdump will be taken if configured.
e Send a SIGTERM to all processes, except for init.
f Will call the oom killer to kill a memory hog process, but do not panic if nothing can be killed.
h Will display help (actually any other key than those listed here will display help. but h is easy to remember :-)
i Send a SIGKILL to all processes, except for init.
j Forcibly “Just thaw it” - filesystems frozen by the FIFREEZE ioctl.
k Secure Access Key (SAK) Kills all programs on the current virtual console.
l Shows a stack backtrace for all active CPUs.
m Will dump current memory info to your console.
n Used to make RT tasks nice-able
o Will shut your system off (if configured and supported).
p Will dump the current registers and flags to your console.
q Will dump per CPU lists of all armed hrtimers (but NOT regular timer_list timers) and detailed information about all clockevent devices.
r Turns off keyboard raw mode and sets it to XLATE.
s Will attempt to sync all mounted filesystems.
t Will dump a list of current tasks and their information to your console.
u Will attempt to remount all mounted filesystems read-only.
w Dumps tasks that are in uninterruptable (blocked) state.
z Dump the ftrace buffer
0-9 Sets the console log level, controlling which kernel messages will be printed to your console. (0, for example would make it so that only emergency messages like PANICs or OOPSes would make it to your console.)

Examples

Here is a list of example output of raspberry pi 3B board:

Get commands supported by current kernel

echo h > /proc/sysrq-trigger or press Alt+SysRq+y if the serial console is not responsive:

[   21.805968] sysrq: SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) show-blocked-tasks(w) dump-ftrace-buffer(z)

I don’t know why Alt+SysRq+h not triggering the kernel to show help, but any other command except listed above can show the help, such as Alt+SysRq+y.

Reboot system

echo b > /proc/sysrq-trigger or press Alt+SysRq+b combination.

[  809.933332] sysrq: SysRq : Resetting
[  809.937014] SMP: stopping secondary CPUs

Note this is not a graceful way to reboot system.

crash kernel

echo c > /proc/sysrq-trigger or press Alt+SysRq+c combination.

[   93.060261] sysrq: SysRq : Trigger a crash
[   93.064456] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[   93.072667] Mem abort info:
[   93.075502]   Exception class = DABT (current EL), IL = 32 bits
[   93.081507]   SET = 0, FnV = 0
[   93.084604]   EA = 0, S1PTW = 0
[   93.087787] Data abort info:
[   93.090706]   ISV = 0, ISS = 0x00000046
[   93.094596]   CM = 0, WnR = 1
[   93.097611] user pgtable: 4k pages, 39-bit VAs, pgd = fffffff2b323f000
[   93.104233] [0000000000000000] *pgd=0000000033222003, *pud=0000000033222003, *pmd=0000000000000000
[   93.113344] Internal error: Oops: 96000046 [#1] PREEMPT SMP
[   93.119001] Modules linked in:
[   93.122108] Process Binder:287_6 (pid: 623, stack limit = 0xffffff800b7d0000)
[   93.129355] CPU: 3 PID: 623 Comm: Binder:287_6 Not tainted 4.14.135-v8+ #7
[   93.136329] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT)
[   93.142687] task: fffffff2b2338000 task.stack: ffffff800b7d0000
[   93.148705] PC is at sysrq_handle_crash+0x28/0x38
[   93.153479] LR is at sysrq_handle_crash+0x14/0x38
[   93.158252] pc : [<ffffff890547ed18>] lr : [<ffffff890547ed04>] pstate: 200001c5
[   93.165754] sp : ffffff800801bb10
[   93.169115] x29: ffffff800801bb10 x28: 0000000000000001
[   93.174513] x27: ffffff8905fbd018 x26: 0000000000000001
[   93.179908] x25: 0000000000000000 x24: 0000000000000001
[   93.185306] x23: 0000000000000007 x22: ffffff890627a000
[   93.190703] x21: ffffff890627a958 x20: 0000000000000063
[   93.196100] x19: ffffff8906239000 x18: 0000000000000010
[   93.201499] x17: 0000000000000000 x16: 000000000000000e
[   93.206896] x15: ffffffffffffffff x14: ffffff89863082c7
[   93.212294] x13: ffffff89063082d5 x12: ffffff8906239000
[   93.217691] x11: 0000000005f5e0ff x10: ffffff800801b830
[   93.223088] x9 : 00000000ffffffd0 x8 : 6172632061207265
[   93.228484] x7 : 6767697254203a20 x6 : 00000000000001c8
[   93.233882] x5 : 0000000000000064 x4 : 0000000000000000
[   93.239281] x3 : 0000000000000000 x2 : 00000069b13fd000
[   93.244680] x1 : 0000000000000000 x0 : 0000000000000001
[   93.250095]
[   93.251605] Call trace:
[   93.254088] Exception stack(0xffffff800801b9d0 to 0xffffff800801bb10)
[   93.260625] b9c0:                                   0000000000000001 0000000000000000
[   93.268572] b9e0: 00000069b13fd000 0000000000000000 0000000000000000 0000000000000064
[   93.276521] ba00: 00000000000001c8 6767697254203a20 6172632061207265 00000000ffffffd0
[   93.284469] ba20: ffffff800801b830 0000000005f5e0ff ffffff8906239000 ffffff89063082d5
[   93.292416] ba40: ffffff89863082c7 ffffffffffffffff 000000000000000e 0000000000000000
[   93.300364] ba60: 0000000000000010 ffffff8906239000 0000000000000063 ffffff890627a958
[   93.308312] ba80: ffffff890627a000 0000000000000007 0000000000000001 0000000000000000
[   93.316259] baa0: 0000000000000001 ffffff8905fbd018 0000000000000001 ffffff800801bb10
[   93.324207] bac0: ffffff890547ed04 ffffff800801bb10 ffffff890547ed18 00000000200001c5
[   93.332155] bae0: 00000000000001c0 ffffff8905e24c78 0000007fffffffff ffffff890547ed04
[   93.340100] bb00: ffffff800801bb10 ffffff890547ed18
[   93.345056] [<ffffff890547ed18>] sysrq_handle_crash+0x28/0x38
[   93.350891] [<ffffff890547f3e8>] __handle_sysrq+0xb0/0x1a8
[   93.356459] [<ffffff890547f880>] sysrq_filter+0x358/0x400
[   93.361944] [<ffffff8905629acc>] input_to_handler+0x64/0x140
[   93.367688] [<ffffff890562bb00>] input_pass_values.part.2+0x148/0x168
[   93.374226] [<ffffff890562bed0>] input_handle_event+0x130/0x5a8
[   93.380234] [<ffffff890562c3bc>] input_event+0x74/0x98
[   93.385452] [<ffffff89056ac41c>] hidinput_report_event+0x4c/0x68
[   93.391549] [<ffffff89056a686c>] hid_report_raw_event+0x25c/0x428
[   93.397732] [<ffffff89056a6b60>] hid_input_report+0x128/0x1a8
[   93.403567] [<ffffff89056b2384>] hid_irq_in+0x20c/0x248
[   93.408875] [<ffffff89055bcee8>] __usb_hcd_giveback_urb+0x68/0x108
[   93.415149] [<ffffff89055bd160>] usb_hcd_giveback_urb+0xf8/0x130
[   93.421248] [<ffffff89055eb560>] completion_tasklet_func+0x68/0xb0
[   93.427524] [<ffffff89055facc8>] tasklet_callback+0x20/0x30
[   93.433183] [<ffffff89050a9544>] tasklet_hi_action+0xb4/0x168
[   93.439016] [<ffffff8905081c38>] __do_softirq+0x158/0x3d0
[   93.444496] [<ffffff89050a8f70>] irq_exit+0xe0/0x100
[   93.449539] [<ffffff8905107808>] __handle_domain_irq+0x90/0xf8
[   93.455460] [<ffffff8905081848>] bcm2836_arm_irqchip_handle_irq+0x68/0xd0
[   93.462347] Exception stack(0xffffff800b7d3ec0 to 0xffffff800b7d4000)
[   93.468884] 3ec0: 0000000000000005 0000000000000000 0000000000000032 0000000000000005
[   93.476832] 3ee0: 0000000072a66a7c 00000000d29f9170 00000000d7b17e00 0000000000000111
[   93.484778] 3f00: 00000000ee376c80 0000000000000003 00000000fffffffe 00000000d29f9194
[   93.492725] 3f20: 0000000000000011 00000000d29f9068 0000000000000000 0000000000000000
[   93.500673] 3f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[   93.508620] 3f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[   93.516566] 3f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[   93.524513] 3fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[   93.532460] 3fc0: 00000000ee377500 00000000800b0010 00000000eb197000 00000000ffffffff
[   93.540406] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[   93.548354] [<ffffff8905083854>] el0_irq_naked+0x50/0x5c
[   93.553750] Code: 52800020 b909c020 d5033e9f d2800001 (39000020)
[   93.559936] ---[ end trace 9d6aae76a13f195f ]---
[   93.564622] Kernel panic - not syncing: Fatal exception in interrupt
[   93.571071] SMP: stopping secondary CPUs
[   93.575061] Kernel Offset: 0x8fd000000 from 0xffffff8008000000
[   93.580981] CPU features: 0x0802004
[   93.584517] Memory Limit: none
[   93.587621] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

Terminate all processes, except for init

echo e > /proc/sysrq-trigger or press Alt+SysRq+e combination.

[ 1086.439084] sysrq: SysRq : Terminate All Tasks

[ 1086.456747] init: Restarting subcontext 'u:r:vendor_init:s0'
[ 1086.482033] init: Forked subcontext for 'u:r:vendor_init:s0' with pid 3670
[ 1086.498876] init: Subcontext received signal 15
143|console:/ $ [ 1086.511216] init: Restarting subcontext 'u:r:vendor_init:s0'
[ 1086.532068] init: Forked subcontext for 'u:r:vendor_init:s0' with pid 3674
[ 1086.532794] logd.auditd: 2 output lines suppressed due to ratelimiting
[ 1086.546956] init: Subcontext received signal 15
[ 1086.552248] init: Service 'ueventd' (pid 3117) received signal 15
[ 1086.563899] init: Sending signal 9 to service 'ueventd' (pid 3117) process group...
[ 1086.573098] libprocessgroup: Successfully killed process cgroup uid 0 pid 3117 in 0ms
[ 1086.583683] init: Service 'adbd' (pid 3119) received signal 15

For android system, init will start all the processes again, that is a soft restart.

Kill memory hog process

echo f > /proc/sysrq-trigger or press Alt+SysRq+f combination.

[  382.854088] sysrq: SysRq : Manual OOM execution
[  382.859100] kworker/2:3 invoked oom-killer: gfp_mask=0x14000c0(GFP_KERNEL), nodemask=(null),  order=-1, oom_score_adj=0
[  382.877692] kworker/2:3 cpuset=/ mems_allowed=0
[  382.882529] CPU: 2 PID: 1308 Comm: kworker/2:3 Not tainted 4.14.135-v8+ #7
[  382.889519] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT)
[  382.895907] Workqueue: events moom_callback
[  382.900165] Call trace:
[  382.902660] [<ffffff8a9a28b5d0>] dump_backtrace+0x0/0x240
[  382.908149] [<ffffff8a9a28b834>] show_stack+0x24/0x30
[  382.913286] [<ffffff8a9ab130f0>] dump_stack+0xac/0xe4
[  382.918425] [<ffffff8a9a3e98d4>] dump_header+0x94/0x1e8
[  382.923740] [<ffffff8a9a3e89b8>] oom_kill_process+0x2c8/0x5d0
[  382.929582] [<ffffff8a9a3e95fc>] out_of_memory+0x104/0x2d0
[  382.935157] [<ffffff8a9a67ecc4>] moom_callback+0x4c/0x78
[  382.940558] [<ffffff8a9a2bfcb8>] process_one_work+0x1d8/0x490
[  382.946395] [<ffffff8a9a2bffbc>] worker_thread+0x4c/0x478
[  382.951886] [<ffffff8a9a2c66c0>] kthread+0x138/0x140
[  382.956934] [<ffffff8a9a285214>] ret_from_fork+0x10/0x1c
[  382.962505] Mem-Info:
[  382.965081] active_anon:64905 inactive_anon:2651 isolated_anon:0
[  382.965081]  active_file:72541 inactive_file:24221 isolated_file:0
[  382.965081]  unevictable:576 dirty:0 writeback:0 unstable:0
[  382.965081]  slab_reclaimable:4792 slab_unreclaimable:9545
[  382.965081]  mapped:70324 shmem:2692 pagetables:4463 bounce:0
[  382.965081]  free:45110 free_pcp:180 free_cma:16228
[  382.999508] Node 0 active_anon:259620kB inactive_anon:10604kB active_file:290164kB inactive_file:96884kB unevictable:2304kB isolated(anon):0kB isolated(file):0kB mapped:281296kB dirty:0kB writeback:0kB shmem:10768kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[  383.023791] DMA free:180440kB min:3720kB low:9056kB high:9988kB active_anon:259620kB inactive_anon:10604kB active_file:290164kB inactive_file:96884kB unevictable:2304kB writepending:0kB present:970752kB managed:935516kB mlocked:2304kB kernel_stack:11680kB pagetables:17852kB bounce:0kB free_pcp:700kB local_pcp:156kB free_cma:64912kB
[  383.053673] lowmem_reserve[]: 0 0 0
[  383.057314] DMA: 4*4kB (UMC) 29*8kB (UMEC) 30*16kB (UEC) 14*32kB (UMEC) 3*64kB (MEC) 5*128kB (UMEC) 1*256kB (E) 2*512kB (UE) 1*1024kB (C) 4*2048kB (UMC) 41*4096kB (MC) = 180440kB
[  383.073723] 99929 total pagecache pages
[  383.077661] 0 pages in swap cache
[  383.081111] Swap cache stats: add 0, delete 0, find 0/0
[  383.086516] Free swap  = 0kB
[  383.089534] Total swap = 0kB
[  383.092489] 242688 pages RAM
[  383.095478] 0 pages HighMem/MovableOnly
[  383.099400] 8809 pages reserved
[  383.102641] 16384 pages cma reserved
[  383.106297] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
[  383.115026] [  163]  2000   163     1122      625       6       2        0         -1000 sh
[  383.123613] [ 1831]     0  1831      721      400       5       2        0         -1000 init
[  383.132345] [ 1832]     0  1832      721      402       3       2        0         -1000 init
[  383.141086] [ 1833]     0  1833      753      461       5       2        0         -1000 ueventd
[  383.150225] [ 1834]  2000  1834      977      259       5       2        0         -1000 adbd
[  383.159102] [ 1835]     0  1835   257090    31475     142       4        0         -1000 main
[  383.167855] [ 1836]  1000  1836     1671      837       8       2        0         -1000 allocator@1.0-s
[  383.177660] [ 1839]  1041  1839    10659     4677      36       2        0         -1000 audioserver
[  383.187103] [ 1841]  1047  1841     6410     2804      22       2        0         -1000 cameraserver
[  383.196602] [ 1842]  1019  1842     4520     2425      19       2        0         -1000 drmserver
[  383.205806] [ 1843]  1000  1843     2540     1277      11       2        0         -1000 gatekeeperd
[  383.215273] [ 1844]     0  1844     1752      863       8       2        0         -1000 healthd
[  383.224379] [ 1846]  1000  1846     2117     1190      10       2        0         -1000 hwservicemanage
[  383.234088] [ 1847]  1067  1847     2382     1025      10       2        0         -1000 Binder:1847_2
[  383.243593] [ 1848]     0  1848     3312     1073      10       2        0         -1000 Binder:1848_2
[  383.253124] [ 1849]  1017  1849     2959     1545      14       2        0         -1000 keystore
[  383.262167] [ 1850]     0  1850     1214      499       6       2        0         -1000 lmkd
[  383.270904] [ 1851]  1036  1851     2754      807      10       2        0         -1000 logd
[  383.279644] [ 1852]  1013  1852     2402     1264      11       2        0         -1000 mediadrmserver
[  383.289265] [ 1853]  1040  1853    11616     3222      25       2        0         -1000 mediaextractor
[  383.298907] [ 1854]  1013  1854     4467     2359      17       2        0         -1000 mediametrics
[  383.309286] [ 1855]  1013  1855    14326     3235      32       2        0         -1000 mediaserver
[  383.318671] [ 1856]     0  1856     6233     1454      19       2        0         -1000 Binder:1856_2
[  383.328221] [ 1858]  1000  1858     1473      789       8       2        0         -1000 servicemanager
[  383.337947] [ 1859]     0  1859     2690     1079      12       2        0         -1000 storaged
[  383.347111] [ 1860]     0  1860     1264      607       6       2        0         -1000 su
[  383.355757] [ 1861]  1000  1861    16970     6260      38       2        0         -1000 surfaceflinger
[  383.365537] [ 1862]  1000  1862     2079      986       9       2        0         -1000 thermalserviced
[  383.375348] [ 1863]  1058  1863     1163      611       6       2        0         -1000 tombstoned
[  383.384707] [ 1864]     0  1864     3261     1454      13       2        0         -1000 Binder:1864_2
[  383.394266] [ 1865]  1010  1865     2328     1135       8       2        0         -1000 wificond
[  383.403449] [ 1866]  1041  1866     3767     2070      14       2        0         -1000 audio@2.0-servi
[  383.413173] [ 1867]  1002  1867     1701      855       8       2        0         -1000 bluetooth@1.0-s
[  383.422979] [ 1868]  1013  1868     1896     1035       8       2        0         -1000 cas@1.0-service
[  383.432698] [ 1869]  1000  1869     2030      894       8       2        0         -1000 configstore@1.1
[  383.442413] [ 1870]  1000  1870     1506      873       8       2        0         -1000 allocator@2.0-s
[  383.452362] [ 1871]  1000  1871     3081     1380      13       2        0         -1000 composer@2.1-se
[  383.462148] [ 1873]  1000  1873     2078     1170       9       2        0         -1000 keymaster@3.0-s
[  383.471941] [ 1876]  1000  1876     1679      837       7       2        0         -1000 light@2.0-servi
[  383.481702] [ 1877]  1046  1877     8618     2782      26       2        0         -1000 omx@1.0-service
[  383.491494] [ 1878]  1000  1878     1719      826       8       2        0         -1000 memtrack@1.0-se
[  383.501200] [ 1879]  1000  1879     1471      831       8       2        0         -1000 power@1.0-servi
[  383.510907] [ 1880]  1000  1880     1954      836       7       2        0         -1000 usb@1.0-service
[  383.520640] [ 1881]  1010  1881     1943     1298       8       2        0         -1000 wifi@1.0-servic
[  383.530442] [ 1882]  1000  1882     1473      784       8       2        0         -1000 vndservicemanag
[  383.540263] [ 1883]  1020  1883      557      244       4       2        0         -1000 mdnsd
[  383.549170] [ 1978]  1000  1978   313573    45104     237       4        0          -800 system_server
[  383.558832] [ 2117]     0  2117     1215      620       6       2        0         -1000 iptables-restor
[  383.568648] [ 2118]     0  2118     1216      616       6       2        0         -1000 ip6tables-resto
[  383.578440] [ 2148] 10059  2148   229362    39246     162       4        0           200 putmethod.latin
[  383.588260] [ 2172] 10017  2172   254370    46955     221       4        0          -800 ndroid.systemui
[  383.598033] [ 2385]  1001  2385   218200    23547     141       4        0          -800 m.android.phone
[  383.607844] [ 2400]  1000  2400   233205    25379     159       4        0           900 ndroid.settings
[  383.617561] [ 2500] 10065  2500   212293    19881     123       4        0           904 droid.deskclock
[  383.627296] [ 2526]  1068  2526   210763    17511     119       4        0          -800 com.android.se
[  383.636953] [ 2540] 10013  2540   211811    17952     121       4        0          -800 neageos.audiofx
[  383.646673] [ 2563] 10008  2563   276852    18249     121       4        0           906 ineageos.eleven
[  383.656388] [ 2579] 10006  2579   230830    37483     181       4        0             0 droid.launcher3
[  383.666096] [ 2594] 10040  2594   210172    16849     118       4        0           100 android.smspush
[  383.675890] [ 2609] 10053  2609   210824    17882     122       4        0           902 id.printspooler
[  383.685615] [ 2619] 10047  2619   211209    18920     124       4        0           500 ageos.lockclock
[  383.695312] [ 2654]  1000  2654   212488    18750     124       4        0           900 os.lineageparts
[  383.705008] [ 2678]  1000  2678   210471    17370     122       4        0           904 ndroid.keychain
[  383.714796] [ 2704] 10003  2704   212014    21684     135       4        0           904 d.process.acore
[  383.724618] [ 2718] 10072  2718   211560    19136     125       4        0           904 ndroid.calendar
[  383.734348] [ 2738] 10007  2738   211808    21817     133       4        0           900 d.process.media
[  383.744087] [ 2756] 10012  2756   212064    18403     125       4        0           904 ndroid.contacts
[  383.753832] [ 2777] 10005  2777   210649    19405     125       4        0           700 viders.calendar
[  383.763519] [ 2805] 10046  2805   213350    20643     133       4        0           500 m.android.email
[  383.773270] [ 2826] 10036  2826   211474    19161     127       4        0           902 ndroid.exchange
[  383.782975] [ 2862]  2000  2862     1856      585       7       2        0         -1000 su
[  383.791539] [ 2864]     0  2864     1328      412       6       2        0         -1000 su
[  383.800080] [ 2867]     0  2867     1360      351       6       2        0         -1000 su
[  383.808735] [ 2868]     0  2868     1122      616       6       2        0         -1000 sh
[  383.817404] [ 2869] 10023  2869   210698    17281     124       4        0           902 gedprovisioning
[  383.827142] [ 2884] 10028  2884   210171    17066     125       4        0           902 timeinitializer
[  383.836916] [ 2902] 10018  2902   212252    17645     127       4        0           902 ackageinstaller
[  383.846636] [ 2923] 10068  2923   211798    19001     128       4        0           900 android.traceur
[  383.856476] [ 2945] 10025  2945   211406    19868     135       4        0           900 neageos.updater
[  383.866291] Out of memory: Kill process 2400 (ndroid.settings) score 1005 or sacrifice child
[  383.875171] Killed process 2400 (ndroid.settings) total-vm:932820kB, anon-rss:11424kB, file-rss:89980kB, shmem-rss:112kB

Kill all processes except for init

echo i > /proc/sysrq-trigger or press Alt+SysRq+i combination.

[ 1022.707078] sysrq: SysRq : Kill All Tasks
[ 1022.756534] init: Restarting subcontext 'u:r:vendor_init:s0'
[ 1022.819923] init: Forked subcontext for 'u:r:vendor_init:s0' with pid 3113
[ 1022.867550] init: Subcontext received signal 9
[ 1022.895539] init: Service 'console' (pid 163) received signal 9
[ 1022.943753] init: Sending signal 9 to service 'console' (pid 163) process group...
[ 1022.979143] logd.auditd: 2 output lines suppressed due to ratelimiting
[ 1022.994745] libprocessgroup: Successfully killed process cgroup uid 2000 pid 163 in 4ms
[ 1023.029230] init: Restarting subcontext 'u:r:vendor_init:s0'
[ 1023.041064] init: Forked subcontext for 'u:r:vendor_init:s0' with pid 3115
[ 1023.054913] init: Subcontext received signal 9
[ 1023.060002] init: Service 'ueventd' (pid 1833) received signal 9

Just thaw it

echo j > /proc/sysrq-trigger or press Alt+SysRq+j combination.

[ 1209.399074] sysrq: SysRq : Emergency Thaw of all frozen filesystems
[ 1209.405632] Emergency Thaw complete

Kill all programs on the current virtual console

echo k > /proc/sysrq-trigger or press Alt+SysRq+k combination.

# [ 1363.838082] sysrq: SysRq : SAK

This is not very useful for android system, since this is used to avoid trojan program running at console which could steal your password.

Show backtrace for all active CPUs

echo l > /proc/sysrq-trigger or press Alt+SysRq+l combination.

[ 1953.598081] sysrq: SysRq : Show backtrace of all active CPUs

Dump memory info

echo m > /proc/sysrq-trigger or press Alt+SysRq+m combination.

# echo m > /proc/sysrq-trigger
[ 1812.347663] sysrq: SysRq : Show Memory
[ 1812.351701] Mem-Info:
[ 1812.354300] active_anon:38083 inactive_anon:2134 isolated_anon:0
[ 1812.354300]  active_file:36846 inactive_file:59751 isolated_file:0
[ 1812.354300]  unevictable:563 dirty:0 writeback:0 unstable:0
[ 1812.354300]  slab_reclaimable:4697 slab_unreclaimable:9943
[ 1812.354300]  mapped:70310 shmem:2683 pagetables:4371 bounce:0
[ 1812.354300]  free:72211 free_pcp:453 free_cma:16228
[ 1812.388921] Node 0 active_anon:152332kB inactive_anon:8536kB active_file:147384kB inactive_file:239004kB unevictable:2252kB isolated(anon):0kB isolated(file):0kB mapped:281240kB dirty:0kB writeback:0kB shmem:10732kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 1812.413049] DMA free:288844kB min:3720kB low:9056kB high:9988kB active_anon:152332kB inactive_anon:8536kB active_file:147384kB inactive_file:239004kB unevictable:2252kB writepending:0kB present:970752kB managed:935516kB mlocked:2252kB kernel_stack:11408kB pagetables:17484kB bounce:0kB free_pcp:1812kB local_pcp:664kB free_cma:64912kB
[ 1812.443018] lowmem_reserve[]: 0 0 0
[ 1812.446612] DMA: 1861*4kB (UMEC) 1073*8kB (UMC) 717*16kB (UMC) 207*32kB (UMC) 88*64kB (MEC) 66*128kB (UMEC) 38*256kB (ME) 17*512kB (M) 5*1024kB (MEC) 6*2048kB (UMC) 50*4096kB (MC) = 288844kB
[ 1812.464088] 99743 total pagecache pages
[ 1812.468022] 0 pages in swap cache
[ 1812.471419] Swap cache stats: add 0, delete 0, find 0/0
[ 1812.476794] Free swap  = 0kB
[ 1812.479768] Total swap = 0kB
[ 1812.482720] 242688 pages RAM
[ 1812.485754] 0 pages HighMem/MovableOnly
[ 1812.489678] 8809 pages reserved
[ 1812.492924] 16384 pages cma reserved

Make RT tasks nice-able

echo n > /proc/sysrq-trigger or press Alt+SysRq+n combination.

# [ 2013.990079] sysrq: SysRq : Nice All RT Tasks

Shut down

echo o > /proc/sysrq-trigger or press Alt+SysRq+o combination.

[ 2602.612076] sysrq: SysRq : Power Off
[ 2602.620540] reboot: Power down

Dump current registers and flags

echo p > /proc/sysrq-trigger or press Alt+SysRq+p combination.

[   32.745593] sysrq: SysRq : Show Regs

Dump per CPU lists of all armed hrtimers

echo q > /proc/sysrq-trigger or press Alt+SysRq+q combination.

[  113.926636] sysrq: SysRq : Show clockevent devices & pending hrtimers (no others)
[  113.934263] Timer List Version: v0.8
[  113.937895] HRTIMER_MAX_CLOCK_BASES: 4
[  113.941700] now at 113930696987 nsecs
[  113.945414]
[  113.946925] cpu: 0
[  113.948964]  clock 0:
[  113.951270]   .base:       0000000000000000
[  113.955512]   .index:      0
[  113.958432]   .resolution: 1 nsecs
[  113.961880]   .get_time:
[  113.961895] ktime_get
[  113.964727]
[  113.968533]   .offset:     0 nsecs
[  113.971980] active timers:
[  113.974723]  #0:
[  113.974732] <0000000000000000>
[  113.976682] ,
[  113.979784] tick_sched_timer
[  113.981470] , S:01
[  113.984389]
[  113.987934]  # expires at 113972000000-113972000000 nsecs [in 41303013 to 41303013 nsecs]
[  113.996231]  #1:
[  113.996244] root_task_group
[  113.998194] ,
[  114.001032] sched_rt_period_timer
[  114.002718] , S:01
[  114.006077]
[  114.009622]  # expires at 114107634687-114107634687 nsecs [in 176937700 to 176937700 nsecs]
[  114.018096]  #2:
[  114.018103] <0000000000000000>
[  114.020054] ,
[  114.023157] hrtimer_wakeup
[  114.024843] , S:01
[  114.027586]
[  114.031131]  # expires at 115947771623-115987771623 nsecs [in 2017074636 to 2057074636 nsecs]
[  114.039780]  #3:
[  114.039787] <0000000000000000>
[  114.041737] ,
[  114.044837] hrtimer_wakeup
[  114.046522] , S:01
[  114.049265]
[  114.052810]  # expires at 135645220177-135671786581 nsecs [in 21714523190 to 21741089594 nsecs]
[  114.061636]  #4:
[  114.061642] <0000000000000000>
[  114.063594] ,
[  114.066698] hrtimer_wakeup
[  114.068388] , S:01
[  114.071133]
[  114.074682]  # expires at 138251793969-138291793969 nsecs [in 24321096982 to 24361096982 nsecs]
[  114.083510]  #5:
[  114.083517] <0000000000000000>
[  114.085470] ,
[  114.088571] hrtimer_wakeup
[  114.090257] , S:01
[  114.092999]
[  114.096544]  # expires at 138251793969-138291793969 nsecs [in 24321096982 to 24361096982 nsecs]
[  114.105373]  #6:
[  114.105380] <0000000000000000>
[  114.107333] ,
[  114.110435] hrtimer_wakeup
[  114.112121] , S:01
[  114.114864]
[  114.118409]  # expires at 140699093239-140799093239 nsecs [in 26768396252 to 26868396252 nsecs]
[  114.127235]  #7:
[  114.127248] sched_clock_timer
[  114.129199] ,
[  114.132211] sched_clock_poll
[  114.133898] , S:01
[  114.136819]
[  114.140363]  # expires at 4398046511078-4398046511078 nsecs [in 4284115814091 to 4284115814091 nsecs]
[  114.149720]  #8:
[  114.149728] <0000000000000000>
[  114.151679] ,
[  114.154781] hrtimer_wakeup
[  114.156468] , S:01
[  114.159212]
[  114.162757]  # expires at 2147483647854776901-2147483647854826901 nsecs [in 2147483533924079914 to 2147483533924129914 nsecs]
[  114.174228]  clock 1:
[  114.176532]   .base:       0000000000000000
[  114.180774]   .index:      1
[  114.183695]   .resolution: 1 nsecs
[  114.187143]   .get_time:
[  114.187153] ktime_get_real
[  114.189985]
[  114.194232]   .offset:     1564823942164342573 nsecs
[  114.199267] active timers:
[  114.202012]  clock 2:
[  114.204316]   .base:       0000000000000000
[  114.208558]   .index:      2
[  114.211479]   .resolution: 1 nsecs
[  114.214927]   .get_time:
[  114.214937] ktime_get_boottime
[  114.217768]
[  114.222367]   .offset:     0 nsecs
[  114.225815] active timers:
[  114.228559]  clock 3:
[  114.230863]   .base:       0000000000000000
[  114.235105]   .index:      3
[  114.238024]   .resolution: 1 nsecs
[  114.241472]   .get_time:
[  114.241483] ktime_get_clocktai
[  114.244316]
[  114.248918]   .offset:     1564823942164342573 nsecs
[  114.253954] active timers:
[  114.256700]   .expires_next   : 114254000000 nsecs
[  114.261560]   .hres_active    : 1
[  114.264921]   .nr_events      : 89513
[  114.268636]   .nr_retries     : 26
[  114.272086]   .nr_hangs       : 0
[  114.275447]   .max_hang_time  : 0
[  114.278810]   .nohz_mode      : 2
[  114.282173]   .last_tick      : 112809000000 nsecs
[  114.287033]   .tick_stopped   : 0
[  114.290395]   .idle_jiffies   : 4294780104
[  114.294549]   .idle_calls     : 77417
[  114.298265]   .idle_sleeps    : 73701
[  114.301980]   .idle_entrytime : 113663783289 nsecs
[  114.306842]   .idle_waketime  : 112813039279 nsecs
[  114.311704]   .idle_exittime  : 112813118602 nsecs
[  114.316565]   .idle_sleeptime : 18227816057 nsecs
[  114.321339]   .iowait_sleeptime: 9271187442 nsecs
[  114.326114]   .last_jiffies   : 4294780108
[  114.330271]   .next_timer     : 112816000000
[  114.334604]   .idle_expires   : 112816000000 nsecs
[  114.339464] jiffies: 4294781631
[  114.342648]
[  114.344161] cpu: 1
[  114.346201]  clock 0:
[  114.348507]   .base:       0000000000000000
[  114.352757]   .index:      0
[  114.355679]   .resolution: 1 nsecs
[  114.359132]   .get_time:
[  114.359142] ktime_get
[  114.361980]
[  114.365793]   .offset:     0 nsecs
[  114.369246] active timers:
[  114.371993]  #0:
[  114.372000] <0000000000000000>
[  114.373952] ,
[  114.377054] tick_sched_timer
[  114.378744] , S:01
[  114.381665]
[  114.385212]  # expires at 114369000000-114369000000 nsecs [in 438303013 to 438303013 nsecs]
[  114.393685]  #1:
[  114.393692] <0000000000000000>
[  114.395644] ,
[  114.398748] hrtimer_wakeup
[  114.400435] , S:01
[  114.403180]
[  114.406728]  # expires at 115229592363-115269592363 nsecs [in 1298895376 to 1338895376 nsecs]
[  114.415382]  #2:
[  114.415390] <0000000000000000>
[  114.417347] ,
[  114.420451] hrtimer_wakeup
[  114.422141] , S:01
[  114.424887]
[  114.428436]  # expires at 115938559123-115942246121 nsecs [in 2007862136 to 2011549134 nsecs]
[  114.437093]  #3:
[  114.437103] <0000000000000000>
[  114.439058] ,
[  114.442160] hrtimer_wakeup
[  114.443848] , S:01
[  114.446591]
[  114.450136]  # expires at 116081636832-116085636831 nsecs [in 2150939845 to 2154939844 nsecs]
[  114.458785]  #4:
[  114.458792] <0000000000000000>
[  114.460743] ,
[  114.463843] hrtimer_wakeup
[  114.465529] , S:01
[  114.468271]
[  114.471816]  # expires at 119266076838-119266126838 nsecs [in 5335379851 to 5335429851 nsecs]
[  114.480466]  #5:
[  114.480473] <0000000000000000>
[  114.482423] ,
[  114.485523] hrtimer_wakeup
[  114.487210] , S:01
[  114.489952]
[  114.493498]  # expires at 131457626281-131457676281 nsecs [in 17526929294 to 17526979294 nsecs]
[  114.502326]  #6:
[  114.502332] <0000000000000000>
[  114.504282] ,
[  114.507381] hrtimer_wakeup
[  114.509068] , S:01
[  114.511811]
[  114.515355]  # expires at 136909656935-136932144933 nsecs [in 22978959948 to 23001447946 nsecs]
[  114.524182]  #7:
[  114.524189] <0000000000000000>
[  114.526138] ,
[  114.529238] hrtimer_wakeup
[  114.530924] , S:01
[  114.533666]
[  114.537211]  # expires at 140773233395-140813233395 nsecs [in 26842536408 to 26882536408 nsecs]
[  114.546037]  #8:
[  114.546044] <0000000000000000>
[  114.547994] ,
[  114.551095] hrtimer_wakeup
[  114.552779] , S:01
[  114.555522]
[  114.559067]  # expires at 142486858030-142486908030 nsecs [in 28556161043 to 28556211043 nsecs]
[  114.567893]  #9:
[  114.567900] <0000000000000000>
[  114.569850] ,
[  114.572951] hrtimer_wakeup
[  114.574637] , S:01
[  114.577379]
[  114.580924]  # expires at 400768445846-400768495846 nsecs [in 286837748859 to 286837798859 nsecs]
[  114.589927]  #10:
[  114.589933] <0000000000000000>
[  114.591971] ,
[  114.595071] hrtimer_wakeup
[  114.596756] , S:01
[  114.599499]
[  114.603045]  # expires at 1800000788293-1800000838293 nsecs [in 1686070091306 to 1686070141306 nsecs]
[  114.612401]  clock 1:
[  114.614706]   .base:       0000000000000000
[  114.618948]   .index:      1
[  114.621867]   .resolution: 1 nsecs
[  114.625315]   .get_time:
[  114.625324] ktime_get_real
[  114.628156]
[  114.632404]   .offset:     1564823942164342573 nsecs
[  114.637439] active timers:
[  114.640182]  clock 2:
[  114.642486]   .base:       0000000000000000
[  114.646727]   .index:      2
[  114.649647]   .resolution: 1 nsecs
[  114.653094]   .get_time:
[  114.653104] ktime_get_boottime
[  114.655935]
[  114.660534]   .offset:     0 nsecs
[  114.663983] active timers:
[  114.666726]  clock 3:
[  114.669029]   .base:       0000000000000000
[  114.673270]   .index:      3
[  114.676189]   .resolution: 1 nsecs
[  114.679637]   .get_time:
[  114.679646] ktime_get_clocktai
[  114.682478]
[  114.687077]   .offset:     1564823942164342573 nsecs
[  114.692112] active timers:
[  114.694856]   .expires_next   : 114692000000 nsecs
[  114.699714]   .hres_active    : 1
[  114.703075]   .nr_events      : 93925
[  114.706790]   .nr_retries     : 27
[  114.710239]   .nr_hangs       : 0
[  114.713601]   .max_hang_time  : 0
[  114.716961]   .nohz_mode      : 2
[  114.720321]   .last_tick      : 113223000000 nsecs
[  114.725181]   .tick_stopped   : 0
[  114.728542]   .idle_jiffies   : 4294780518
[  114.732697]   .idle_calls     : 64023
[  114.736410]   .idle_sleeps    : 60150
[  114.740125]   .idle_entrytime : 114736013914 nsecs
[  114.744985]   .idle_waketime  : 113223043185 nsecs
[  114.749844]   .idle_exittime  : 113223096102 nsecs
[  114.754703]   .idle_sleeptime : 15424201483 nsecs
[  114.759474]   .iowait_sleeptime: 9294936340 nsecs
[  114.764245]   .last_jiffies   : 4294782056
[  114.768399]   .next_timer     : 114144000000
[  114.772729]   .idle_expires   : 114144000000 nsecs
[  114.777588] jiffies: 4294782070
[  114.780770]
[  114.782280] cpu: 2
[  114.784318]  clock 0:
[  114.786621]   .base:       0000000000000000
[  114.790862]   .index:      0
[  114.793781]   .resolution: 1 nsecs
[  114.797229]   .get_time:
[  114.797236] ktime_get
[  114.800067]
[  114.803873]   .offset:     0 nsecs
[  114.807320] active timers:
[  114.810063]  #0:
[  114.810070] <0000000000000000>
[  114.812021] ,
[  114.815118] tick_sched_timer
[  114.816804] , S:01
[  114.819724]
[  114.823268]  # expires at 113924000000-113924000000 nsecs [in -6696987 to -6696987 nsecs]
[  114.831564]  #1:
[  114.831570] <0000000000000000>
[  114.833521] ,
[  114.836621] hrtimer_wakeup
[  114.838307] , S:01
[  114.841050]
[  114.844594]  # expires at 114504266207-114508265206 nsecs [in 573569220 to 577568219 nsecs]
[  114.853067]  #2:
[  114.853073] <0000000000000000>
[  114.855023] ,
[  114.858122] hrtimer_wakeup
[  114.859808] , S:01
[  114.862551]
[  114.866096]  # expires at 117098217927-117106862925 nsecs [in 3167520940 to 3176165938 nsecs]
[  114.874746]  #3:
[  114.874752] <0000000000000000>
[  114.876701] ,
[  114.879800] hrtimer_wakeup
[  114.881485] , S:01
[  114.884228]
[  114.887774]  # expires at 117815373654-117819373652 nsecs [in 3884676667 to 3888676665 nsecs]
[  114.896423]  #4:
[  114.896429] <0000000000000000>
[  114.898379] ,
[  114.901478] hrtimer_wakeup
[  114.903164] , S:01
[  114.905907]
[  114.909451]  # expires at 120893170479-120905852477 nsecs [in 6962473492 to 6975155490 nsecs]
[  114.918101]  #5:
[  114.918107] <0000000000000000>
[  114.920056] ,
[  114.923156] hrtimer_wakeup
[  114.924841] , S:01
[  114.927583]
[  114.931127]  # expires at 673233351260-673333351260 nsecs [in 559302654273 to 559402654273 nsecs]
[  114.940129]  #6:
[  114.940135] <0000000000000000>
[  114.942085] ,
[  114.945185] hrtimer_wakeup
[  114.946871] , S:01
[  114.949613]
[  114.953157]  # expires at 3612220424943-3612320424943 nsecs [in 3498289727956 to 3498389727956 nsecs]
[  114.962512]  #7:
[  114.962518] <0000000000000000>
[  114.964467] ,
[  114.967566] hrtimer_wakeup
[  114.969252] , S:01
[  114.971994]
[  114.975540]  # expires at 2147483647980256131-2147483647980306131 nsecs [in 2147483534049559144 to 2147483534049609144 nsecs]
[  114.987009]  clock 1:
[  114.989311]   .base:       0000000000000000
[  114.993553]   .index:      1
[  114.996472]   .resolution: 1 nsecs
[  114.999919]   .get_time:
[  114.999928] ktime_get_real
[  115.002760]
[  115.007008]   .offset:     1564823942164342573 nsecs
[  115.012041] active timers:
[  115.014784]  clock 2:
[  115.017088]   .base:       0000000000000000
[  115.021329]   .index:      2
[  115.024249]   .resolution: 1 nsecs
[  115.027697]   .get_time:
[  115.027706] ktime_get_boottime
[  115.030536]
[  115.035137]   .offset:     0 nsecs
[  115.038584] active timers:
[  115.041328]  clock 3:
[  115.043631]   .base:       0000000000000000
[  115.047872]   .index:      3
[  115.050791]   .resolution: 1 nsecs
[  115.054238]   .get_time:
[  115.054248] ktime_get_clocktai
[  115.057079]
[  115.061679]   .offset:     1564823942164342573 nsecs
[  115.066712] active timers:
[  115.069456]   .expires_next   : 113924000000 nsecs
[  115.074316]   .hres_active    : 1
[  115.077676]   .nr_events      : 91385
[  115.081389]   .nr_retries     : 50
[  115.084839]   .nr_hangs       : 0
[  115.088199]   .max_hang_time  : 0
[  115.091559]   .nohz_mode      : 2
[  115.094920]   .last_tick      : 113664000000 nsecs
[  115.099780]   .tick_stopped   : 0
[  115.103141]   .idle_jiffies   : 4294780959
[  115.107294]   .idle_calls     : 69027
[  115.111008]   .idle_sleeps    : 64438
[  115.114721]   .idle_entrytime : 113666333862 nsecs
[  115.119581]   .idle_waketime  : 113666018081 nsecs
[  115.124440]   .idle_exittime  : 113666333862 nsecs
[  115.129298]   .idle_sleeptime : 21080342589 nsecs
[  115.134069]   .iowait_sleeptime: 6621761156 nsecs
[  115.138841]   .last_jiffies   : 4294780962
[  115.142995]   .next_timer     : 114000000000
[  115.147325]   .idle_expires   : 114000000000 nsecs
[  115.152184] jiffies: 4294782444
[  115.155368]
[  115.156878] cpu: 3
[  115.158915]  clock 0:
[  115.161218]   .base:       0000000000000000
[  115.165460]   .index:      0
[  115.168380]   .resolution: 1 nsecs
[  115.171827]   .get_time:
[  115.171834] ktime_get
[  115.174667]
[  115.178472]   .offset:     0 nsecs
[  115.181919] active timers:
[  115.184663]  #0:
[  115.184669] <0000000000000000>
[  115.186618] ,
[  115.189717] tick_sched_timer
[  115.191404] , S:01
[  115.194323]
[  115.197868]  # expires at 115182000000-115182000000 nsecs [in 1251303013 to 1251303013 nsecs]
[  115.206517]  #1:
[  115.206523] <0000000000000000>
[  115.208473] ,
[  115.211571] hrtimer_wakeup
[  115.213257] , S:01
[  115.215999]
[  115.219543]  # expires at 115230312924-115230512922 nsecs [in 1299615937 to 1299815935 nsecs]
[  115.228193]  #2:
[  115.228198] <0000000000000000>
[  115.230148] ,
[  115.233247] hrtimer_wakeup
[  115.234933] , S:01
[  115.237676]
[  115.241219]  # expires at 122627199715-122727199715 nsecs [in 8696502728 to 8796502728 nsecs]
[  115.249868]  #3:
[  115.249875] <0000000000000000>
[  115.251824] ,
[  115.254923] hrtimer_wakeup
[  115.256609] , S:01
[  115.259353]
[  115.262897]  # expires at 124368101258-124368151258 nsecs [in 10437404271 to 10437454271 nsecs]
[  115.271723]  #4:
[  115.271729] <0000000000000000>
[  115.273678] ,
[  115.276778] hrtimer_wakeup
[  115.278463] , S:01
[  115.281206]
[  115.284751]  # expires at 132154639555-132154689555 nsecs [in 18223942568 to 18223992568 nsecs]
[  115.293576]  #5:
[  115.293582] <0000000000000000>
[  115.295532] ,
[  115.298631] hrtimer_wakeup
[  115.300317] , S:01
[  115.303060]
[  115.306604]  # expires at 135861244438-135861294438 nsecs [in 21930547451 to 21930597451 nsecs]
[  115.315429]  #6:
[  115.315435] <0000000000000000>
[  115.317385] ,
[  115.320484] hrtimer_wakeup
[  115.322170] , S:01
[  115.324914]
[  115.328458]  # expires at 135861244438-135861294438 nsecs [in 21930547451 to 21930597451 nsecs]
[  115.337284]  #7:
[  115.337289] <0000000000000000>
[  115.339239] ,
[  115.342339] hrtimer_wakeup
[  115.344024] , S:01
[  115.346767]
[  115.350312]  # expires at 164104975946-164157330944 nsecs [in 50174278959 to 50226633957 nsecs]
[  115.359138]  #8:
[  115.359144] <0000000000000000>
[  115.361093] ,
[  115.364192] hrtimer_wakeup
[  115.365876] , S:01
[  115.368619]
[  115.372164]  # expires at 10012211306089-10012311306089 nsecs [in 9898280609102 to 9898380609102 nsecs]
[  115.381694]  clock 1:
[  115.383997]   .base:       0000000000000000
[  115.388238]   .index:      1
[  115.391157]   .resolution: 1 nsecs
[  115.394604]   .get_time:
[  115.394613] ktime_get_real
[  115.397444]
[  115.401690]   .offset:     1564823942164342573 nsecs
[  115.406724] active timers:
[  115.409468]  clock 2:
[  115.411771]   .base:       0000000000000000
[  115.416012]   .index:      2
[  115.418931]   .resolution: 1 nsecs
[  115.422379]   .get_time:
[  115.422388] ktime_get_boottime
[  115.425220]
[  115.429817]   .offset:     0 nsecs
[  115.433264] active timers:
[  115.436007]  #0:
[  115.436013] <0000000000000000>
[  115.437963] ,
[  115.441063] timerfd_tmrproc
[  115.442748] , S:01
[  115.445578]
[  115.449122]  # expires at 117836000000-117836000000 nsecs [in 3905303013 to 3905303013 nsecs]
[  115.457772]  #1:
[  115.457778] <0000000000000000>
[  115.459727] ,
[  115.462825] alarmtimer_fired
[  115.464511] , S:01
[  115.467430]
[  115.470974]  # expires at 123179000000-123179000000 nsecs [in 9248303013 to 9248303013 nsecs]
[  115.479622]  clock 3:
[  115.481926]   .base:       0000000000000000
[  115.486168]   .index:      3
[  115.489088]   .resolution: 1 nsecs
[  115.492536]   .get_time:
[  115.492545] ktime_get_clocktai
[  115.495375]
[  115.499974]   .offset:     1564823942164342573 nsecs
[  115.505009] active timers:
[  115.507753]   .expires_next   : 115505000000 nsecs
[  115.512612]   .hres_active    : 1
[  115.515972]   .nr_events      : 95098
[  115.519686]   .nr_retries     : 22
[  115.523135]   .nr_hangs       : 0
[  115.526496]   .max_hang_time  : 0
[  115.529857]   .nohz_mode      : 2
[  115.533217]   .last_tick      : 114071000000 nsecs
[  115.538076]   .tick_stopped   : 0
[  115.541437]   .idle_jiffies   : 4294781366
[  115.545591]   .idle_calls     : 57460
[  115.549305]   .idle_sleeps    : 51148
[  115.553018]   .idle_entrytime : 115549011883 nsecs
[  115.557877]   .idle_waketime  : 114073028706 nsecs
[  115.562736]   .idle_exittime  : 114073486362 nsecs
[  115.567595]   .idle_sleeptime : 13404003264 nsecs
[  115.572365]   .iowait_sleeptime: 9129755111 nsecs
[  115.577137]   .last_jiffies   : 4294782869
[  115.581291]   .next_timer     : 193504000000
[  115.585621]   .idle_expires   : 193504000000 nsecs
[  115.590480] jiffies: 4294782882
[  115.593663]
[  115.595174] Tick Device: mode:     1
[  115.598799] Broadcast device
[  115.601719] Clock Event Device:
[  115.601725] bc_hrtimer
[  115.607384]  max_delta_ns:   9223372036854775807
[  115.612065]  min_delta_ns:   1
[  115.615161]  mult:           1
[  115.618257]  shift:          0
[  115.621353]  mode:           1
[  115.624449]  next_event:     9223372036854775807 nsecs
[  115.629660]  set_next_event:
[  115.629667] <0000000000000000>
[  115.632674]
[  115.637271]  shutdown:
[  115.637279] bc_shutdown
[  115.639758]
[  115.643740]  event_handler:
[  115.643748] tick_handle_oneshot_broadcast
[  115.646756]
[  115.652326]  retries:        0
[  115.655421]
[  115.656933] tick_broadcast_mask: 0
[  115.660383] tick_broadcast_oneshot_mask: 0
[  115.664535]
[  115.666046] Tick Device: mode:     1
[  115.669671] Per CPU device: 0
[  115.672678] Clock Event Device:
[  115.672684] arch_sys_timer
[  115.678695]  max_delta_ns:   111848106728
[  115.682761]  min_delta_ns:   1000
[  115.686121]  mult:           82463372
[  115.689833]  shift:          32
[  115.693018]  mode:           3
[  115.696115]  next_event:     115693000000 nsecs
[  115.700708]  set_next_event:
[  115.700720] arch_timer_set_next_event_phys
[  115.703727]
[  115.709384]  shutdown:
[  115.709393] arch_timer_shutdown_phys
[  115.711872]
[  115.717000]  oneshot stopped:
[  115.717009] arch_timer_shutdown_phys
[  115.720106]
[  115.725232]  event_handler:
[  115.725239] hrtimer_interrupt
[  115.728246]
[  115.732758]  retries:        96
[  115.735942]
[  115.737451] Tick Device: mode:     1
[  115.741076] Per CPU device: 1
[  115.744082] Clock Event Device:
[  115.744088] arch_sys_timer
[  115.750097]  max_delta_ns:   111848106728
[  115.754163]  min_delta_ns:   1000
[  115.757524]  mult:           82463372
[  115.761237]  shift:          32
[  115.764421]  mode:           3
[  115.767517]  next_event:     115764000000 nsecs
[  115.772110]  set_next_event:
[  115.772120] arch_timer_set_next_event_phys
[  115.775128]
[  115.780786]  shutdown:
[  115.780795] arch_timer_shutdown_phys
[  115.783274]
[  115.788401]  oneshot stopped:
[  115.788410] arch_timer_shutdown_phys
[  115.791505]
[  115.796632]  event_handler:
[  115.796639] hrtimer_interrupt
[  115.799645]
[  115.804156]  retries:        45
[  115.807338]
[  115.808848] Tick Device: mode:     1
[  115.812472] Per CPU device: 2
[  115.815479] Clock Event Device:
[  115.815485] arch_sys_timer
[  115.821495]  max_delta_ns:   111848106728
[  115.825560]  min_delta_ns:   1000
[  115.828921]  mult:           82463372
[  115.832633]  shift:          32
[  115.835818]  mode:           3
[  115.838914]  next_event:     113924000000 nsecs
[  115.843507]  set_next_event:
[  115.843517] arch_timer_set_next_event_phys
[  115.846524]
[  115.852178]  shutdown:
[  115.852187] arch_timer_shutdown_phys
[  115.854666]
[  115.859792]  oneshot stopped:
[  115.859801] arch_timer_shutdown_phys
[  115.862897]
[  115.868026]  event_handler:
[  115.868032] hrtimer_interrupt
[  115.871040]
[  115.875550]  retries:        4427
[  115.878910]
[  115.880420] Tick Device: mode:     1
[  115.884044] Per CPU device: 3
[  115.887051] Clock Event Device:
[  115.887057] arch_sys_timer
[  115.893067]  max_delta_ns:   111848106728
[  115.897132]  min_delta_ns:   1000
[  115.900493]  mult:           82463372
[  115.904206]  shift:          32
[  115.907390]  mode:           3
[  115.910486]  next_event:     115907000000 nsecs
[  115.915080]  set_next_event:
[  115.915089] arch_timer_set_next_event_phys
[  115.918096]
[  115.923753]  shutdown:
[  115.923763] arch_timer_shutdown_phys
[  115.926242]
[  115.931370]  oneshot stopped:
[  115.931379] arch_timer_shutdown_phys
[  115.934474]
[  115.939600]  event_handler:
[  115.939607] hrtimer_interrupt
[  115.942614]
[  115.947124]  retries:        25
[  115.950308]

Turn off keyboard raw mode and sets it to XLATE

echo r > /proc/sysrq-trigger or press Alt+SysRq+r combination.

[  239.059598] sysrq: SysRq : Keyboard mode set to system default

Sync all mounted filesystems

echo s > /proc/sysrq-trigger or press Alt+SysRq+s combination.

[  291.483595] sysrq: SysRq : Emergency Sync
[  291.487974] Emergency Sync complete

Dump a list of current tasks with details

echo t > /proc/sysrq-trigger or press Alt+SysRq+t combination. This is a huge list, below is only part of it:

[  971.473344] sysrq: SysRq : Show State
[  971.477084]   task                        PC stack   pid father
[  971.483093] init            S    0     1      0 0x00400000
[  971.488667] Call trace:
[  971.491164] [<ffffff90ef486678>] __switch_to+0xa8/0xf0
[  971.496385] [<ffffff90efd2602c>] __schedule+0x21c/0x888
[  971.501691] [<ffffff90efd266cc>] schedule+0x34/0x98
[  971.506647] [<ffffff90efd2a648>] schedule_hrtimeout_range_clock+0xe8/0xf0
[  971.513539] [<ffffff90efd2a688>] schedule_hrtimeout_range+0x38/0x48
[  971.519903] [<ffffff90ef6b53e4>] SyS_epoll_wait+0x2ec/0x450
[  971.525559] [<ffffff90ef6b57ac>] compat_SyS_epoll_pwait+0xf4/0x144
[  971.531832] Exception stack(0xffffff8008023ec0 to 0xffffff8008024000)
[  971.538370] 3ec0: 0000000000000005 00000000ffd37be8 0000000000000001 00000000ffffffff
[  971.546318] 3ee0: 0000000000000000 0000000000000008 0000000000000000 000000000000015a
[  971.554267] 3f00: 00000000ffd37b00 000000000017f008 00000000ffd37b00 00000000ffffffff
[  971.562214] 3f20: 00000000ffd37aa0 00000000ffd37a90 000000000010a875 0000000000000000
[  971.570162] 3f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  971.578110] 3f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  971.586058] 3f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  971.594006] 3fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  971.601955] 3fc0: 000000000010ff28 0000000020070010 0000000000000005 000000000000015a
[  971.609903] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  971.617857] [<ffffff90ef483ac0>] el0_svc_naked+0x34/0x38
[  971.623248] kthreadd        S    0     2      0 0x00000020
[  971.628820] Call trace:
[  971.631305] [<ffffff90ef486678>] __switch_to+0xa8/0xf0
[  971.636521] [<ffffff90efd2602c>] __schedule+0x21c/0x888
[  971.641826] [<ffffff90efd266cc>] schedule+0x34/0x98
[  971.646782] [<ffffff90ef4c77c4>] kthreadd+0x1cc/0x1e8
[  971.651910] [<ffffff90ef485214>] ret_from_fork+0x10/0x1c
[  971.657301] kworker/0:0H    I    0     4      2 0x00000020
[  971.662889] Call trace:
[  971.665376] [<ffffff90ef486678>] __switch_to+0xa8/0xf0
[  971.670594] [<ffffff90efd2602c>] __schedule+0x21c/0x888
[  971.675898] [<ffffff90efd266cc>] schedule+0x34/0x98
[  971.680851] [<ffffff90ef4c0044>] worker_thread+0xd4/0x478
[  971.686332] [<ffffff90ef4c66c0>] kthread+0x138/0x140
[  971.691372] [<ffffff90ef485214>] ret_from_fork+0x10/0x1c
[  971.696762] kworker/u8:0    I    0     5      2 0x00000020
[  971.702343] Call trace:
[  971.704831] [<ffffff90ef486678>] __switch_to+0xa8/0xf0
[  971.710049] [<ffffff90efd2602c>] __schedule+0x21c/0x888
[  971.715353] [<ffffff90efd266cc>] schedule+0x34/0x98
[  971.720303] [<ffffff90ef4c0044>] worker_thread+0xd4/0x478
[  971.725784] [<ffffff90ef4c66c0>] kthread+0x138/0x140
[  971.730824] [<ffffff90ef485214>] ret_from_fork+0x10/0x1c
[  971.736213] mm_percpu_wq    I    0     6      2 0x00000020
[  971.741791] Call trace:
[  971.744276] [<ffffff90ef486678>] __switch_to+0xa8/0xf0
[  971.749493] [<ffffff90efd2602c>] __schedule+0x21c/0x888
[  971.754799] [<ffffff90efd266cc>] schedule+0x34/0x98
[  971.759750] [<ffffff90ef4c06d0>] rescuer_thread+0x2e8/0x350
[  971.765407] [<ffffff90ef4c66c0>] kthread+0x138/0x140
[  971.770447] [<ffffff90ef485214>] ret_from_fork+0x10/0x1c
[  971.775837] ksoftirqd/0     S    0     7      2 0x00000020

Remount all filesystems read-only

echo u > /proc/sysrq-trigger or press Alt+SysRq+u combination.

[  159.951339] sysrq: SysRq : Emergency Remount R/O
[  159.956193] Emergency Remount complete

Dump uninterruptable tasks

echo w > /proc/sysrq-trigger or press Alt+SysRq+w combination.

[  513.753333] sysrq: SysRq : Show Blocked State
[  513.757778]   task                        PC stack   pid father

Dump ftrace buffer

echo z > /proc/sysrq-trigger or press Alt+SysRq+z combination.

[  396.745343] sysrq: SysRq : Dump ftrace buffer
[  396.749793] Dumping ftrace buffer:
[  396.753252]    (ftrace buffer empty)

Change log level

echo 0-9 > /proc/sysrq-trigger or press Alt+SysRq+numpad 0-9 combination.

[ 2500.020077] sysrq: SysRq : Changing Loglevel
[ 2500.024435] sysrq: Loglevel set to 0
[ 2507.980085] sysrq: SysRq : Changing Loglevel
[ 2507.984444] sysrq: Loglevel set to 1
[ 2509.188092] sysrq: SysRq : Changing Loglevel
[ 2509.192451] sysrq: Loglevel set to 2
[ 2509.804077] sysrq: SysRq : Changing Loglevel
[ 2509.808433] sysrq: Loglevel set to 3
[ 2510.428077] sysrq: SysRq : Changing Loglevel
[ 2510.432435] sysrq: Loglevel set to 4
[ 2510.852081] sysrq: SysRq : Changing Loglevel
[ 2510.856438] sysrq: Loglevel set to 5
[ 2511.156079] sysrq: SysRq : Changing Loglevel
[ 2511.160436] sysrq: Loglevel set to 6
[ 2511.580081] sysrq: SysRq : Changing Loglevel
[ 2511.584438] sysrq: Loglevel set to 7
[ 2511.844078] sysrq: SysRq : Changing Loglevel
[ 2511.848437] sysrq: Loglevel set to 8
[ 2512.092077] sysrq: SysRq : Changing Loglevel
[ 2512.096434] sysrq: Loglevel set to 9

NOTE: this changes kernel log level, if you don’t see log dumped by sysrq, then you’d better check if the log level is set too low.

References