Linux kernel debug print custom identifier

Sometimes it is easier to have our own identifier in print messages to easily filter/grep messages from the dmesg. It is also useful when we want to have some fixed data (e.g. __func__ or __LINE__). This also helps in debugging.

We can format pr_messages by defining pr_fmt macro.

#define pr_fmt "<custom-fmt>" fmt, <custom-param-for-custom-fmt>

e.g. you can print kernel module name using below:

    #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

Example

I generally define it like  below format for my debug messages. It prints out “rmv” and function name before any pr_* messages.

#define pr_fmt " rmv: %s() %d " fmt, __func__, __LINE__

Below is print statement at line 25 in my_func()

pr_info("Hello World\n");

Its output would like below after timestamp:

rmv: my_func()  25 Hello World

 

 

Advertisements

Passing Commandline arguments to Kernel module

Just like a C program, we can also pass command line arguments to kernel module. However, mechanisms are different in both the case.

In C, commandline arguments are stored in argv[] array while in linux kernel module, we set value of parameter from commandline.

In C, we pass arguments below way:

# ./prog 1 2 3 4  Hello

Here char * argv[] is set as {“prog”, 1″ , “2”, “3”, “4”, “Five”}.

In kernel module, to pass arguments to module, variables that need to take values from command line arguments are declared as global and then module_param() is used to setup command line mechanism.

# insmod kernel_module.ko mychar='a' myint=2345 mystring="Hello"

This will set intial values of variables mychar = ‘a’, myint = 2345 and mystring = “Hello”.

module_param(value, type, perm) macro takes three paramters:

* @value: the variable to alter, and exposed parameter name.
* @type: the type of the parameter
* @perm: visibility in sysfs.

Example:

static mychar = 'A';
module_param(mychar, unsigned char, 0);
MODULE_PARM_DESC(mychar,"this is the unsigned char variable");

Linux kernel: Print Device tree nodes

To print node name, there are different specifiers in Linux kernel. You can pass device node pointer (struct device_node *) to print function to print device node in different formats.

Lets say I have node named “my-node” in device tree:

/{
...
    my-node {
        my-sub-node@0x12345678 {
            compatible = "test,compatible-string";
        };
    };
...
};


Printing node (lets say sub-node) using device node pointer:

%pOF => /my-node/my-sub-node@0x12345678
%pOFf => /my-node/my-sub-node@0x12345678
%pOFfp => /my-node/my-sub-node@0x12345678:52
(52 is phandle)
%pOFfcF => /my-node/my-sub-node@0x12345678:test,compatible-string:–P-
Last are flags as below:
D – dynamic
d – detached
P – Populated
B – Populated bus

reference: Documentation/printk-formats.txt

Tested on: linux-4.14

Linux Kernel: Print Callee and Caller

Print the callback function name

If you cannot trace which function gets called when you call from core driver using function pointer, you can print function name using pf format specifier.
Linux maintains symbol table so based on function address, it can identify function name based on address pointed by function pointer.
e.g. Core driver calls hardware specific enable/disable functions:
core->ops->enable(). If you are not sure, which function is executed as a result of this call, you can print them using %pf or %pF:
pr_info("callee function name is: %pf,  core->ops->enable)
%pf prints without offset (e.g. foo) while %pF prints with offset (e.g. foo+0x00/0x80)
Reference: https://www.kernel.org/doc/Documentation/printk-formats.txt.
Thanks Bhargav(br13patel) for sharing this!

Print caller name

There are few ways to trace caller in Linux kernel.
  • Print stack backtrace
    • dump_stack();
      • To enable the dump_stack() function in the kernel config the following options must be set.
        1. Kernel hacking -> Kernel debugging
        2. Kernel hacking -> Verbose kernel error messages
    • WARN_ON(1);
  • Print caller name
    • pr_info(“caller function name is: %pf,  __builtin_return_address(0))

Device Tree Debugging

DTB-DTS conversion (Linux kernel)

  • Convert DTS file to DTB
  • ./scripts/dtc/dtc -I dts -O dtb -o .dtb .dts
  • Convert DTB file to DTS

  • ./scripts/dtc/dtc -I dtb -O dts -o .dts .dtb

 U-boot FDT commands

  • print fdt address
  • print $fdt_addr_r (or $fdt_addr)
  • Set FDT node property
  • fdt addr $fdt_addr
    fdt set /testnode testprop testvalue
  • remove FDT node property
  • fdt addr $fdt_addr
    fdt rm /testnode testprop
  • Remove DT node
  • fdt rm /testnode
  • List FDT nodes
  • fdt list                         /* All nodes */
    fdt list /cpus                   /* All nodes under cups node */
  • Create node
  • fdt mknode / testnode

References:

Linux Kernel – Tools/Scripts

Here are few tools/scripts which are useful for static analysis of code:

  • Checkpatch
    # ./script/checkpath.pl <patch_file(s)>
    # ./script/checkpatch.pl --file <file_name(s)>
    
  • kmemleak
    Check kernel memory leak using kmemleak:

    # cppcheck <source-dir>

    Kmemleak usage

  • Coccicheck
    • make coccicheck MODE=<mode>

    Where,

    ‘patch’ proposes a fix, when possible.

    ‘report’ generates a list in the following format: file:line:column-column: message

    ‘context’ highlights lines of interest and their context in a diff-like style.Lines of interest are indicated with ‘-‘

    ‘org’ generates a report in the Org mode format of Emacs.

    • To apply Coccinelle to a specific directory, M= can be used.
      For example, to check drivers/net/wireless/ one may write:
    make coccicheck M=drivers/net/wireless/
    
    • To apply Coccinelle on a file basis, instead of a directory basis, the
      following command may be used:

      make C=1 CHECK="scripts/coccicheck"
    • To check only newly edited code, use the value 2 for the C flag, i.e.
      make C=2 CHECK="scripts/coccicheck"
  • kselftest
    • Run kselftest install tool in tools/testing/selftests
       # cd tools/testing/selftests
       # ./kselftest_install.sh [ install_location ]
    • Default install tool: in tools/testing/selftests
    • Specify install location:
    # ./kselftest_install.sh /tmp
    • Kselftest install creates run_kselftest.sh
    • Run tests:
# cd install_dir
# ./run_kselftest.sh

How to check Memory Leak in linux kernel

  • Compile kernel with
    CONFIG_DEBUG_KMEMLEAK=y
    #CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF=y
    CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=4096
    
  • Mount debugfs (if it is not mounted)
    mount -t debugfs nodev /sys/kernel/debug/
    
  • Check memory leaks
    # echo scan > /sys/kernel/debug/kmemleak
    # cat /sys/kernel/debug/kmemleak
    

Example Trace:

unreferenced object 0xffff88021481d600 (size 16):
comm “swapper/0”, pid 1, jiffies 4294667622 (age 9531.310s)
hex dump (first 16 bytes):
70 6e 70 20 30 30 3a 30 31 00 38 81 ff ff ff ff pnp 00:01.8…..
backtrace:
[<ffffffff811619cf>] create_object+0xff/0x2a0
[<ffffffff8189ba63>] kmemleak_alloc+0x23/0x40
[<ffffffff8115bf37>] kmem_cache_alloc_trace+0xb7/0x160
[<ffffffff81395fe9>] reserve_range+0x59/0xf0
[<ffffffff81396102>] system_pnp_probe+0x82/0x90
[<ffffffff81392447>] pnp_device_probe+0x67/0xd0
[<ffffffff814c2a7f>] driver_probe_device+0x13f/0x2a0
[<ffffffff814c2ea5>] __device_attach_driver+0x85/0xa0
[<ffffffff814c0fa7>] bus_for_each_drv+0x57/0x90
[<ffffffff814c28d0>] __device_attach+0xb0/0xf0
[<ffffffff814c2ece>] device_initial_probe+0xe/0x10
[<ffffffff814c1f85>] bus_probe_device+0x95/0xa0
[<ffffffff814c00ce>] device_add+0x36e/0x570
[<ffffffff814c0469>] device_register+0x19/0x20
[<ffffffff813913d6>] __pnp_add_device+0x96/0xe0
[<ffffffff8139145b>] pnp_add_device+0x3b/0x100