A very good article on linux program execution flow

Linux x86 Program Start Up


The C++ compilation process

Copied from: http://faculty.cs.niu.edu/~mcmahon/CS241/Notes/compile.html

The C++ compilation process

C++ compilation process

Compiling a source code file in C++ is a four-step process. For example, if you have a C++ source code file named prog1.cpp and you execute the compile command

   g++ -Wall -ansi -o prog1 prog1.cpp

the compilation process looks like this:

  1. The C++ preprocessor copies the contents of the included header files into the source code file, generates macro code, and replaces symbolic constants defined using #define with their values.
  2. The expanded source code file produced by the C++ preprocessor is compiled into the assembly language for the platform.
  3. The assembler code generated by the compiler is assembled into the object code for the platform.
  4. The object code file generated by the assembler is linked together with the object code files for any library functions used to produce an executable file.

By using appropriate compiler options, we can stop this process at any stage.

  1. To stop the process after the preprocessor step, you can use the -E option:
       g++ -E prog1.cpp

    The expanded source code file will be printed on standard output (the screen by default); you can redirect the output to a file if you wish. Note that the expanded source code file is often incredibly large – a 20 line source code file can easily produce an expanded file of 20,000 lines or more, depending on which header files were included.

  2. To stop the process after the compile step, you can use the -S option:
       g++ -Wall -ansi -S prog1.cpp

    By default, the assembler code for a source file named filename.cpp will be placed in a file namedfilename.s.

  3. To stop the process after the assembly step, you can use the -c option:
       g++ -Wall -ansi -c prog1.cpp

    By default, the assembler code for a source file named filename.cpp will be placed in a file namedfilename.o.


Misc driver is the simplest character driver. I am giving  a very simple example of misc driver.

Example misc driver creates a /dev/miscdev node which can be read or write. When read it reads kernel buffer data and when written, it updates kernel data with new data.

More information regarding misc driver can be found at http://www.linuxjournal.com/article/2920

#include <linux/module.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/miscdevice.h>
#include <linux/uaccess.h>

static char kern_data[] = "This is kernel data\n";

static struct miscdevice misc_dev;

static int misc_open(struct inode *inode, struct file *filp)
	pr_info("%s() misc device open\n", __func__);
	return 0;

static int misc_release(struct inode *inode, struct file *filp)
	pr_info("%s() misc device release\n", __func__);
	return 0;

static ssize_t misc_read(struct file *filp, char __user *buf,
					size_t count, loff_t *f_pos)
	pr_info("%s() misc device read\n", __func__);

	return simple_read_from_buffer(buf, strlen(kern_data), f_pos,
						kern_data, strlen(kern_data));

static ssize_t misc_write(struct file *filp, const char __user *buf,
				size_t count, loff_t *f_pos)
	int nbytes;

	pr_info("%s() misc device write\n", __func__);

	nbytes = simple_write_to_buffer(kern_data, sizeof(kern_data),
							f_pos, buf, count);
	kern_data[count] = '';

	return count;

static const struct file_operations misc_fops = {
	.owner = THIS_MODULE,
	.read = misc_read,
	.write = misc_write,
	.open = misc_open,
	.release = misc_release,

static struct miscdevice misc_dev = {
	.name = "miscdev",
	.fops = &misc_fops,

static __init int misc_driver_init(void)
	int rc = 0;

	pr_info("%s()\n", __func__);

	rc = misc_register(&misc_dev);
	if (rc) {
		pr_err("%s() misc_register() fail %d\n", __func__, rc);
		return rc;

	pr_info("%s() misc dev minor number = %u\n", __func__, misc_dev.minor);

	return rc;

static __exit void misc_driver_exit(void)
	pr_info("%s()\n", __func__);



MODULE_AUTHOR("Rajan Vaja");
MODULE_DESCRIPTION("Simple Misc Character Driver");


$ cat /dev/miscdev
This is kernel data

$ echo "This is new data" > /dev/miscdev

$ cat /dev/miscdev
This is new data


Some of them are:

  • seq_lock
  • RCU (Read-Copy-Update)

Seqlocks work in situations where the resource to be protected is small, simple, and frequently accessed, and where write access is rare but must be fast. Essentially, they work by allowing readers free access to the resource but requiring those readers to check for collisions with writers and, when such a collision happens, retry their access. Seqlocks generally cannot be used to protect data structures involving pointers, because the reader may be following a pointer that is invalid while the writer is changing the data structure.

How it works:

  • Reader obtains sequence value before reading
  • Reader reads value
  • Readers compares obtained sequence value with current value
  • If there is mismatch, read access is retried
unsigned int seq;
do {
seq = read_seqbegin(&seq_lock);
/* Do what you need to do */
} while read_seqretry(&seq_lock, seq);

– Writers need to get seqlock before writing data. Writers can use below callbacks (as well as other variants based on requirements) to lock and unlock seqlock.

void write_seqlock(seqlock_t *lock);
void write_sequnlock(seqlock_t *lock);

If it is required to use seqlock from IRQ, IRQ-safe versions can be used.

unsigned int read_seqbegin_irqsave(seqlock_t *lock, unsigned long flags);
int read_seqretry_irqrestore(seqlock_t *lock, unsigned int seq, unsigned long flags);



Provides high performance when used at proper place:

  • There are many constrains on data structures which RCU can protect.
  • Optimized for situations where reads are common and writes are rare
  • The resources being protected should be accessed via pointer
  • All references to those resources must be held only by atomic code

When data structure need to be changed, writing thread makes a copy of current
data, changes data in copy and change the read pointer.

  • On the reader side,
    – rcu_read_lock();
    – data = dereference_data_using_rcu_callback(…);
    – do_something_with_data(data);
    – rcu_read_unlock();
  • On Write side,- allocate memory for new data
    – copy old data to new memory
    – update data in new memory
    – update rcu data pointer using rcu callback
    – register a RCU call for to free old data memory (Alternate is
    synchronize_rcu() but this will block thread until all readers are done)

How to make sure that all readers are done before freeling old memory:

  1. This can be done using synchronize_rcu() callback which will block thread
    until all readers are done.
  2. call_rcu() through which a callback can be registered will be invoked after
    the completion of all RCU read-side critical sections that might be
    referencing that data item.



My first kernel module


#include <linux/module.h>
#include <linux/init.h>

static __exit void hello_world_exit(void)
	printk(KERN_DEBUG "Bye Bye World!\n");

static __init int hello_world_init(void)
	printk(KERN_DEBUG "Hello World!\n");
	return 0;



ifneq (${KERNELRELEASE},)                                                        
	obj-m := hello_world.o
KERNELDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
	$(MAKE) -C $(KERNELDIR) M=$(PWD) modules
	$(MAKE) -C $(KERNELDIR) M=$(PWD) clean 

Command to load module:

sudo insmod hello_world.ko

Command to unload module:

sudo rmmod hello_world.ko

Token Parsing Macro

Token parsing macros are useful to avoid tying same thing multiple times.

For an example, if you have multiple function and some condition checking is required in all functions, this type of macro can be used.

Below code is an example:

#include <stdio.h>
#define TEST(var1, var2) \
void test_func_##var1##_##var2##_test(int i) \
{                                            \
    /* Common part can be written here */
    printf("%s(): var = %d\n", __func__, i);\

    /* Task function can be called here */

int main()
    return 0;