diff --git a/index.html b/index.html index 06c24cc..7d6bc83 100644 --- a/index.html +++ b/index.html @@ -17,7 +17,7 @@

The Linux Kernel Module Programming Guide

Peter Jay Salzman, Michael Burian, Ori Pomerantz, Bob Mottram, Jim Huang

-
August 16, 2021
+
August 17, 2021
@@ -280,7 +280,7 @@ module.
1/* 
 2 * hello-1.c - The simplest kernel module. 
 3 */ 
-4#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+4#include <linux/kernel.h> /* Needed for pr_info() */ 
 5#include <linux/module.h> /* Needed by all modules */ 
 6 
 7int init_module(void) 
@@ -425,7 +425,7 @@ technique:
 3 * This is preferred over using init_module() and cleanup_module(). 
 4 */ 
 5#include <linux/init.h>   /* Needed for the macros */ 
-6#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+6#include <linux/kernel.h> /* Needed for pr_info() */ 
 7#include <linux/module.h> /* Needed by all modules */ 
 8 
 9static int __init hello_2_init(void) 
@@ -462,7 +462,7 @@ is as simple as this:
 see, some things get hardwired into the kernel (obj-y) but where are all those obj-m
 gone? Those familiar with shell scripts will easily be able to spot them. For those not,
 the obj-$(CONFIG_FOO) entries you see everywhere expand into obj-y or obj-m,
-depending on whether the CONFIG_FOO variable has been set to y or m. While we are
+depending on whether the CONFIG_FOO variable has been set to y or m. While we are
 at it, those were exactly the kind of variables that you have set in the .config file in
 the top-level directory of Linux kernel source tree, the last time when you said
  make menuconfig
@@ -494,7 +494,7 @@ memory: 236k freed, this is precisely what the kernel is freeing.
 2 * hello-3.c - Illustrating the __init, __initdata and __exit macros. 
 3 */ 
 4#include <linux/init.h>   /* Needed for the macros */ 
-5#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+5#include <linux/kernel.h> /* Needed for pr_info() */ 
 6#include <linux/module.h> /* Needed by all modules */ 
 7 
 8static int hello3_data __initdata = 3; 
@@ -543,7 +543,7 @@ example.
 2 * hello-4.c - Demonstrates module documentation. 
 3 */ 
 4#include <linux/init.h>   /* Needed for the macros */ 
-5#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+5#include <linux/kernel.h> /* Needed for pr_info() */ 
 6#include <linux/module.h> /* Needed by all modules */ 
 7 
 8MODULE_LICENSE("GPL"); 
@@ -800,7 +800,7 @@ error as follows:
 insmod: error inserting 'poet_atkm.ko': -1 Invalid module format
 

-

Less cryptical information are logged to the systemd journal: +

Less cryptic information is logged to the systemd journal: @@ -833,8 +833,8 @@ name:           hello_4 vermagic:       5.4.0-70-generic SMP mod_unload modversions

-

To overcome this problem we could resort to the --force-vermagic -option, but this solution is potentially unsafe, and unquestionably inacceptable +

To overcome this problem we could resort to the --force-vermagic option, +but this solution is potentially unsafe, and unquestionably unacceptable in production modules. Consequently, we want to compile our module in an environment which was identical to the one in which our precompiled kernel was built. How to do this, is the subject of the remainder of this @@ -847,11 +847,11 @@ tree: cp /boot/config- Let’s focus again on the previous error message: a closer look at the version magic strings suggests that, even with two configuration files which are exactly the same, a -slight difference in the version magic could be possible, and it is sufficient to -prevent insertion of the module into the kernel. That slight difference, namely -the custom string which appears in the module’s version magic and not in -the kernel’s one, is due to a modification with respect to the original, in -the makefile that some distribution include. Then, examine your Makefile, +slight difference in the version magic could be possible, and it is sufficient to prevent +insertion of the module into the kernel. That slight difference, namely the +custom string which appears in the module’s version magic and not in the +kernel’s one, is due to a modification with respect to the original, in the +makefile that some distributions include. Then, examine your Makefile, and make sure that the specified version information matches exactly the one used for your current kernel. For example, you makefile could start as follows: @@ -974,7 +974,7 @@ data into strings and write the string data using the low-level system call 6    return 0; 7}

with gcc -Wall -o hello hello.c -. Run the exectable with strace ./hello +. Run the executable with strace ./hello . Are you impressed? Every line you see corresponds to a system call. strace is a handy program that gives you details about what system calls a program is making, including which call is made, what its arguments are and what it @@ -996,7 +996,7 @@ with (like cosh()

You can even write modules to replace the kernel’s system calls, which we will do shortly. Crackers often make use of this sort of thing for backdoors or trojans, but you can write your own modules to do more benign things, like have the kernel -write Tee hee, that tickles! everytime someone tries to delete a file on your +write Tee hee, that tickles! every time someone tries to delete a file on your system.

@@ -1036,8 +1036,8 @@ symbols. entire kernel, so this is definitely an issue. The best way to deal with this is to declare all your variables as static and to use a well-defined prefix for your symbols. By convention, all kernel prefixes are lowercase. If you do not want to declare everything -as static, another option is to declare a symbol table and register it with a kernel. We -will get to this later. +as static, another option is to declare a symbol table and register it with the kernel. +We will get to this later.

The file /proc/kallsyms holds all the symbols that the kernel knows about and which are therefore accessible to your modules since they share the kernel’s codespace. @@ -1583,7 +1583,7 @@ conditional compilation directives. The way to do this to compare the macro information to processes — the /proc file system. Originally designed to allow easy access to information about processes (hence the name), it is now used by every bit of the kernel which has something interesting to report, such as /proc/modules -which provides the list of modules and /proc/meminfo which stats memory usage +which provides the list of modules and /proc/meminfo which gathers memory usage statistics.

The method to use the proc file system is very similar to the one used with device drivers — a structure is created with all the information needed for the /proc file, @@ -1620,7 +1620,7 @@ function procfile_read . The return value is a struct proc_dir_entry , and it will be used to configure the file /proc/helloworld (for example, the owner of this file). A null return value means that the creation has failed. -

Each time, everytime the file /proc/helloworld is read, the function +

Every time the file /proc/helloworld is read, the function procfile_read is called. Two parameters of this function are very important: the buffer (the second parameter) and the offset (the fourth one). The content of the @@ -2200,7 +2200,7 @@ the same way as in the previous example.

If you want more information, you can read this web page:

You can also read the code of fs/seq_file.c in the linux kernel. @@ -2327,7 +2327,7 @@ device file to write things to the modem (either modem commands or data to be sent through the phone line) and read things from the modem (either responses for commands or the data received through the phone line). However, this leaves open the question of what to do when you need to talk to the -serial port itself, for example to send the rate at which data is sent and +serial port itself, for example to configure the rate at which data is sent and received. @@ -2360,7 +2360,7 @@ ioctl.c.

If you want to use ioctls in your own kernel modules, it is best to receive an official ioctl assignment, so if you accidentally get somebody else’s ioctls, or if they get yours, you’ll know something is wrong. For more information, consult the kernel -source tree at Documentation/driver-api/ioctl.rst. +source tree at Documentation/userspace-api/ioctl/ioctl-number.rst.

1/* 
@@ -2843,30 +2843,30 @@ source tree at 196 
 197MODULE_LICENSE("GPL"); 
 198MODULE_DESCRIPTION("This is test_ioctl module");
-

+

0.10 System Calls

-

So far, the only thing we’ve done was to use well defined kernel mechanisms to +

So far, the only thing we’ve done was to use well defined kernel mechanisms to register /proc files and device handlers. This is fine if you want to do something the kernel programmers thought you’d want, such as write a device driver. But what if you want to do something unusual, to change the behavior of the system in some way? Then, you are mostly on your own. -

If you are not being sensible and using a virtual machine then this is where kernel +

If you are not being sensible and using a virtual machine then this is where kernel programming can become hazardous. While writing the example below, I killed the open() system call. This meant I could not open any files, I could not run any programs, and I could not shutdown the system. I had to restart the virtual -machine. No important files got anihilated, but if I was doing this on some live +machine. No important files got annihilated, but if I was doing this on some live mission critical system then that could have been a possible outcome. To ensure you do not lose any files, even within a test environment, please run sync right before you do the insmod and the rmmod . -

Forget about /proc files, forget about device files. They are just minor details. +

Forget about /proc files, forget about device files. They are just minor details. Minutiae in the vast expanse of the universe. The real process to kernel communication mechanism, the one used by all processes, is system calls. When a process requests a service from the kernel (such as opening a file, forking to a new @@ -2875,11 +2875,11 @@ change the behaviour of the kernel in interesting ways, this is the place to do it. By the way, if you want to see which system calls a program uses, run strace <arguments> . -

In general, a process is not supposed to be able to access the kernel. It can not +

In general, a process is not supposed to be able to access the kernel. It can not access kernel memory and it can’t call kernel functions. The hardware of the CPU enforces this (that is the reason why it is called “protected mode” or “page protection”). -

System calls are an exception to this general rule. What happens is that the +

System calls are an exception to this general rule. What happens is that the process fills the registers with the appropriate values and then calls a special instruction which jumps to a previously defined location in the kernel (of course, that location is readable by user processes, it is not writable by them). Under Intel CPUs, @@ -2887,7 +2887,7 @@ this is done by means of interrupt 0x80. The hardware knows that once you jump t this location, you are no longer running in restricted user mode, but as the operating system kernel — and therefore you’re allowed to do whatever you want. -

The location in the kernel a process can jump to is called system_call. The +

The location in the kernel a process can jump to is called system_call. The procedure at that location checks the system call number, which tells the kernel what service the process requested. Then, it looks at the table of system calls ( sys_call_table @@ -2900,7 +2900,7 @@ different process, if the process time ran out). If you want to read this code, at the source file arch/$(architecture)/kernel/entry.S, after the line ENTRY(system_call) . -

So, if we want to change the way a certain system call works, what we need to do +

So, if we want to change the way a certain system call works, what we need to do is to write our own function to implement it (usually by adding a bit of our own code, and then calling the original function) and then change the pointer at sys_call_table @@ -2908,7 +2908,7 @@ code, and then calling the original function) and then change the pointer at don’t want to leave the system in an unstable state, it’s important for cleanup_module to restore the table to its original state. -

The source code here is an example of such a kernel module. We want to “spy” on a certain +

The source code here is an example of such a kernel module. We want to “spy” on a certain user, and to pr_info() a message whenever that user opens a file. Towards this end, we replace the system call to open a file with our own function, called @@ -2918,7 +2918,7 @@ spy on, it calls pr_info() to display the name of the file to be opened. Then, either way, it calls the original open() function with the same parameters, to actually open the file. -

The init_module +

The init_module function replaces the appropriate location in sys_call_table and keeps the original pointer in a variable. The @@ -2936,7 +2936,7 @@ with B_open , which will call what it thinks is the original system call, A_open , when it’s done. -

Now, if B is removed first, everything will be well — it will simply restore the system +

Now, if B is removed first, everything will be well — it will simply restore the system call to A_open , which calls the original. However, if A is removed and then B is removed, the system will crash. A’s removal will restore the system call to the original, @@ -2959,7 +2959,7 @@ problem. When A is removed, it sees that the system call was changed to will still try to call A_open which is no longer there, so that even without removing B the system would crash. -

Note that all the related problems make syscall stealing unfeasiable for +

Note that all the related problems make syscall stealing unfeasible for production use. In order to keep people from doing potential harmful things sys_call_table is no longer exported. This means, if you want to do something more than a mere @@ -2967,8 +2967,8 @@ dry run of this example, you will have to patch your current kernel in order to sys_call_table exported. In the example directory you will find a README and the patch. As you can imagine, such modifications are not to be taken lightly. Do not try this on -valueable systems (ie systems that you do not own - or cannot restore easily). You -will need to get the complete sourcecode of this guide as a tarball in order to get the +valuable systems (ie systems that you do not own - or cannot restore easily). You will +need to get the complete sourcecode of this guide as a tarball in order to get the patch and the README. Depending on your kernel version, you might even need to hand apply the patch.

@@ -3110,13 +3110,13 @@ hand apply the patch. 135module_exit(syscall_end); 136 137MODULE_LICENSE("GPL"); -

+

0.11 Blocking Processes and threads

-

+

0.11.1 Sleep

-

What do you do when somebody asks you for something you can not do right +

What do you do when somebody asks you for something you can not do right away? If you are a human being and you are bothered by a human being, the only thing you can say is: "Not right now, I’m busy. Go away!". But if you are a kernel module and you are bothered by a process, you have another @@ -3127,21 +3127,21 @@ processes are being put to sleep by the kernel and woken up all the time (that is the way multiple processes appear to run on the same time on a single CPU). -

This kernel module is an example of this. The file (called /proc/sleep) can only +

This kernel module is an example of this. The file (called /proc/sleep) can only be opened by a single process at a time. If the file is already open, the kernel module calls wait_event_interruptible . The easiest way to keep a file open is to open it with:

1tail -f
-

This function changes the status of the task (a task is the kernel data structure +

This function changes the status of the task (a task is the kernel data structure which holds information about a process and the system call it is in, if any) to TASK_INTERRUPTIBLE , which means that the task will not run until it is woken up somehow, and adds it to WaitQ, the queue of tasks waiting to access the file. Then, the function calls the scheduler to context switch to a different process, one which has some use for the CPU. -

When a process is done with the file, it closes it, and +

When a process is done with the file, it closes it, and module_close is called. That function wakes up all the processes in the queue (there’s no mechanism to only wake up one of them). It then returns and the process which just @@ -3151,20 +3151,20 @@ Eventually, one of the processes which was in the queue will be given control of the CPU by the scheduler. It starts at the point right after the call to module_interruptible_sleep_on . -

This means that the process is still in kernel mode - as far as the process +

This means that the process is still in kernel mode - as far as the process is concerned, it issued the open system call and the system call has not returned yet. The process does not know somebody else used the CPU for most of the time between the moment it issued the call and the moment it returned. -

It can then proceed to set a global variable to tell all the other processes that the +

It can then proceed to set a global variable to tell all the other processes that the file is still open and go on with its life. When the other processes get a piece of the CPU, they’ll see that global variable and go back to sleep. -

So we will use tail -f +

So we will use tail -f to keep the file open in the background, while trying to access it with another process (again in the background, so that we need not switch to a different vt). As soon as the first background process is killed with kill %1 , the second is woken up, is able to access the file and finally terminates. -

To make our life more interesting, module_close +

To make our life more interesting, module_close does not have a monopoly on waking up the processes which wait to access the file. A signal, such as Ctrl +c (SIGINT) can also wake up a process. This is because we used module_interruptible_sleep_on @@ -3174,11 +3174,11 @@ used module_interruptible_sleep_on instead, but that would have resulted in extremely angry users whose Ctrl+c’s are ignored. -

In that case, we want to return with +

In that case, we want to return with -EINTR immediately. This is important so users can, for example, kill the process before it receives the file. -

There is one more point to remember. Some times processes don’t want to sleep, they want +

There is one more point to remember. Some times processes don’t want to sleep, they want either to get what they want immediately, or to be told it cannot be done. Such processes use the O_NONBLOCK flag when opening the file. The kernel is supposed to respond by returning with the error @@ -3214,7 +3214,7 @@ $ cat_nonblock /proc/sleep Last input: $ -

+

1/* 
@@ -3498,14 +3498,14 @@ $
 57 
 58    return 0; 
 59}
-

+

0.11.2 Completions

-

Sometimes one thing should happen before another within a module having multiple threads. +

Sometimes one thing should happen before another within a module having multiple threads. Rather than using /bin/sleep commands, the kernel has another way to do this which allows timeouts or interrupts to also happen. -

In the following example two threads are started, but one needs to start before +

In the following example two threads are started, but one needs to start before another.

@@ -3588,31 +3588,31 @@ another. 74 75MODULE_DESCRIPTION("Completions example"); 76MODULE_LICENSE("GPL"); -

The machine +

The machine structure stores the completion states for the two threads. At the exit point of each thread the respective completion state is updated, and wait_for_completion is used by the flywheel thread to ensure that it does not begin prematurely. -

So even though flywheel_thread +

So even though flywheel_thread is started first you should notice if you load this module and run dmesg that turning the crank always happens first because the flywheel thread waits for it to complete. -

There are other variations upon the +

There are other variations upon the wait_for_completion function, which include timeouts or being interrupted, but this basic mechanism is enough for many common situations without adding a lot of complexity. -

+

0.12 Avoiding Collisions and Deadlocks

-

If processes running on different CPUs or in different threads try to access the same +

If processes running on different CPUs or in different threads try to access the same memory, then it is possible that strange things can happen or your system can lock up. To avoid this, various types of mutual exclusion kernel functions are available. These indicate if a section of code is "locked" or "unlocked" so that simultaneous attempts to run it can not happen.

0.12.1 Mutex

-

You can use kernel mutexes (mutual exclusions) in much the same manner that you +

You can use kernel mutexes (mutual exclusions) in much the same manner that you might deploy them in userland. This may be all that is needed to avoid collisions in most cases.

@@ -3658,18 +3658,18 @@ most cases. 39 40MODULE_DESCRIPTION("Mutex example"); 41MODULE_LICENSE("GPL"); -

+

0.12.2 Spinlocks

-

As the name suggests, spinlocks lock up the CPU that the code is running on, +

As the name suggests, spinlocks lock up the CPU that the code is running on, taking 100% of its resources. Because of this you should only use the spinlock mechanism around code which is likely to take no more than a few milliseconds to -run and so will not noticably slow anything down from the user’s point of +run and so will not noticeably slow anything down from the user’s point of view. -

The example here is "irq safe" in that if interrupts happen during the lock then +

The example here is "irq safe" in that if interrupts happen during the lock then they will not be forgotten and will activate when the unlock happens, using the flags variable to retain their state. @@ -3738,10 +3738,10 @@ they will not be forgotten and will activate when the unlock happens, using the 61 62MODULE_DESCRIPTION("Spinlock example"); 63MODULE_LICENSE("GPL"); -

+

0.12.3 Read and write locks

-

Read and write locks are specialised kinds of spinlocks so that you can exclusively +

Read and write locks are specialised kinds of spinlocks so that you can exclusively read from something or write to something. Like the earlier spinlocks example, the one below shows an "irq safe" situation in which if other functions were triggered from irqs which might also read and write to whatever you are concerned with @@ -3806,14 +3806,14 @@ module. 53 54MODULE_DESCRIPTION("Read/Write locks example"); 55MODULE_LICENSE("GPL"); -

Of course, if you know for sure that there are no functions triggered by irqs +

Of course, if you know for sure that there are no functions triggered by irqs which could possibly interfere with your logic then you can use the simpler read_lock(&myrwlock) and read_unlock(&myrwlock) or the corresponding write functions.

0.12.4 Atomic operations

-

If you are doing simple arithmetic: adding, subtracting or bitwise operations, then +

If you are doing simple arithmetic: adding, subtracting or bitwise operations, then there is another way in the multi-CPU and multi-hyperthreaded world to stop other parts of the system from messing with your mojo. By using atomic operations you can be confident that your addition, subtraction or bit flip did actually happen @@ -3898,21 +3898,21 @@ below. -

+

0.13 Replacing Print Macros

-

+

0.13.1 Replacement

-

In Section 2, I said that X Window System and kernel module programming do not +

In Section 2, I said that X Window System and kernel module programming do not mix. That is true for developing kernel modules. But in actual use, you want to be able to send messages to whichever tty the command to load the module came from. -

"tty" is an abbreviation of teletype: originally a combination keyboard-printer +

"tty" is an abbreviation of teletype: originally a combination keyboard-printer used to communicate with a Unix system, and today an abstraction for the text stream used for a Unix program, whether it is a physical terminal, an xterm on an X display, a network connection used with ssh, etc. -

The way this is done is by using current, a pointer to the currently running task, +

The way this is done is by using current, a pointer to the currently running task, to get the current task’s tty structure. Then, we look inside that tty structure to find a pointer to a string write function, which we use to write a string to the tty. @@ -3995,16 +3995,16 @@ tty. 75module_exit(print_string_exit); 76 77MODULE_LICENSE("GPL"); -

+

0.13.2 Flashing keyboard LEDs

-

In certain conditions, you may desire a simpler and more direct way to communicate +

In certain conditions, you may desire a simpler and more direct way to communicate to the external world. Flashing keyboard LEDs can be such a solution: It is an immediate way to attract attention or to display a status condition. Keyboard LEDs are present on every hardware, they are always visible, they do not need any setup, and their use is rather simple and non-intrusive, compared to writing to a tty or a file. -

The following source code illustrates a minimal kernel module which, when +

The following source code illustrates a minimal kernel module which, when loaded, starts blinking the keyboard LEDs until it is unloaded.

@@ -4100,7 +4100,7 @@ loaded, starts blinking the keyboard LEDs until it is unloaded. -

If none of the examples in this chapter fit your debugging needs, +

If none of the examples in this chapter fit your debugging needs, there might yet be some other tricks to try. Ever wondered what CONFIG_LL_DEBUG in make menuconfig @@ -4111,22 +4111,22 @@ everything what your code does over a serial line. If you find yourself porting kernel to some new and former unsupported architecture, this is usually amongst the first things that should be implemented. Logging over a netconsole might also be worth a try. -

While you have seen lots of stuff that can be used to aid debugging here, there are +

While you have seen lots of stuff that can be used to aid debugging here, there are some things to be aware of. Debugging is almost always intrusive. Adding debug code can change the situation enough to make the bug seem to dissappear. Thus you should try to keep debug code to a minimum and make sure it does not show up in production code. -

+

0.14 Scheduling Tasks

-

There are two main ways of running tasks: tasklets and work queues. Tasklets are a +

There are two main ways of running tasks: tasklets and work queues. Tasklets are a quick and easy way of scheduling a single function to be run. For example, when triggered from an interrupt, whereas work queues are more complicated but also better suited to running multiple things in a sequence. -

+

0.14.1 Tasklets

-

Here is an example tasklet module. The +

Here is an example tasklet module. The tasklet_fn function runs for a few seconds and in the mean time execution of the example_tasklet_init @@ -4170,7 +4170,7 @@ better suited to running multiple things in a sequence. 35 36MODULE_DESCRIPTION("Tasklet example"); 37MODULE_LICENSE("GPL"); -

So with this example loaded dmesg +

So with this example loaded dmesg should show: @@ -4182,11 +4182,11 @@ Example tasklet starts Example tasklet init continues... Example tasklet ends -

-

+

+

0.14.2 Work queues

-

To add a task to the scheduler we can use a workqueue. The kernel then uses the +

To add a task to the scheduler we can use a workqueue. The kernel then uses the Completely Fair Scheduler (CFS) to execute work within the queue.

@@ -4224,19 +4224,19 @@ Completely Fair Scheduler (CFS) to execute work within the queue. 32 33MODULE_LICENSE("GPL"); 34MODULE_DESCRIPTION("Workqueue example"); -

+

0.15 Interrupt Handlers

-

+

0.15.1 Interrupt Handlers

-

Except for the last chapter, everything we did in the kernel so far we have done as a +

Except for the last chapter, everything we did in the kernel so far we have done as a response to a process asking for it, either by dealing with a special file, sending an ioctl() , or issuing a system call. But the job of the kernel is not just to respond to process requests. Another job, which is every bit as important, is to speak to the hardware connected to the machine. -

There are two types of interaction between the CPU and the rest of the +

There are two types of interaction between the CPU and the rest of the computer’s hardware. The first type is when the CPU gives orders to the hardware, the order is when the hardware needs to tell the CPU something. The second, called interrupts, is much harder to implement because it has to be dealt with when @@ -4246,14 +4246,14 @@ lost. -

Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There +

Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There are two types of IRQ’s, short and long. A short IRQ is one which is expected to take a very short period of time, during which the rest of the machine will be blocked and no other interrupts will be handled. A long IRQ is one which can take longer, and during which other interrupts may occur (but not interrupts from the same device). If at all possible, it is better to declare an interrupt handler to be long. -

When the CPU receives an interrupt, it stops whatever it is doing (unless it is +

When the CPU receives an interrupt, it stops whatever it is doing (unless it is processing a more important interrupt, in which case it will deal with this one only when the more important one is done), saves certain parameters on the stack and calls the interrupt handler. This means that certain things @@ -4265,10 +4265,10 @@ the new information at a later time (this is called the "bottom half") and return. The kernel is then guaranteed to call the bottom half as soon as possible – and when it does, everything allowed in kernel modules will be allowed. -

The way to implement this is to call +

The way to implement this is to call request_irq() to get your interrupt handler called when the relevant IRQ is received. -

In practice IRQ handling can be a bit more complex. Hardware is often +

In practice IRQ handling can be a bit more complex. Hardware is often designed in a way that chains two interrupt controllers, so that all the IRQs from interrupt controller B are cascaded to a certain IRQ from interrupt controller A. Of course, that requires that the kernel finds out which IRQ it @@ -4278,11 +4278,11 @@ them requires handlers to be written in assembler, so they do not really fit into the kernel. They can be made to work similar to the others, but after that procedure, they are no longer any faster than "common" IRQs. SMP enabled kernels running on systems with more than one processor -need to solve another truckload of problems. It is not enough to know if -a certain IRQs has happend, it’s also important for what CPU(s) it was +need to solve another truckload of problems. It is not enough to know if a +certain IRQs has happened, it’s also important to know what CPU(s) it was for. People still interested in more details, might want to refer to "APIC" now. -

This function receives the IRQ number, the name of the function, +

This function receives the IRQ number, the name of the function, flags, a name for /proc/interrupts and a parameter to be passed to the interrupt handler. Usually there is a certain number of IRQs available. How many IRQs there are is hardware-dependent. The flags can include @@ -4295,16 +4295,16 @@ already a handler on this IRQ, or if you are both willing to share. -

+

0.15.2 Detecting button presses

-

Many popular single board computers, such as Raspberry Pi or Beagleboards, have a +

Many popular single board computers, such as Raspberry Pi or Beagleboards, have a bunch of GPIO pins. Attaching buttons to those and then having a button press do something is a classic case in which you might need to use interrupts, so that instead of having the CPU waste time and battery power polling for a change in input state, it is better for the input to trigger the CPU to then run a particular handling function. -

Here is an example where buttons are connected to GPIO numbers 17 and 18 and +

Here is an example where buttons are connected to GPIO numbers 17 and 18 and an LED is connected to GPIO 4. You can change those numbers to whatever is appropriate for your board.

@@ -4454,14 +4454,14 @@ appropriate for your board. 143 144MODULE_LICENSE("GPL"); 145MODULE_DESCRIPTION("Handle some GPIO interrupts"); -

+

0.15.3 Bottom Half

-

Suppose you want to do a bunch of stuff inside of an interrupt routine. A common +

Suppose you want to do a bunch of stuff inside of an interrupt routine. A common way to do that without rendering the interrupt unavailable for a significant duration is to combine it with a tasklet. This pushes the bulk of the work off into the scheduler. -

The example below modifies the previous example to also run an additional task +

The example below modifies the previous example to also run an additional task when an interrupt is triggered.

@@ -4626,10 +4626,10 @@ when an interrupt is triggered. 159 160MODULE_LICENSE("GPL"); 161MODULE_DESCRIPTION("Interrupt with top and bottom half"); -

+

0.16 Crypto

-

At the dawn of the internet, everybody trusted everybody completely…but that did +

At the dawn of the internet, everybody trusted everybody completely…but that did not work out so well. When this guide was originally written, it was a more innocent era in which almost nobody actually gave a damn about crypto - least of all kernel developers. That is certainly no longer the case now. To handle crypto stuff, the @@ -4638,10 +4638,10 @@ favourite hash functions. -

+

0.16.1 Hash functions

-

Calculating and checking the hashes of things is a common operation. Here is a +

Calculating and checking the hashes of things is a common operation. Here is a demonstration of how to calculate a sha256 hash within a kernel module.

@@ -4707,21 +4707,21 @@ demonstration of how to calculate a sha256 hash within a kernel module. 60 61MODULE_DESCRIPTION("sha256 hash test"); 62MODULE_LICENSE("GPL"); -

Make and install the module: +

Make and install the module:

1make 
 2sudo insmod cryptosha256.ko 
 3dmesg
-

And you should see that the hash was calculated for the test string. -

Finally, remove the test module: +

And you should see that the hash was calculated for the test string. +

Finally, remove the test module:

1sudo rmmod cryptosha256
-

+

0.16.2 Symmetric key encryption

-

Here is an example of symmetrically encrypting a string using the AES algorithm +

Here is an example of symmetrically encrypting a string using the AES algorithm and a password.

@@ -4922,13 +4922,13 @@ and a password. 195 196MODULE_DESCRIPTION("Symmetric key encryption example"); 197MODULE_LICENSE("GPL"); -

+

0.17 Standardizing the interfaces: The Device Model

-

Up to this point we have seen all kinds of modules doing all kinds of things, but there +

Up to this point we have seen all kinds of modules doing all kinds of things, but there was no consistency in their interfaces with the rest of the kernel. To impose some consistency such that there is at minimum a standardized way to start, suspend and -resume a device a device model was added. An example is show below, and you can +resume a device a device model was added. An example is shown below, and you can use this as a template to add your own suspend, resume or other interface functions.

@@ -5035,24 +5035,23 @@ functions. -

+

0.18 Optimizations

-

+

0.18.1 Likely and Unlikely conditions

-

Sometimes you might want your code to run as quickly as possible, +

Sometimes you might want your code to run as quickly as possible, especially if it is handling an interrupt or doing something which might -cause noticible latency. If your code contains boolean conditions and if you -know that the conditions are almost always likely to evaluate as either +cause noticeable latency. If your code contains boolean conditions and if +you know that the conditions are almost always likely to evaluate as either true or false , then you can allow the compiler to optimize for this using the likely and unlikely - macros. -

For example, when allocating memory you are almost always expecting this to -succeed. + macros. For example, when allocating memory you are almost always expecting this +to succeed.

1bvl = bvec_alloc(gfp_mask, nr_iovecs, &idx); 
@@ -5061,50 +5060,50 @@ succeed.
 4    bio = NULL; 
 5    goto out; 
 6}
-

When the unlikely +

When the unlikely macro is used, the compiler alters its machine instruction output, so that it continues along the false branch and only jumps if the condition is true. That avoids flushing the processor pipeline. The opposite happens if you use the likely macro. -

+

0.19 Common Pitfalls

-

+

0.19.1 Using standard libraries

-

You can not do that. In a kernel module, you can only use kernel functions which are +

You can not do that. In a kernel module, you can only use kernel functions which are the functions you can see in /proc/kallsyms. -

+

0.19.2 Disabling interrupts

-

You might need to do this for a short time and that is OK, but if you do not enable +

You might need to do this for a short time and that is OK, but if you do not enable them afterwards, your system will be stuck and you will have to power it off. -

+

0.20 Where To Go From Here?

-

For people seriously interested in kernel programming, I recommend kernelnewbies.org +

For people seriously interested in kernel programming, I recommend kernelnewbies.org and the Documentation subdirectory within the kernel source code which is not always easy to understand but can be a starting point for further investigation. Also, as Linus Torvalds said, the best way to learn the kernel is to read the source code yourself. -

If you are interested in more examples of short kernel modules then searching on +

If you are interested in more examples of short kernel modules then searching on sites such as Github and Gitlab is a good way to start, although there is a lot of duplication of older LKMPG examples which may not compile with newer kernel versions. You will also be able to find examples of the use of kernel modules to attack or compromise systems or exfiltrate data and those can be useful for thinking about how to defend systems and learning about existing security mechanisms within the kernel. -

I hope I have helped you in your quest to become a better programmer, or at +

I hope I have helped you in your quest to become a better programmer, or at least to have fun through technology. And, if you do write useful kernel modules, I hope you publish them under the GPL, so I can use them too. -

If you would like to contribute to this guide or notice anything glaringly wrong, +

If you would like to contribute to this guide or notice anything glaringly wrong, please create an issue at https://github.com/sysprog21/lkmpg. -

Happy hacking! +

Happy hacking!

diff --git a/lkmpg-for-ht.html b/lkmpg-for-ht.html index 06c24cc..7d6bc83 100644 --- a/lkmpg-for-ht.html +++ b/lkmpg-for-ht.html @@ -17,7 +17,7 @@

The Linux Kernel Module Programming Guide

Peter Jay Salzman, Michael Burian, Ori Pomerantz, Bob Mottram, Jim Huang

-
August 16, 2021
+
August 17, 2021
@@ -280,7 +280,7 @@ module.
1/* 
 2 * hello-1.c - The simplest kernel module. 
 3 */ 
-4#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+4#include <linux/kernel.h> /* Needed for pr_info() */ 
 5#include <linux/module.h> /* Needed by all modules */ 
 6 
 7int init_module(void) 
@@ -425,7 +425,7 @@ technique:
 3 * This is preferred over using init_module() and cleanup_module(). 
 4 */ 
 5#include <linux/init.h>   /* Needed for the macros */ 
-6#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+6#include <linux/kernel.h> /* Needed for pr_info() */ 
 7#include <linux/module.h> /* Needed by all modules */ 
 8 
 9static int __init hello_2_init(void) 
@@ -462,7 +462,7 @@ is as simple as this:
 see, some things get hardwired into the kernel (obj-y) but where are all those obj-m
 gone? Those familiar with shell scripts will easily be able to spot them. For those not,
 the obj-$(CONFIG_FOO) entries you see everywhere expand into obj-y or obj-m,
-depending on whether the CONFIG_FOO variable has been set to y or m. While we are
+depending on whether the CONFIG_FOO variable has been set to y or m. While we are
 at it, those were exactly the kind of variables that you have set in the .config file in
 the top-level directory of Linux kernel source tree, the last time when you said
  make menuconfig
@@ -494,7 +494,7 @@ memory: 236k freed, this is precisely what the kernel is freeing.
 2 * hello-3.c - Illustrating the __init, __initdata and __exit macros. 
 3 */ 
 4#include <linux/init.h>   /* Needed for the macros */ 
-5#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+5#include <linux/kernel.h> /* Needed for pr_info() */ 
 6#include <linux/module.h> /* Needed by all modules */ 
 7 
 8static int hello3_data __initdata = 3; 
@@ -543,7 +543,7 @@ example.
 2 * hello-4.c - Demonstrates module documentation. 
 3 */ 
 4#include <linux/init.h>   /* Needed for the macros */ 
-5#include <linux/kernel.h> /* Needed for KERN_INFO */ 
+5#include <linux/kernel.h> /* Needed for pr_info() */ 
 6#include <linux/module.h> /* Needed by all modules */ 
 7 
 8MODULE_LICENSE("GPL"); 
@@ -800,7 +800,7 @@ error as follows:
 insmod: error inserting 'poet_atkm.ko': -1 Invalid module format
 

-

Less cryptical information are logged to the systemd journal: +

Less cryptic information is logged to the systemd journal: @@ -833,8 +833,8 @@ name:           hello_4 vermagic:       5.4.0-70-generic SMP mod_unload modversions

-

To overcome this problem we could resort to the --force-vermagic -option, but this solution is potentially unsafe, and unquestionably inacceptable +

To overcome this problem we could resort to the --force-vermagic option, +but this solution is potentially unsafe, and unquestionably unacceptable in production modules. Consequently, we want to compile our module in an environment which was identical to the one in which our precompiled kernel was built. How to do this, is the subject of the remainder of this @@ -847,11 +847,11 @@ tree: cp /boot/config- Let’s focus again on the previous error message: a closer look at the version magic strings suggests that, even with two configuration files which are exactly the same, a -slight difference in the version magic could be possible, and it is sufficient to -prevent insertion of the module into the kernel. That slight difference, namely -the custom string which appears in the module’s version magic and not in -the kernel’s one, is due to a modification with respect to the original, in -the makefile that some distribution include. Then, examine your Makefile, +slight difference in the version magic could be possible, and it is sufficient to prevent +insertion of the module into the kernel. That slight difference, namely the +custom string which appears in the module’s version magic and not in the +kernel’s one, is due to a modification with respect to the original, in the +makefile that some distributions include. Then, examine your Makefile, and make sure that the specified version information matches exactly the one used for your current kernel. For example, you makefile could start as follows: @@ -974,7 +974,7 @@ data into strings and write the string data using the low-level system call 6    return 0; 7}

with gcc -Wall -o hello hello.c -. Run the exectable with strace ./hello +. Run the executable with strace ./hello . Are you impressed? Every line you see corresponds to a system call. strace is a handy program that gives you details about what system calls a program is making, including which call is made, what its arguments are and what it @@ -996,7 +996,7 @@ with (like cosh()

You can even write modules to replace the kernel’s system calls, which we will do shortly. Crackers often make use of this sort of thing for backdoors or trojans, but you can write your own modules to do more benign things, like have the kernel -write Tee hee, that tickles! everytime someone tries to delete a file on your +write Tee hee, that tickles! every time someone tries to delete a file on your system.

@@ -1036,8 +1036,8 @@ symbols. entire kernel, so this is definitely an issue. The best way to deal with this is to declare all your variables as static and to use a well-defined prefix for your symbols. By convention, all kernel prefixes are lowercase. If you do not want to declare everything -as static, another option is to declare a symbol table and register it with a kernel. We -will get to this later. +as static, another option is to declare a symbol table and register it with the kernel. +We will get to this later.

The file /proc/kallsyms holds all the symbols that the kernel knows about and which are therefore accessible to your modules since they share the kernel’s codespace. @@ -1583,7 +1583,7 @@ conditional compilation directives. The way to do this to compare the macro information to processes — the /proc file system. Originally designed to allow easy access to information about processes (hence the name), it is now used by every bit of the kernel which has something interesting to report, such as /proc/modules -which provides the list of modules and /proc/meminfo which stats memory usage +which provides the list of modules and /proc/meminfo which gathers memory usage statistics.

The method to use the proc file system is very similar to the one used with device drivers — a structure is created with all the information needed for the /proc file, @@ -1620,7 +1620,7 @@ function procfile_read . The return value is a struct proc_dir_entry , and it will be used to configure the file /proc/helloworld (for example, the owner of this file). A null return value means that the creation has failed. -

Each time, everytime the file /proc/helloworld is read, the function +

Every time the file /proc/helloworld is read, the function procfile_read is called. Two parameters of this function are very important: the buffer (the second parameter) and the offset (the fourth one). The content of the @@ -2200,7 +2200,7 @@ the same way as in the previous example.

If you want more information, you can read this web page:

You can also read the code of fs/seq_file.c in the linux kernel. @@ -2327,7 +2327,7 @@ device file to write things to the modem (either modem commands or data to be sent through the phone line) and read things from the modem (either responses for commands or the data received through the phone line). However, this leaves open the question of what to do when you need to talk to the -serial port itself, for example to send the rate at which data is sent and +serial port itself, for example to configure the rate at which data is sent and received. @@ -2360,7 +2360,7 @@ ioctl.c.

If you want to use ioctls in your own kernel modules, it is best to receive an official ioctl assignment, so if you accidentally get somebody else’s ioctls, or if they get yours, you’ll know something is wrong. For more information, consult the kernel -source tree at Documentation/driver-api/ioctl.rst. +source tree at Documentation/userspace-api/ioctl/ioctl-number.rst.

1/* 
@@ -2843,30 +2843,30 @@ source tree at 196 
 197MODULE_LICENSE("GPL"); 
 198MODULE_DESCRIPTION("This is test_ioctl module");
-

+

0.10 System Calls

-

So far, the only thing we’ve done was to use well defined kernel mechanisms to +

So far, the only thing we’ve done was to use well defined kernel mechanisms to register /proc files and device handlers. This is fine if you want to do something the kernel programmers thought you’d want, such as write a device driver. But what if you want to do something unusual, to change the behavior of the system in some way? Then, you are mostly on your own. -

If you are not being sensible and using a virtual machine then this is where kernel +

If you are not being sensible and using a virtual machine then this is where kernel programming can become hazardous. While writing the example below, I killed the open() system call. This meant I could not open any files, I could not run any programs, and I could not shutdown the system. I had to restart the virtual -machine. No important files got anihilated, but if I was doing this on some live +machine. No important files got annihilated, but if I was doing this on some live mission critical system then that could have been a possible outcome. To ensure you do not lose any files, even within a test environment, please run sync right before you do the insmod and the rmmod . -

Forget about /proc files, forget about device files. They are just minor details. +

Forget about /proc files, forget about device files. They are just minor details. Minutiae in the vast expanse of the universe. The real process to kernel communication mechanism, the one used by all processes, is system calls. When a process requests a service from the kernel (such as opening a file, forking to a new @@ -2875,11 +2875,11 @@ change the behaviour of the kernel in interesting ways, this is the place to do it. By the way, if you want to see which system calls a program uses, run strace <arguments> . -

In general, a process is not supposed to be able to access the kernel. It can not +

In general, a process is not supposed to be able to access the kernel. It can not access kernel memory and it can’t call kernel functions. The hardware of the CPU enforces this (that is the reason why it is called “protected mode” or “page protection”). -

System calls are an exception to this general rule. What happens is that the +

System calls are an exception to this general rule. What happens is that the process fills the registers with the appropriate values and then calls a special instruction which jumps to a previously defined location in the kernel (of course, that location is readable by user processes, it is not writable by them). Under Intel CPUs, @@ -2887,7 +2887,7 @@ this is done by means of interrupt 0x80. The hardware knows that once you jump t this location, you are no longer running in restricted user mode, but as the operating system kernel — and therefore you’re allowed to do whatever you want. -

The location in the kernel a process can jump to is called system_call. The +

The location in the kernel a process can jump to is called system_call. The procedure at that location checks the system call number, which tells the kernel what service the process requested. Then, it looks at the table of system calls ( sys_call_table @@ -2900,7 +2900,7 @@ different process, if the process time ran out). If you want to read this code, at the source file arch/$(architecture)/kernel/entry.S, after the line ENTRY(system_call) . -

So, if we want to change the way a certain system call works, what we need to do +

So, if we want to change the way a certain system call works, what we need to do is to write our own function to implement it (usually by adding a bit of our own code, and then calling the original function) and then change the pointer at sys_call_table @@ -2908,7 +2908,7 @@ code, and then calling the original function) and then change the pointer at don’t want to leave the system in an unstable state, it’s important for cleanup_module to restore the table to its original state. -

The source code here is an example of such a kernel module. We want to “spy” on a certain +

The source code here is an example of such a kernel module. We want to “spy” on a certain user, and to pr_info() a message whenever that user opens a file. Towards this end, we replace the system call to open a file with our own function, called @@ -2918,7 +2918,7 @@ spy on, it calls pr_info() to display the name of the file to be opened. Then, either way, it calls the original open() function with the same parameters, to actually open the file. -

The init_module +

The init_module function replaces the appropriate location in sys_call_table and keeps the original pointer in a variable. The @@ -2936,7 +2936,7 @@ with B_open , which will call what it thinks is the original system call, A_open , when it’s done. -

Now, if B is removed first, everything will be well — it will simply restore the system +

Now, if B is removed first, everything will be well — it will simply restore the system call to A_open , which calls the original. However, if A is removed and then B is removed, the system will crash. A’s removal will restore the system call to the original, @@ -2959,7 +2959,7 @@ problem. When A is removed, it sees that the system call was changed to will still try to call A_open which is no longer there, so that even without removing B the system would crash. -

Note that all the related problems make syscall stealing unfeasiable for +

Note that all the related problems make syscall stealing unfeasible for production use. In order to keep people from doing potential harmful things sys_call_table is no longer exported. This means, if you want to do something more than a mere @@ -2967,8 +2967,8 @@ dry run of this example, you will have to patch your current kernel in order to sys_call_table exported. In the example directory you will find a README and the patch. As you can imagine, such modifications are not to be taken lightly. Do not try this on -valueable systems (ie systems that you do not own - or cannot restore easily). You -will need to get the complete sourcecode of this guide as a tarball in order to get the +valuable systems (ie systems that you do not own - or cannot restore easily). You will +need to get the complete sourcecode of this guide as a tarball in order to get the patch and the README. Depending on your kernel version, you might even need to hand apply the patch.

@@ -3110,13 +3110,13 @@ hand apply the patch. 135module_exit(syscall_end); 136 137MODULE_LICENSE("GPL"); -

+

0.11 Blocking Processes and threads

-

+

0.11.1 Sleep

-

What do you do when somebody asks you for something you can not do right +

What do you do when somebody asks you for something you can not do right away? If you are a human being and you are bothered by a human being, the only thing you can say is: "Not right now, I’m busy. Go away!". But if you are a kernel module and you are bothered by a process, you have another @@ -3127,21 +3127,21 @@ processes are being put to sleep by the kernel and woken up all the time (that is the way multiple processes appear to run on the same time on a single CPU). -

This kernel module is an example of this. The file (called /proc/sleep) can only +

This kernel module is an example of this. The file (called /proc/sleep) can only be opened by a single process at a time. If the file is already open, the kernel module calls wait_event_interruptible . The easiest way to keep a file open is to open it with:

1tail -f
-

This function changes the status of the task (a task is the kernel data structure +

This function changes the status of the task (a task is the kernel data structure which holds information about a process and the system call it is in, if any) to TASK_INTERRUPTIBLE , which means that the task will not run until it is woken up somehow, and adds it to WaitQ, the queue of tasks waiting to access the file. Then, the function calls the scheduler to context switch to a different process, one which has some use for the CPU. -

When a process is done with the file, it closes it, and +

When a process is done with the file, it closes it, and module_close is called. That function wakes up all the processes in the queue (there’s no mechanism to only wake up one of them). It then returns and the process which just @@ -3151,20 +3151,20 @@ Eventually, one of the processes which was in the queue will be given control of the CPU by the scheduler. It starts at the point right after the call to module_interruptible_sleep_on . -

This means that the process is still in kernel mode - as far as the process +

This means that the process is still in kernel mode - as far as the process is concerned, it issued the open system call and the system call has not returned yet. The process does not know somebody else used the CPU for most of the time between the moment it issued the call and the moment it returned. -

It can then proceed to set a global variable to tell all the other processes that the +

It can then proceed to set a global variable to tell all the other processes that the file is still open and go on with its life. When the other processes get a piece of the CPU, they’ll see that global variable and go back to sleep. -

So we will use tail -f +

So we will use tail -f to keep the file open in the background, while trying to access it with another process (again in the background, so that we need not switch to a different vt). As soon as the first background process is killed with kill %1 , the second is woken up, is able to access the file and finally terminates. -

To make our life more interesting, module_close +

To make our life more interesting, module_close does not have a monopoly on waking up the processes which wait to access the file. A signal, such as Ctrl +c (SIGINT) can also wake up a process. This is because we used module_interruptible_sleep_on @@ -3174,11 +3174,11 @@ used module_interruptible_sleep_on instead, but that would have resulted in extremely angry users whose Ctrl+c’s are ignored. -

In that case, we want to return with +

In that case, we want to return with -EINTR immediately. This is important so users can, for example, kill the process before it receives the file. -

There is one more point to remember. Some times processes don’t want to sleep, they want +

There is one more point to remember. Some times processes don’t want to sleep, they want either to get what they want immediately, or to be told it cannot be done. Such processes use the O_NONBLOCK flag when opening the file. The kernel is supposed to respond by returning with the error @@ -3214,7 +3214,7 @@ $ cat_nonblock /proc/sleep Last input: $ -

+

1/* 
@@ -3498,14 +3498,14 @@ $
 57 
 58    return 0; 
 59}
-

+

0.11.2 Completions

-

Sometimes one thing should happen before another within a module having multiple threads. +

Sometimes one thing should happen before another within a module having multiple threads. Rather than using /bin/sleep commands, the kernel has another way to do this which allows timeouts or interrupts to also happen. -

In the following example two threads are started, but one needs to start before +

In the following example two threads are started, but one needs to start before another.

@@ -3588,31 +3588,31 @@ another. 74 75MODULE_DESCRIPTION("Completions example"); 76MODULE_LICENSE("GPL"); -

The machine +

The machine structure stores the completion states for the two threads. At the exit point of each thread the respective completion state is updated, and wait_for_completion is used by the flywheel thread to ensure that it does not begin prematurely. -

So even though flywheel_thread +

So even though flywheel_thread is started first you should notice if you load this module and run dmesg that turning the crank always happens first because the flywheel thread waits for it to complete. -

There are other variations upon the +

There are other variations upon the wait_for_completion function, which include timeouts or being interrupted, but this basic mechanism is enough for many common situations without adding a lot of complexity. -

+

0.12 Avoiding Collisions and Deadlocks

-

If processes running on different CPUs or in different threads try to access the same +

If processes running on different CPUs or in different threads try to access the same memory, then it is possible that strange things can happen or your system can lock up. To avoid this, various types of mutual exclusion kernel functions are available. These indicate if a section of code is "locked" or "unlocked" so that simultaneous attempts to run it can not happen.

0.12.1 Mutex

-

You can use kernel mutexes (mutual exclusions) in much the same manner that you +

You can use kernel mutexes (mutual exclusions) in much the same manner that you might deploy them in userland. This may be all that is needed to avoid collisions in most cases.

@@ -3658,18 +3658,18 @@ most cases. 39 40MODULE_DESCRIPTION("Mutex example"); 41MODULE_LICENSE("GPL"); -

+

0.12.2 Spinlocks

-

As the name suggests, spinlocks lock up the CPU that the code is running on, +

As the name suggests, spinlocks lock up the CPU that the code is running on, taking 100% of its resources. Because of this you should only use the spinlock mechanism around code which is likely to take no more than a few milliseconds to -run and so will not noticably slow anything down from the user’s point of +run and so will not noticeably slow anything down from the user’s point of view. -

The example here is "irq safe" in that if interrupts happen during the lock then +

The example here is "irq safe" in that if interrupts happen during the lock then they will not be forgotten and will activate when the unlock happens, using the flags variable to retain their state. @@ -3738,10 +3738,10 @@ they will not be forgotten and will activate when the unlock happens, using the 61 62MODULE_DESCRIPTION("Spinlock example"); 63MODULE_LICENSE("GPL"); -

+

0.12.3 Read and write locks

-

Read and write locks are specialised kinds of spinlocks so that you can exclusively +

Read and write locks are specialised kinds of spinlocks so that you can exclusively read from something or write to something. Like the earlier spinlocks example, the one below shows an "irq safe" situation in which if other functions were triggered from irqs which might also read and write to whatever you are concerned with @@ -3806,14 +3806,14 @@ module. 53 54MODULE_DESCRIPTION("Read/Write locks example"); 55MODULE_LICENSE("GPL"); -

Of course, if you know for sure that there are no functions triggered by irqs +

Of course, if you know for sure that there are no functions triggered by irqs which could possibly interfere with your logic then you can use the simpler read_lock(&myrwlock) and read_unlock(&myrwlock) or the corresponding write functions.

0.12.4 Atomic operations

-

If you are doing simple arithmetic: adding, subtracting or bitwise operations, then +

If you are doing simple arithmetic: adding, subtracting or bitwise operations, then there is another way in the multi-CPU and multi-hyperthreaded world to stop other parts of the system from messing with your mojo. By using atomic operations you can be confident that your addition, subtraction or bit flip did actually happen @@ -3898,21 +3898,21 @@ below. -

+

0.13 Replacing Print Macros

-

+

0.13.1 Replacement

-

In Section 2, I said that X Window System and kernel module programming do not +

In Section 2, I said that X Window System and kernel module programming do not mix. That is true for developing kernel modules. But in actual use, you want to be able to send messages to whichever tty the command to load the module came from. -

"tty" is an abbreviation of teletype: originally a combination keyboard-printer +

"tty" is an abbreviation of teletype: originally a combination keyboard-printer used to communicate with a Unix system, and today an abstraction for the text stream used for a Unix program, whether it is a physical terminal, an xterm on an X display, a network connection used with ssh, etc. -

The way this is done is by using current, a pointer to the currently running task, +

The way this is done is by using current, a pointer to the currently running task, to get the current task’s tty structure. Then, we look inside that tty structure to find a pointer to a string write function, which we use to write a string to the tty. @@ -3995,16 +3995,16 @@ tty. 75module_exit(print_string_exit); 76 77MODULE_LICENSE("GPL"); -

+

0.13.2 Flashing keyboard LEDs

-

In certain conditions, you may desire a simpler and more direct way to communicate +

In certain conditions, you may desire a simpler and more direct way to communicate to the external world. Flashing keyboard LEDs can be such a solution: It is an immediate way to attract attention or to display a status condition. Keyboard LEDs are present on every hardware, they are always visible, they do not need any setup, and their use is rather simple and non-intrusive, compared to writing to a tty or a file. -

The following source code illustrates a minimal kernel module which, when +

The following source code illustrates a minimal kernel module which, when loaded, starts blinking the keyboard LEDs until it is unloaded.

@@ -4100,7 +4100,7 @@ loaded, starts blinking the keyboard LEDs until it is unloaded. -

If none of the examples in this chapter fit your debugging needs, +

If none of the examples in this chapter fit your debugging needs, there might yet be some other tricks to try. Ever wondered what CONFIG_LL_DEBUG in make menuconfig @@ -4111,22 +4111,22 @@ everything what your code does over a serial line. If you find yourself porting kernel to some new and former unsupported architecture, this is usually amongst the first things that should be implemented. Logging over a netconsole might also be worth a try. -

While you have seen lots of stuff that can be used to aid debugging here, there are +

While you have seen lots of stuff that can be used to aid debugging here, there are some things to be aware of. Debugging is almost always intrusive. Adding debug code can change the situation enough to make the bug seem to dissappear. Thus you should try to keep debug code to a minimum and make sure it does not show up in production code. -

+

0.14 Scheduling Tasks

-

There are two main ways of running tasks: tasklets and work queues. Tasklets are a +

There are two main ways of running tasks: tasklets and work queues. Tasklets are a quick and easy way of scheduling a single function to be run. For example, when triggered from an interrupt, whereas work queues are more complicated but also better suited to running multiple things in a sequence. -

+

0.14.1 Tasklets

-

Here is an example tasklet module. The +

Here is an example tasklet module. The tasklet_fn function runs for a few seconds and in the mean time execution of the example_tasklet_init @@ -4170,7 +4170,7 @@ better suited to running multiple things in a sequence. 35 36MODULE_DESCRIPTION("Tasklet example"); 37MODULE_LICENSE("GPL"); -

So with this example loaded dmesg +

So with this example loaded dmesg should show: @@ -4182,11 +4182,11 @@ Example tasklet starts Example tasklet init continues... Example tasklet ends -

-

+

+

0.14.2 Work queues

-

To add a task to the scheduler we can use a workqueue. The kernel then uses the +

To add a task to the scheduler we can use a workqueue. The kernel then uses the Completely Fair Scheduler (CFS) to execute work within the queue.

@@ -4224,19 +4224,19 @@ Completely Fair Scheduler (CFS) to execute work within the queue. 32 33MODULE_LICENSE("GPL"); 34MODULE_DESCRIPTION("Workqueue example"); -

+

0.15 Interrupt Handlers

-

+

0.15.1 Interrupt Handlers

-

Except for the last chapter, everything we did in the kernel so far we have done as a +

Except for the last chapter, everything we did in the kernel so far we have done as a response to a process asking for it, either by dealing with a special file, sending an ioctl() , or issuing a system call. But the job of the kernel is not just to respond to process requests. Another job, which is every bit as important, is to speak to the hardware connected to the machine. -

There are two types of interaction between the CPU and the rest of the +

There are two types of interaction between the CPU and the rest of the computer’s hardware. The first type is when the CPU gives orders to the hardware, the order is when the hardware needs to tell the CPU something. The second, called interrupts, is much harder to implement because it has to be dealt with when @@ -4246,14 +4246,14 @@ lost. -

Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There +

Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There are two types of IRQ’s, short and long. A short IRQ is one which is expected to take a very short period of time, during which the rest of the machine will be blocked and no other interrupts will be handled. A long IRQ is one which can take longer, and during which other interrupts may occur (but not interrupts from the same device). If at all possible, it is better to declare an interrupt handler to be long. -

When the CPU receives an interrupt, it stops whatever it is doing (unless it is +

When the CPU receives an interrupt, it stops whatever it is doing (unless it is processing a more important interrupt, in which case it will deal with this one only when the more important one is done), saves certain parameters on the stack and calls the interrupt handler. This means that certain things @@ -4265,10 +4265,10 @@ the new information at a later time (this is called the "bottom half") and return. The kernel is then guaranteed to call the bottom half as soon as possible – and when it does, everything allowed in kernel modules will be allowed. -

The way to implement this is to call +

The way to implement this is to call request_irq() to get your interrupt handler called when the relevant IRQ is received. -

In practice IRQ handling can be a bit more complex. Hardware is often +

In practice IRQ handling can be a bit more complex. Hardware is often designed in a way that chains two interrupt controllers, so that all the IRQs from interrupt controller B are cascaded to a certain IRQ from interrupt controller A. Of course, that requires that the kernel finds out which IRQ it @@ -4278,11 +4278,11 @@ them requires handlers to be written in assembler, so they do not really fit into the kernel. They can be made to work similar to the others, but after that procedure, they are no longer any faster than "common" IRQs. SMP enabled kernels running on systems with more than one processor -need to solve another truckload of problems. It is not enough to know if -a certain IRQs has happend, it’s also important for what CPU(s) it was +need to solve another truckload of problems. It is not enough to know if a +certain IRQs has happened, it’s also important to know what CPU(s) it was for. People still interested in more details, might want to refer to "APIC" now. -

This function receives the IRQ number, the name of the function, +

This function receives the IRQ number, the name of the function, flags, a name for /proc/interrupts and a parameter to be passed to the interrupt handler. Usually there is a certain number of IRQs available. How many IRQs there are is hardware-dependent. The flags can include @@ -4295,16 +4295,16 @@ already a handler on this IRQ, or if you are both willing to share. -

+

0.15.2 Detecting button presses

-

Many popular single board computers, such as Raspberry Pi or Beagleboards, have a +

Many popular single board computers, such as Raspberry Pi or Beagleboards, have a bunch of GPIO pins. Attaching buttons to those and then having a button press do something is a classic case in which you might need to use interrupts, so that instead of having the CPU waste time and battery power polling for a change in input state, it is better for the input to trigger the CPU to then run a particular handling function. -

Here is an example where buttons are connected to GPIO numbers 17 and 18 and +

Here is an example where buttons are connected to GPIO numbers 17 and 18 and an LED is connected to GPIO 4. You can change those numbers to whatever is appropriate for your board.

@@ -4454,14 +4454,14 @@ appropriate for your board. 143 144MODULE_LICENSE("GPL"); 145MODULE_DESCRIPTION("Handle some GPIO interrupts"); -

+

0.15.3 Bottom Half

-

Suppose you want to do a bunch of stuff inside of an interrupt routine. A common +

Suppose you want to do a bunch of stuff inside of an interrupt routine. A common way to do that without rendering the interrupt unavailable for a significant duration is to combine it with a tasklet. This pushes the bulk of the work off into the scheduler. -

The example below modifies the previous example to also run an additional task +

The example below modifies the previous example to also run an additional task when an interrupt is triggered.

@@ -4626,10 +4626,10 @@ when an interrupt is triggered. 159 160MODULE_LICENSE("GPL"); 161MODULE_DESCRIPTION("Interrupt with top and bottom half"); -

+

0.16 Crypto

-

At the dawn of the internet, everybody trusted everybody completely…but that did +

At the dawn of the internet, everybody trusted everybody completely…but that did not work out so well. When this guide was originally written, it was a more innocent era in which almost nobody actually gave a damn about crypto - least of all kernel developers. That is certainly no longer the case now. To handle crypto stuff, the @@ -4638,10 +4638,10 @@ favourite hash functions. -

+

0.16.1 Hash functions

-

Calculating and checking the hashes of things is a common operation. Here is a +

Calculating and checking the hashes of things is a common operation. Here is a demonstration of how to calculate a sha256 hash within a kernel module.

@@ -4707,21 +4707,21 @@ demonstration of how to calculate a sha256 hash within a kernel module. 60 61MODULE_DESCRIPTION("sha256 hash test"); 62MODULE_LICENSE("GPL"); -

Make and install the module: +

Make and install the module:

1make 
 2sudo insmod cryptosha256.ko 
 3dmesg
-

And you should see that the hash was calculated for the test string. -

Finally, remove the test module: +

And you should see that the hash was calculated for the test string. +

Finally, remove the test module:

1sudo rmmod cryptosha256
-

+

0.16.2 Symmetric key encryption

-

Here is an example of symmetrically encrypting a string using the AES algorithm +

Here is an example of symmetrically encrypting a string using the AES algorithm and a password.

@@ -4922,13 +4922,13 @@ and a password. 195 196MODULE_DESCRIPTION("Symmetric key encryption example"); 197MODULE_LICENSE("GPL"); -

+

0.17 Standardizing the interfaces: The Device Model

-

Up to this point we have seen all kinds of modules doing all kinds of things, but there +

Up to this point we have seen all kinds of modules doing all kinds of things, but there was no consistency in their interfaces with the rest of the kernel. To impose some consistency such that there is at minimum a standardized way to start, suspend and -resume a device a device model was added. An example is show below, and you can +resume a device a device model was added. An example is shown below, and you can use this as a template to add your own suspend, resume or other interface functions.

@@ -5035,24 +5035,23 @@ functions. -

+

0.18 Optimizations

-

+

0.18.1 Likely and Unlikely conditions

-

Sometimes you might want your code to run as quickly as possible, +

Sometimes you might want your code to run as quickly as possible, especially if it is handling an interrupt or doing something which might -cause noticible latency. If your code contains boolean conditions and if you -know that the conditions are almost always likely to evaluate as either +cause noticeable latency. If your code contains boolean conditions and if +you know that the conditions are almost always likely to evaluate as either true or false , then you can allow the compiler to optimize for this using the likely and unlikely - macros. -

For example, when allocating memory you are almost always expecting this to -succeed. + macros. For example, when allocating memory you are almost always expecting this +to succeed.

1bvl = bvec_alloc(gfp_mask, nr_iovecs, &idx); 
@@ -5061,50 +5060,50 @@ succeed.
 4    bio = NULL; 
 5    goto out; 
 6}
-

When the unlikely +

When the unlikely macro is used, the compiler alters its machine instruction output, so that it continues along the false branch and only jumps if the condition is true. That avoids flushing the processor pipeline. The opposite happens if you use the likely macro. -

+

0.19 Common Pitfalls

-

+

0.19.1 Using standard libraries

-

You can not do that. In a kernel module, you can only use kernel functions which are +

You can not do that. In a kernel module, you can only use kernel functions which are the functions you can see in /proc/kallsyms. -

+

0.19.2 Disabling interrupts

-

You might need to do this for a short time and that is OK, but if you do not enable +

You might need to do this for a short time and that is OK, but if you do not enable them afterwards, your system will be stuck and you will have to power it off. -

+

0.20 Where To Go From Here?

-

For people seriously interested in kernel programming, I recommend kernelnewbies.org +

For people seriously interested in kernel programming, I recommend kernelnewbies.org and the Documentation subdirectory within the kernel source code which is not always easy to understand but can be a starting point for further investigation. Also, as Linus Torvalds said, the best way to learn the kernel is to read the source code yourself. -

If you are interested in more examples of short kernel modules then searching on +

If you are interested in more examples of short kernel modules then searching on sites such as Github and Gitlab is a good way to start, although there is a lot of duplication of older LKMPG examples which may not compile with newer kernel versions. You will also be able to find examples of the use of kernel modules to attack or compromise systems or exfiltrate data and those can be useful for thinking about how to defend systems and learning about existing security mechanisms within the kernel. -

I hope I have helped you in your quest to become a better programmer, or at +

I hope I have helped you in your quest to become a better programmer, or at least to have fun through technology. And, if you do write useful kernel modules, I hope you publish them under the GPL, so I can use them too. -

If you would like to contribute to this guide or notice anything glaringly wrong, +

If you would like to contribute to this guide or notice anything glaringly wrong, please create an issue at https://github.com/sysprog21/lkmpg. -

Happy hacking! +

Happy hacking!