1895 lines
120 KiB
TeX
1895 lines
120 KiB
TeX
\documentclass[10pt, oneside]{book}
|
||
\usepackage[Bjornstrup]{fncychap}
|
||
\usepackage[utf8]{inputenc}
|
||
\usepackage[T1]{fontenc}
|
||
\usepackage{graphicx}
|
||
\usepackage{fancyhdr}
|
||
\usepackage{xparse}
|
||
\usepackage{ifthen}
|
||
\usepackage{pdfpages}
|
||
\usepackage{CJKutf8}
|
||
|
||
% TikZ设置
|
||
\usepackage{tikz}
|
||
\usetikzlibrary{shapes.geometric, arrows, shadows, decorations.text}
|
||
\tikzstyle{startstop} = [rectangle, rounded corners, minimum width=3cm, minimum height=1cm,text centered, draw=black, fill=red!30, drop shadow]
|
||
\tikzstyle{io} = [trapezium, trapezium left angle=70, trapezium right angle=110, minimum width=3cm, minimum height=1cm, text centered, draw=black, fill=blue!30]
|
||
\tikzstyle{process} = [rectangle, minimum width=3cm, minimum height=1cm, text centered, text width=3cm, draw=black, fill=orange!30]
|
||
\tikzstyle{decision} = [diamond, minimum width=1cm, minimum height=1cm, text centered, draw=black, fill=green!30]
|
||
\tikzstyle{arrow} = [thick,->,>=stealth]
|
||
\tikzstyle{line} = [draw, -latex']
|
||
|
||
% 代码包
|
||
\usepackage{verbatim}
|
||
\usepackage{minted}
|
||
% 引用
|
||
\usepackage{cite}
|
||
\usepackage[colorlinks,citecolor=green]{hyperref}
|
||
\usepackage{xcolor}
|
||
% 超链接
|
||
\hypersetup{
|
||
colorlinks,
|
||
linkcolor = {red!50!black},
|
||
citecolor = {blue!50!black},
|
||
urlcolor = {blue!80!black}
|
||
}
|
||
|
||
\input{lib/codeblock}
|
||
\input{lib/kernelsrc}
|
||
|
||
\renewcommand{\thesection}{\arabic{section}}
|
||
|
||
\title{Linux内核模块的编程指南}
|
||
\begin{document}
|
||
\begin{CJK*}{UTF8}{gbsn}
|
||
|
||
% 目录
|
||
\maketitle
|
||
\pagestyle{empty}
|
||
\tableofcontents
|
||
|
||
\section{引言}
|
||
\label{sec:引言}
|
||
|
||
《Linux内核模块编程指南》是一本免费的书；你可以根据 \href{https://opensource.org/licenses/OSL-3.0}{开放软件许可证(Open Software License)} 3.0版的条款复制和/或修改它。
|
||
|
||
本书的发行是希望它会有用, 但没有任何担保, 甚至没有对适销性或特定用途的适用性的暗示担保。
|
||
|
||
作者鼓励广泛分发本书供个人或商业使用, 前提是上述版权声明保持不变, 并且该方法符合\href{https://opensource.org/licenses/OSL-3.0}{开放软件许可证(Open Software License)}的规定。总之, 你可以免费复制和分发本书, 也可以赢利。以任何媒介、物理或电子形式复制本书, 无需获得作者的明确许可。
|
||
|
||
本文档的衍生作品和翻译必须置于开放软件许可证下, 并且原始版权声明必须保持不变。如果你为本书贡献了新材料, 则必须为你的修订提供材料和源代码。请直接向文档维护者Jim Huang提供修订和更新<jserv@ccns.ncku.edu.tw>。这将允许合并更新, 并为Linux社区提供一致的修订。
|
||
|
||
如果你以商业方式出版或分发本书, 作者和\href{https://tldp.org/}{Linux文档项目(LDP)}将非常感谢你的捐赠、版税和/或印刷本。以这种方式贡献表明你支持自由软件和LDP。如果你有任何问题或意见, 请联系上述地址。
|
||
|
||
\subsection{作者}
|
||
\label{sec:作者}
|
||
|
||
《Linux内核模块编程指南》最初由Ori Pomerantz for Linux v2.2编写。随着Linux内核的发展, Ori维护文档的可用性降低了。因此, Peter Jay Salzman承担了维护者的角色, 并更新了Linuxv2.4指南。在跟踪Linux v2.6中的开发时, Peter也受到了类似的限制, 导致Michael Burian加入为共同维护者, 以使该指南与Linux v2.6同步。Bob Mottram通过更新Linux v3.8和更高版本的示例对该指南做出了贡献。Jim Huang随后承担了更新最新Linux版本(v5.0及更高版本)指南的任务, 并修订LaTeX文档。
|
||
|
||
\subsection{致谢}
|
||
\label{sec:致谢}
|
||
|
||
以下人员为本书提供了一些更正与良好的建议:
|
||
|
||
\begin{flushleft}
|
||
\input{contrib}
|
||
\end{flushleft}
|
||
|
||
\subsection{什么是内核模块?}
|
||
\label{sec:kernelmod}
|
||
|
||
参与Linux内核模块开发需要扎实的C编程语言基础和创建用于进程执行的传统程序的丰富经验。这一追求涉及到一个领域, 如果处理不当, 一个未受限制的指针可能会导致整个文件系统的崩溃, 导致系统必须重启。
|
||
|
||
Linux 内核模块的确切定义是: 能够根据需要在内核中动态加载和卸载的代码段。
|
||
这些模块增强了内核功能, 而无需重新启动系统。
|
||
一个值得注意的例子是设备驱动程序模块, 它促进了内核与链接到系统的硬件组件的交互。
|
||
在没有模块的情况下, 主流的方法倾向于使用单内核( Monolithic Kernel 也可以称之为宏内核 Macrokernel ), 需要将新功能直接集成到内核镜像中。这种方法会导致内核变得更大, 并在需要新增功能时, 必须重新构建内核并进行系统重启。
|
||
|
||
译者注: 单内核( Monolithic Kernel) 有时候也被称之为宏内核(Macrokernel), 与其对应的另一种内核形式是微内核, Linux内核是微内核和内核的混合产物
|
||
|
||
\subsection{内核模块包}
|
||
\label{sec:packages}
|
||
|
||
Linux发行版在一个包中提供了命令 \sh|modprobe|, \sh|insmod| 和 \sh|depmod|。
|
||
|
||
在安同上:
|
||
\begin{codebash}
|
||
sudo oma install gcc kmod
|
||
\end{codebash}
|
||
|
||
在 Debian 与其衍生发行版(乌班图ubuntu, 深度deebin, kali)上:
|
||
\begin{codebash}
|
||
sudo apt-get install build-essential kmod
|
||
\end{codebash}
|
||
|
||
在 Arch Linux系上:
|
||
\begin{codebash}
|
||
sudo pacman -S gcc kmod
|
||
\end{codebash}
|
||
|
||
\subsection{我的内核中有哪些模块?}
|
||
\label{sec:modutils}
|
||
|
||
要发现当前内核中已经加载了哪些模块可以使用命令 \sh|lsmod|.
|
||
\begin{codebash}
|
||
sudo lsmod
|
||
\end{codebash}
|
||
|
||
模块存储在文件 \verb|/proc/modules| 中, 因此你也可以使用以下命令查看它们:
|
||
\begin{codebash}
|
||
sudo cat /proc/modules
|
||
\end{codebash}
|
||
|
||
这可能是一个很长的列表, 可以使用grep命令搜索某类模块, 比如使用下述命令搜索网络模块
|
||
\begin{codebash}
|
||
sudo lsmod | grep net
|
||
\end{codebash}
|
||
|
||
\subsection{是否需要下载和编译内核?}
|
||
\label{sec:buildkernel}
|
||
本指南并没有对此有强制要求。但建议本指南的示例使用虚拟机运行, 从而减少可能对系统造成的任何风险
|
||
|
||
\subsection{开始之前}
|
||
\label{sec:preparation}
|
||
在深入研究代码之前, 需要注意某些事项。每个人的主机都有所不同, 所以在不同的内核与发行版之间都可能存在一些差异, 成功编译和加载首个 "hello world" 程序有时可能会是一个挑战。 克服最初的障碍是一个令人欣慰的过程, 它会为你后续的努力铺平道路。
|
||
|
||
\begin{enumerate}
|
||
\item Modversioning.
|
||
如果引导不同的内核, 则为一个内核编译的模块将不会加载, 除非在内核中启用了 \cpp|CONFIG_MODVERSIONS| 本指南稍后将讨论模块版本化问题。在讨论模块版本控制之前, 如果在启用了 modversioning 的情况下运行内核, 则本指南中的示例可能无法正常工作。然而, 大多数Linux发行版内核都启用了 modversioning 。如果由于版本控制错误而在加载模块时出现问题, 请考虑在关闭 modversioning 的情况下编译内核。
|
||
|
||
\item 使用 X Window System.
|
||
强烈建议使用控制台(终端)中获取、编译、运行本指南中讨论的所有示例, 不建议使用可视化窗口运行
|
||
|
||
模块不能像 \cpp|printf()| 那样直接打印到屏幕上, 但它们可以记录最终显示在屏幕上的信息和警告, 特别是在控制台中。如果从 \sh|xTER| 加载模块, 则信息和警告将被记录在systemd日志中。只有查询 \sh|journalctl|才能看到这些日志。 有关详细信息, 请看 \ref{sec:helloworld} 。为了即时访问此信息, 建议从控制台执行所有任务。
|
||
|
||
\item SecureBoot.
|
||
许多现代计算机都预先配置了UEFI SecureBoot, 这是一个基本的安全标准, 确保仅通过原始设备制造商认可的可信软件引导。某些Linux发行版甚至附带配置为支持SecureBoot的默认Linux内核。在这些情况下, 内核模块需要一个签名的安全密钥。
|
||
|
||
如果在启动"hello world"模块时出现错误信息: ``\emph{ERROR: could not insert module(错误:无法插入模块)}''。如果此消息 \emph{Lockdown: insmod: unsigned module loading is restricted; see man kernel lockdown.7(未签名的模块的装载限制, 请参阅 man kernel lockdown.7 )} 出现在 \sh|dmesg| 输出, 最简单的方法是从主机的引导菜单中禁用UEFI SecureBoot。当然, 另一种方法涉及复杂的过程, 如生成密钥、系统密钥安装和模块签名, 以实现功能。然而, 这种复杂的过程不太适合初学者。如果有兴趣, 可以探索和遵循\href{https://wiki.debian.org/SecureBoot}{SecureBoot}的更详细步骤。
|
||
\end{enumerate}
|
||
|
||
\section{开始(头部)}
|
||
\label{sec:headers}
|
||
在构建任何东西之前, 必须安装内核的头文件。
|
||
|
||
对于几乎所有的发行版都可以使用如下命令来确定是否已有内核头文件
|
||
\begin{codebash}
|
||
ls /lib/modules/$(uname -r)/build
|
||
\end{codebash}
|
||
|
||
如果上述命令返回不为空则代表存在相关内核头文件, 其他情况需要从包管理器中手动下载, 内核头文件的包名在Debian系或Arch系中名称一般是 linux-headers、linux-headers-(内核版本号)-其他, 而在红帽系中一般名称是 kernel-devel和kernel-headers
|
||
|
||
\section{示例}
|
||
\label{sec:examples}
|
||
本文档中的所有示例都位于 \verb|examples| 子目录中。
|
||
|
||
如果发生编译错误, 可能是由于正在使用较新的内核版本, 或者可能需要安装相应的内核头文件。
|
||
|
||
\section{Hello World}
|
||
\label{sec:helloworld}
|
||
\subsection{最简单的模块}
|
||
\label{sec:org2d3e245}
|
||
大多数人开始他们的编程旅程通常从 \emph{hello world} 示例的某种变体开始。偏离这一传统的人目前尚未可知会产生什么结果, 但坚持这一传统似乎是明智的。 学习过程将从一系列hello world程序开始, 这些程序说明了编写内核模块的各个基本方面。
|
||
|
||
接下来介绍的可能是最简单的模块。
|
||
|
||
创建测试目录:
|
||
\begin{codebash}
|
||
mkdir -p ~/develop/kernel/hello-1
|
||
cd ~/develop/kernel/hello-1
|
||
\end{codebash}
|
||
|
||
将下述代码粘贴到你喜爱的编辑器中, 并将其命名为 \verb|hello-1.c|:
|
||
|
||
\samplec{examples/hello-1.c}
|
||
|
||
现在你需要一个 \verb|Makefile| 。如果复制并粘贴此内容, 请将缩进更改为使用 \textit{制表符(tabs)}, 而不是空格。
|
||
|
||
\begin{code}
|
||
obj-m += hello-1.o
|
||
|
||
PWD := $(CURDIR)
|
||
|
||
all:
|
||
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
|
||
|
||
clean:
|
||
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
|
||
\end{code}
|
||
|
||
在 \verb|Makefile| 中, \verb|$(CURDIR)| 可以设置为当前工作目录的绝对路径名(在处理所有 \verb|-C| 选项后, 如果有)。请参阅 \href{https://www.gnu.org/software/make/manual/make.html}{GNU make manual} 中有关 \verb|CURDIR| 的更多信息。
|
||
|
||
后续直接运行make即可
|
||
|
||
\begin{codebash}
|
||
make
|
||
\end{codebash}
|
||
|
||
如果在Makefile中没有 \verb|PWD := $(CURDIR)| 这样的语句, 那么使用\verb|sudo make| 可能无法正确编译。这是因为一些环境变量受到安全策略的限制, 不能被继承。默认的安全策略是 \verb|sudoers| 。在 \verb|sudoers| 安全策略中, 默认启用了 \verb|env_reset| , 这会限制环境变量的使用。具体来说, 路径变量不会从用户环境中保留, 而是被设置为默认值(更多信息请参考\href{https://www.sudo.ws/docs/man/sudoers.man/}{sudoers 手册})。你可以通过以下方式查看环境变量的设置:
|
||
|
||
\begin{verbatim}
|
||
$ sudo -s
|
||
# sudo -V
|
||
\end{verbatim}
|
||
|
||
下面以一个简单的 Makefile 为例, 演示上述问题。
|
||
|
||
\begin{code}
|
||
all:
|
||
echo $(PWD)
|
||
\end{code}
|
||
|
||
我们可以使用 \verb|-p| 标志从Makefile中打印出环境变量值。
|
||
|
||
\begin{verbatim}
|
||
$ make -p | grep PWD
|
||
PWD = /home/ubuntu/temp
|
||
OLDPWD = /home/ubuntu
|
||
echo $(PWD)
|
||
\end{verbatim}
|
||
|
||
\verb|PWD| 变量不会与 \verb|sudo| 一起继承。
|
||
|
||
\begin{verbatim}
|
||
$ sudo make -p | grep PWD
|
||
echo $(PWD)
|
||
\end{verbatim}
|
||
|
||
有三种方法可以解决这个问题。
|
||
|
||
\begin{enumerate}
|
||
\item {
|
||
可以使用 \verb|-E| 标志临时保留它们。
|
||
|
||
\begin{codebash}
|
||
$ sudo -E make -p | grep PWD
|
||
PWD = /home/ubuntu/temp
|
||
OLDPWD = /home/ubuntu
|
||
echo $(PWD)
|
||
\end{codebash}
|
||
}
|
||
|
||
\item {
|
||
可以通过执行\verb|visudo|命令, 编辑\verb|/etc/sudoers|来禁用\verb|env_reset|(环境重置)。
|
||
|
||
\begin{code}
|
||
## sudoers 文件.
|
||
##
|
||
...
|
||
Defaults env_reset
|
||
## 将env_reset更改为!env_reset, 其他环境变量不做更改
|
||
\end{code}
|
||
|
||
然后分别执行 \verb|env|和\verb|sudo env|
|
||
|
||
\begin{codebash}
|
||
# 禁用 env_reset
|
||
echo "user:" > non-env_reset.log; env >> non-env_reset.log
|
||
echo "root:" >> non-env_reset.log; sudo env >> non-env_reset.log
|
||
# 启用 the env_reset
|
||
echo "user:" > env_reset.log; env >> env_reset.log
|
||
echo "root:" >> env_reset.log; sudo env >> env_reset.log
|
||
\end{codebash}
|
||
|
||
可以查看上述日志发现 \verb|env_reset| 和 \verb|!env_reset|的差异
|
||
}
|
||
|
||
\item {可以通过将环境变量附加到\verb|/etc/sudoers|中的\verb|env_keep|来保留环境变量
|
||
|
||
\begin{code}
|
||
Defaults env_keep += "PWD"
|
||
\end{code}
|
||
|
||
应用上述更改后, 可以通过以下方式检查环境变量设置:
|
||
|
||
\begin{verbatim}
|
||
$ sudo -s
|
||
# sudo -V
|
||
\end{verbatim}
|
||
}
|
||
\end{enumerate}
|
||
|
||
如果一切顺利, 你应该会发现你有一个已编译的\verb|hello-1.ko|模块。你可以使用以下命令查找有关它的信息:
|
||
\begin{codebash}
|
||
modinfo hello-1.ko
|
||
\end{codebash}
|
||
|
||
使用此命令:
|
||
\begin{codebash}
|
||
sudo lsmod | grep hello
|
||
\end{codebash}
|
||
|
||
应不返回任何内容。但尝试接下来的命令加载新模块:
|
||
\begin{codebash}
|
||
sudo insmod hello-1.ko
|
||
\end{codebash}
|
||
|
||
载入模块时中划线(hello-1)将会被替换成下划线(hello\_1), 因此当你再次尝试时:
|
||
\begin{codebash}
|
||
sudo lsmod | grep hello
|
||
\end{codebash}
|
||
|
||
你应该可以看到已加载的模块。可以使用以下命令再次移除它(请注意, 中划线已被下划线替换):
|
||
\begin{codebash}
|
||
sudo rmmod hello_1
|
||
\end{codebash}
|
||
|
||
要查看日志中刚刚发生的情况, 请执行以下操作:
|
||
\begin{codebash}
|
||
sudo journalctl --since "1 hour ago" | grep kernel
|
||
\end{codebash}
|
||
|
||
现在你已经了解了创建、编译、安装和删除模块的基础知识。现在, 更多地描述该模块的工作原理。
|
||
|
||
内核模块必须至少有两个函数: 一个名为 \cpp|init_module()| 的 "init"(初始化) 函数, 当模块被插入内核时调用；另一个名为主模块的 "cleanup"(清理) 函数, 在将其从内核中删除之前调用。实际上, 从内核2.3.13开始, 情况发生了变化。现在, 你可以为模块的开始和结束函数使用任何名称, 你将在第\ref{hello_n_goodbye}节中学习如何做到这一点。事实上, 新方法是首选方法。然而, 许多人仍然使用 \cpp|init_module()| 和 \cpp|cleanup_module()| 作为他们的开始和结束函数。
|
||
|
||
通常, \cpp|init_module()| 要么向内核注册某个对象的处理程序, 要么用自己的代码替换其中一个内核函数(通常是执行某些操作的代码, 然后调用原始函数)。\cpp|cleanup_module()| 函数的作用是撤消 \cpp|init_module()| 所做的任何操作, 因此可以安全地卸载该模块。
|
||
|
||
最后, 每个内核模块都需要包含\verb|<linux/module.h>|。我们只需要为 \cpp|pr_alert()| 日志级别的宏扩展包含\verb|<linux/printk.h>|, 你将在第\ref{sec:printk}节中了解这一点。
|
||
|
||
\begin{enumerate}
|
||
\item 关于编码风格。
|
||
对任何开始内核编程的人来说都不容易被注意到的事情是, 代码中的缩进应该使用\textbf{制表符(tabs)}, 而不是空格。它是内核的编码约定之一。你可能不喜欢它, 但如果你想要向上游提交补丁, 则需要习惯它。并且内核中的注释无论是单行还是多行都习惯使用 /**/ 的方式而非 //
|
||
|
||
\item 介绍打印(print)宏。
|
||
\label{sec:printk}
|
||
一开始是 \cpp|printk| , 后面通常是优先级, 如\cpp|KERN_INFO| 或者 \cpp|KERN_DEBUG|。在较新的内核也可以使用一组打印宏(如\cpp|pr_info| 和 \cpp|pr_debug|)以缩写形式表示。这只是省去了一些无意义的键盘敲击, 看起来更加整洁。它们可以在\src{include/linux/printk.h}中找到。请花时间阅读可用的优先级宏(更多信息请参考\href{https://www.kernel.org/doc/html/latest/translations/zh_CN/core-api/printk-basics.html}{中文版 printk 手册}/\href{https://www.kernel.org/doc/html/latest/core-api/printk-basics.html}{英文版 printk 手册})。
|
||
|
||
\item 关于编译。
|
||
内核模块的编译需要与常规用户空间应用程序稍有不同。以前的内核版本要求我们非常关心这些设置, 这些设置通常存储在Makefiles中。尽管分层组织, 但许多冗余设置积累在子级Makefiles中, 使它们变得很大, 很难维护。幸运的是, 有一种新的方法来完成这些事情, 称为kbuild, 并且外部可加载模块的构建过程现在完全集成到标准内核构建机制中。要了解有关如何编译不属于官方内核的模块(如本指南中的所有示例)的更多信息, 请参阅文件\src{Documentation/kbuild/modules.rst}。
|
||
|
||
有关内核模块的Makefiles的其他详细信息, 请参阅\src{Documentation/kbuild/makefiles.rst}。在开始修改Makefile之前, 请务必阅读此文件和相关文件。它可能会为你节省大量工作。
|
||
|
||
\begin{quote}
|
||
这里是读者的另一个练习。看到\cpp|init_module()|中return语句上方的注释了吗？将返回值更改为负值, 重新编译并再次加载模块。发生了什么？
|
||
\end{quote}
|
||
\end{enumerate}
|
||
|
||
\subsection{你好, 再见}
|
||
\label{hello_n_goodbye}
|
||
在早期的内核版本中, 你必须使用 \cpp|init_module| 和 \cpp|cleanup_module|函数, 就像在第一个hello world示例中一样, 但现在你可以通过使用\cpp|module_init|和 \cpp|module_exit|宏来命名这些函数。这些宏在 \src{include/linux/module.h} 中定义。唯一的要求是必须在调用这些宏之前定义init和cleanup函数, 否则将出现编译错误。下面是该技术的一个示例:
|
||
|
||
\samplec{examples/hello-2.c}
|
||
|
||
现在我们有两个真正的内核模块。添加另一个模块非常简单:
|
||
|
||
\begin{code}
|
||
obj-m += hello-1.o
|
||
obj-m += hello-2.o
|
||
|
||
PWD := $(CURDIR)
|
||
|
||
all:
|
||
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
|
||
|
||
clean:
|
||
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
|
||
\end{code}
|
||
|
||
现在来看一个现实的示例 \src{drivers/char/Makefile} 。如你所见, 有些东西已经被硬编码到内核中(\verb|obj-y|), 但所有这些\verb|obj-m|都到哪里去了？熟悉shell脚本的人将很容易发现它们。对于那些不熟悉的人来说, 你在各处看到的 obj-\$(CONFIG\_FOO) 条目会根据 CONFIG\_FOO 变量是设置为 y 还是 m, 展开为\verb|obj-y| 或 \verb|obj-m|。顺便提一下, 这些正是你在 Linux 内核源代码树的顶级目录下使用 make menuconfig 或类似命令时, 在 .config 文件中设置的那种变量。
|
||
|
||
\subsection{ \_\_init 和 \_\_exit 宏}
|
||
\label{init_n_exit}
|
||
\cpp|__init|宏会导致在初始化函数为内置驱动程序(但不是可加载模块)完成后丢弃初始化函数并释放其内存。如果你思考一下 初始化 函数何时被调用就会发现这是非常合理的操作, 因为内置模块不需要再次加载, 而可加载的模块则需要。
|
||
|
||
还有一个\cpp|__initdata|, 其工作方式类似于\cpp|__init|, 但用于初始化变量而不是函数。
|
||
|
||
\cpp|__exit| 宏会在将模块内置到内核时也会被省略, 与 \cpp|__init| 类似, 对可加载模块没有影响。考虑到 退出 函数的作用这种做法也是完全合理的。
|
||
|
||
这些宏在\src{include/linux/init.h}中定义, 用于释放内核内存。当引导内核并看到类似"释放未使用的内核内存: 236k已释放"的内容时, 这正是内核正在释放的内容。
|
||
|
||
\samplec{examples/hello-3.c}
|
||
|
||
\subsection{许可(Licensing)和模块文档}
|
||
\label{modlicense}
|
||
老实说, 谁加载甚至关心专有模块？如果你这样做, 你可能会看到这样的情况:
|
||
\begin{verbatim}
|
||
$ sudo insmod xxxxxx.ko
|
||
loading out-of-tree module taints kernel.
|
||
module license 'unspecified' taints kernel.
|
||
\end{verbatim}
|
||
|
||
你可以使用几个宏来指示模块的许可证。常见示例有"GPL"、"GPL v2"、"GPL和附加权限(GPL and additional rights)"、"Dual BSD/GPL", "Dual MIT/GPL""Dual MPL/GPL"和"Proprietary"。它们在\src{include/linux/module.h}中定义。
|
||
|
||
要引用你正在使用的许可证, 可以使用名为\cpp|MODULE_LICENSE|的宏。下面的示例中说明了该宏和几个描述该模块的其他宏。
|
||
|
||
\samplec{examples/hello-4.c}
|
||
|
||
\subsection{将命令行参数传递到模块}
|
||
\label{modparam}
|
||
模块可以采用命令行参数, 但不能使用你可能习惯的argc/argv。
|
||
|
||
要允许将参数传递给模块, 请声明将采用命令行参数值的变量作为全局变量, 然后使用 \cpp|module_param()| 宏(在\src{include/linux/moduleparam.h}中定义)来设置机制。在运行时, \sh|insmod| 将使用给定的任何命令行参数填充变量, 如 \sh|insmod mymodule.ko myvariable=5| 。为了清楚起见, 变量声明和宏应该放在模块的开头。这块解释的很糟糕, 还是无法理解可以试着查看实例代码。
|
||
|
||
\cpp|module_param()|宏接受3个参数: 变量的名称、类型和sysfs中相应文件的权限。整数类型可以像平常一样有符号, 也可以无符号。如果要使用整数数组或字符串数组, 请使用 \cpp|module_param_array()| 和 \cpp|module_param_string()|。
|
||
|
||
\begin{code}
|
||
int myint = 3;
|
||
module_param(myint, int, 0);
|
||
\end{code}
|
||
|
||
数组也得到了支持, 但与过去相比, 现在的处理方式有所不同。为了跟踪参数数量, 你需要将一个指向计数变量的指针作为第三个参数传递。你也可以选择忽略计数并传递 \cpp|NULL|。这里我们展示了两种可能性:
|
||
|
||
\begin{code}
|
||
int myintarray[2];
|
||
module_param_array(myintarray, int, NULL, 0); /* 不需求计数器 */
|
||
|
||
short myshortarray[4];
|
||
int count;
|
||
module_param_array(myshortarray, short, &count, 0); /* 将计数器的值传给count */
|
||
\end{code}
|
||
|
||
这种用法的一个好处是可以设置模块变量的默认值, 如端口或 IO 地址。如果变量包含默认值, 则执行自动检测(在其他地方解释)。否则, 保留当前值。这将在后面详细说明。
|
||
|
||
最后, 有一个宏函数\cpp|MODULE_PARM_DESC()|, 用于记录模块可以接受的参数。它接受两个参数: 变量名和描述该变量的自由格式字符串。
|
||
|
||
\samplec{examples/hello-5.c}
|
||
|
||
推荐使用下述方法测试参数传递代码:
|
||
\begin{verbatim}
|
||
$ sudo insmod hello-5.ko mystring="bebop" myintarray=-1
|
||
$ sudo dmesg -t | tail -7
|
||
myshort is a short integer: 1
|
||
myint is an integer: 420
|
||
mylong is a long integer: 9999
|
||
mystring is a string: bebop
|
||
myintarray[0] = -1
|
||
myintarray[1] = 420
|
||
got 1 arguments for myintarray.
|
||
|
||
$ sudo rmmod hello-5
|
||
$ sudo dmesg -t | tail -1
|
||
Goodbye, world 5
|
||
|
||
$ sudo insmod hello-5.ko mystring="supercalifragilisticexpialidocious" myintarray=-1,-1
|
||
$ sudo dmesg -t | tail -7
|
||
myshort is a short integer: 1
|
||
myint is an integer: 420
|
||
mylong is a long integer: 9999
|
||
mystring is a string: supercalifragilisticexpialidocious
|
||
myintarray[0] = -1
|
||
myintarray[1] = -1
|
||
got 2 arguments for myintarray.
|
||
|
||
$ sudo rmmod hello-5
|
||
$ sudo dmesg -t | tail -1
|
||
Goodbye, world 5
|
||
|
||
$ sudo insmod hello-5.ko mylong=hello
|
||
insmod: ERROR: could not insert module hello-5.ko: Invalid parameters
|
||
\end{verbatim}
|
||
|
||
\subsection{跨多文件的模块}
|
||
\label{modfiles}
|
||
有时将内核模块划分为几个源文件是有意义的。
|
||
下面是这样一个内核模块的示例。
|
||
\samplec{examples/start.c}
|
||
|
||
下一个文件:
|
||
\samplec{examples/stop.c}
|
||
|
||
最后, 生成文件:
|
||
|
||
\begin{code}
|
||
obj-m += hello-1.o
|
||
obj-m += hello-2.o
|
||
obj-m += hello-3.o
|
||
obj-m += hello-4.o
|
||
obj-m += hello-5.o
|
||
obj-m += startstop.o
|
||
startstop-objs := start.o stop.o
|
||
|
||
PWD := $(CURDIR)
|
||
|
||
all:
|
||
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
|
||
|
||
clean:
|
||
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
|
||
\end{code}
|
||
|
||
这是我们迄今为止看到的所有示例的完整 Makefile。前五行没什么特别的, 但对于最后一个示例, 我们将需要两行。首先, 我们为我们的组合模块发明一个对象名称, 其次我们告诉 \sh|make| 哪些对象文件是该模块的一部分。
|
||
|
||
\subsection{为预编译内核构建模块}
|
||
\label{precompiled}
|
||
我们强烈建议你重新编译内核, 以便你可以启用许多有用的调试功能, 例如强制模块卸载(\cpp|MODULE_FORCE_UNLOAD|): 当启用此选项时, 你可以通过\sh|sudo rmmod -f module|命令强制内核卸载模块, 即使它认为模块不安全。在模块开发过程中, 此选项可以节省大量时间和多次重新启动。如果不想重新编译内核, 则应该考虑在虚拟机上的测试发行版中运行这些示例。如果你搞坏了什么, 可以快速重新启动或恢复虚拟机(VM)。
|
||
|
||
在许多情况下, 你可能希望将模块加载到预编译的运行内核中, 例如常见Linux发行版附带的内核, 或者你过去编译过的内核。在某些情况下, 你可能需要编译模块并将其插入到不允许重新编译的正在运行的内核中, 或者在不希望重新启动的机器上。如果你想不出一种情况会迫使你为预编译内核使用模块, 那么你可能想跳过本节剩下的内容, 并将本节的其余部分视为一个大脚注。
|
||
|
||
如果你只是安装内核源代码树(kernel source tree), 请使用它来编译内核模块, 然后尝试将模块插入内核, 在大多数情况下, 你将获得如下错误:
|
||
|
||
\begin{verbatim}
|
||
insmod: ERROR: could not insert module poet.ko: Invalid module format
|
||
报错翻译: insmod: 错误:无法插入poet.ko模块: 无效的模块格式
|
||
\end{verbatim}
|
||
|
||
一些隐秘信息被记录到systemd日志中:
|
||
|
||
\begin{verbatim}
|
||
kernel: poet: disagrees about version of symbol module_layout
|
||
报错翻译: kernel: poet: 对 module_layout 的版本符号存在分歧
|
||
\end{verbatim}
|
||
|
||
换句话说, 内核拒绝接受模块, 因为版本信息(更准确的翻译应该是 \textit{版本魔术(version magic)}, 在这里版本信息更有助于理解, 后续所有version magic的翻译都会翻译成版本信息, 请参阅\src{include/linux/vermagic.h})不匹配。顺便说一句, 版本信息以静态字符串的形式存储在模块对象中, 从\cpp|vermagic:|开始。当版本信息链接到\verb|kernel/module.o|文件时, 它会插入到模块中。要检查存储在给定模块中的版本信息和其他字符串, 请发出命令sh|modinfo module.ko|:
|
||
|
||
\begin{verbatim}
|
||
$ modinfo hello-4.ko
|
||
description: A sample driver
|
||
author: LKMPG
|
||
license: GPL
|
||
srcversion: B2AA7FBFCC2C39AED665382
|
||
depends:
|
||
retpoline: Y
|
||
name: hello_4
|
||
vermagic: 5.4.0-70-generic SMP mod_unload modversions
|
||
\end{verbatim}
|
||
|
||
为了克服这个问题, 我们可以使用\verb|--force-vermagic|选项, 但这种解决方案可能是不安全的, 并且在生产环境中是不可接受的。因此, 我们希望在与构建预编译内核的环境相同的环境中编译模块。如何做到这一点, 是本章其余部分的主题。
|
||
|
||
首先, 确保有一个内核源代码树可用, 其版本与当前内核完全相同。然后, 找到用于编译预编译内核的配置文件。这通常在\verb|/boot|目录下, 文件名类似于 \verb|config-5.14.x|。你可能只需将其复制到内核源代码树中: 执行命令 \sh|cp /boot/config-`uname -r` .config| 。
|
||
|
||
观察之前的版本信息, 即使两个配置文件完全相同, 版本信息还是可能会出现一些差异阻止模块插入到内核中。这种差异是由于对某些发行版修改了Makefile中定义的版本信息所致。查看你的内核源码根目录下的Makefile, 确保指定的版本信息与当前内核使用的完全匹配。发行版的\verb|Makefile|是类似如下格式:
|
||
|
||
\begin{verbatim}
|
||
VERSION = 5
|
||
PATCHLEVEL = 14
|
||
SUBLEVEL = 0
|
||
EXTRAVERSION = -rc2
|
||
\end{verbatim}
|
||
|
||
假设出现上述情况时, 我们需要将\verb|Makefile| 中的 \textbf{EXTRAVERSION} 的值配置为 \textbf{-rc2}。可以使用如下命令保留编译内核的makefile的备份。
|
||
|
||
\begin{codebash}
|
||
cp /lib/modules/`uname -r`/build/Makefile linux-`uname -r`
|
||
\end{codebash}
|
||
\sh|linux-`uname -r`| 是你试图构建的linux内核源代码, 可以之间在命令行中执行 \sh|`uname -r`| 查看返回值。
|
||
|
||
最后, 请运行make来更新配置、头文件:
|
||
|
||
\begin{verbatim}
|
||
$ make
|
||
SYNC include/config/auto.conf.cmd
|
||
HOSTCC scripts/basic/fixdep
|
||
HOSTCC scripts/kconfig/conf.o
|
||
HOSTCC scripts/kconfig/confdata.o
|
||
HOSTCC scripts/kconfig/expr.o
|
||
LEX scripts/kconfig/lexer.lex.c
|
||
YACC scripts/kconfig/parser.tab.[ch]
|
||
HOSTCC scripts/kconfig/preprocess.o
|
||
HOSTCC scripts/kconfig/symbol.o
|
||
HOSTCC scripts/kconfig/util.o
|
||
HOSTCC scripts/kconfig/lexer.lex.o
|
||
HOSTCC scripts/kconfig/parser.tab.o
|
||
HOSTLD scripts/kconfig/conf
|
||
\end{verbatim}
|
||
|
||
如果你不想编译完整的内核, 在SPLIT行之后中断构建 (CTRL-C) 即可, 后续返回到模块的目录并编译模块即可: 它将完全根据你当前的内核设置构建, 并且它将在没有任何错误的情况下加载。
|
||
|
||
\section{准备工作}
|
||
\subsection{模块开始和结束方式}
|
||
\label{sec:module_init_exit}
|
||
|
||
典型的程序以 \cpp|main()| 函数开始, 执行一系列指令, 并在完成这些指令后终止。然而内核模块遵循模式有所不同。模块总是以\cpp|init_module|函数或\cpp|module_init|调用指定的函数开头。该函数是模块的入口, 会通知内核模块的功能, 并在必要时让内核做好使用模块功能的准备。完成这些任务后, 入口函数返回, 模块将保持不活动状态, 直到内核使用其代码。
|
||
|
||
所有模块都通过 \cpp|cleanup_module|函数 或 \cpp|module_exit| 调用指定的函数来结束。用作模块的退出功能, 通过注销以前注册的功能来逆转入口函数操作。
|
||
|
||
每个模块都必须具有进入和退出功能。虽然有多种方法来定义这些函数, 但通常使用术语"进入函数(entry function)"和"退出函数(exit function)"。它们有时可能被称为\cpp|init_module| 和 \cpp|cleanup_module| 这两个模块的含义是相同的。
|
||
|
||
\subsection{模块可用的功能}
|
||
\label{sec:avail_func}
|
||
程序员经常使用不是自己定义的函数。这方面的一个常见的例子是 \cpp|printf()|。在使用标准C库libc提供的这些库函数实际上直到链接阶段才会进入你的程序, 这确保代码(例如 \cpp|printf()|的代码)可用, 并将调用指令指向那段代码。
|
||
|
||
内核模块在这里也有所不同。在"Hello World"示例中, 你可能注意到我们使用了函数\cpp|pr_info()|, 但没有包括标准I/O库。这是因为模块是对象文件, 其符号在运行\sh|insmod| 或 \sh|modprobe|时得到解析。符号的定义来自内核本身；你只能使用内核提供的外部函数。如果你对内核已导出的符号感兴趣, 可以查看\verb|/proc/kallsyms|。
|
||
|
||
需要记住的一点是库函数和系统调用之间的区别。库函数是更上层的, 完全在用户空间中运行, 并为程序员提供了一个更方便的接口, 以实现真正的工作系统调用。
|
||
系统调用代表用户在内核模式下运行, 并由内核本身提供。库函数\cpp|printf()|可能看起来像一个非常通用的打印函数, 但它真正做的只是将数据格式化为字符串, 并使用底层系统调用\cpp|write()|写入字符串数据, 然后将数据发送到标准输出。
|
||
|
||
可以使用下述方法看看\cpp|printf()|都进行了那些系统调用。编译以下程序:
|
||
|
||
\begin{code}
|
||
#include <stdio.h>
|
||
|
||
int main(void)
|
||
{
|
||
printf("hello");
|
||
return 0;
|
||
}
|
||
\end{code}
|
||
|
||
使用 \sh|gcc -Wall -o hello hello.c|。使用 \sh|strace ./hello| 运行可执行文件。你看到的每一行都对应于一个系统调用。
|
||
\href{https://strace.io/}{strace} 是一个非常实用的程序, 它可以告诉你一个程序正在进行的系统调用的详细信息, 包括调用的是哪个系统函数、其参数是什么以及它返回了什么。
|
||
这是一个极其宝贵的工具, 用来了解诸如程序试图访问哪些文件之类的问题。在输出的最后, 你会看到像 \cpp|write(1, "hello", 5hello)| 这样的一行。这就是 \cpp|printf()| 函数背后的真实系统调用。你可能不熟悉 \cpp|write| , 因为大多数人会使用库函数来进行文件 I/O (如 \cpp|fopen|, \cpp|fputs|, \cpp|fclose|)。
|
||
如果想要了解更详细的信息可以查看 \sh|man 2 write|。第二部分的手册专门讨论系统调用(如 \cpp|kill()| 和 \cpp|read()|), 而第三部分的手册则专门讨论库调用, 你可能会更熟悉这部分(如 \cpp|cosh()| 和 \cpp|random()|)。
|
||
|
||
你甚至可以编写模块来替换内核的系统调用, 我们很快就会这样做。这种操作常被骇客用于制作后门或木马程序, 但你可以编写自己的模块来做无害的事情, 比如每当有人试图删除你系统上的文件时, 让内核输出"哎呀！你干～嘛？"
|
||
|
||
\subsection{用户空间vs内核空间}
|
||
\label{sec:user_kernl_space}
|
||
内核主要管理对资源的访问, 无论是显卡、硬盘还是内存。程序经常争夺相同的资源。例如, 在保存文档时, updatedb可能会开始更新locate数据库。vim等编辑器中的会话和updatedb等进程可以同时利用硬盘。内核的作用是维护秩序, 确保用户不会随意访问资源。
|
||
|
||
为了管理这些资源, CPU 在不同模式下运行, 每个模式都提供不同级别的系统控制。例如, 英特尔的80386架构就具有四种这样的模式, 称为环。然而, Unix仅使用其中两个环: 最高环(环0, 也称为"内核模式", 其中允许所有操作)和最低环, 称为"用户模式"。
|
||
|
||
回顾关于库函数与系统调用的讨论。通常会在用户模式下使用库函数。库函数调用一个或多个系统调用, 并且这些系统调用以库函数的名义执行, 但在内核模式下执行, 因为它们是内核本身的一部分。一旦系统调用完成其任务, 它将返回, 并将执行权转交回用户模式。
|
||
|
||
\subsection{命名空间}
|
||
\label{sec:namespace}
|
||
当你在编写一个小规模项目时, 使用通俗易懂的变量名通常是为了增加程序的可读性。但如果参与一个多人维护的庞大项目, 你的全局变量会和其他人的全局变量共存, 可能会出现全局变量名冲突的情况。当一个程序拥有大量全局变量且这些变量名不足以准确描述其用途时, 就会出现命名空间污染。在大型团队项目中, 在确保能记住保留的名称的同时, 需要找到一种制定方案, 以便为变量名和符号命名保持唯一性。
|
||
|
||
在编写内核代码时, 即使是最小的模块也会链接到整个内核, 因此需要避免命名空间污染这个问题。处理此问题的最佳方法是将所有变量声明为静态, 并使用一个精准不会重复的前缀。按照惯例, 所有内核前缀都是小写的。如果你不想将所有内容都声明为静态, 另一个方法是声明符号表并将其注册到内核。我们稍后将讨论这个问题。
|
||
|
||
文件 \verb|/proc/kallsyms|包含了内核所知道的所有符号, 因此这些符号对你的模块来说是可访问的, 因为它们共享了内核的代码空间。
|
||
|
||
\subsection{代码空间}
|
||
\label{sec:codespace}
|
||
内存管理是一个非常复杂的主题, O'Reilly 出版的 \href{https://www.oreilly.com/library/view/understanding-the-linux/0596005652/}{Understanding The Linux Kernel(理解 Linux 内核)}一书中大部分内容都专注于内存管理！我们的目标不是成为内存管理方面的专家, 但我们确实需要掌握一些基本知识, 才能开始着手编写实际的模块。
|
||
|
||
如果你没有深入思考过段错误(segfault)的真正含义, 你可能会惊讶地发现指针并不真正指向内存位置——至少不是真实的内存位置。当一个进程被创建时, 内核会划分一部分真实的物理内存并分配给该进程, 用于其执行代码、变量、栈、堆以及计算机科学家所熟知的其他内容。这部分内存从0x00000000开始, 延伸到所需的任何地址。由于任何两个进程的内存空间不会重叠, 任何可以访问某个内存地址(比如0xbffff978)的进程, 实际上访问的是物理内存中的不同位置！这些进程实际上访问的是名为0xbffff978的索引, 该索引指向为该特定进程预留的内存区域中的某个偏移量。大多数情况下, 像我们的"Hello, World"程序这样的进程不能访问其他进程的空间, 我们稍后会讨论一些可以做到这一点的方法。
|
||
|
||
内核也有自己的内存空间。由于模块是可以在内核中动态插入和删除的代码(与半独立对象相反), 因此它共享内核的代码空间, 而不是拥有自己的代码空间。因此, 如果模块段出错, 则内核段出错。如果你因为一个"边界错误(off-by-one)"开始覆写数据, 那么你就是在破坏内核数据(或代码)。这种情况比听起来的还要危险, 所以尽量小心处理。
|
||
|
||
译者注: "半独立"强调的是既有独立操作的能力, 同时又不完全脱离外部控制或支持的状态, 内核模块直接运行在内核空间, 与内核共享相同的内存和执行环境, 而半独立的对象通常在用户空间运行, 拥有自己独立的内存和资源管理, 这使得它们在出现错误时不会直接影响到内核的稳定性。
|
||
|
||
应当注意, 上述讨论适用于任何使用单块式内核(monolithic kernel)的操作系统。这个概念与"将所有模块构建到内核中"略有不同, 尽管底层原理类似。相反, 在微内核模块会分配自己的代码空间。微内核的著名的例子有\href{https://www.gnu.org/software/hurd/}{GNU Hurd}和谷歌Fuchsia的\href{https://fuchsia.dev/fuchsia-src/concepts/kernel}{锆石内核(Zircon kernel)}。
|
||
|
||
\subsection{设备驱动}
|
||
\label{sec:device_drivers}
|
||
有一类模块是设备驱动程序, 它为硬件(如串行端口)提供功能。在Unix上, 每个硬件都通过位于\verb|/dev|目录下的设备文件来表示, 这些文件提供了与硬件通信的手段。设备驱动程序代表用户程序提供通信。因此, es1370.ko声卡设备驱动程序可以将/dev/sound设备文件连接到Ensoniq ES1370声卡。像mp3blaster这样的用户空间程序可以使用/dev/sound, 而无需知道安装了何种声卡。
|
||
|
||
我们可以看一些设备文件。以下是表示串行端口(tty-teletypewriter)上的前三个端口的设备文件:
|
||
|
||
\begin{verbatim}
|
||
$ ls -l /dev/tty[1-3]
|
||
crw--w---- 1 root tty 4, 1 7月28日 15:44 /dev/tty1
|
||
crw--w---- 1 root tty 4, 2 7月28日 15:44 /dev/tty2
|
||
crw--w---- 1 root tty 4, 3 7月28日 15:44 /dev/tty3
|
||
\end{verbatim}
|
||
|
||
请注意日期前方由逗号分隔的数字列(4, 1 ; 4, 2 ; 4, 3)。第一个数字称为设备的主编号。第二个数字是次要数字。主编号告诉你使用哪个驱动程序访问硬件。每个驱动程序都有唯一的一个主编号码；主编号码相同的设备文件都由同一个驱动程序控制。上述所有主编号码都是 4, 因为它们都由同一个驱动程序控制。
|
||
|
||
次要编号由驱动程序用于区分其控制的各种硬件。回到上面的例子, 尽管所有三个设备都由相同的驱动程序处理, 但它们具有唯一的次要编号, 因为驱动程序将它们视为不同的硬件。
|
||
|
||
设备分为两种类型: 字符设备和块设备。区别在于块设备具有请求的缓冲区, 因此它们可以选择响应请求的最佳顺序。这一点对存储设备尤为重要, 因为读写相邻的扇区比读写距离较远的扇区更快。另一个区别是, 块设备只能按块接收输入和返回输出(块的大小可以根据设备而变), 而字符设备则可以按需要使用任意数量的字节。世界上大多数设备都是字符型的, 它们不需要这种缓冲不以固定的块大小运行。你可以通过查看\sh|ls -l|输出中的第一个字符来判断设备文件是用于块设备还是字符设备。如果它是"b", 则它是块设备, 如果它是c, 则它为字符设备。上面看到的设备是字符设备。以下是一些块设备(存储):
|
||
|
||
\begin{verbatim}
|
||
brw-rw---- 1 root disk 7, 0 7月28日 15:44 loop0
|
||
brw-rw---- 1 root disk 7, 1 7月28日 15:44 loop1
|
||
brw-rw---- 1 root disk 7, 2 7月28日 15:44 loop2
|
||
brw-rw---- 1 root disk 7, 3 7月28日 15:44 loop3
|
||
brw-rw---- 1 root disk 7, 4 7月28日 15:44 loop4
|
||
brw-rw---- 1 root disk 7, 5 7月28日 15:44 loop5
|
||
brw-rw---- 1 root disk 7, 6 7月28日 15:44 loop6
|
||
brw-rw---- 1 root disk 7, 7 7月28日 15:44 loop7
|
||
brw-rw---- 1 root disk 259, 0 7月28日 15:44 nvme0n1
|
||
brw-rw---- 1 root disk 259, 1 7月28日 15:44 nvme0n1p1
|
||
brw-rw---- 1 root disk 8, 0 7月28日 15:44 sda
|
||
brw-rw---- 1 root disk 8, 1 7月28日 15:44 sda1
|
||
brw-rw---- 1 root disk 8, 2 7月28日 15:44 sda2
|
||
\end{verbatim}
|
||
|
||
如果要查看已分配的主编号, 可以查看\src{Documentation/admin-guide/devices.txt}。
|
||
|
||
安装系统时, 所有设备文件都是通过\sh|mknod|命令创建的。要创建一个名为\verb|coffee|的新字符设备, 主为12和次为2, 只需执行\sh|mknod /dev/coffee c 12 2|。你不必将设备文件放入\verb|/dev|中, 但这是按照约定完成的。Linus将他的设备文件放在\verb|/dev|中, 你也应该这样。然而, 当为测试目的创建设备文件时, 将其放置在你编译内核模块的工作目录中通常是可以的。但完成设备驱动程序编写后, 确保将其放在正确的位置。
|
||
|
||
最后需要明确的是当访问设备文件时, 内核使用该文件的主编号来识别处理访问对应的驱动程序。这意味着内核不必依赖或了解次设备号。处理次设备号的是驱动程序本身, 它用次设备号来区分不同的硬件设备。
|
||
|
||
需要注意的是, 当提到"\emph{"硬件(hardware)"}"时, 该术语的使用比较抽象, 他指的不仅仅是PCI设备。看看以下两个设备文件:
|
||
|
||
\begin{verbatim}
|
||
ls -l /dev/nvme0n1* /dev/sda
|
||
brw-rw---- 1 root disk 259, 0 7月28日 15:44 /dev/nvme0n1
|
||
brw-rw---- 1 root disk 259, 1 7月28日 15:44 /dev/nvme0n1p1
|
||
\end{verbatim}
|
||
|
||
从这两个设备文件的信息你应该能看出它们是块设备, 并且他们是由相同的驱动程序处理(块, 主259)。但是上述具有相同主编号但不同次号的两个设备文件实际上是表示相同的物理硬件。所以请注意, 我们讨论中的"\emph{"硬件(hardware)"}"一词可能意味着一些抽象的东西。
|
||
|
||
\section{字符设备驱动程序}
|
||
\label{sec:chardev}
|
||
\subsection{文件操作结构}
|
||
\label{sec:file_operations}
|
||
\cpp|file_operations|结构在\src{include/linux/fs.h}中定义并保存指向驱动程序定义的函数的指针, 这些函数在设备上执行各种操作。结构的每个字段对应于驱动程序定义的某个函数的地址, 以处理请求的操作。
|
||
|
||
每个字符驱动程序都需要定义从设备读取的函数。\cpp|file_operations|结构保存执行该操作的模块函数的地址。下面是内核5.4的定义:
|
||
|
||
\begin{code}
|
||
struct file_operations {
|
||
struct module *owner;
|
||
loff_t (*llseek) (struct file *, loff_t, int);
|
||
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
|
||
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
|
||
ssize_t (*read_iter) (struct kiocb *, struct iov_iter *);
|
||
ssize_t (*write_iter) (struct kiocb *, struct iov_iter *);
|
||
int (*iopoll)(struct kiocb *kiocb, bool spin);
|
||
int (*iterate) (struct file *, struct dir_context *);
|
||
int (*iterate_shared) (struct file *, struct dir_context *);
|
||
__poll_t (*poll) (struct file *, struct poll_table_struct *);
|
||
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
|
||
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
|
||
int (*mmap) (struct file *, struct vm_area_struct *);
|
||
unsigned long mmap_supported_flags;
|
||
int (*open) (struct inode *, struct file *);
|
||
int (*flush) (struct file *, fl_owner_t id);
|
||
int (*release) (struct inode *, struct file *);
|
||
int (*fsync) (struct file *, loff_t, loff_t, int datasync);
|
||
int (*fasync) (int, struct file *, int);
|
||
int (*lock) (struct file *, int, struct file_lock *);
|
||
ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int);
|
||
unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
|
||
int (*check_flags)(int);
|
||
int (*flock) (struct file *, int, struct file_lock *);
|
||
ssize_t (*splice_write)(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int);
|
||
ssize_t (*splice_read)(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int);
|
||
int (*setlease)(struct file *, long, struct file_lock **, void **);
|
||
long (*fallocate)(struct file *file, int mode, loff_t offset,
|
||
loff_t len);
|
||
void (*show_fdinfo)(struct seq_file *m, struct file *f);
|
||
ssize_t (*copy_file_range)(struct file *, loff_t, struct file *,
|
||
loff_t, size_t, unsigned int);
|
||
loff_t (*remap_file_range)(struct file *file_in, loff_t pos_in,
|
||
struct file *file_out, loff_t pos_out,
|
||
loff_t len, unsigned int remap_flags);
|
||
int (*fadvise)(struct file *, loff_t, loff_t, int);
|
||
} __randomize_layout;
|
||
\end{code}
|
||
|
||
一些操作不是由驱动程序实现的。例如, 显卡的驱动程序不需要从目录结构中读取(\cpp|ssize\_t (*read) (struct file *, char \_\_user *, size\_t, loff\_t *);|)。\cpp|file\_operations|结构中的相应条目应设置为\cpp|NULL|。
|
||
|
||
有一个GCC扩展使分配到此结构更方便。你会在现代的驱动程序中看到它, 这可能会让你感到意外。下面是这种新方式对结构赋值的样子:
|
||
|
||
\begin{code}
|
||
struct file_operations fops = {
|
||
read: device_read,
|
||
write: device_write,
|
||
open: device_open,
|
||
release: device_release
|
||
};
|
||
\end{code}
|
||
|
||
但也有一种更常用基于C99标准的方式来分配结构元素, 即\href{https://gcc.gnu.org/onlinedocs/gcc/Designated-Inits.html}{指定初始化器(designated initializers)}, 与使用GNU扩展相比这种基于C99的分配方法适用性更加广泛。使用此语法有助于增加驱动的兼容性:
|
||
|
||
\begin{code}
|
||
struct file_operations fops = {
|
||
.read = device_read,
|
||
.write = device_write,
|
||
.open = device_open,
|
||
.release = device_release
|
||
};
|
||
\end{code}
|
||
|
||
没有显式分配的结构的任何成员都将由GCC初始化为 \cpp|NULL|。
|
||
|
||
结构 \cpp|struct file\_operations| 的实例包含指向用于实现 \cpp|读取(read)|, \cpp|写入(write)|, \cpp|打开(open)|, \ldots{}系统调用的函数的指针, 通常称为\cpp|fops|。
|
||
|
||
从 Linux v3.14 开始, 通过使用特定的 \cpp|f\_pos| 锁, 读、写和寻址操作保证了线程安全, 这使得文件位置的更新变成了互斥操作。因此, 我们可以在不需要额外加锁的情况下安全地实现这些操作。
|
||
|
||
此外, 从Linux v5.6开始, 引入了 \cpp|proc\_ops| 结构, 以取代在注册proc处理程序时使用的 \cpp|file\_operations|结构。请参阅\ref{sec:proc_ops}节中的更多信息。
|
||
|
||
\subsection{文件结构}
|
||
\label{sec:file_struct}
|
||
|
||
每个设备在内核中由一个文件结构表示, 该文件结构在 \src{include/linux/fs.h}中定义。这里的文件是内核级结构, 从不出现在用户空间程序中。他与我们常见的由\cpp|glibc|定义\cpp|FILE|是不同的, 由\cpp|glibc|定义的\cpp|FILE|不会出现在内核空间函数中。内核这里的文件(file)命名会产生一些误导；它表示一个抽象的打开"文件(file)", 而不是磁盘上的文件, 该文件由一个名为\cpp|inode|的结构表示。
|
||
|
||
内核中的结构体 \cpp|文件(file)| 实例通常被称为 filp。你也会看到它被称为 \cpp|结构体文件对象(struct file object)|。不要随意更改这种叫法。
|
||
|
||
现在我们来看一下"文件(file)"的定义。你会看到许多条目, 例如"结构目录项(struct dentry)", 这些通常不被设备驱动程序使用, 因此你可以忽略它们。这是因为驱动程序并不直接填充"文件(file)"; 它们仅使用文件中的结构体, 这些结构体是在其他地方创建的。
|
||
|
||
\subsection{注册设备}
|
||
\label{sec:register_device}
|
||
如之前所讨论的, 按照约定俗成的做法字符设备通过位于 \verb|/dev| 目录下的设备文件来访问。在编写驱动程序时, 将设备文件放在当前目录是可以的。但请确保在生产环境中将其放置在 \verb|/dev| 目录。主编号用于标识哪个驱动程序处理哪个设备文件。次号仅由驱动程序本身使用, 在一个驱动程序处理多个设备时区分它正在操作的设备。
|
||
|
||
向系统中添加驱动程序意味着向内核注册它。这等同于在模块初始化期间为其分配主编号。你可以通过使用 \cpp|register_chrdev| 函数来实现这一点, 该函数由 \src{include/linux/fs.h} 定义。
|
||
|
||
\begin{code}
|
||
int register_chrdev(unsigned int major, const char *name, struct file_operations *fops);
|
||
\end{code}
|
||
其中, \cpp|unsigned int major|是要请求的主编号, \cpp|const char *name|是设备的名称, 它将出现在\verb|/proc/devices|中, \cpp|struct file_operations *fops| 是指向驱动程序的 \cpp|file_operations|表的指针。负返回值表示注册失败。我们没有将次号传递给\cpp|register_chrdev|。因为内核不关心次号, 只有我们的驱动使用它。
|
||
|
||
现在的问题是, 如何获得一个不占用已经在使用中的主编号？最简单的方法是查看 \src{Documentation/admin-guide/devices.txt} 文件选择一个未被使用的编号。但这种做法并不理想, 因为你无法确保你选择的号码以后不会被分配。正确的做法是请求内核为你分配一个动态的主编号。
|
||
|
||
如果传递给 \cpp|register_chrdev|的值是0, 返回值将是动态分配的主编号。不过, 这样做的缺点是你无法提前创建设备文件, 因为你不知道将会获得什么主编号。处理这种情况有几种方法: 驱动程序本身可以打印新分配的编号, 我们可以手动制作设备文件; 新注册的设备将在\verb|/proc/devices|中有一个条目, 我们可以手动创建设备文件, 也可以编写一个shell脚本来读取文件并创建设备文件; 还有一种方法是, 我们可以让驱动程序在成功注册后使用 \cpp|device_create|函数生成设备文件, 并在调用 \cpp|cleanup_module|期间使用\cpp|device_destroy|。
|
||
|
||
\cpp|register_chrdev()| 函数会占用与指定主编号相关联的一系列次设备号。为了减少字符设备注册时的资源浪费, 推荐使用 cdev 接口。
|
||
|
||
新接口通过两个步骤完成字符设备的注册。首先注册一系列设备号, 可以用\cpp|register_chrdev_region| 或 \cpp|alloc_chrdev_region|来完成。
|
||
|
||
\begin{code}
|
||
int register_chrdev_region(dev_t from, unsigned count, const char *name);
|
||
int alloc_chrdev_region(dev_t *dev, unsigned baseminor, unsigned count, const char *name);
|
||
\end{code}
|
||
|
||
两种不同功能之间的选择取决于你是否知道设备的主编号。如果知道设备主编号, 则使用\cpp|register_chrdev_region|；如果要分配动态分配的主编号, 请使用\cpp|alloc_chrdev_region|。
|
||
|
||
其次, 我们应该初始化字符驱动的数据结构\cpp|struct cdev|并将其与设备号相关联。初始化 \cpp|struct cdev| 的代码序列可以参照以下示例来实现。
|
||
|
||
\begin{code}
|
||
struct cdev *my_dev = cdev_alloc();
|
||
my_cdev->ops = &my_fops;
|
||
\end{code}
|
||
|
||
通常的做法是将 \cpp|struct cdev| 嵌入到设备特定的结构中。在这种情况下, 我们需要使用 \cpp|cdev_init| 来进行初始化。
|
||
|
||
\begin{code}
|
||
void cdev_init(struct cdev *cdev, const struct file_operations *fops);
|
||
\end{code}
|
||
|
||
一旦完成初始化, 就可以使用\cpp|cdev_add|将字符驱动添加到系统中。
|
||
|
||
\begin{code}
|
||
int cdev_add(struct cdev *p, dev_t dev, unsigned count);
|
||
\end{code}
|
||
|
||
要查找使用该接口的示例, 可以参阅第\ref{sec:device_files}节中描述的\verb|ioctl.c|。
|
||
|
||
\subsection{注销设备}
|
||
\label{sec:unregister_device}
|
||
我们不能让内核模块在root的情况下随意使用 \sh|rmmod| 删除模块。如果有一个进程打开了设备文件, 然后我们移除了内核模块, 再使用该文件将会导致调用已删除函数(如读/写)的内存位置。如果幸运的话, 那里没有加载其他代码, 我们将收到一条错误消息。如果我们运气不好, 另一个内核模块被加载到相同的位置, 这意味着跳转到内核中另一个函数的中间。这种情况的结果是无法预测的, 但肯定不会有什么好结果。
|
||
|
||
当你不想允许某些操作时, 应该从尝试执行该操作的函数返回一个错误代码(一个负数)。但 \cpp|cleanup_module| 是一个无返回值的函数。然而, 有一个计数器可以跟踪有多少进程正在使用你的模块。你可以通过执行命令 \sh|cat /proc/modules| 或 \sh|sudo lsmod| 查看该计数器的值, 它显示在第三个字段。该数字不为零的时候\sh|rmmod|将执行失败。需要注意的是你不必检查\cpp|cleanup_module|这个计数器, 因为检查将由在\src{include/linux/syscalls.h}中定义的系统调用\cpp|sys_delete_module|中执行。虽然不建议直接使用这个计数器, 但在 \src{include/linux/module.h} 中定义了一些函数, 允许你增加、减少和显示这个计数器的值：
|
||
|
||
\begin{itemize}
|
||
\item \cpp|try_module_get(THIS_MODULE)|: 增加当前模块的引用计数。
|
||
\item \cpp|module_put(THIS_MODULE)|: 减少当前模块的引用计数。
|
||
\item \cpp|module_refcount(THIS_MODULE)|: 返回当前模块的引用计数的值。
|
||
\end{itemize}
|
||
|
||
保持计数器的准确性是很重要的时期, 如果你让计数器数值出现了异常可能会导致模块永远无法被卸载了, 朋友如果你碰到这种情况了, 那只能靠重启来解决问题了。不过在模块开发过程中, 这种情况迟早会发生。
|
||
|
||
\subsection{chardev.c}
|
||
\label{sec:chardev_c}
|
||
下一个代码示例创建一个名为\verb|chardev|的字符驱动程序。你可以转储其设备文件。
|
||
|
||
\begin{codebash}
|
||
cat /proc/devices
|
||
\end{codebash}
|
||
|
||
驱动程序会将设备文件被读取(或用程序打开文件)的次数记录到文件中。这里不支持写入文件(如 \sh|echo "hi" > /dev/hello|), 但会捕获这些尝试并告诉用户该操作不被支持。不用担心为什么你没有看到这里对读入缓冲区的数据进行了什么操作，因为并没有什么处理。这里只是简单地读入数据后打印一条消息来确认我们收到了它。
|
||
|
||
在多线程环境中, 如果没有任何保护, 对同一内存的并发访问可能会导致争用情况, 并且会影响性能。在内核模块中, 这个问题可能由于多个实例访问共享资源而发生。因此, 一种解决方案是强制独占访问。我们使用原子性的 CAS(比较和交换, Compare-And-Swap) 来维护状态\cpp|CDEV\_NOT\_USED| 和 \cpp|CDEV\_EXCLUSIVE\_OPEN| 以确定文件当前是否被打开。CAS将存储器位置的内容与期望值进行比较, 并且仅当它们相同时, 才将该存储器位置的内容修改为期望值。有关并发的更多细节，请参阅 \ref{sec:synchronization} 部分。
|
||
|
||
\samplec{examples/chardev.c}
|
||
|
||
\subsection{为多个内核版本编写模块}
|
||
\label{sec:modules_for_versions}
|
||
系统调用是进程调用内核的主要接口，在不同版本中通常保持不变。在添加新的系统调用时往往会保证旧的系统调用不会发生变化，这对于系统的向后兼容性是非常重要的。内核新版本不会破坏旧进程的正常运行。在大多数情况下，设备文件也应当保持不变。另一方面，内核内部的接口在不同版本之间会存在一些变化。
|
||
|
||
不同的内核版本之间存在差异, 如果想支持多个内核版本, 则需要编写条件编译指令。实现这一点的方法是比较 \cpp|LINUX_VERSION_CODE| 与 \cpp|KERNEL_VERSION| 这两个宏。在内核的a.b.c版本中, 该宏的值为\(2^{16}a+2^{8}b+c\)
|
||
|
||
\section{The /proc File System}
|
||
\label{sec:procfs}
|
||
In Linux, there is an additional mechanism for the kernel and kernel modules to send information to processes --- the \verb|/proc| file system.
|
||
Originally designed to allow easy access to information about processes (hence the name), it is now used by every bit of the kernel which has something interesting to report, such as \verb|/proc/modules| which provides the list of modules and \verb|/proc/meminfo| which gathers memory usage statistics.
|
||
|
||
The method to use the proc file system is very similar to the one used with device drivers --- a structure is created with all the information needed for the \verb|/proc| file, including pointers to any handler functions (in our case there is only one, the one called when somebody attempts to read from the \verb|/proc| file).
|
||
Then, \cpp|init_module| registers the structure with the kernel and \cpp|cleanup_module| unregisters it.
|
||
|
||
Normal file systems are located on a disk, rather than just in memory (which is where \verb|/proc| is), and in that case the index-node (inode for short) number is a pointer to a disk location where the file's inode is located.
|
||
The inode contains information about the file, for example the file's permissions, together with a pointer to the disk location or locations where the file's data can be found.
|
||
|
||
Because we don't get called when the file is opened or closed, there's nowhere for us to put \cpp|try_module_get| and \cpp|module_put| in this module, and if the file is opened and then the module is removed, there's no way to avoid the consequences.
|
||
|
||
Here a simple example showing how to use a \verb|/proc| file.
|
||
This is the HelloWorld for the \verb|/proc| filesystem.
|
||
There are three parts: create the file \verb|/proc/helloworld| in the function \cpp|init_module|, return a value (and a buffer) when the file \verb|/proc/helloworld| is read in the callback function \cpp|procfile_read|, and delete the file \verb|/proc/helloworld| in the function \cpp|cleanup_module|.
|
||
|
||
The \verb|/proc/helloworld| is created when the module is loaded with the function \cpp|proc_create|.
|
||
The return value is a pointer to \cpp|struct proc_dir_entry|, and it will be used to configure the file \verb|/proc/helloworld| (for example, the owner of this file).
|
||
A null return value means that the creation has failed.
|
||
|
||
Every time the file \verb|/proc/helloworld| is read, the function \cpp|procfile_read| is called.
|
||
Two parameters of this function are very important: the buffer (the second parameter) and the offset (the fourth one).
|
||
The content of the buffer will be returned to the application which read it (for example the \sh|cat| command).
|
||
The offset is the current position in the file.
|
||
If the return value of the function is not null, then this function is called again.
|
||
So be careful with this function, if it never returns zero, the read function is called endlessly.
|
||
|
||
\begin{verbatim}
|
||
$ cat /proc/helloworld
|
||
HelloWorld!
|
||
\end{verbatim}
|
||
|
||
\samplec{examples/procfs1.c}
|
||
|
||
\subsection{The proc\_ops Structure}
|
||
\label{sec:proc_ops}
|
||
The \cpp|proc_ops| structure is defined in \src{include/linux/proc\_fs.h} in Linux v5.6+.
|
||
In older kernels, it used \cpp|file_operations| for custom hooks in \verb|/proc| file system, but it contains some members that are unnecessary in VFS, and every time VFS expands \cpp|file_operations| set, \verb|/proc| code comes bloated.
|
||
On the other hand, not only the space, but also some operations were saved by this structure to improve its performance.
|
||
For example, the file which never disappears in \verb|/proc| can set the \cpp|proc_flag| as \cpp|PROC_ENTRY_PERMANENT| to save 2 atomic ops, 1 allocation, 1 free in per open/read/close sequence.
|
||
|
||
\subsection{Read and Write a /proc File}
|
||
\label{sec:read_write_procfs}
|
||
We have seen a very simple example for a \verb|/proc| file where we only read the file \verb|/proc/helloworld|.
|
||
It is also possible to write in a \verb|/proc| file.
|
||
It works the same way as read, a function is called when the \verb|/proc| file is written.
|
||
But there is a little difference with read, data comes from user, so you have to import data from user space to kernel space (with \cpp|copy_from_user| or \cpp|get_user|)
|
||
|
||
The reason for \cpp|copy_from_user| or \cpp|get_user| is that Linux memory (on Intel architecture, it may be different under some other processors) is segmented.
|
||
This means that a pointer, by itself, does not reference a unique location in memory, only a location in a memory segment, and you need to know which memory segment it is to be able to use it.
|
||
There is one memory segment for the kernel, and one for each of the processes.
|
||
|
||
The only memory segment accessible to a process is its own, so when writing regular programs to run as processes, there is no need to worry about segments.
|
||
When you write a kernel module, normally you want to access the kernel memory segment, which is handled automatically by the system.
|
||
However, when the content of a memory buffer needs to be passed between the currently running process and the kernel, the kernel function receives a pointer to the memory buffer which is in the process segment.
|
||
The \cpp|put_user| and \cpp|get_user| macros allow you to access that memory.
|
||
These functions handle only one character, you can handle several characters with \cpp|copy_to_user| and \cpp|copy_from_user|.
|
||
As the buffer (in read or write function) is in kernel space, for write function you need to import data because it comes from user space, but not for the read function because data is already in kernel space.
|
||
|
||
\samplec{examples/procfs2.c}
|
||
|
||
\subsection{Manage /proc file with standard filesystem}
|
||
\label{sec:manage_procfs}
|
||
We have seen how to read and write a \verb|/proc| file with the \verb|/proc| interface.
|
||
But it is also possible to manage \verb|/proc| file with inodes.
|
||
The main concern is to use advanced functions, like permissions.
|
||
|
||
In Linux, there is a standard mechanism for file system registration.
|
||
Since every file system has to have its own functions to handle inode and file operations, there is a special structure to hold pointers to all those functions, \cpp|struct inode_operations|, which includes a pointer to \cpp|struct proc_ops|.
|
||
|
||
The difference between file and inode operations is that file operations deal with the file itself whereas inode operations deal with ways of referencing the file, such as creating links to it.
|
||
|
||
In \verb|/proc|, whenever we register a new file, we're allowed to specify which \cpp|struct inode_operations| will be used to access to it.
|
||
This is the mechanism we use, a \cpp|struct inode_operations| which includes a pointer to a \cpp|struct proc_ops| which includes pointers to our \cpp|procfs_read| and \cpp|procfs_write| functions.
|
||
|
||
Another interesting point here is the \cpp|module_permission| function.
|
||
This function is called whenever a process tries to do something with the \verb|/proc| file, and it can decide whether to allow access or not.
|
||
Right now it is only based on the operation and the uid of the current user (as available in current, a pointer to a structure which includes information on the currently running process), but it could be based on anything we like, such as what other processes are doing with the same file, the time of day, or the last input we received.
|
||
|
||
It is important to note that the standard roles of read and write are reversed in the kernel.
|
||
Read functions are used for output, whereas write functions are used for input.
|
||
The reason for that is that read and write refer to the user's point of view --- if a process reads something from the kernel, then the kernel needs to output it, and if a process writes something to the kernel, then the kernel receives it as input.
|
||
|
||
\samplec{examples/procfs3.c}
|
||
|
||
Still hungry for procfs examples?
|
||
Well, first of all keep in mind, there are rumors around, claiming that procfs is on its way out, consider using \verb|sysfs| instead.
|
||
Consider using this mechanism, in case you want to document something kernel related yourself.
|
||
|
||
\subsection{Manage /proc file with seq\_file}
|
||
\label{sec:manage_procfs_with_seq_file}
|
||
As we have seen, writing a \verb|/proc| file may be quite ``complex''.
|
||
So to help people writing \verb|/proc| file, there is an API named \cpp|seq_file| that helps formatting a \verb|/proc| file for output.
|
||
It is based on sequence, which is composed of 3 functions: \cpp|start()|, \cpp|next()|, and \cpp|stop()|.
|
||
The \cpp|seq_file| API starts a sequence when a user read the \verb|/proc| file.
|
||
|
||
A sequence begins with the call of the function \cpp|start()|.
|
||
If the return is a non \cpp|NULL| value, the function \cpp|next()| is called; otherwise, the \cpp|stop()| function is called directly.
|
||
This function is an iterator, the goal is to go through all the data.
|
||
Each time \cpp|next()| is called, the function \cpp|show()| is also called.
|
||
It writes data values in the buffer read by the user.
|
||
The function \cpp|next()| is called until it returns \cpp|NULL|.
|
||
The sequence ends when \cpp|next()| returns \cpp|NULL|, then the function \cpp|stop()| is called.
|
||
|
||
BE CAREFUL: when a sequence is finished, another one starts.
|
||
That means that at the end of function \cpp|stop()|, the function \cpp|start()| is called again.
|
||
This loop finishes when the function \cpp|start()| returns \cpp|NULL|.
|
||
You can see a scheme of this in the Figure~\ref{img:seqfile}.
|
||
|
||
\begin{figure}[h]
|
||
\center
|
||
\begin{tikzpicture}[node distance=2cm, thick]
|
||
\node (start) [startstop] {start() treatment};
|
||
\node (branch1) [decision, below of=start, yshift=-1cm] {return is NULL?};
|
||
\node (next) [process, below of=branch1, yshift=-1cm] {next() treatment};
|
||
\node (branch2) [decision, below of=next, yshift=-1cm] {return is NULL?};
|
||
\node (stop) [startstop, below of=branch2, yshift=-1cm] {stop() treatment};
|
||
|
||
\draw [->] (start) -- (branch1);
|
||
\draw [->] (branch1.east) to [out=135, in=-135, bend left=45] node [right] {Yes} (stop.east);
|
||
\draw [->] (branch1) -- node[left=2em, anchor=south] {No} (next);
|
||
\draw [->] (next) -- (branch2);
|
||
\draw [->] (branch2.west) to [out=135, in=-135, bend left=45] node [left] {No} (next.west);
|
||
\draw [->] (branch2) -- node[left=2em, anchor=south] {Yes} (stop);
|
||
\draw [->] (stop.west) to [out=135, in=-135] node [left] {} (start.west);
|
||
\end{tikzpicture}
|
||
\caption{How seq\_file works}
|
||
\label{img:seqfile}
|
||
\end{figure}
|
||
|
||
The \cpp|seq_file| provides basic functions for \cpp|proc_ops|, such as \cpp|seq_read|, \cpp|seq_lseek|, and some others.
|
||
But nothing to write in the \verb|/proc| file.
|
||
Of course, you can still use the same way as in the previous example.
|
||
|
||
\samplec{examples/procfs4.c}
|
||
|
||
If you want more information, you can read this web page:
|
||
|
||
\begin{itemize}
|
||
\item \url{https://lwn.net/Articles/22355/}
|
||
\item \url{https://kernelnewbies.org/Documents/SeqFileHowTo}
|
||
\end{itemize}
|
||
|
||
You can also read the code of \src{fs/seq\_file.c} in the linux kernel.
|
||
|
||
\section{sysfs: Interacting with your module}
|
||
\label{sec:sysfs}
|
||
\emph{sysfs} allows you to interact with the running kernel from userspace by reading or setting variables inside of modules.
|
||
This can be useful for debugging purposes, or just as an interface for applications or scripts.
|
||
You can find sysfs directories and files under the \verb|/sys| directory on your system.
|
||
|
||
\begin{codebash}
|
||
ls -l /sys
|
||
\end{codebash}
|
||
|
||
Attributes can be exported for kobjects in the form of regular files in the filesystem.
|
||
Sysfs forwards file I/O operations to methods defined for the attributes, providing a means to read and write kernel attributes.
|
||
|
||
An attribute definition in simply:
|
||
|
||
\begin{code}
|
||
struct attribute {
|
||
char *name;
|
||
struct module *owner;
|
||
umode_t mode;
|
||
};
|
||
|
||
int sysfs_create_file(struct kobject * kobj, const struct attribute * attr);
|
||
void sysfs_remove_file(struct kobject * kobj, const struct attribute * attr);
|
||
\end{code}
|
||
|
||
For example, the driver model defines \cpp|struct device_attribute| like:
|
||
|
||
\begin{code}
|
||
struct device_attribute {
|
||
struct attribute attr;
|
||
ssize_t (*show)(struct device *dev, struct device_attribute *attr,
|
||
char *buf);
|
||
ssize_t (*store)(struct device *dev, struct device_attribute *attr,
|
||
const char *buf, size_t count);
|
||
};
|
||
|
||
int device_create_file(struct device *, const struct device_attribute *);
|
||
void device_remove_file(struct device *, const struct device_attribute *);
|
||
\end{code}
|
||
|
||
To read or write attributes, \cpp|show()| or \cpp|store()| method must be specified when declaring the attribute.
|
||
For the common cases \src{include/linux/sysfs.h} provides convenience macros (\cpp|__ATTR|, \cpp|__ATTR_RO|, \cpp|__ATTR_WO|, etc.) to make defining attributes easier as well as making code more concise and readable.
|
||
|
||
An example of a hello world module which includes the creation of a variable accessible via sysfs is given below.
|
||
|
||
\samplec{examples/hello-sysfs.c}
|
||
|
||
Make and install the module:
|
||
|
||
\begin{codebash}
|
||
make
|
||
sudo insmod hello-sysfs.ko
|
||
\end{codebash}
|
||
|
||
Check that it exists:
|
||
|
||
\begin{codebash}
|
||
sudo lsmod | grep hello_sysfs
|
||
\end{codebash}
|
||
|
||
What is the current value of \cpp|myvariable| ?
|
||
|
||
\begin{codebash}
|
||
sudo cat /sys/kernel/mymodule/myvariable
|
||
\end{codebash}
|
||
|
||
Set the value of \cpp|myvariable| and check that it changed.
|
||
|
||
\begin{codebash}
|
||
echo "32" | sudo tee /sys/kernel/mymodule/myvariable
|
||
sudo cat /sys/kernel/mymodule/myvariable
|
||
\end{codebash}
|
||
|
||
Finally, remove the test module:
|
||
|
||
\begin{codebash}
|
||
sudo rmmod hello_sysfs
|
||
\end{codebash}
|
||
|
||
In the above case, we use a simple kobject to create a directory under sysfs, and communicate with its attributes.
|
||
Since Linux v2.6.0, the \cpp|kobject| structure made its appearance.
|
||
It was initially meant as a simple way of unifying kernel code which manages reference counted objects.
|
||
After a bit of mission creep, it is now the glue that holds much of the device model and its sysfs interface together.
|
||
For more information about kobject and sysfs, see \src{Documentation/driver-api/driver-model/driver.rst} and \url{https://lwn.net/Articles/51437/}.
|
||
|
||
\section{Talking To Device Files}
|
||
\label{sec:device_files}
|
||
Device files are supposed to represent physical devices.
|
||
Most physical devices are used for output as well as input, so there has to be some mechanism for device drivers in the kernel to get the output to send to the device from processes.
|
||
This is done by opening the device file for output and writing to it, just like writing to a file.
|
||
In the following example, this is implemented by \cpp|device_write|.
|
||
|
||
This is not always enough.
|
||
Imagine you had a serial port connected to a modem (even if you have an internal modem, it is still implemented from the CPU's perspective as a serial port connected to a modem, so you don't have to tax your imagination too hard).
|
||
The natural thing to do would be to use the device file to write things to the modem (either modem commands or data to be sent through the phone line) and read things from the modem (either responses for commands or the data received through the phone line).
|
||
However, this leaves open the question of what to do when you need to talk to the serial port itself, for example to configure the rate at which data is sent and received.
|
||
|
||
The answer in Unix is to use a special function called \cpp|ioctl| (short for Input Output ConTroL).
|
||
Every device can have its own \cpp|ioctl| commands, which can be read ioctl's (to send information from a process to the kernel), write ioctl's (to return information to a process), both or neither.
|
||
Notice here the roles of read and write are reversed again, so in ioctl's read is to send information to the kernel and write is to receive information from the kernel.
|
||
|
||
The ioctl function is called with three parameters: the file descriptor of the appropriate device file, the ioctl number, and a parameter, which is of type long so you can use a cast to use it to pass anything.
|
||
You will not be able to pass a structure this way, but you will be able to pass a pointer to the structure.
|
||
Here is an example:
|
||
|
||
\samplec{examples/ioctl.c}
|
||
|
||
You can see there is an argument called \cpp|cmd| in \cpp|test_ioctl_ioctl()| function.
|
||
It is the ioctl number.
|
||
The ioctl number encodes the major device number, the type of the ioctl, the command, and the type of the parameter.
|
||
This ioctl number is usually created by a macro call (\cpp|_IO|, \cpp|_IOR|, \cpp|_IOW| or \cpp|_IOWR| --- depending on the type) in a header file.
|
||
This header file should then be included both by the programs which will use ioctl (so they can generate the appropriate ioctl's) and by the kernel module (so it can understand it).
|
||
In the example below, the header file is \verb|chardev.h| and the program which uses it is \verb|userspace_ioctl.c|.
|
||
|
||
If you want to use ioctls in your own kernel modules, it is best to receive an official ioctl assignment, so if you accidentally get somebody else's ioctls, or if they get yours, you'll know something is wrong.
|
||
For more information, consult the kernel source tree at \src{Documentation/userspace-api/ioctl/ioctl-number.rst}.
|
||
|
||
Also, we need to be careful that concurrent access to the shared resources will lead to the race condition.
|
||
The solution is using atomic Compare-And-Swap (CAS), which we mentioned at \ref{sec:chardev_c} section, to enforce the exclusive access.
|
||
|
||
\samplec{examples/chardev2.c}
|
||
|
||
\samplec{examples/chardev.h}
|
||
|
||
\samplec{examples/other/userspace_ioctl.c}
|
||
|
||
\section{System Calls}
|
||
\label{sec:syscall}
|
||
So far, the only thing we've done was to use well defined kernel mechanisms to register \verb|/proc| files and device handlers.
|
||
This is fine if you want to do something the kernel programmers thought you'd want, such as write a device driver.
|
||
But what if you want to do something unusual, to change the behavior of the system in some way?
|
||
Then, you are mostly on your own.
|
||
|
||
Should one choose not to use a virtual machine, kernel programming can become risky.
|
||
For example, while writing the code below, the \cpp|open()| system call was inadvertently disrupted.
|
||
This resulted in an inability to open any files, run programs, or shut down the system, necessitating a restart of the virtual machine.
|
||
Fortunately, no critical files were lost in this instance.
|
||
However, if such modifications were made on a live, mission-critical system, the consequences could be severe.
|
||
To mitigate the risk of file loss, even in a test environment, it is advised to execute \sh|sync| right before using \sh|insmod| and \sh|rmmod|.
|
||
|
||
Forget about \verb|/proc| files, forget about device files.
|
||
They are just minor details.
|
||
Minutiae in the vast expanse of the universe.
|
||
The real process to kernel communication mechanism, the one used by all processes, is \emph{system calls}.
|
||
When a process requests a service from the kernel (such as opening a file, forking to a new process, or requesting more memory), this is the mechanism used.
|
||
If you want to change the behaviour of the kernel in interesting ways, this is the place to do it.
|
||
By the way, if you want to see which system calls a program uses, run \sh|strace <arguments>|.
|
||
|
||
In general, a process is not supposed to be able to access the kernel.
|
||
It can not access kernel memory and it can't call kernel functions.
|
||
The hardware of the CPU enforces this (that is the reason why it is called ``protected mode'' or ``page protection'').
|
||
|
||
System calls are an exception to this general rule.
|
||
What happens is that the process fills the registers with the appropriate values and then calls a special instruction which jumps to a previously defined location in the kernel (of course, that location is readable by user processes, it is not writable by them).
|
||
Under Intel CPUs, this is done by means of interrupt 0x80. The hardware knows that once you jump to this location, you are no longer running in restricted user mode, but as the operating system kernel --- and therefore you're allowed to do whatever you want.
|
||
|
||
% FIXME: recent kernel changes the system call entries
|
||
The location in the kernel a process can jump to is called \verb|system_call|.
|
||
The procedure at that location checks the system call number, which tells the kernel what service the process requested.
|
||
Then, it looks at the table of system calls (\cpp|sys_call_table|) to see the address of the kernel function to call.
|
||
Then it calls the function, and after it returns, does a few system checks and then return back to the process (or to a different process, if the process time ran out).
|
||
If you want to read this code, it is at the source file \verb|arch/$(architecture)/kernel/entry.S|, after the line \cpp|ENTRY(system_call)|.
|
||
|
||
So, if we want to change the way a certain system call works, what we need to do is to write our own function to implement it (usually by adding a bit of our own code, and then calling the original function) and then change the pointer at \cpp|sys_call_table| to point to our function.
|
||
Because we might be removed later and we don't want to leave the system in an unstable state, it's important for \cpp|cleanup_module| to restore the table to its original state.
|
||
|
||
To modify the content of \cpp|sys_call_table|, we need to consider the control register.
|
||
A control register is a processor register that changes or controls the general behavior of the CPU.
|
||
For x86 architecture, the \verb|cr0| register has various control flags that modify the basic operation of the processor.
|
||
The \verb|WP| flag in \verb|cr0| stands for write protection.
|
||
Once the \verb|WP| flag is set, the processor disallows further write attempts to the read-only sections
|
||
Therefore, we must disable the \verb|WP| flag before modifying \cpp|sys_call_table|.
|
||
Since Linux v5.3, the \cpp|write_cr0| function cannot be used because of the sensitive \verb|cr0| bits pinned by the security issue, the attacker may write into CPU control registers to disable CPU protections like write protection.
|
||
As a result, we have to provide the custom assembly routine to bypass it.
|
||
|
||
However, \cpp|sys_call_table| symbol is unexported to prevent misuse.
|
||
But there have few ways to get the symbol, manual symbol lookup and \cpp|kallsyms_lookup_name|.
|
||
Here we use both depend on the kernel version.
|
||
|
||
Because of the \textit{control-flow integrity}, which is a technique to prevent the redirect execution code from the attacker, for making sure that the indirect calls go to the expected addresses and the return addresses are not changed.
|
||
Since Linux v5.7, the kernel patched the series of \textit{control-flow enforcement} (CET) for x86,
|
||
and some configurations of GCC, like GCC versions 9 and 10 in Ubuntu Linux,
|
||
will add with CET (the \verb|-fcf-protection| option) in the kernel by default.
|
||
Using that GCC to compile the kernel with retpoline off may result in CET being enabled in the kernel.
|
||
You can use the following command to check out the \verb|-fcf-protection| option is enabled or not:
|
||
\begin{verbatim}
|
||
$ gcc -v -Q -O2 --help=target | grep protection
|
||
Using built-in specs.
|
||
COLLECT_GCC=gcc
|
||
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
|
||
...
|
||
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
|
||
COLLECT_GCC_OPTIONS='-v' '-Q' '-O2' '--help=target' '-mtune=generic' '-march=x86-64'
|
||
/usr/lib/gcc/x86_64-linux-gnu/9/cc1 -v ... -fcf-protection ...
|
||
GNU C17 (Ubuntu 9.3.0-17ubuntu1~20.04) version 9.3.0 (x86_64-linux-gnu)
|
||
...
|
||
\end{verbatim}
|
||
But CET should not be enabled in the kernel, it may break the Kprobes and bpf.
|
||
Consequently, CET is disabled since v5.11.
|
||
To guarantee the manual symbol lookup worked, we only use up to v5.4.
|
||
|
||
Unfortunately, since Linux v5.7 \cpp|kallsyms_lookup_name| is also unexported, it needs certain trick to get the address of \cpp|kallsyms_lookup_name|.
|
||
If \cpp|CONFIG_KPROBES| is enabled, we can facilitate the retrieval of function addresses by means of Kprobes to dynamically break into the specific kernel routine.
|
||
Kprobes inserts a breakpoint at the entry of function by replacing the first bytes of the probed instruction.
|
||
When a CPU hits the breakpoint, registers are stored, and the control will pass to Kprobes.
|
||
It passes the addresses of the saved registers and the Kprobe struct to the handler you defined, then executes it.
|
||
Kprobes can be registered by symbol name or address.
|
||
Within the symbol name, the address will be handled by the kernel.
|
||
|
||
Otherwise, specify the address of \cpp|sys_call_table| from \verb|/proc/kallsyms| and \verb|/boot/System.map| into \cpp|sym| parameter.
|
||
Following is the sample usage for \verb|/proc/kallsyms|:
|
||
\begin{verbatim}
|
||
$ sudo grep sys_call_table /proc/kallsyms
|
||
ffffffff82000280 R x32_sys_call_table
|
||
ffffffff820013a0 R sys_call_table
|
||
ffffffff820023e0 R ia32_sys_call_table
|
||
$ sudo insmod syscall-steal.ko sym=0xffffffff820013a0
|
||
\end{verbatim}
|
||
|
||
Using the address from \verb|/boot/System.map|, be careful about \verb|KASLR| (Kernel Address Space Layout Randomization).
|
||
\verb|KASLR| may randomize the address of kernel code and data at every boot time, such as the static address listed in \verb|/boot/System.map| will offset by some entropy.
|
||
The purpose of \verb|KASLR| is to protect the kernel space from the attacker.
|
||
Without \verb|KASLR|, the attacker may find the target address in the fixed address easily.
|
||
Then the attacker can use return-oriented programming to insert some malicious codes to execute or receive the target data by a tampered pointer.
|
||
\verb|KASLR| mitigates these kinds of attacks because the attacker cannot immediately know the target address, but a brute-force attack can still work.
|
||
If the address of a symbol in \verb|/proc/kallsyms| is different from the address in \verb|/boot/System.map|, \verb|KASLR| is enabled with the kernel, which your system running on.
|
||
\begin{verbatim}
|
||
$ grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub
|
||
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
|
||
$ sudo grep sys_call_table /boot/System.map-$(uname -r)
|
||
ffffffff82000300 R sys_call_table
|
||
$ sudo grep sys_call_table /proc/kallsyms
|
||
ffffffff820013a0 R sys_call_table
|
||
# Reboot
|
||
$ sudo grep sys_call_table /boot/System.map-$(uname -r)
|
||
ffffffff82000300 R sys_call_table
|
||
$ sudo grep sys_call_table /proc/kallsyms
|
||
ffffffff86400300 R sys_call_table
|
||
\end{verbatim}
|
||
If \verb|KASLR| is enabled, we have to take care of the address from \verb|/proc/kallsyms| each time we reboot the machine.
|
||
In order to use the address from \verb|/boot/System.map|, make sure that \verb|KASLR| is disabled.
|
||
You can add the \verb|nokaslr| for disabling \verb|KASLR| in next booting time:
|
||
\begin{verbatim}
|
||
$ grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub
|
||
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
|
||
$ sudo perl -i -pe 'm/quiet/ and s//quiet nokaslr/' /etc/default/grub
|
||
$ grep quiet /etc/default/grub
|
||
GRUB_CMDLINE_LINUX_DEFAULT="quiet nokaslr splash"
|
||
$ sudo update-grub
|
||
\end{verbatim}
|
||
|
||
For more information, check out the following:
|
||
|
||
\begin{itemize}
|
||
\item \href{https://lwn.net/Articles/804849/}{Cook: Security things in Linux v5.3}
|
||
\item \href{https://lwn.net/Articles/12211/}{Unexporting the system call table}
|
||
\item \href{https://lwn.net/Articles/810077/}{Control-flow integrity for the kernel}
|
||
\item \href{https://lwn.net/Articles/813350/}{Unexporting kallsyms\_lookup\_name()}
|
||
\item \href{https://www.kernel.org/doc/Documentation/kprobes.txt}{Kernel Probes (Kprobes)}
|
||
\item \href{https://lwn.net/Articles/569635/}{Kernel address space layout randomization}
|
||
\end{itemize}
|
||
|
||
The source code here is an example of such a kernel module.
|
||
We want to ``spy'' on a certain user, and to \cpp|pr_info()| a message whenever that user opens a file.
|
||
Towards this end, we replace the system call to open a file with our own function, called \cpp|our_sys_openat|.
|
||
This function checks the uid (user's id) of the current process, and if it is equal to the uid we spy on, it calls \cpp|pr_info()| to display the name of the file to be opened.
|
||
Then, either way, it calls the original \cpp|openat()| function with the same parameters, to actually open the file.
|
||
|
||
The \cpp|init_module| function replaces the appropriate location in \cpp|sys_call_table| and keeps the original pointer in a variable.
|
||
The \cpp|cleanup_module| function uses that variable to restore everything back to normal.
|
||
This approach is dangerous, because of the possibility of two kernel modules changing the same system call.
|
||
Imagine we have two kernel modules, A and B. A's openat system call will be \cpp|A_openat| and B's will be \cpp|B_openat|.
|
||
Now, when A is inserted into the kernel, the system call is replaced with \cpp|A_openat|, which will call the original \cpp|sys_openat| when it is done.
|
||
Next, B is inserted into the kernel, which replaces the system call with \cpp|B_openat|, which will call what it thinks is the original system call, \cpp|A_openat|, when it's done.
|
||
|
||
Now, if B is removed first, everything will be well --- it will simply restore the system call to \cpp|A_openat|, which calls the original.
|
||
However, if A is removed and then B is removed, the system will crash.
|
||
A's removal will restore the system call to the original, \cpp|sys_openat|, cutting B out of the loop.
|
||
Then, when B is removed, it will restore the system call to what it thinks is the original, \cpp|A_openat|, which is no longer in memory.
|
||
At first glance, it appears we could solve this particular problem by checking if the system call is equal to our open function and if so not changing it at all (so that B won't change the system call when it is removed), but that will cause an even worse problem.
|
||
When A is removed, it sees that the system call was changed to \cpp|B_openat| so that it is no longer pointing to \cpp|A_openat|, so it will not restore it to \cpp|sys_openat| before it is removed from memory.
|
||
Unfortunately, \cpp|B_openat| will still try to call \cpp|A_openat| which is no longer there, so that even without removing B the system would crash.
|
||
|
||
For x86 architecture, the system call table cannot be used to invoke a system call after commit
|
||
\href{https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1e3ad78334a69b36e107232e337f9d693dcc9df2}{1e3ad78} since v6.9.
|
||
This commit has been backported to long term stable kernels, like v5.15.154+, v6.1.85+, v6.6.26+ and v6.8.5+, see this \href{https://stackoverflow.com/a/78607015}{answer} for more details.
|
||
In this case, thanks to Kprobes, a hook can be used instead on the system call entry to intercept the system call.
|
||
|
||
Note that all the related problems make syscall stealing unfeasible for production use.
|
||
In order to keep people from doing potential harmful things \cpp|sys_call_table| is no longer exported.
|
||
This means, if you want to do something more than a mere dry run of this example, you will have to patch your current kernel in order to have \cpp|sys_call_table| exported.
|
||
|
||
\samplec{examples/syscall-steal.c}
|
||
|
||
\section{Blocking Processes and threads}
|
||
\label{sec:blocking_process_thread}
|
||
\subsection{Sleep}
|
||
\label{sec:sleep}
|
||
What do you do when somebody asks you for something you can not do right away?
|
||
If you are a human being and you are bothered by a human being, the only thing you can say is: "\emph{Not right now, I'm busy. Go away!}".
|
||
But if you are a kernel module and you are bothered by a process, you have another possibility.
|
||
You can put the process to sleep until you can service it.
|
||
After all, processes are being put to sleep by the kernel and woken up all the time (that is the way multiple processes appear to run on the same time on a single CPU).
|
||
|
||
This kernel module is an example of this.
|
||
The file (called \verb|/proc/sleep|) can only be opened by a single process at a time.
|
||
If the file is already open, the kernel module calls \cpp|wait_event_interruptible|.
|
||
The easiest way to keep a file open is to open it with:
|
||
|
||
\begin{codebash}
|
||
tail -f
|
||
\end{codebash}
|
||
|
||
This function changes the status of the task (a task is the kernel data structure which holds information about a process and the system call it is in,
|
||
if any) to \cpp|TASK_INTERRUPTIBLE|, which means that the task will not run until it is woken up somehow, and adds it to WaitQ, the queue of tasks waiting to access the file.
|
||
Then, the function calls the scheduler to context switch to a different process, one which has some use for the CPU.
|
||
|
||
When a process is done with the file, it closes it, and \cpp|module_close| is called.
|
||
That function wakes up all the processes in the queue (there's no mechanism to only wake up one of them).
|
||
It then returns and the process which just closed the file can continue to run.
|
||
In time, the scheduler decides that that process has had enough and gives control of the CPU to another process.
|
||
Eventually, one of the processes which was in the queue will be given control of the CPU by the scheduler.
|
||
It starts at the point right after the call to \cpp|wait_event_interruptible|.
|
||
|
||
This means that the process is still in kernel mode - as far as the process is concerned, it issued the open system call and the system call has not returned yet.
|
||
The process does not know somebody else used the CPU for most of the time between the moment it issued the call and the moment it returned.
|
||
|
||
It can then proceed to set a global variable to tell all the other processes that the file is still open and go on with its life.
|
||
When the other processes get a piece of the CPU, they'll see that global variable and go back to sleep.
|
||
|
||
So we will use \sh|tail -f| to keep the file open in the background, while trying to access it with another process (again in the background, so that we need not switch to a different vt).
|
||
As soon as the first background process is killed with kill \%1 , the second is woken up, is able to access the file and finally terminates.
|
||
|
||
To make our life more interesting, \cpp|module_close| does not have a monopoly on waking up the processes which wait to access the file.
|
||
A signal, such as \emph{Ctrl +c} (\textbf{SIGINT}) can also wake up a process. This is because we used \cpp|wait_event_interruptible|.
|
||
We could have used \cpp|wait_event| instead, but that would have resulted in extremely angry users whose \emph{Ctrl+c}'s are ignored.
|
||
|
||
In that case, we want to return with \cpp|-EINTR| immediately. This is important so users can, for example, kill the process before it receives the file.
|
||
|
||
There is one more point to remember. Some times processes don't want to sleep, they want either to get what they want immediately, or to be told it cannot be done.
|
||
Such processes use the \cpp|O_NONBLOCK| flag when opening the file.
|
||
The kernel is supposed to respond by returning with the error code \cpp|-EAGAIN| from operations which would otherwise block, such as opening the file in this example. The program \sh|cat_nonblock|, available in the \verb|examples/other| directory, can be used to open a file with \cpp|O_NONBLOCK|.
|
||
|
||
\begin{verbatim}
|
||
$ sudo insmod sleep.ko
|
||
$ cat_nonblock /proc/sleep
|
||
Last input:
|
||
$ tail -f /proc/sleep &
|
||
Last input:
|
||
Last input:
|
||
Last input:
|
||
Last input:
|
||
Last input:
|
||
Last input:
|
||
Last input:
|
||
tail: /proc/sleep: file truncated
|
||
[1] 6540
|
||
$ cat_nonblock /proc/sleep
|
||
Open would block
|
||
$ kill %1
|
||
[1]+ Terminated tail -f /proc/sleep
|
||
$ cat_nonblock /proc/sleep
|
||
Last input:
|
||
$
|
||
\end{verbatim}
|
||
|
||
\samplec{examples/sleep.c}
|
||
|
||
\samplec{examples/other/cat_nonblock.c}
|
||
|
||
\subsection{Completions}
|
||
\label{sec:completion}
|
||
Sometimes one thing should happen before another within a module having multiple threads.
|
||
Rather than using \sh|/bin/sleep| commands, the kernel has another way to do this which allows timeouts or interrupts to also happen.
|
||
|
||
Completions as code synchronization mechanism have three main parts, initialization of struct completion synchronization object, the waiting or barrier part through \cpp|wait_for_completion()|, and the signalling side through a call to \cpp|complete()|.
|
||
|
||
In the subsequent example, two threads are initiated: crank and flywheel.
|
||
It is imperative that the crank thread starts before the flywheel thread.
|
||
A completion state is established for each of these threads, with a distinct completion defined for both the crank and flywheel threads.
|
||
At the exit point of each thread the respective completion state is updated, and \cpp|wait_for_completion| is used by the flywheel thread to ensure that it does not begin prematurely.
|
||
The crank thread uses the \cpp|complete_all()| function to update the completion, which lets the flywheel thread continue.
|
||
|
||
So even though \cpp|flywheel_thread| is started first you should notice when you load this module and run \sh|dmesg|, that turning the crank always happens first because the flywheel thread waits for the crank thread to complete.
|
||
|
||
There are other variations of the \cpp|wait_for_completion| function, which include timeouts or being interrupted, but this basic mechanism is enough for many common situations without adding a lot of complexity.
|
||
|
||
\samplec{examples/completions.c}
|
||
|
||
\section{Avoiding Collisions and Deadlocks}
|
||
\label{sec:synchronization}
|
||
If processes running on different CPUs or in different threads try to access the same memory, then it is possible that strange things can happen or your system can lock up.
|
||
To avoid this, various types of mutual exclusion kernel functions are available.
|
||
These indicate if a section of code is "locked" or "unlocked" so that simultaneous attempts to run it can not happen.
|
||
\subsection{Mutex}
|
||
\label{sec:mutex}
|
||
You can use kernel mutexes (mutual exclusions) in much the same manner that you might deploy them in userland.
|
||
This may be all that is needed to avoid collisions in most cases.
|
||
|
||
\samplec{examples/example_mutex.c}
|
||
|
||
\subsection{Spinlocks}
|
||
\label{sec:spinlock}
|
||
As the name suggests, spinlocks lock up the CPU that the code is running on, taking 100\% of its resources.
|
||
Because of this you should only use the spinlock mechanism around code which is likely to take no more than a few milliseconds to run and so will not noticeably slow anything down from the user's point of view.
|
||
|
||
The example here is \verb|"irq safe"| in that if interrupts happen during the lock then they will not be forgotten and will activate when the unlock happens, using the \cpp|flags| variable to retain their state.
|
||
|
||
\samplec{examples/example_spinlock.c}
|
||
|
||
Taking 100\% of a CPU's resources comes with greater responsibility.
|
||
Situations where the kernel code monopolizes a CPU are called \textbf{atomic contexts}.
|
||
Holding a spinlock is one of those situations.
|
||
Sleeping in atomic contexts may leave the system hanging, as the occupied CPU devotes 100\% of its resources doing nothing but sleeping.
|
||
In some worse cases the system may crash.
|
||
Thus, sleeping in atomic contexts is considered a bug in the kernel.
|
||
They are sometimes called ``sleep-in-atomic-context'' in some materials.
|
||
|
||
Note that sleeping here is not limited to calling the sleep functions explicitly.
|
||
If subsequent function calls eventually invoke a function that sleeps, it is also considered sleeping.
|
||
Thus, it is important to pay attention to functions being used in atomic context.
|
||
There's no documentation recording all such functions, but code comments may help.
|
||
Sometimes you may find comments in kernel source code stating that a function ``may sleep'', ``might sleep'', or more explicitly ``the caller should not hold a spinlock''.
|
||
Those comments are hints that a function may implicitly sleep and must not be called in atomic contexts.
|
||
|
||
\subsection{Read and write locks}
|
||
\label{sec:rwlock}
|
||
Read and write locks are specialised kinds of spinlocks so that you can exclusively read from something or write to something.
|
||
Like the earlier spinlocks example, the one below shows an "irq safe" situation in which if other functions were triggered from irqs which might also read and write to whatever you are concerned with then they would not disrupt the logic.
|
||
As before it is a good idea to keep anything done within the lock as short as possible so that it does not hang up the system and cause users to start revolting against the tyranny of your module.
|
||
|
||
\samplec{examples/example_rwlock.c}
|
||
|
||
Of course, if you know for sure that there are no functions triggered by irqs which could possibly interfere with your logic then you can use the simpler \cpp|read_lock(&myrwlock)| and \cpp|read_unlock(&myrwlock)| or the corresponding write functions.
|
||
\subsection{Atomic operations}
|
||
\label{sec:atomics}
|
||
If you are doing simple arithmetic: adding, subtracting or bitwise operations, then there is another way in the multi-CPU and multi-hyperthreaded world to stop other parts of the system from messing with your mojo.
|
||
By using atomic operations you can be confident that your addition, subtraction or bit flip did actually happen and was not overwritten by some other shenanigans.
|
||
An example is shown below.
|
||
|
||
\samplec{examples/example_atomic.c}
|
||
|
||
Before the C11 standard adopts the built-in atomic types, the kernel already provided a small set of atomic types by using a bunch of tricky architecture-specific codes.
|
||
Implementing the atomic types by C11 atomics may allow the kernel to throw away the architecture-specific codes and letting the kernel code be more friendly to the people who understand the standard.
|
||
But there are some problems, such as the memory model of the kernel doesn't match the model formed by the C11 atomics.
|
||
For further details, see:
|
||
\begin{itemize}
|
||
\item \href{https://www.kernel.org/doc/Documentation/atomic_t.txt}{kernel documentation of atomic types}
|
||
\item \href{https://lwn.net/Articles/691128/}{Time to move to C11 atomics?}
|
||
\item \href{https://lwn.net/Articles/698315/}{Atomic usage patterns in the kernel}
|
||
\end{itemize}
|
||
|
||
% FIXME: we should rewrite this section
|
||
\section{Replacing Print Macros}
|
||
\label{sec:print_macros}
|
||
\subsection{Replacement}
|
||
% FIXME: cross-reference
|
||
In Section \ref{sec:preparation}, it was noted that the X Window System and kernel module programming are not conducive to integration.
|
||
This remains valid during the development of kernel modules.
|
||
However, in practical scenarios, the necessity emerges to relay messages to the tty (teletype) originating the module load command.
|
||
|
||
The term ``tty'' originates from \emph{teletype}, which initially referred to a combined keyboard-printer for Unix system communication.
|
||
Today, it signifies a text stream abstraction employed by Unix programs, encompassing physical terminals,
|
||
xterms in X displays, and network connections like SSH.
|
||
|
||
To achieve this, the ``current'' pointer is leveraged to access the active task's tty structure.
|
||
Within this structure lies a pointer to a string write function, facilitating the string's transmission to the tty.
|
||
|
||
\samplec{examples/print_string.c}
|
||
|
||
\subsection{Flashing keyboard LEDs}
|
||
\label{sec:flash_kb_led}
|
||
In certain conditions, you may desire a simpler and more direct way to communicate to the external world.
|
||
Flashing keyboard LEDs can be such a solution: It is an immediate way to attract attention or to display a status condition.
|
||
Keyboard LEDs are present on every hardware, they are always visible, they do not need any setup, and their use is rather simple and non-intrusive, compared to writing to a tty or a file.
|
||
|
||
From v4.14 to v4.15, the timer API made a series of changes to improve memory safety.
|
||
A buffer overflow in the area of a \cpp|timer_list| structure may be able to overwrite the \cpp|function| and \cpp|data| fields, providing the attacker with a way to use return-oriented programming (ROP) to call arbitrary functions within the kernel.
|
||
Also, the function prototype of the callback, containing a \cpp|unsigned long| argument, will prevent work from any type checking.
|
||
Furthermore, the function prototype with \cpp|unsigned long| argument may be an obstacle to the forward-edge protection of \textit{control-flow integrity}.
|
||
Thus, it is better to use a unique prototype to separate from the cluster that takes an \cpp|unsigned long| argument.
|
||
The timer callback should be passed a pointer to the \cpp|timer_list| structure rather than an \cpp|unsigned long| argument.
|
||
Then, it wraps all the information the callback needs, including the \cpp|timer_list| structure, into a larger structure, and it can use the \cpp|container_of| macro instead of the \cpp|unsigned long| value.
|
||
For more information see: \href{https://lwn.net/Articles/735887/}{Improving the kernel timers API}.
|
||
|
||
Before Linux v4.14, \cpp|setup_timer| was used to initialize the timer and the \cpp|timer_list| structure looked like:
|
||
\begin{code}
|
||
struct timer_list {
|
||
unsigned long expires;
|
||
void (*function)(unsigned long);
|
||
unsigned long data;
|
||
u32 flags;
|
||
/* ... */
|
||
};
|
||
|
||
void setup_timer(struct timer_list *timer, void (*callback)(unsigned long),
|
||
unsigned long data);
|
||
\end{code}
|
||
|
||
Since Linux v4.14, \cpp|timer_setup| is adopted and the kernel step by step converting to \cpp|timer_setup| from \cpp|setup_timer|.
|
||
One of the reasons why API was changed is it need to coexist with the old version interface.
|
||
Moreover, the \cpp|timer_setup| was implemented by \cpp|setup_timer| at first.
|
||
\begin{code}
|
||
void timer_setup(struct timer_list *timer,
|
||
void (*callback)(struct timer_list *), unsigned int flags);
|
||
\end{code}
|
||
|
||
The \cpp|setup_timer| was then removed since v4.15.
|
||
As a result, the \cpp|timer_list| structure had changed to the following.
|
||
\begin{code}
|
||
struct timer_list {
|
||
unsigned long expires;
|
||
void (*function)(struct timer_list *);
|
||
u32 flags;
|
||
/* ... */
|
||
};
|
||
\end{code}
|
||
|
||
The following source code illustrates a minimal kernel module which, when loaded, starts blinking the keyboard LEDs until it is unloaded.
|
||
|
||
\samplec{examples/kbleds.c}
|
||
|
||
If none of the examples in this chapter fit your debugging needs, there might yet be some other tricks to try.
|
||
Ever wondered what \cpp|CONFIG_LL_DEBUG| in \sh|make menuconfig| is good for?
|
||
If you activate that you get low level access to the serial port.
|
||
While this might not sound very powerful by itself, you can patch \src{kernel/printk.c} or any other essential syscall to print ASCII characters, thus making it possible to trace virtually everything what your code does over a serial line.
|
||
If you find yourself porting the kernel to some new and former unsupported architecture, this is usually amongst the first things that should be implemented.
|
||
Logging over a netconsole might also be worth a try.
|
||
|
||
While you have seen lots of stuff that can be used to aid debugging here, there are some things to be aware of. Debugging is almost always intrusive.
|
||
Adding debug code can change the situation enough to make the bug seem to disappear.
|
||
Thus, you should keep debug code to a minimum and make sure it does not show up in production code.
|
||
|
||
\section{Scheduling Tasks}
|
||
\label{sec:scheduling_tasks}
|
||
There are two main ways of running tasks: tasklets and work queues.
|
||
Tasklets are a quick and easy way of scheduling a single function to be run.
|
||
For example, when triggered from an interrupt, whereas work queues are more complicated but also better suited to running multiple things in a sequence.
|
||
|
||
It is possible that in future tasklets may be replaced by \textit{threaded irqs}.
|
||
However, discussion about that has been ongoing since 2007 (\href{https://lwn.net/Articles/239633}{Eliminating tasklets}), so do not hold your breath.
|
||
See the section \ref{sec:irq} if you wish to avoid the tasklet debate.
|
||
|
||
\subsection{Tasklets}
|
||
\label{sec:tasklet}
|
||
Here is an example tasklet module.
|
||
The \cpp|tasklet_fn| function runs for a few seconds.
|
||
In the meantime, execution of the \cpp|example_tasklet_init| function may continue to the exit point, depending on whether it is interrupted by \textbf{softirq}.
|
||
|
||
\samplec{examples/example_tasklet.c}
|
||
|
||
So with this example loaded \sh|dmesg| should show:
|
||
|
||
\begin{verbatim}
|
||
tasklet example init
|
||
Example tasklet starts
|
||
Example tasklet init continues...
|
||
Example tasklet ends
|
||
\end{verbatim}
|
||
Although tasklet is easy to use, it comes with several drawbacks, and developers are discussing about getting rid of tasklet in linux kernel.
|
||
The tasklet callback runs in atomic context, inside a software interrupt, meaning that it cannot sleep or access user-space data, so not all work can be done in a tasklet handler.
|
||
Also, the kernel only allows one instance of any given tasklet to be running at any given time; multiple different tasklet callbacks can run in parallel.
|
||
|
||
In recent kernels, tasklets can be replaced by workqueues, timers, or threaded interrupts.\footnote{The goal of threaded interrupts is to push more of the work to separate threads, so that the minimum needed for acknowledging an interrupt is reduced, and therefore the time spent handling the interrupt (where it can't handle any other interrupts at the same time) is reduced.
|
||
See \url{https://lwn.net/Articles/302043/}.}
|
||
While the removal of tasklets remains a longer-term goal, the current kernel contains more than a hundred uses of tasklets.
|
||
Now developers are proceeding with the API changes and the macro \cpp|DECLARE_TASKLET_OLD| exists for compatibility.
|
||
For further information, see \url{https://lwn.net/Articles/830964/}.
|
||
|
||
\subsection{Work queues}
|
||
\label{sec:workqueue}
|
||
To add a task to the scheduler we can use a workqueue.
|
||
The kernel then uses the Completely Fair Scheduler (CFS) to execute work within the queue.
|
||
|
||
\samplec{examples/sched.c}
|
||
|
||
\section{Interrupt Handlers}
|
||
\label{sec:interrupt_handler}
|
||
\subsection{Interrupt Handlers}
|
||
\label{sec:irq}
|
||
Except for the last chapter, everything we did in the kernel so far we have done as a response to a process asking for it, either by dealing with a special file, sending an \cpp|ioctl()|, or issuing a system call.
|
||
But the job of the kernel is not just to respond to process requests.
|
||
Another job, which is every bit as important, is to speak to the hardware connected to the machine.
|
||
|
||
There are two types of interaction between the CPU and the rest of the computer's hardware.
|
||
The first type is when the CPU gives orders to the hardware, the other is when the hardware needs to tell the CPU something.
|
||
The second, called interrupts, is much harder to implement because it has to be dealt with when convenient for the hardware, not the CPU.
|
||
Hardware devices typically have a very small amount of RAM, and if you do not read their information when available, it is lost.
|
||
|
||
Under Linux, hardware interrupts are called IRQ's (Interrupt ReQuests).
|
||
There are two types of IRQ's, short and long.
|
||
A short IRQ is one which is expected to take a very short period of time, during which the rest of the machine will be blocked and no other interrupts will be handled.
|
||
A long IRQ is one which can take longer, and during which other interrupts may occur (but not interrupts from the same device).
|
||
If at all possible, it is better to declare an interrupt handler to be long.
|
||
|
||
When the CPU receives an interrupt, it stops whatever it is doing (unless it is processing a more important interrupt, in which case it will deal with this one only when the more important one is done),
|
||
saves certain parameters on the stack and calls the interrupt handler.
|
||
This means that certain things are not allowed in the interrupt handler itself, because the system is in an unknown state.
|
||
% TODO: add some diagrams
|
||
Linux kernel solves the problem by splitting interrupt handling into two parts.
|
||
The first part executes right away and masks the interrupt line.
|
||
Hardware interrupts must be handled quickly, and that is why we need the second part to handle the heavy work deferred from an interrupt handler.
|
||
Historically, BH (Linux naming for \textit{Bottom Halves}) statistically book-keeps the deferred functions.
|
||
\textbf{Softirq} and its higher level abstraction, \textbf{Tasklet}, replace BH since Linux 2.3.
|
||
|
||
The way to implement this is to call \cpp|request_irq()| to get your interrupt handler called when the relevant IRQ is received.
|
||
|
||
In practice IRQ handling can be a bit more complex.
|
||
Hardware is often designed in a way that chains two interrupt controllers, so that all the IRQs from interrupt controller B are cascaded to a certain IRQ from interrupt controller A.
|
||
Of course, that requires that the kernel finds out which IRQ it really was afterwards and that adds overhead. Other architectures offer some special, very low overhead, so called "fast IRQ" or FIQs.
|
||
To take advantage of them requires handlers to be written in assembly language, so they do not really fit into the kernel.
|
||
They can be made to work similar to the others, but after that procedure, they are no longer any faster than "common" IRQs.
|
||
SMP enabled kernels running on systems with more than one processor need to solve another truckload of problems.
|
||
It is not enough to know if a certain IRQs has happened, it's also important to know what CPU(s) it was for.
|
||
People still interested in more details, might want to refer to "APIC" now.
|
||
|
||
This function receives the IRQ number, the name of the function, flags, a name for \verb|/proc/interrupts| and a parameter to be passed to the interrupt handler.
|
||
Usually there is a certain number of IRQs available.
|
||
How many IRQs there are is hardware-dependent.
|
||
|
||
The flags can be used for specify behaviors of the IRQ.
|
||
For example, use \cpp|IRQF_SHARED| to indicate you are willing to share the IRQ with other interrupt handlers (usually because a number of hardware devices sit on the same IRQ); use the \cpp|IRQF_ONESHOT| to indicate that the IRQ is not reenabled after the handler finished.
|
||
It should be noted that in some materials, you may encouter another set of IRQ flags named with the \cpp|SA| prefix.
|
||
For example, the \cpp|SA_SHIRQ| and the \cpp|SA_INTERRUPT|.
|
||
Those are the the IRQ flags in the older kernels.
|
||
They have been removed completely.
|
||
Today only the \cpp|IRQF| flags are in use.
|
||
This function will only succeed if there is not already a handler on this IRQ, or if you are both willing to share.
|
||
|
||
\subsection{Detecting button presses}
|
||
\label{sec:detect_button}
|
||
Many popular single board computers, such as Raspberry Pi or Beagleboards, have a bunch of GPIO pins.
|
||
Attaching buttons to those and then having a button press do something is a classic case in which you might need to use interrupts,
|
||
so that instead of having the CPU waste time and battery power polling for a change in input state, it is better for the input to trigger the CPU to then run a particular handling function.
|
||
|
||
Here is an example where buttons are connected to GPIO numbers 17 and 18 and an LED is connected to GPIO 4.
|
||
You can change those numbers to whatever is appropriate for your board.
|
||
|
||
\samplec{examples/intrpt.c}
|
||
|
||
\subsection{Bottom Half}
|
||
\label{sec:bottom_half}
|
||
Suppose you want to do a bunch of stuff inside of an interrupt routine.
|
||
A common way to do that without rendering the interrupt unavailable for a significant duration is to combine it with a tasklet.
|
||
This pushes the bulk of the work off into the scheduler.
|
||
|
||
The example below modifies the previous example to also run an additional task when an interrupt is triggered.
|
||
|
||
\samplec{examples/bottomhalf.c}
|
||
|
||
\subsection{Threaded IRQ}
|
||
|
||
Threaded IRQ is a mechanism to organize both top-half and bottom-half of an IRQ at once.
|
||
A threaded IRQ splits the one handler in \cpp|request_irq()| into two: one for the top-half, the other for the bottom-half.
|
||
The \cpp|request_threaded_irq()| is the function for using threaded IRQs.
|
||
Two handlers are registered at once in the \cpp|request_threaded_irq()|.
|
||
|
||
Those two handlers run in different context.
|
||
The top-half handler runs in interrupt context.
|
||
It's the equivalence of the handler passed to the \cpp|request_irq()|.
|
||
The bottom-half handler on the other hand runs in its own thread.
|
||
This thread is created on registration of a threaded IRQ.
|
||
Its sole purpose is to run this bottom-half handler.
|
||
This is where a threaded IRQ is ``threaded''.
|
||
If \cpp|IRQ_WAKE_THREAD| is returned by the top-half handler, that bottom-half serving thread will wake up.
|
||
The thread then runs the bottom-half handler.
|
||
|
||
Here is an example of how to do the same thing as before, with top and bottom halves, but using threads.
|
||
|
||
\samplec{examples/bh_threaded.c}
|
||
|
||
A threaded IRQ is registered using \cpp|request_threaded_irq()|.
|
||
This function only takes one additional parameter than the \cpp|request_irq()| -- the bottom-half handling function that runs in its own thread.
|
||
In this example it is the \cpp|button_bottom_half()|.
|
||
Usage of other parameters are the same as \cpp|request_irq()|.
|
||
|
||
Presence of both handlers is not mandatory.
|
||
If either of them is not needed, pass the \cpp|NULL| instead.
|
||
A \cpp|NULL| top-half handler implies that no action is taken except to wake up the bottom-half serving thread, which runs the bottom-half handler.
|
||
Similarly, a \cpp|NULL| bottom-half handler effectively acts as if \cpp|request_irq()| were used.
|
||
In fact, this is how \cpp|request_irq()| is implemented.
|
||
|
||
Note that passing \cpp|NULL| to both handlers is considered an error and will make registration fail.
|
||
|
||
\section{Virtual Input Device Driver}
|
||
\label{sec:vinput}
|
||
The input device driver is a module that provides a way to communicate with the interaction device via the event.
|
||
For example, the keyboard can send the press or release event to tell the kernel what we want to do.
|
||
The input device driver will allocate a new input structure with \cpp|input_allocate_device()| and sets up input bitfields, device id, version, etc.
|
||
After that, registers it by calling \cpp|input_register_device()|.
|
||
|
||
Here is an example, vinput,
|
||
It is an API to allow easy development of virtual input drivers.
|
||
The drivers needs to export a \cpp|vinput_device()| that contains the virtual device name and \cpp|vinput_ops| structure that describes:
|
||
|
||
\begin{itemize}
|
||
\item the init function: \cpp|init()|
|
||
\item the input event injection function: \cpp|send()|
|
||
\item the readback function: \cpp|read()|
|
||
\end{itemize}
|
||
|
||
Then using \cpp|vinput_register_device()| and \cpp|vinput_unregister_device()| will add a new device to the list of support virtual input devices.
|
||
|
||
\begin{code}
|
||
int init(struct vinput *);
|
||
\end{code}
|
||
|
||
This function is passed a \cpp|struct vinput| already initialized with an allocated \cpp|struct input_dev|.
|
||
The \cpp|init()| function is responsible for initializing the capabilities of the input device and register it.
|
||
|
||
\begin{code}
|
||
int send(struct vinput *, char *, int);
|
||
\end{code}
|
||
|
||
This function will receive a user string to interpret and inject the event using the \cpp|input_report_XXXX| or \cpp|input_event| call.
|
||
The string is already copied from user.
|
||
|
||
\begin{code}
|
||
int read(struct vinput *, char *, int);
|
||
\end{code}
|
||
|
||
This function is used for debugging and should fill the buffer parameter with the last event sent in the virtual input device format.
|
||
The buffer will then be copied to user.
|
||
|
||
vinput devices are created and destroyed using sysfs.
|
||
And, event injection is done through a \verb|/dev| node.
|
||
The device name will be used by the userland to export a new virtual input device.
|
||
|
||
The \cpp|class_attribute| structure is similar to other attribute types we talked about in section \ref{sec:sysfs}:
|
||
|
||
\begin{code}
|
||
struct class_attribute {
|
||
struct attribute attr;
|
||
ssize_t (*show)(struct class *class, struct class_attribute *attr,
|
||
char *buf);
|
||
ssize_t (*store)(struct class *class, struct class_attribute *attr,
|
||
const char *buf, size_t count);
|
||
};
|
||
\end{code}
|
||
|
||
In \verb|vinput.c|, the macro \cpp|CLASS_ATTR_WO(export/unexport)| defined in \src{include/linux/device.h} (in this case, \verb|device.h| is included in \src{include/linux/input.h}) will generate the \cpp|class_attribute| structures which are named \verb|class_attr_export/unexport|.
|
||
Then, put them into \cpp|vinput_class_attrs| array and the macro \cpp|ATTRIBUTE_GROUPS(vinput_class)| will generate the \cpp|struct attribute_group vinput_class_group| that should be assigned in \cpp|vinput_class|.
|
||
Finally, call \cpp|class_register(&vinput_class)| to create attributes in sysfs.
|
||
|
||
To create a \verb|vinputX| sysfs entry and \verb|/dev| node.
|
||
|
||
\begin{codebash}
|
||
echo "vkbd" | sudo tee /sys/class/vinput/export
|
||
\end{codebash}
|
||
|
||
To unexport the device, just echo its id in unexport:
|
||
|
||
\begin{codebash}
|
||
echo "0" | sudo tee /sys/class/vinput/unexport
|
||
\end{codebash}
|
||
|
||
\samplec{examples/vinput.h}
|
||
\samplec{examples/vinput.c}
|
||
|
||
Here the virtual keyboard is one of example to use vinput.
|
||
It supports all \cpp|KEY_MAX| keycodes.
|
||
The injection format is the \cpp|KEY_CODE| such as defined in \src{include/linux/input.h}.
|
||
A positive value means \cpp|KEY_PRESS| while a negative value is a \cpp|KEY_RELEASE|.
|
||
The keyboard supports repetition when the key stays pressed for too long.
|
||
The following demonstrates how simulation work.
|
||
|
||
Simulate a key press on "g" (\cpp|KEY_G| = 34):
|
||
|
||
\begin{codebash}
|
||
echo "+34" | sudo tee /dev/vinput0
|
||
\end{codebash}
|
||
|
||
Simulate a key release on "g" (\cpp|KEY_G| = 34):
|
||
|
||
\begin{codebash}
|
||
echo "-34" | sudo tee /dev/vinput0
|
||
\end{codebash}
|
||
|
||
\samplec{examples/vkbd.c}
|
||
|
||
% TODO: Add vts.c and vmouse.c example
|
||
|
||
\section{Standardizing the interfaces: The Device Model}
|
||
\label{sec:device_model}
|
||
Up to this point we have seen all kinds of modules doing all kinds of things, but there was no consistency in their interfaces with the rest of the kernel.
|
||
To impose some consistency such that there is at minimum a standardized way to start, suspend and resume a device model was added.
|
||
An example is shown below, and you can use this as a template to add your own suspend, resume or other interface functions.
|
||
|
||
\samplec{examples/devicemodel.c}
|
||
|
||
\section{Optimizations}
|
||
\label{sec:optimization}
|
||
\subsection{Likely and Unlikely conditions}
|
||
\label{sec:likely_unlikely}
|
||
Sometimes you might want your code to run as quickly as possible, especially if it is handling an interrupt or doing something which might cause noticeable latency.
|
||
If your code contains boolean conditions and if you know that the conditions are almost always likely to evaluate as either \cpp|true| or \cpp|false|,
|
||
then you can allow the compiler to optimize for this using the \cpp|likely| and \cpp|unlikely| macros.
|
||
For example, when allocating memory you are almost always expecting this to succeed.
|
||
|
||
\begin{code}
|
||
bvl = bvec_alloc(gfp_mask, nr_iovecs, &idx);
|
||
if (unlikely(!bvl)) {
|
||
mempool_free(bio, bio_pool);
|
||
bio = NULL;
|
||
goto out;
|
||
}
|
||
\end{code}
|
||
|
||
When the \cpp|unlikely| macro is used, the compiler alters its machine instruction output, so that it continues along the false branch and only jumps if the condition is true.
|
||
That avoids flushing the processor pipeline.
|
||
The opposite happens if you use the \cpp|likely| macro.
|
||
|
||
\subsection{Static keys}
|
||
\label{sec:static_keys}
|
||
Static keys allow us to enable or disable kernel code paths based on the runtime state of key. Its APIs have been available since 2010 (most architectures are already supported), use self-modifying code to eliminate the overhead of cache and branch prediction.
|
||
The most typical use case of static keys is for performance-sensitive kernel code, such as tracepoints, context switching, networking, etc. These hot paths of the kernel often contain branches and can be optimized easily using this technique.
|
||
Before we can use static keys in the kernel, we need to make sure that gcc supports \cpp|asm goto| inline assembly, and the following kernel configurations are set:
|
||
|
||
\begin{code}
|
||
CONFIG_JUMP_LABEL=y
|
||
CONFIG_HAVE_ARCH_JUMP_LABEL=y
|
||
CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE=y
|
||
\end{code}
|
||
|
||
To declare a static key, we need to define a global variable using the \cpp|DEFINE_STATIC_KEY_FALSE| or \cpp|DEFINE_STATIC_KEY_TRUE| macro defined in \src{include/linux/jump\_label.h}.
|
||
This macro initializes the key with the given initial value, which is either false or true, respectively. For example, to declare a static key with an initial value of false, we can use the following code:
|
||
|
||
\begin{code}
|
||
DEFINE_STATIC_KEY_FALSE(fkey);
|
||
\end{code}
|
||
|
||
Once the static key has been declared, we need to add branching code to the module that uses the static key.
|
||
For example, the code includes a fastpath, where a no-op instruction will be generated at compile time as the key is initialized to false and the branch is unlikely to be taken.
|
||
|
||
\begin{code}
|
||
pr_info("fastpath 1\n");
|
||
if (static_branch_unlikely(&fkey))
|
||
pr_alert("do unlikely thing\n");
|
||
pr_info("fastpath 2\n");
|
||
\end{code}
|
||
|
||
If the key is enabled at runtime by calling \cpp|static_branch_enable(&fkey)|, the fastpath will be patched with an unconditional jump instruction to the slowpath code \cpp|pr_alert|, so the branch will always be taken until the key is disabled again.
|
||
|
||
The following kernel module derived from \verb|chardev.c|, demonstrates how the static key works.
|
||
|
||
\samplec{examples/static_key.c}
|
||
|
||
To check the state of the static key, we can use the \verb|/dev/key_state| interface.
|
||
|
||
\begin{codebash}
|
||
cat /dev/key_state
|
||
\end{codebash}
|
||
|
||
This will display the current state of the key, which is disabled by default.
|
||
|
||
To change the state of the static key, we can perform a write operation on the file:
|
||
|
||
\begin{codebash}
|
||
echo enable > /dev/key_state
|
||
\end{codebash}
|
||
|
||
This will enable the static key, causing the code path to switch from the fastpath to the slowpath.
|
||
|
||
In some cases, the key is enabled or disabled at initialization and never changed, we can declare a static key as read-only, which means that it can only be toggled in the module init function. To declare a read-only static key, we can use the \cpp|DEFINE_STATIC_KEY_FALSE_RO| or \cpp|DEFINE_STATIC_KEY_TRUE_RO| macro instead. Attempts to change the key at runtime will result in a page fault.
|
||
For more information, see \href{https://www.kernel.org/doc/Documentation/static-keys.txt}{Static keys}
|
||
|
||
\section{Common Pitfalls}
|
||
\label{sec:opitfall}
|
||
|
||
\subsection{Using standard libraries}
|
||
\label{sec:using_stdlib}
|
||
You can not do that.
|
||
In a kernel module, you can only use kernel functions which are the functions you can see in \verb|/proc/kallsyms|.
|
||
|
||
\subsection{Disabling interrupts}
|
||
\label{sec:disabling_interrupts}
|
||
You might need to do this for a short time and that is OK, but if you do not enable them afterwards, your system will be stuck and you will have to power it off.
|
||
|
||
\section{Where To Go From Here?}
|
||
\label{sec:where_to_go}
|
||
For those deeply interested in kernel programming,
|
||
\href{https://kernelnewbies.org}{kernelnewbies.org} and the \src{Documentation} subdirectory within the kernel source code are highly recommended.
|
||
Although the latter may not always be straightforward, it serves as a valuable initial step for further exploration.
|
||
Echoing Linus Torvalds' perspective, the most effective method to understand the kernel is through personal examination of the source code.
|
||
|
||
Contributions to this guide are welcome, especially if there are any significant inaccuracies identified.
|
||
To contribute or report an issue, please initiate an issue at \url{https://github.com/sysprog21/lkmpg}.
|
||
Pull requests are greatly appreciated.
|
||
|
||
Happy hacking!
|
||
\end{CJK*}
|
||
\end{document}
|