Kernel Compilation, Kernel Module, andSystem Call Implementation1 IntroductionThe objective of this assignment is to familiarize yourself with the Linux kernel source code. Specifically,you will:• Compile your own Linux kernel.• Create your own kernel module.• Add a new kernel system call and then test this system call from a user-space program.All the assignment’s steps must be performed using your Debian virtual machine from programmingassignment 1. The notions from the course involved in this assignment are the following:• Linux source code exploration and compilation.• Installing and running a modified kernel, as well as creating and testing a module.• The printk kernel function.• User space / kernel space communication using a system call.2 Finding and Booting Your “Known Good” KernelStart the Debian VM, log in, open a terminal window, and execute the following command to find theexact version of the kernel you are currently running: uname -rWrite down the result somewhere and remember this exact kernel version for the rest of the semester.This is your “known good” kernel which is known to work correctly. Also execute ls -l /boot and youshould see one “vmlinuz” file in the /boot directory that has the exact same version in its name. This isthe file containing your “known good” kernel. Never delete or modify this file in any way.Use Debian’s battery icon menu (upper right corner of the Debian desktop) to restart the virtual machine.As soon as you see the white-on-blue-green menu of the GRUB boot loader, press the down arrow key onthe keyboard to select the second menu entry named “Advanced options for Debian GNU/Linux”. Youhave only 5 seconds to do this! Then press Enter to get a sub-menu. In the sub-menu, select the “knowngood” kernel, based on the version you learned just above (but not the “recovery mode” one, which issimilar to Microsoft Windows’s “safe mode”, which is not what we want here) and press Enter to bootthis ‘known good” kernel. From now on this is what you will do every time you boot or reboot the VM, tomake sure you always know exactly which kernel you are booting! If later you have a problem when testingyour own modified “working” kernel (see below), such as a kernel that panics (crashes) on boot, you willalways be able to boot this “known good” kernel instead to get back to work.3 Compiling Your Own “Working” Kernel1) Before compiling your own “working” kernel, you must first install some tools and the Linux kernelsource code. So, after booting the “known good” kernel (as indicated above), log in, open a terminalwindow, and execute the following command to install all the software required to configure andcompile the Linux kernel source code:sudo apt install build-essential pahole libelf-dev libncurses-dev libssl-dev flex bison(build-essential is a software package that depends on all the development libraries and headerfiles and basic development tools that you need, so installing it guarantees that all those things arepresent on your VM; pahole and libelf-dev are a set of tools and a library to handle the ELFstandard file format for executable files; libncurses-dev is a library to create text-based userinterfaces, which is used by “make menuconfig” below; libssl-dev is a library that providescryptographic functions and secure networking; and flex and bison are a lexer / scanner generatorand parser generator, respectively).2) Install the Linux kernel source code: sudo apt install linux-sourceThis automatically selects the right version of the Linux kernel source code, based on the version ofthe “known good” kernel which is currently executing on the VM. Then execute ls -l /usr/srcand you should see one file named “linux-source-X.Y.tar.xz” (“tar” is the standard file archive formatfor Unix, and “xz” is one among many compressed file formats on Unix; other common ones are “gz”and “bz2”; together “tar” and “xz” give you a compressed file archive, similar in spirit to the “zip”compressed file archive format which is used a lot on Microsoft Windows).3) Execute cd to make sure that you are in your home directory. Then decompress and extract the kernelsource code you just installed: tar -xavf /usr/src/linux-source-X.Y.tar.xz(the -x option means “extract”; -a means auto-compress or auto-decompress, as the case may be; -v means “verbose”, to see the names of the extracted files as the extraction happens; -f is followedby the name of the compresses archive file). After doing this, you should have in your home directorya new directory named “linux-source-X.Y” (use ls to see it).Use du -s -h linux-source-X.Y to see how big the Linux kernel source code is.4) Change the name of this new “linux-source-X.Y” directory: mv linux-source-X.Y pa25) Execute cd pa2 to move into the top directory of your Linux kernel source code. Use ls to have alook at the different directories there (see slides 20-21 in the “Introduction to Linux” lecture notes onCanvas for an explanation of the content of those directories).6) Next we need to create a configuration file for the kernel (see slides 34 and 36 of the “Introduction toLinux” lecture notes). The simplest method is to just copy and then modify the configuration file forthe “known good” kernel which is currently executing on your VM. Again, the command uname -rtells you exactly which kernel version you are currently executing on the VM, so we can use the resultof this command to copy the right kernel configuration file from the /boot directory into the currentdirectory containing your Linux kernel source code: cp /boot/config-$(uname -r) .config7) We can now modify this kernel configuration file using: make menuconfigYou should then get a text-based user interface (see slide 35 in “Introduction to Linux”) which allowsyou to easily modify your kernel configuration file:a) Press Enter to go into the “General setup” sub-menu, press the down arrow twice on yourkeyboard to go down two lines, then Enter to select “Local version - append to kernel release”. Inthe new text window that appears, enter a minus sign - followed by your login name (which isalso your Stevens login name), followed by “-pa2”. For example: -pmeunier-pa2 (replace mylogin name with yours, obviously). Do not add any space before, inside, or after this text. Thispiece of text is going to be added to the version of the Linux kernel you are going to compile, sothat later you can easily recognize which kernel is yours. This will be your “working” kernel. PressEnter to select “OK” and you should be back in the previous sub-menu, with the text you justtyped is now showing between () parentheses in front of the name of the “Local version - appendto kernel release” sub-menu. Press the right arrow on your keyboard to select “Exit” and thenEnter to go back to the previous top-level menu.b) Select Save and press Enter. In the new window, check that the default “.config” file name iscorrect and press Enter to save your new configuration file. Then select “Exit” to quit the“menuconfig” user interface and go back to the usual Unix shell.c) Double check the differences between the original kernel configuration file and the one you justmodified: diff /boot/config-$(uname -r) .configIn the output you should see that the kernel variable CONFIG_LOCALVERSION is now defined tobe the piece of text “-pmeunier-pa2” (except with your own login name) while before it was theempty string (ignore any change for CONFIG_CC_VERSION_TEXT and for SALT, SIG, and KEY stuff,these changes are irrelevant). You are now ready to compile the Linux kernel source code for your“working” kernel!8) Execute nproc to see how many processors your VM is using (this should be the same number youindicated in the settings of the VM in VirtualBox). The more the better.You might also want to go into the settings of your host operating system (Microsoft Windows orApple macOS) and make sure that the power settings of your computer are set for maximumperformance (for example, in Microsoft Windows, click on the battery icon at the right end of thebottom taskbar, then move the slider to “Best performance”); you can undo this setting later againafter you are finished compiling the kernel.Also make sure that your host OS is not set up to automatically suspend or shut down your hostcomputer if you do not move the mouse or whatnot for a while. Again you can undo this later whenyou are finished compiling the kernel.If you are using a laptop computer, also make sure that it is plugged into an electric socket, otherwiseyour host OS might automatically throttle your computer’s performance to save the battery.Finally, click with your right mouse button on the Debian desktop background, select “Settings”, then“Power”, then click on “Automatic suspend” and turn off both “On Battery Power” and “Plugged In”;this will guarantee that Debian does not suspend itself while you are in the middle of compiling. Thenclose the settings window.Now we are ready to compile the whole Linux kernel source code using all the VM’s processors inparallel, to speed up things: make -j $(nproc) allWait an hour or two (or three). You will know the compilation is over when your computer’s fan stopsmaking more noise than usual… If for some reason you need to stop the compilation before it isfinished, just press Ctrl-c on the keyboard. You can then re-start the compilation later using the samecommand again, and it will automatically restart from where it left off.If for some reason the compilation fails, it might be hard to see why it failed because you are compilingdifferent parts of the kernel code in parallel. In that case, you can execute just make all to restartthe compilation from where it failed but this time in sequential mode on a single CPU, which will makeit much easier to see what the problem is. Then contact your nearest Course Assistant.If you look at the output of the make command while it is compiling the code, you will see differentabbreviations: CC is when compiling a C file to get a “something.o” object file (a file containing binaryCPU instructions), LD is when linking (gluing) multiple object files together, AR is when creating alibrary (a “something.a” archive file of object files, which later will be linked with the rest of thekernel), [M] means that this code is part of a dynamically loadable kernel module, not part of themonolithic Linux kernel proper.9) Execute du -s -h . to see how much disk space the current directory containing the whole compiledLinux source code takes (it should be around 19GB).10) When compiling the kernel, all the kernel modules are also compiled at the same time. We now needto install these new kernel modules: sudo make INSTALL_MOD_STRIP=1 modules_installThis must be done as root (the system administrator) hence the use of “sudo” here. TheINSTALL_MOD_STRIP=1 argument given to the make command tells make to strip extra debugginginformation from the modules when installing them, which saves around 2GB of disk space.Once the modules are installed, you can execute ls -l /usr/lib/modules and you should see inthe output a directory with a name that has your login name and “pa2” at the end, which is where themodules were installed. The command du -s -h /usr/lib/modules/* should show you that allthe different directories there (the one for your own modules, as well as the one for the “known good”kernel) are about 400MB in size.11) Now we can install the new “working” kernel itself: sudo make installDo not worry about any “Please install the Linux kernel "header" files matching the current kernel”message in the command’s output; we would need this only if we were to try to use our Debian VMas the host for another VM on top of it!Execute ls -l /boot and you will see that make installed several files there, all of which have yourlogin name and “pa2” at the end of their name: “config-X.Y…-pmeunier-pa2” which is a copy of the“.config” file you used for compiling your kernel (you can check this using the diff command, forexample; see above), “initrd.img-X.Y…-pmeunier-pa2” which is the kernel’s “initial RAM disk” whichthe kernel only uses at boot time (see here for more information if you are curious), “System.map-X.Y…-pmeunier-pa2” which contains a list of the kernel’s symbols (the function names and variablenames inside the kernel) for your kernel (see slide 44 in “Introduction to Linux” lecture notes), andfinally “vmlinuz-X.Y…-pmeunier-pa2” which is your very own “working” kernel!As part of the kernel’s installation, the configuration file for the GRUB bootloader is also automaticallyupdated so that the bootloader now knows about your new kernel.12) Use Debian’s battery icon menu to restart the virtual machine. Again, as soon as you see the white-on-blue-green menu of the GRUB boot loader, press the down arrow key on the keyboard to selectthe second menu entry named “Advanced options for Debian GNU/Linux”. You have only 5 secondsto do this! Then press Enter to get a sub-menu. In the sub-menu, select now your own “working”kernel, which has your login name and “pa2” at the end of its name (but again not the “recoverymode” one) and press Enter to boot your own kernel. Happiness ensues.13) After booting your own “working” kernel, log in, open a terminal window, use Ctrl-+ to increase thefont size of the terminal window, and execute uname -a (which should show a kernel name withyour own login name and “pa2” at the end of it) and then id. Take a screenshot and save it as a picturesomewhere on your host computer, you will need to submit it later on Canvas (more details aboutthis at the end of this document). Make sure the kernel version and your login name (Stevens loginname) are clearly visible inside the terminal window in your screenshot. For example:Congratulations on compiling and booting your first Linux kernel!Now create in your home directory another new directory that you will use for submitting your assignment(again, use your own Stevens login name): cd; mkdir pmeunier-pa2Then copy your “working” kernel configuration file into your submission directory:cp pa2/.config pmeunier-pa2To finish this part of the assignment, create a backup copy of the kernel source code you just compiled,just in case (-r means to do a recursive copy that will automatically copy the directory, its sub-directories,its sub-sub-directories, etc.; -p preserves the file time stamps when copying which the make commanduses to decide whether to compile a file again or not, so this is important when copying compiled codethat has been created using make; this command takes a couple of minutes to complete because you arecopying about 19GB of data): cp -p -r pa2 pa2-backupLater, when doing the rest of this assignment, if you make a big mistake inside your “pa2” directory, suchas accidentally deleting some files, you can always find the original files inside this backup directory. Inthe worst case, if you do not know how to fix the mistake, you can always completely delete your “pa2”directory (cd; rm -rf pa2) and re-create it using the backup copy (cp -p -r pa2-backup pa2).4 Creating a Kernel ModuleIn your submission directory (cd pmeunier-pa2), create a new directory (mkdir module). In that newdirectory (cd module), write a Linux kernel module in a file named “pmeunier.c” (use your own loginname). This module must have a printk() statement that outputs “Hello World from NAME (LOGIN)” inthe kernel log when the module is loaded into the kernel, where NAME is your full legal name and LOGINis your Stevens login name. When the kernel module is unloaded it must print “PID is XYZ and programname is NAME” where XYZ is the PID number and NAME the program name of the current process. (Seeyour “Introduction to Linux” and “Processes in Linux” lecture notes.)Compile your kernel module using the appropriate “Makefile”. (Make sure that you are still running yourown “working” kernel when doing this!)You must provide a single screenshot that shows two Debian terminal windows side by side: one windowmust show the kernel log (with your module’s output visible in it), and the other window must show theoutput of the uname -a command followed by commands to load-unload your module at least twice (sowe can see in the kernel log that the PID printed when unloading the module changes during each unload).Save the screenshot somewhere on your host computer, you will need to submit it later on Canvas (moredetails about this at the end of this document). For example (partly censored):5 Adding a System Call to Your “Working” Kernel and Testing It5.1 Adding The System CallIn the directory containing the compiled source code of your “working” kernel (cd ~/pa2; “~” is a shellspecial character that always represents your home directory) create a new directory (mkdirmy_syscall). In that new directory (cd my_syscall), in a file named “my_syscall.c”, write the C codefor a new system call named “LOGIN_syscall” (replace LOGIN with your Stevens login name) that takes assingle parameter a pointer to a character array containing a string. Make sure you use the proper C macroto define your system call (see the “System Calls” lecture notes). The code of your system call must alwaysreturn a signed long integer (long) as result.Your system call must do the following:1) If the string pointer given as argument is NULL then your system call must immediately return -1.2) If the string length is larger than 32 (where the string length is its total number of characters, includingthe ’\0’ string terminator character), then your system call must immediately return -1.3) Copy the string from user space to kernel space.4) Use printk to print “before: “ followed by the string. Note: whenever you use printk, always makesure that the string you want to print is terminated with a ‘\n’ newline character, otherwise the stringwill not immediately appear in the kernel logs when your code is executed.5) Replace all occurrences of a lowercase vowel letter (a, e, i, o, u, y) in the string with the first letter ofyour login name, in uppercase (for example, given my “pmeunier” login name, the letters a, e, i, o, u,and y are all replaced with the uppercase letter P).6) Use printk to print “after: “ followed by the modified string.7) Copy the modified string from kernel space to user space.8) The system call then returns as result the number of character replacements performed in step 5.In your code, use the functions listed on slide 25 of the “System Calls” lecture notes as much as possible.Make sure your code properly checks for errors when using each of these functions (see the examplesystem call code on slide 21 in those same lecture notes). Do not use a module to implement your systemcall. Also create a “Makefile” in the same directory (see slide 27).Modify other files in the Linux kernel source code as necessary (see slides 26 and 27; use gettimeofdayfrom slide 21 as an example of how to add your own system call to the various Linux files). Give yoursystem call the number 451. Keep a list somewhere of all the files you modify, because later you will needto copy them to your submission directory.Recompile your “working” kernel, the same way you did it in section 3 of this assignment. Note: some ofthe files you need to modify are referenced throughout the kernel’s code, so, when recompiling, the makecommand then needs to recompile most of the kernel code again, which again takes a very long time. Somake sure you modify the files correctly on the first try, so you do not have to recompile the whole kernelover and over. If you only need to change your my_syscall/my_syscall.c file then recompiling the kernelshould take only about 5 minutes since the rest of the kernel’s code will not need to be recompiled againin this case.If the compilation succeeds then reinstall your “working” kernel and its associated modules (do NOT tryto install anything if the compilation failed!) Note that, when installing your modified “working” kernel,the previous version of your “working” kernel will be renamed with an extra “.old” extension at the endof its name. You can just ignore this one.Reboot the virtual machine and make sure that you use the GRUB bootloader to select the correct kernelwhen rebooting (your “working” kernel with the “-pmeunier-pa2” name, that now contains the code foryour own system call).If your “working” kernel panics (crashes) on boot for some reason, reboot with your “known good” kerneland fix the problem in the code of your “working” kernel.After you have booted the correct kernel, execute the following command to double-check that the kernelyou are now running actually knows about your system call (see slide 44 of the “Introduction to Linux”lecture notes; replace “pmeunier” with your own login name): grep pmeunier /proc/kallsymsIf the output of this command is empty then the current kernel does not know about your system call andso you did something wrong somewhere. Re-read this whole section and try again.5.2 Testing The System CallIn your submission directory (cd ~/pmeunier-pa2), write the C code for a user-space test programcalled “syscall.c” that invokes your system call. Since your system call does not have a corresponding Cwrapper function available in the C standard library, you must directly use the “syscall” function of the Cstandard library to call your system call (see slide 18 of the “OS Concepts and Structure” lecture notes foran example). Your code must invoke your system call twice: once for the case where the string size islarger than 32, and once for the case where the string size is less than 32. In both cases, your C programmust print on the screen:1) The string given to the syscall, before the system call happens.2) The return value of the syscall.3) The string after the system call, even if it has not been modified.Note: you can directly use the system call number 451 in your C code.Important note: the strings that you use in your code must be defined as local variables (the same way itis done for example on slide 18 of the “OS Concepts and Structure” lecture notes), not as string constantsthat you directly give to the system call as argument. This is because, when your code executes inside aprocess, all the string constants from your code are stored in a part of the “data” segment of the process’saddress space which is read-only (since string constants are… constant), which in turn means that yourkernel system call would fail when trying to modify those strings. Local variables are stored in the process’sstack, which is read-write and your system call will then be able to modify those strings.You must provide a single screenshot that shows two Debian terminal windows side by side: one windowmust show the kernel log (with your system call’s output visible in it, when the string given as argumentto the system call is short enough), and the other window must show the output of the uname -acommand followed by the execution of your user-space test program. Save the screenshot somewhereon your host computer, you will need to submit it later on Canvas (more details about this below). Forexample:6 What to Submit For This AssignmentOnce both your system call and your user-space test program work, copy the code of the system call, plusany other kernel file you modified, into your submission directory (your user-space test program mustalready be there): cd ~/pa2; cp -p -r my_syscall … any other file you modified … ~/pmeunier-pa2In the same submission directory, create a PDF file named “screenshots.pdf” that contains:1) Your full name.2) The Stevens Honor pledge.3) The three screenshots you created above: one showing that you compiled your own “working” kernel(section 3), one showing how your kernel module works (section 4), and one showing how your systemcall and your user-space test program work (section 5.2). Make sure the screenshots are clearlyreadable.4) Also add a short explanation before each screenshot so the Course Assistants know what you aretrying to show on those screenshots.At this point the submission directory (“pmeunier-pa2”) must contain all the files you have created ormodified during this assignment (including the kernel .config configuration file, which you can only see inthe submission directory by using the ls -a command). Install the zip program on your virtual machine(sudo apt install zip) and then create a ZIP file of your submission directory (the -r option meansto zip all the subdirectories recursively: do not forget it; also use your own student login name, twice,obviously): cd; zip -r pmeunier-pa2.zip pmeunier-pa2Once you have correctly created the file “pmeunier-pa2.zip”, copy it to the host OS using your sharedfolder, double-check its content to make sure it contains everything, then submit it on Canvas.After the deadline for this assignment has passed, you can delete the backup copy of your “working”kernel: rm -rf ~/pa2-backupIt is up to you whether you delete the compiled source code of your “working” kernel or not (rm -rf~/pa2), you will not need it anymore. You can delete it after the deadline for this assignment has passed,if you want to save VM disk space, or delete it only at the end of the semester, or keep it for ever as asouvenir!Done!7 Rubric1) Correct .config file: 5%2) Screenshot showing the corresponding “working” kernel version: 5%3) Code of module/pmeunier.c: 15%4) Code of module/Makefile: 5%5) Screenshot of kernel log with module loading-unloading (twice): 10%6) Code of my_syscall/my_syscall.c: 25%7) Code of my_syscall/Makefile: 5%8) Code of syscall.c: 15%9) Other modified kernel files: 5%10) Screenshot of kernel log and output of syscall.c: 10%Important note: you do not get points for screenshots unless the corresponding files are submitted too.So for example you will not get points for the first screenshot listed above if you do not also provide thecorresponding .config file. Screenshots alone will not get you any points at all. So make sure you doublecheck everything before you submit on Canvas!
...