How to build a Systemtap kernel module ?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 9
  • Red Hat Enterprise Linux 10
  • systemtap-runtime

Issue

  • It is not possible to install kernel-debuginfo and kernel-devel packages on a production server. Is there any alternate way to run SystemStap script on production machine without installing kernel-debuginfo and kernel-devel packages?

Resolution

Table of contents

The following are the requirements to build and run a systemtap kernel module:

  • Development machine having the same architecture as the target (production) system.
  • The kernel-devel, kernel-debuginfo and, kernel-debuginfo-common-x86_64 package versions installed on the development machine must match with the kernel version of the target (production) system.
  • The systemtap-runtime package must be installed on the target (production) system.
    Note: Only systemtap-runtime is required on the target (production) system.

Section 1: Development Machine

A. Build a module for a currently running kernel version

Step 1: Install the following packages:

# yum install systemtap kernel-devel kernel-debuginfo kernel-debuginfo-common-x86_64

If debug repository is not enabled then refer the steps to enable it here.

Note: The kernel-devel, kernel-debuginfo and kernel-debuginfo-common-x86_64 package versions on development machine must match the kernel version of target (production) system.

Step 2: Build a pre-compiled module for currently running kernel version:

Syntax
# stap <systemtap.stp> -m <name-for-module> -p 4

Where;

-m MODULE
Use the given name for the generated kernel object module, instead of a unique randomized name.  The generated kernel object module is copied to the current directory.

-p NUM 
Stop after pass NUM.  The passes are numbered 1-5: parse, elaborate, translate,  compile, run.

Note: some systemtap scripts might require uprobes.ko on the production machine , for that, build the module with --save-uprobes flag.

Step 2.1: Check the current kernel version:

# uname -r
4.18.0-60.el8.x86_64
Step 2.2: Create a sample systemtap script say file-exec.stp with below contents:
#!/usr/bin/stap

#To trace new process(es) created via exec() only and show command line agruments:

probe begin 
{ 
  printf("Monitoring new process(es): Start\n"); 
}

probe syscall.execve.return
{
  printf ("%-25s: PID: [%d] COMM: %s\n", ctime(gettimeofday_s()), pid(), cmdline_str());
}

probe timer.ms(40000)
{
  printf("Monitoring new process(es): Stop\n");
  exit()
}

Step 2.3: Build the above systemtap script:

# stap file-exec.stp -m stap_module_file_exec -p 4
stap_module_file_exec.ko

The above command will create a module file stap_module_file_exec.ko in the current directory.

Step 3: Verify the details of the module:

# modinfo stap_module_file_exec.ko
filename:       /root/systemtap/stap_module_file_exec.ko
license:        GPL
description:    systemtap-generated probe
license:        GPL
rhelversion:    8.0
srcversion:     362B2169EF67E9E2DF1C8DB
depends:        
name:           stap_module_file_exec
vermagic:       4.18.0-60.el8.x86_64 SMP mod_unload modversions 
parm:           _stp_bufsize:buffer size (int)

B. Build a module for a different kernel version:

The following part highlights the steps to build a module for different kernel versions. If the module has been already built using the steps mentioned in Section A then skip this section and proceed to Section 2 for deploying it on Production Machine.

Step 1: Check the current kernel version:

# uname -r
4.18.0-60.el8.x86_64

Step 2: Install thekernel-devel package of the corresponding kernel:

# yum install kernel-devel-4.18.0-32.el8

Step 3: Build the sample systemtap script (sample file file-exec.stp in step 2.2 of above section). Then compile the kernel module using the below step. Here -r option is used to specify the target kernel version:

# stap -r 4.18.0-32.el8.x86_64 file-exec.stp -m stap_module_file_exec -p 4
stap_module_file_exec.ko

The above command will create a module file stap_module_file_exec.ko in the current directory.

Step 4: Verify the details of the module:

# modinfo stap_module_file_exec.ko
filename:       /root/systemtap/stap_module_file_exec.ko
license:        GPL
description:    systemtap-generated probe
license:        GPL
rhelversion:    8.0
srcversion:     6BAD97388246AAFA7B62F44
depends:        
retpoline:      Y
name:           stap_module_file_exec
vermagic:       4.18.0-32.el8.x86_64 SMP mod_unload modversions 
parm:           _stp_bufsize:buffer size (int)

Note: The kernel-devel package of the corresponding kernel is required to build a module.

Section 2: Deploying on Production Machine

Step 1: Copy the pre-compiled module (eg: stap_module_file_exec.ko) from the development machine to the target (production) system.

Step 2: Install the following package:

# yum install systemtap-runtime

Step 3: Run the pre-compiled module:

Syntax
# staprun <name-of-module>.ko
  • To display the output on console:
# staprun stap_module_file_exec.ko
Monitoring new process(es): Start
Tue Feb 19 09:04:25 2019 : PID: [10849] COMM: ls --color=auto
Monitoring new process(es): Stop
  • To redirect the output to a file:
# staprun stap_module_file_exec.ko -o output.txt

# cat output.txt 
Monitoring new process(es): Start
Tue Feb 19 09:23:20 2019 : PID: [29413] COMM: ls --color=auto
Monitoring new process(es): Stop
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.