Kali Linux Long Term Usage Part 2: Systemd, Partitions, and Swap Space

2025-04-06 1354 words 7 minutes

Contents

I am using Kali Linux on a regular basis inside a virtual machine (VirtualBox) and I have maintained my image for a couple of years.

Over the last months and years, the booting time got worse and worse, until it reached the following startup time:

1
2
3
└─$ systemd-analyze time                                                           
Startup finished in 34.355s (kernel) + 1min 42.261s (userspace) = 2min 16.617s 
graphical.target reached after 1min 42.261s in userspace.

In the first part of the series, we covered some optimizations with regard to disk usage. In this part, we want to determine why the system takes so much time to start up, even though it has sufficient hardware resources.

Journalctl to the Rescue

A good start is the journalctl command, which will print log entries from the systemd journal. The systemd journal logs detailed startup events, including timeouts and failures.

Since we are interested in the boot process, we can look for potential errors and timeouts with the following command, which shows only messages for the current boot:

1
2
3
4
5
└─$  journalctl -b

Apr 05 17:26:00 kalipen kernel: Linux version 6.12.20-amd64 (devel@kali.org) (x86_64-linux-gnu-gcc-14 (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 SMP PREEMPT_DYNAMIC Kali 6.12.20-1kali1 (2025-03-26)
Apr 05 17:26:00 kalipen kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-6.12.20-amd64 root=UUID=dad034ac-b23b-466f-9402-cd385b400121 ro initrd=/install/initrd.gz quiet splash
...

The output can be overwhelming, so we refine our search by looking at higher priority events only:

1
journalctl -b -p 3

-p 3: Shows priority level 3 (errors) and above, often highlighting startup failures.

1
2
3
4
└─$ sudo journalctl -b -p 3                                                        
[...]
Apr 05 17:27:35 kalipen systemd[1]: Timed out waiting for device dev-disk-by\x2duuid-8191879a\x2df03c\x2d4f6c\x2d9d59\x2dc217cfca09ab.device - /dev/disk/by-uuid/8191879a-f03c-4f6c-9d59-c217cfca09ab.
[...]

This is already a promising hint: systemd waits for a certain disk or device. What is going on here?

Systemd Timeouts

First of all, we note that it takes over 90 seconds for systemd to time out (note the timestamps from above, which start at 17:26:00, whereas the systemd error message shows up only at 17:27:35). Let us examine the timeout values configured for systemd:

1
2
3
4
5
└─$ grep -i timeout /etc/systemd/system.conf                                                                                                                          
#DefaultTimeoutStartSec=90s
#DefaultTimeoutStopSec=90s
#DefaultTimeoutAbortSec=
#DefaultDeviceTimeoutSec=90s

Since no timeout value is configured, systemd’s built-in default is 90 seconds (1min 30s), as per the systemd-system.conf man page.

The DefaultTimeoutStartSec configuration in systemd sets the default time limit that systemd will wait for a service to start before considering it failed.

Let us configure now a shorter timeout value for both DefaultDeviceTimeoutSec, which configures the default timeout for waiting for devices, and DefaultTimeoutStartSec. Note that this will not fix the underlying problem, but maybe it will reduce the boot loading time a bit.

To make it more uniform, we apply the same value (10 seconds) to the DefaultTimeoutStopSec setting as well:

1
2
3
4
5
└─$ grep -i timeout /etc/systemd/system.conf                                       
DefaultTimeoutStartSec=10s
DefaultTimeoutStopSec=10s
#DefaultTimeoutAbortSec=
DefaultDeviceTimeoutSec=10s

After rebooting, we check the results again:

1
2
3
└─$ systemd-analyze time             
Startup finished in 35.915s (kernel) + 27.617s (userspace) = 1min 3.532s 
graphical.target reached after 27.616s in userspace.

Success! We reduced our system startup time from 2 minutes 16 seconds to 1 minute 3 seconds, a reduction of more than 50%.

The Swap Problem

Let us now check /etc/fstab for configured disk devices:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
└─$ cat /etc/fstab  
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=dad034ac-b23b-466f-9402-cd385b400121 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=8191879a-f03c-4f6c-9d59-c217cfca09ab none            swap    sw              0       0
/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0

Here we see also the UUID of the device that systemd is waiting for: UUID=8191879a-f03c-4f6c-9d59-c217cfca09ab

This disk is supposed to serve as a swap partition. We continue to inspect the available disks and disk IDs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
└─$ lsblk         
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 144.1G  0 disk 
├─sda1   8:1    0 142.1G  0 part /
└─sda2   8:2    0     2G  0 part 


└─$ blkid
/dev/sda2: UUID="a8f66941-83aa-465d-a649-5ad990d7a197" TYPE="swap" PARTUUID="f831f204-02"
/dev/sda1: UUID="dad034ac-b23b-466f-9402-cd385b400121" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="f831f204-01"

The UUID of the swap partition changed, but why? As it turns out, when you resize your disk, for example with GParted, the UUIDs change (see https://techlabs.blog/categories/debian-linux/expand-linux-virtual-machine-disk-partition-using-gparted).

Thus, the static references in /etc/fstab will not update and you have to adjust them manually.

This is described in detail on https://linux-blog.anracom.com/2020/11/15/long-boot-time-of-kali-linux-after-swap-partition-changes-swap-settings-for-the-initramfs/.

Thus, we have to make sure each time after resizing a disk that has swap partitions to update at least the following files:

/etc/fstab
/etc/initramfs-tools/conf.d/resume (this file is used when resuming from hibernation, and it can cause boot delays if we no longer have a swap partition, see https://www.linuxuprising.com/2018/08/how-to-use-swap-file-instead-of-swap.html)
/etc/default/grub (for Grub2 configuration)

Back to our situation: Since we had originally configured a swap partition, but the swap partition UUID changed and it was not updated in /etc/fstab we currently do not have any swap space at all!

1
2
3
4
└─$ free -h                      
               total        used        free      shared  buff/cache   available
Mem:            14Gi       1.2Gi        11Gi        40Mi       2.0Gi        13Gi
Swap:             0B          0B          0B

Due to the lack of swap space, programs could fail or crash when they run out of memory. In our case it was not noticeable due to the large amount of available RAM. But there can be several problems:

Programs may not run fully or crash when running of memory.
The system might crash completely when running out of physical RAM.
There may be a segmentation fault when writing to a page when there are no physical pages.

In modern Linux kernel versions swap files are virtually as fast as swap partitions. Of course, the swap file should be contiguous to provide the same speed as a partition. When using swap files instead of swap partitions, we also mitigate the problem of changing disk UUIDs.

First, let us create a swap file and use it in /etc/fstab. With a RAM size of 16 GB, we set the size of the swap file to 4 GB (see https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/managing_storage_devices/getting-started-with-swap_managing-storage-devices#overview-of-swap-space_getting-started-with-swap):

1
2
3
4
5
6
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

echo "/swapfile none swap sw 0 0" | sudo tee -a /etc/fstab

We can immediately verify that our system has swap space again:

1
2
3
4
5
6
7
8
9
└─$ free -h                                                                        
               total        used        free      shared  buff/cache   available
Mem:            14Gi       1.2Gi        11Gi        40Mi       2.0Gi        13Gi
Swap:          4.0Gi          0B       4.0Gi


└─$ sudo swapon --show            
NAME      TYPE SIZE USED PRIO
/swapfile file   4G   0B   -2

Secondly, we comment out the line in /etc/fstab with the swap partition:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=dad034ac-b23b-466f-9402-cd385b400121 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
#UUID=8191879a-f03c-4f6c-9d59-c217cfca09ab none            swap    sw              0       0
/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0
/swapfile none swap sw 0 0

After rebooting, we can notice another improvement in the system’s startup time:

1
2
3
└─$ systemd-analyze time  
Startup finished in 38.529s (kernel) + 13.637s (userspace) = 52.167s 
graphical.target reached after 13.637s in userspace.

Thirdly, we boot with a GParted ISO image, remove the swap partition entirely and reize our main partition /dev/sda1:

Finally, we finish with a startup time of roughly 47 seconds. This is still a lot and we will tackle the underlying problems in the next blog post:

1
2
3
└─$ systemd-analyze time          
Startup finished in 33.775s (kernel) + 13.879s (userspace) = 47.655s 
graphical.target reached after 13.879s in userspace.