Loading

RTF Troubleshooting systemd-tmpfiles-clean.service

Дата публикации: Aug 1, 2025
Задача

GOAL

Troubleshooting steps for RTF degraded state due to systemd-tmpfiles-clean.service.

Действия
1. Enter gravity 
gravity enter

2. Check if the last execution of systemd-tmpfiles-clean.service (runs once per day) was successful. In below sample output it is successful. 
$ systemctl status systemd-tmpfiles-clean.service
● systemd-tmpfiles-clean.service - Cleanup of Temporary Directories
   Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-clean.service; static; vendor preset: enabled)
   Active: inactive (dead) since Thu 2021-12-02 00:49:08 UTC; 4h 28min ago
     Docs: man:tmpfiles.d(5)
           man:systemd-tmpfiles(8)
  Process: 64118 ExecStart=/bin/systemd-tmpfiles --clean (code=exited, status=0/SUCCESS)
 Main PID: 64118 (code=exited, status=0/SUCCESS)

Dec 02 00:49:08 ip-172-31-0-9.ap-northeast-2.compute.internal systemd[1]: Starting Cleanup of Temporary Directories...
Dec 02 00:49:08 ip-172-31-0-9.ap-northeast-2.compute.internal systemd[1]: systemd-tmpfiles-clean.service: Succeeded.
Dec 02 00:49:08 ip-172-31-0-9.ap-northeast-2.compute.internal systemd[1]: Started Cleanup of Temporary Directories.
3. If the last execution failed, trigger the service again manually (by starting it) and confirm (check status again) if it is still failing. 
systemctl start systemd-tmpfiles-clean.service
systemctl status systemd-tmpfiles-clean.service
4. If the service/job is still failing, run the below command to execute it manually with debug logging. 
SYSTEMD_LOG_LEVEL=debug systemd-tmpfiles --clean
Sample output: 
$ SYSTEMD_LOG_LEVEL=debug systemd-tmpfiles --clean
Looking for configuration files in (higher priority first):
	/etc/tmpfiles.d
	/run/tmpfiles.d
	/usr/local/lib/tmpfiles.d
	/usr/lib/tmpfiles.d
	/lib/tmpfiles.d
Reading config file "/usr/lib/tmpfiles.d/dbus.conf"…
Reading config file "/usr/lib/tmpfiles.d/debian.conf"…
Reading config file "/usr/lib/tmpfiles.d/home.conf"…
Reading config file "/usr/lib/tmpfiles.d/journal-nocow.conf"…
Reading config file "/usr/lib/tmpfiles.d/legacy.conf"…
Ignoring entry r! "/forcefsck" because --boot is not specified.
Ignoring entry r! "/fastboot" because --boot is not specified.
Ignoring entry r! "/forcequotacheck" because --boot is not specified.
Reading config file "/usr/lib/tmpfiles.d/lvm2.conf"…
...
Running clean action for entry d /run/sendsigs.omit.d
Running clean action for entry L /etc/mtab
Running clean action for entry Q /home
Running clean action for entry q /srv
Running clean action for entry L /var/lock
...
etc.
Analyse the output to determine why the job is failing.

5. If the service isn't failing right now, proceed to next steps to enable debug logging in the service itself. The next time the service runs (once per day) it will run with debug logging.

6. Edit the "systemd-tmpfiles-clean.service" file. 
vi /lib/systemd/system/systemd-tmpfiles-clean.service
7. Add (Environment="SYSTEMD_LOG_LEVEL=debug") to the [Service] section of the file.

Sample output: 
[Unit]
Description=Cleanup of Temporary Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
DefaultDependencies=no
Conflicts=shutdown.target
After=local-fs.target time-sync.target
Before=shutdown.target

[Service]
Type=oneshot
ExecStart=/bin/systemd-tmpfiles --clean
SuccessExitStatus=65
IOSchedulingClass=idle
Environment="SYSTEMD_LOG_LEVEL=debug"
8. Reload the daemon and start the service to make sure debug is working. In the output of the status command, there should be more details. 
systemctl daemon-reload
systemctl start systemd-tmpfiles-clean
systemctl status systemd-tmpfiles-clean
The additional debug logs can be viewed in full using journalctl.
Sample output: 
> journalctl -u systemd-tmpfiles-clean.service
...
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry z /var/log/journal
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry d /var/log
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry d /var/spool
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry d /var/cache
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Failed to determine whether '/var/tmp/systemd-private-18308ee298584729b5ae914296
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry X /var/tmp/systemd-private-18308ee298584729b5ae91
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Failed to determine whether '/var/tmp/systemd-private-18308ee298584729b5ae914296
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry x /var/tmp/systemd-private-18308ee298584729b5ae91
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry d /var/lib
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry L /var/run
Dec 03 06:04:40 ip-172-31-0-9.ap-northeast-2.compute.internal systemd-tmpfiles[36897]: Running clean action for entry q /var
...

Further Troubleshooting
If further troubleshooting is required after analysing the debug output with the error, the configuration files/tasks can be isolated to narrow down the problem.

a) View systemd-tmpfiles configuration. 
/bin/systemd-tmpfiles --cat-config
Sample output: 
$ /bin/systemd-tmpfiles --cat-config | grep -iv "^#\|^$"
L /var/lib/dbus/machine-id      -       -       -       -       /etc/machine-id
L /run/shm     -    -    -    -   /dev/shm
d /run/sendsigs.omit.d 0755 root root -
L+ /etc/mtab   -    -    -    -  ../proc/self/mounts
Q /home 0755 - - -
q /srv 0755 - - -
h /var/log/journal - - - - +C
h /var/log/journal/%m - - - - +C
h /var/log/journal/remote - - - - +C
L /var/lock - - - - ../run/lock
d /run/lock/subsys 0755 root root -
r! /forcefsck
r! /fastboot
r! /forcequotacheck
d /run/lock/lvm 0700 root root -
d /run/lvm 0700 root root -
r! /etc/gshadow.lock
r! /etc/shadow.lock
...
etc.
b) There are many different types of tasks executed. More details about the types of tasks can be found in the tmpfiles.d man page (e.g. https://man.archlinux.org/man/tmpfiles.d.5.en).

Here is a short 1 sentence summary of some types, refer to man page for full details. 
L, L+
Create a symlink if it does not exist yet. 

d
Create a directory.

Q
Create the subvolume or directory the same as v, but assign the new subvolume to a new leaf quota group.

q
Create a subvolume or directory the same as v, but assign the subvolume to the same higher-level quota groups as the parent.

h
Set Linux file/directory attributes. Lines of this type accept shell-style globs in place of normal path names.

c, c+
Create a character device node if it does not exist yet.

d
Create a directory.

r
Remove a file or directory if it exists.

a, a+
Set POSIX ACLs (access control lists), see acl(5).

z
Adjust the access mode, user and group ownership, and restore the SELinux security context of a file or directory, if it exists.

x
Ignore a path during cleaning.

If the exclamation mark ("!") is used, this line is only safe to execute during boot, and can break a running system.

c) The systemd-tmpfiles-clean service in Gravity reads the configuration from 2 directories. 
$ ls -ltr /run/tmpfiles.d/
total 4
-rw-r--r--. 1 root root 564 Nov  9 01:03 kmod.conf

$ ls -ltr /usr/lib/tmpfiles.d/
total 52
-rw-r--r--. 1 planet planet   61 Mar 17  2017 lvm2.conf
-rw-r--r--. 1 planet planet   77 May  5  2017 rpcbind.conf
-rw-r--r--. 1 planet planet  239 May 17  2017 passwd.conf
-rw-r--r--. 1 planet planet  655 Feb 14  2019 x11.conf
-rw-r--r--. 1 planet planet  596 Feb 14  2019 systemd-nologin.conf
-rw-r--r--. 1 planet planet 1096 Feb 14  2019 journal-nocow.conf
-rw-r--r--. 1 planet planet  362 Feb 14  2019 home.conf
-rw-r--r--. 1 planet planet  568 Jun  7  2019 var.conf
-rw-r--r--. 1 planet planet  756 Jun  7  2019 tmp.conf
-rw-r--r--. 1 planet planet 1735 Jun  7  2019 systemd.conf
-rw-r--r--. 1 planet planet  812 Jun  7  2019 legacy.conf
-rw-r--r--. 1 planet planet  517 Jun  7  2019 debian.conf
-rw-r--r--. 1 planet planet  153 Jul  2  2020 dbus.conf

d) The configuration files can be removed to try and isolate the problem. For example, move all configuration files to a backup directory and add 1-2 configuration files back at a time to see if the job still fails.

Make a backup directory and move all the files. 
cd /usr/lib/tmpfiles.d/
mkdir /var/lib/gravity/planet/share/my_backup
mv * /var/lib/gravity/planet/share/my_backup/
ls -ltr /usr/lib/tmpfiles.d/

cd /run/tmpfiles.d/
mkdir /var/lib/gravity/planet/share/my_backup2
mv * /var/lib/gravity/planet/share/my_backup2/
ls -ltr /run/tmpfiles.d/
Copy 1-2 files back at a time. 
cp /var/lib/gravity/planet/share/my_backup/tmp.conf /usr/lib/tmpfiles.d/
ls -ltr /usr/lib/tmpfiles.d/
Номер статьи базы знаний

001116648

 
Загрузка
Salesforce Help | Article