Tiny Core Base > TCB Q&A Forum
/var/log/wtmp rotating?
Rich:
Hi Stefann
For text files, here is how you can limit length (I'll address wtmp later):
--- Code: ---#Limit log file to 100 lines.
echo "$(tail -n 100 xyzzy.log)" > xyzzy.log
--- End code ---
Or conditionally:
--- Code: ---#Limit log file to 100 lines if it's greater than 150 lines.
[ $(wc -l xyzzy.log | cut -d " " -f1) -gt 150 ] && echo "$(tail -n 100 xyzzy.log)" > xyzzy.log
--- End code ---
Limiting wtmp needs to be handled differently since it's stored in a
binary format, not a text format. Examining the file in hex suggested
fixed length entries of 384 bytes, confirmed here:
https://askubuntu.com/questions/325491/how-to-properly-display-the-contents-of-the-utmp-wtmp-and-btmp-files
This means wtmp needs to be trimmed in multiples of 384 bytes.
Here is the current status of my wtmp file:
--- Code: ---tc@E310:~$ last -f /var/log/wtmp | tail -n 1
wtmp begins Sun Jun 23 18:33:39 2024
tc@E310:~$ last -f /var/log/wtmp | head -n 1
tc pts/7 :0.0 Sun Oct 13 08:26 still logged in
tc@E310:~$ wc -c /var/log/wtmp
181248 /var/log/wtmp
tc@E310:~$ calc 181248/384
472
tc@E310:~$
--- End code ---
Its first entry is dated June 23 and the last entry is dated Sun Oct 13 08:26.
The file contains 472 entries each 384 bytes long.
Lets say I want to shorten it to 275 entries:
--- Code: ---tc@E310:~$ calc 275*384
105600
tc@E310:~$ tail -c 105600 /var/log/wtmp > NewWtmp
tc@E310:~$ sudo mv NewWtmp /var/log/wtmp
tc@E310:~$ last -f /var/log/wtmp | tail -n 1
wtmp begins Wed Aug 21 10:35:48 2024
tc@E310:~$ last -f /var/log/wtmp | head -n 1
tc pts/7 :0.0 Sun Oct 13 08:26 still logged in
tc@E310:~$ wc -c /var/log/wtmp
105600 /var/log/wtmp
tc@E310:~$
--- End code ---
The size has been reduced to 105600 bytes (275*384).
Its first entry is now dated Aug 21 and the last entry is still dated Sun Oct 13 08:26.
The file now contains 275 entries each 384 bytes long.
You can launch something like this to periodically trim certain files in the background.
TrimLogs.sh:
--- Code: ---#!/bin/sh
# List of text logs to trim formatted as /Path/Filename:MinSize:MaxSize
# MinSize and MaxSize refer to number of lines.
# Example:
TextLogs="/var/log/messages:300:350 /var/log/Xorg.0.log:526:700 AnotherPath/Filename:MinSize:MaxSize"
# Variables for trimming /var/log/wtmp.
# How many entries to keep.
Entries=275
#
EntrySize=384
FileSize=$(($Entries*$EntrySize))
# Run forever.
while true
do
# Process the list of text based logs.
for F in $TextLogs
do
LogFile="$(echo $F | cut -d ":" -f1)"
MinSize=$(echo $F | cut -d ":" -f2)
MaxSize=$(echo $F | cut -d ":" -f3)
[ $(wc -l $LogFile | cut -d " " -f1) -gt $MaxSize ] && echo "$(tail -n $MinSize $LogFile)" > $LogFile
done
# Process /var/log/wtmp.
tail -c $FileSize /var/log/wtmp > NewWtmp
sudo mv NewWtmp /var/log/wtmp
# Sleep for 8 hours, then trim again.
sleep 8h
done
--- End code ---
Then launch it like this:
--- Code: ---TrimLogs.sh &
--- End code ---
Stefann:
Rich, thanks a lot!
3 questions/remarks.
1/
On your 1st in-line script you run tail on a file and write the result back in the original file without using a temp intermediate file. Isn’t that dangerous? Does that not have a risk of getting entangled and conflict?
It’s a very nice code-line. I can use that in my rsyslog.conf file instead of calling a rotate script (I like your approach because that reduces the amount of small interlinked cluttering files)
2/
Thanks for wtmp explaination, I’ll follow it, but that will be next weekend as a workweek is showing up now.
3/
Yes I understand the shell script and that will certainly work. In fact (apart from the 384byte handling of wtmp) I have things nicely setup now.
But….
Point was different…
Should we not include something like this in next TC release?
For reason that in fact one can consider this a memory leak: “vanilla tc is growing ram-usage if not restarted”.
As I said earlier, the risk is quite minor. You yourself had grown to 100k in 3.5month. My file had grown to 22k in about a month. Still it’s a bit ugly.
I’m afraid I’m not yet a skills level I could contribute on that.
And with that said, it’s only a community concern, I have my own stuff covered now.
Rich:
Hi Stefann
--- Quote from: Stefann on October 13, 2024, 01:39:14 PM --- ... On your 1st in-line script you run tail on a file and write the result back in the original file without using a temp intermediate file. Isn’t that dangerous? ...
--- End quote ---
If you try to do it this way:
--- Code: ---tail -n 100 xyzzy.log > xyzzy.log
--- End code ---
It won't work
When you do it this way:
--- Code: ---echo "$(tail -n 100 xyzzy.log)" > xyzzy.log
--- End code ---
The tail command executes first. The results are then presented
to the echo command and then redirected into the original file.
(The actual mechanics may be more complex than that).
--- Quote --- ... Should we not include something like this in next TC release?
For reason that in fact one can consider this a memory leak: ...
--- End quote ---
This particular machine has 3 Gig of RAM. So a couple of hundred Kbytes
of RAM gets lost in the noise, especially compared to a pig like Firefox
which needs to be restarted every so often to force it to release some of
the RAM it's hogging.
Stefann:
Rich
Thanks for explanation on “no need of intermediate file”
I like the single line command
I will probably use that in my rsyslog.conf file for rotating logs.
I “just managed” to get that working this weekend by using an extra script file. The “1 line command” opens the door to do it in the rsyslog.conf without needing that extra script.
I like that.
It gets rid of an “undocumented script file” and brings all in the rsyslog.conf that is a clear documented need for rsyslog
I feel the wtmp rotation can be converted similarly in a 1 line command. I will be able to figure that out. Just following the logic of the other command.
patrikg
Also thanks for the “1 byte comment” and the -n suggestion.
You are very right, I indeed did see the file to be 1 byte big after flush.
This morning I checked and it’s now 2688 bytes, 7x384. So “somehow things corrected itself”.
At least this means there is no immediate need to change things.
Anyway
Weekend is over. Things will have to wait until next week.
I will definitely followup and report on how I got things finally solved.
curaga:
The right place for this functionality would probably be busybox. We don't have default crontabs, cron isn't even running by default. If you know C, you could submit a patch to busybox.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version