Thursday, March 7, 2013

rrd illegal time fix, and editing xml with sed


And now for some BASHing...

If you use rrd, and do a reboot of your computer, on rare occasion you might run into an error like the following (I put xxx in some places for privacy).  

ERROR: Cannot update /var/www/mrtg/xxx/10.x.x.5_xx.rrd with '1359389221:4100149299:563322753' /var/www/mrtg/xxx/10.x.x.5_xx.rrd: illegal attempt to update using time 1359389221 when last update time is 1359402962 (minimum one second step)

That long 10 digit number starting with 1359... is a unix time stamp, which is just the number of seconds that have elapsed since january 1st 1970.  If you can't figure out the cause of this error, you can edit the rrd files.  Unfortunately they are in a binary format and need to first be exported to xml, edited, and then converted back to binary format.  While there is a python script for editing xml files exported from rrd databases, it required a library which I was unable to properly install on my linux machine.  So I decided to create a bash script using sed, and not bother with python (or it's libraries)


Remember this is a BASH script, not a BATCH script.  So if you're going to run this on a windows machine you'll need to install cygwin first, and run it from inside of cygwin.  



#!/bin/bash
# you will probably want to run this as
# find . -iname "*.rrd" -print0 |xargs -0 -I {} rrd-fixer.sh {}

#first we convert the rrd file to xml format
rrdtool dump $1 > $1.xml
#we need to figure out the current unix date (epoc format, number of seconds
#since 1970)
udate=`date +%s`
#the heart and soul, take out the bad last update stamp, and replace it with
# one mrtg (or other rrd based tools), won't barf on
sed -e "s/<lastupdate>1[0-9]\{9\}/<lastupdate>$udate/" $1.xml > $1.fixed.xml
#rrdtool won't overwrite the bad rrd file on it's own, so have to remove
#it ourselves
rm $1
#convert the .xml file back to .rrd format
rrdtool restore $1.fixed.xml $1
#cleanup the xml files
rm $1.xml
rm $1.fixed.xml

# ---------- end of bash script --------------------------

note: if you want to run this on a folder with many rrd files (like mrtg or smokeping will often create), you will probably want to run the script in the following manner:
find . -name "*.rrd" -print0 |xargs -0 -I {} rrd-fixer.sh {}
or if you want to simplify it:
ls -1 *.rrd |xargs rrd-fixer.sh



fyi, the python script I came across is at: http://pierre.palatin.fr/old/rrd-repair

1 comment:

  1. Thanks for posting this. I was also unable to get the ElementTree library sorted out so your script was a good option.

    ReplyDelete