QueueMetrics > MySQL storage and Qloaderd/Uniloader

Manually dump old log files into qloader database

(1/1)

jhfitness:
I have data from the last couple of weeks going into the sql:P001 database. However I have some old logs such as queue_log.2, queue_log.4, etc, that I would like to manually dump into the database before they are rotated away.

I found this in the user manuals, is this what I want to do?


--- Quote ---To test that the qloader script is working, try the following command:

./qloader.pl /var/log/asterisk/queue_log P02 /var/log/asterisk/qloader.log

This command will try uploading the contents of the file /var/log/asterisk/queue_log to partition P02 of the chosen database, writing its own log on /var/log/asterisk/qloader.log. When the program is running no output is written on stdout; if you want to know how the loading is going you have to consult its own log file.

--- End quote ---

but with /queue_log.X P001 ?

Thanks

jhfitness:
Actually, I got it to work once I passed the correct database connection parameters but the log says "Ignoring timestamps below XXXXXXXX" and everything is skipped.

I found another topic that says it won't accept data older than existing data and this seems to confirm that. Is there a quick temporary to override that or do I need to go the route of deleting the database records and importing my 5 logs one by one?

jhfitness:
OK, so here's what I did. P001 was my only partition so I loaded the missing logs into a new partition, P002, except I forgot to do it in reverse time order so I went on P003:


--- Code: ---./qloader.pl -h localhost -d queuemetrics -u queuemetrics -p (password) /var/log/asterisk/queue_log.5 P003 /var/log/asterisk/qloader.log
./qloader.pl -h localhost -d queuemetrics -u queuemetrics -p (password) /var/log/asterisk/queue_log.4 P003 /var/log/asterisk/qloader.log
./qloader.pl -h localhost -d queuemetrics -u queuemetrics -p (password) /var/log/asterisk/queue_log.2 P003 /var/log/asterisk/qloader.log
--- End code ---

Then I went into mysql and changed the partition name.


--- Code: ---update queue_log set partition='P001' where partition='P003';
--- End code ---

Also, of course, deleting where partition was P002. Now all of my data shows up in P001 like I wanted.

Hope this can help someone else... Feel free to let me know if I've messed something up and should change something. Everything looks good though.

marcos:
Hi,
What you did is correct.
Morehover, we provided a script that you can use for inserting timestamps in an already present queue log.
The script is called queuePartialUpdater.pl and you can use it in this way:

queuePartialUpdater.pl /my/queue_log/file partition_name /my/queue_log/logfile

The script takes the /my/queue_log/file as input and merge the events found with the information stored in the database for the specified partition.
You have to customize it a little bit, before to run it for the first time, in order to better adapt to your database configuration.

Thank you and regards,
Marco Signorini.

Navigation

[0] Message Index