QueueMetrics > MySQL storage and Qloaderd/Uniloader

qloader.pl patch for better timestamp handling

(1/1)

gb_delti:
In the past few days we had three incidents where the qloaderd wrote timestamps into the database that were very far in the future. This lead to a complete standstill of all statistics and the inability of agents to make call classifications. We haven't found the cause yet (probably a broken line in queue_log).

To prevent this from happening again and to analyze the source of the problem, I made the following modifications to qloader.pl (posted as a diff):

--- Code: ------ old/qloader.pl        2009-08-19 16:28:56.000000000 +0200
+++ qloader.pl  2010-03-02 16:00:30.574029731 +0100
@@ -46,6 +46,7 @@
 my $log_queries   = 0;            # set to 1 to log all queries

 my $timezone_offset  = 0 * 3600;  # in seconds
+my $max_future_ts    = 15;        # in seconds, how many seconds a timestamp can lie in the future
 my $heartbeat_delay  = 15 * 60;   # in seconds
 my $use_subqueue     = 0;         # 0 no; 1 yes
 my $split_subq_name  = 0;         # 0 no; 1 yes - turn a subqueue name from 'xxx/yyy" to "xxx"
@@ -116,7 +117,11 @@
                        $rowdata[3] = agentRewriteByDB( $rowdata[3] );
                }

-               if ( $tst < $highWaterMark ) {
+               if ( $tst > time() + $max_future_ts ) {
+                       syslog("Timestamp $tst is too far in the future. Skipping line '$_'");
+                       skipRow( @rowdata );
+               }
+               elsif ( $tst < $highWaterMark ) {
                        # salta le righe obsolete
                        skipRow( @rowdata );
                }

--- End code ---

If I get broken timestamps again I will post the results from the log output. Until then I'd love to hear your comments.

QueueMetrics:
I think that the latest version of the qloaderd includes something like that.

It is revision 1.18 and the relevant bug was #792.

In the cases we tested, it was caused by wrong data written on the queue_log.

AndyML:
I have been experiencing this problem but can't seem to find any download-links for 1.18. Is this indeed fixed in 1.18? Has it been released?

QueueMetrics:
You can get that from the tgz file of QM 1.6.1.0, as per:

QueueMetrics-1.6.1.0-trial.tar.gz\queuemetrics-1.6.1.0\WEB-INF\mysql-utils\qloader

This contains 1.18.

Navigation

[0] Message Index