Serious problems with buffered accounting

23 Nov 2015

      Hi,

   Nowadays, I have tons of Accounting packets as stats below.

# radsniff -i eth0 -f "port 1813" -W10

######### Stats Iteration 2 #########
Interface capture rate:
        eth0      : 42.700/s
Accounting-Request counters:
        Total     : 21.400/s
        Linked    : 21.200/s
        Unlinked  : 0.000/s
Accounting-Request latency:
        High      : 113.875ms
        Low       : 7.446ms
        Average   : 31.365ms
        MA        : 31.365ms
Accounting-Request retransmits & loss:
        RT (0)    : 24.600/s
Accounting-Response counters:
        Total     : 21.200/s
        Linked    : 21.200/s
        Unlinked  : 0.000/s
Accounting-Response latency:
        High      : 113.875ms
        Low       : 7.446ms
        Average   : 31.365ms
        MA        : 31.365ms
#

And my approach was to use the buffered accounting as example below.

# Simple accounting server
server default_accounting {
        listen {
                ipaddr = *
                port = 0
                type = acct
        }

        accounting {
              detail-buffered
        }
}

# written by
detail detail-buffered {
        filename =
${radacctdir}/buffered/detail-%Y%m%d-%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}
        escape_filenames = no
        permissions = 0600
        header = "%t"
        log_packet_header = yes
        suppress {
                User-Password
        }
}

# consumed by
server buffered-accounting {
    listen {
        type = detail
        filename = "${radacctdir}/buffered/detail-*"
        load_factor = 30
        retry_interval = 5
#      poll_interval = 15
#      track = yes
    }

    preacct {
        preprocess
        acct_unique
    }

    accounting {
        -sql_accounting
        if (!ok) {
                linelog-fail-sql
                ok
        }
    }
}

My problem is: The process that receive and write the packets is completely
fast them who consume (buffered-accounting). This causes a big
queue of files in /var/log/freeradius/buffered/ resulting in a giant delay.
I have thinking about some issue related to race-condition or dead-lock.

I really appreciate some tips and suggestions.

--
Jorge Pereira