JSON logging from pfSense Applications

I use pfSense as my home router and firewall with the pfBlockerNG package to eliminate ads and trackers online. I love everything about it, except the reporting interface. It’s slow and clunky. I wanted to get the data into ClickHouse so I can create dashboards with Grafana. Unfortunately, pfBlockerNG only logs data to the local filesystem.

This post is for folks who want to export data in log files from pfSense to a central log server. We’ll cover how to do this for the pfBlockerNG DNSBL log, but it will work for any other service logs that don’t use syslog, like pfSense zeek.

Prequisites

Before we start, you’ll need to install and configure pfBlockerNG. If you are unfamiliar with it, here are a few good tutorials to help you install it:

• nguvu guide
• Privacy Affairs Guide

You’ll need to install the syslog-ng package from the Packages section of the pfSense UI.

You will also need a central log server somewhere nearby to receive the logs.

Goals

The goal of this setup is to forward the pfBlockerNG DNSBL logs from the pfSense appliance to the central log server. The DNSBL logs live on the filesystem as /var/log/pfblockerng/dnsbl.log in CSV format. We’re going to forward them as JSON using the Common Event Expression CEE format. I use processors to extract that data and insert it into ClickHouse, but you can also use Elasticsearch or OpenSearch.

Enabling syslog-ng

Once you install syslog-ng, navigate to “Services” > “Syslog-ng”. In the “General” tab, tick the “Enable” checkbox and “Include SCL” checkbox, then click “Save.” The other options on the General tab don’t really matter for our use case.

Add the Source

Next we’ll need to add the pfb_dnsbl source to syslog-ng. The UI here isn’t intuitive even if you’re familiar with syslog-ng. Here’s a screen shot:

This configuration will generate the following syslog-ng snippet:

source pfb_dnsbl {
    file("/var/log/pfblockerng/dnsbl.log"
        default-facility(security)
        default-priority(notice)
        program-override("pfBlockerNG")
        flags(no-parse)
    );
};

There’s nothing too scary here, but a few important pieces:

• program-override("pfBlockerNG") sets the $PROGRAM macro for the resulting log output
• flags(no-parse) disables the default syslog format parser as the file is CSV and we’ll need to build our own parser

Create a Parser

We’ll need to create our own CSV parser for the file. Again, the UI is terrible, but it looks like this:

This configuration will generate the following syslog-ng snippet:

parser pfb_dnsbl {
    csv-parser(
        columns("service","time","dst","src_ip","call","action","class","sld","rule_id","hitormiss")
        delimiters(",")
        prefix(".SDATA.dnsbl.")
    );
};

Again, nothing too complicated here, but there’s a few notes:

• columns(...) contains the key names of columns in the dnsbl.log file
• prefix(".SDATA.dnsbl.") creates the keys in the .SDATA.dnsbl. namespace. The trailing . is important!

Create a Destination

Next we’ll need a destination to send the logs. This destination is going to take the .SDATA. namespace, convert it to JSON and add a $MSGHDR to the front and forward it along. Here’s a screenshot:

destination CentralJSON {
    network("10.0.1.9"
        port(514)
        transport("tcp")
        template("<${PRI}>${ISODATE} ${HOST} ${MSGHDR}$(format-json --scope sdata --rekey .* --shift-levels 2)\n")
        persist-name("central-logs-json")
    );
};

The hardest part of getting this configuration to work was sorting out the format-json parameters. The --scope parameter selects the variables that are available to JSONify. The docs were a little hard to find so that link is as much for me to be able to find it as it is for you! By specifying --scope sdata, only the .SDATA.* fields are available. If we stopped there, that part of the message would look like {".SDATA":{"dnsdbl":{...}}}. I wanted my messages to skip that .SDATA key, so I applied a --rekey .* --shift-levels 2. The rekey applies to all variables since I used .*. The --shift-levels 2 shifts the key space by 2 dots. So, .SDATA.dnsbl.* becomes dnsbl.*. The resulting messages have the {"dnsbl":{...}} format.

Putting It All Together

The last step is to combine these pieces into a log statement in syslog-ng. The pfSense UI automatically rebuilds the configuration and restarts the service when you click Save. Once you save, you should start to see the logs land on the destination server.

In the UI, the log set looks like this:

log {
   source(pfb_dnsbl);
   parser(pfb_dnsbl);
   destination(CentralJSON);
};

That’s it! Now logs should be arriving in JSON format at our central log server. I setup a quick Grafana dashboard:

There are a few issues with my current setup. I’m relying on the timestamp of the syslog message, so I get lots of spikes in the data. I could fix that by using the time field from the JSON itself, but this is a great start!