Recent content on Inliniac
I’ve just pushed out a minor update to Vuurmuur. Basically a single important bug fix, and lots of minor cleanups. Both in Github Actions and my local CI I’ve updated the build tests and the scanners like cppcheck. There are quite a few debs for Ubuntu and Debian. On the rpm side, just Fedora. Since libnetfilter_log is still missing from EPEL, creating CentOS rpms isn’t feasible…
After many years of hosting this blog at Wordpress, I’ve decided to convert it to a static hugo site. I like that I can just write a simple markdown file and publish it. I’ve done the conversion using https://github.com/ashishb/wp2hugo , which was very easy. There is no option to comment anymore. This feature wasn’t used much anyway, but if you have feedback please see contact .…
I’ve just pushed the 0.8 release. See my announcement here . Get it from github or the ftp server . Largest changes: ipv6 support using ip6tables logging uses nflog - initial work by Fred Leeflang connection logging and viewer add rpfilter and improved helper support a ‘dialog’ based setup wizard single code base / package massive code cleanup I plan to continue to work on…
In Suricata we’re experimenting with implementing app-layer parser in Rust. See Pierre Chifflier’s presentation at the last SuriCon: [ pdf ]. The first experimental parsers will soon land in master. So coming from a C world I often use a pattern like: value = hash_lookup(hashtable, key) if ( ! value) { hash_insert(hashtable, key, somevalue); } Playing with Rust and it’s HashMap…
Over the holidays I’ve spent some time refreshing the Vuurmuur code. One major thing that is now done is that the 3 different ‘projects’ (libvuurmuur, vuurmuur and vuurmuur-conf) are now merged into a single ‘project’. This means that a single ‘./configure && make && make install’ now installs everything. When I originally started Vuurmuur I had much…
Since the beginning of the project we’ve spoken about variables on multiple levels. Of course flowbits defined by the Snort language came first, but other flow based variables quickly followed: flowints for basic counting, and vars for extracting data using pcre expressions. I’ve always thought of the pcre data extraction using substring capture as a potentially powerful feature.…
Yesterday I wrote about fuzzing Suricata with AFL . Today I’m going to show another way. Since early in the project, we’ve shipped a perl based fuzzer called ‘wirefuzz’. The tool is very simple. It takes a list of pcaps, changes random bits in them using Wiresharks editcap and runs them through Suricata. Early in the project Will Metcalf, who wrote the tool, found a lot of…
AFL is a very powerful fuzzer, that tries to be smarter than random input generating fuzzers. It’s cool, but needs a bit more baby sitting. I’ve added some support to Suricata to assist AFL. Here’s how to get started on fuzzing pcaps. mkdir ~/ tmp / fuzz git clone https: // github . com / inliniac / suricata - b dev - afl - v5 cd suricata git clone https: // github . com / OISF /…
Today, almost 2 years after the release of Suricata 2.0, we released 3.0! This new version of Suricata improves performance, scalability, accuracy and general robustness. Next to this, it brings a lot of new features. New features are too numerous to mention here, but I’d like to highlight a few: netmap support: finally a high speed capture method for our FreeBSD friends, IDS and IPS…
As the team is back from a very successful week in Barcelona, I’d like to take a moment on what we discussed and decided on with regards to development. One thing no one was happy with is how the release schedules are working. Releases were meant to reasonably frequent, but the time between major releases was growing longer and longer. The 2.0 branch for example, is closing in on 2 years as…
If you like fiddling with Suricata development, maybe you can get paid to do it. Companies ask me regularly if I can recommend Suricata developers. I’m going to assemble a list of people who are interested in such work. If you like me to consider you in such cases, drop me an email. If you really want me to *recommend* you, it’s important that I actually know you somewhat. So becoming…
Due to a ‘administrative problem’ between my registrar Xs4all and their US-partner Network Solutions, my domain has been offline since Sunday. Resolving the issue took them some time, and there was a technical issue after the administrative one was resolved. Add long DNS TTL values into the mix, and the disruption was quite lengthy. The domain is back up, although it may still take…
Thanks to the hard work of Arturo Borrero Gonzalez, Suricata has just been added to the Debian ‘backports’ repository. This allows users of Debian stable to run up to date versions of Suricata. The ‘Backports’ repository makes the Suricata and libhtp packages from Debian Testing available to ‘stable’ users. As ’testing’ is currently in a freeze, it…
JEMALLOC is a memory allocation library: http://www.canonware.com/jemalloc/ It offers many interesting things for a tool like Suricata. Ken Steele of EZchip (formerly Tilera) made me aware of it . In Ken’s testing it helps performance. Install wget http: // www . canonware . com / download / jemalloc / jemalloc - 3.6 . 0. tar . bz2 tar xvfj jemalloc - 3.6 . 0. tar . bz2 cd jemalloc - 3.6 . 0…
At it’s core, Suricata is a packet processor. It reads packets and pushes them through a configurable pipeline. The 2nd most important processing unit in Suricata is the flow. In Suricata we use the term flow for the bidirectional flows of packets with the same 5 tuple (proto, src ip, dst ip, sp, dp. Vlans can be added as well). In fact, much of Suricata’s threading effort revolves…
In 2.1beta2 the long awaited SMTP file extraction support for Suricata finally appeared. It has been a long development cycle. Originally started by BAE Systems, it was picked up by Tom Decanio of FireEye Forensics Group (formerly nPulse Technologies) followed by a last round of changes from my side. But it’s here now. It contains: a MIME decoder updates to the SMTP parser to use the MIME…
After a lot of preparations, it’s finally going to happen: official Suricata trainings ! In the next couple of months I’ll be doing at least 3 sessions: a home match (Amsterdam) , a workshop in Luxembourg and a session at DeepSec . Next to this, we’re planning various US based sessions on the East coast and West coast. I’m really looking forward to doing these sessions.…
In Suricata we’re often not printing malloc errors. The reason is that we’re not willing to print such errors based on (attacker controlled) traffic. So often such cases are silently handled. We came across a bug though, where a integer underflow led to -1/0xffffffff being passed to malloc. Luckily, malloc just failed by returning NULL, and this return was properly handled. Still,…
Pretty much from the start of the project, Suricata has been able to track flows. In Suricata the term ‘flow’ means the bidirectional flow of packets with the same 5 tuple. Or 7 tuple when vlan tags are counted as well. Such a flow is created when the first packet comes in and is stored in the flow hash. Each new packet does a hash look-up and attaches the flow to the packet. Through…
The OpenSSL heartbleed vulnerability is a pretty serious weakness in OpenSSL that can lead to information disclosure, in some cases even to to private key leaking. Please see this post here http://blog.existentialize.com/diagnosis-of-the-openssl-heartbleed-bug.html for more info. This is a case where an IDS is able to detect the vuln, even though we’re talking about TLS. LUA I’ve…
I’ve made a video on installing Suricata 2.0 on Debian Wheezy. The video does the installation, quick setup, ethtool config and shows a simple way to test the IDS. It’s the first time I’ve made such a video. Feedback is welcome.
Today I finally released Suricata 2.0 . The 2.0 branch opened in December 2012. In the little over a year that it’s development lasted, we have closed 183 tickets. We made 1174 commits, with the following stats: 582 files changed, 94782 insertions(+), 63243 deletions(-) So, a significant update! In total, 17 different people made commits. I’m really happy with how much code and…
For my Suricata QA setup, I’m using tcpreplay on a dual port gigabit NIC. The idea is to blast out packets on one port and then have Suricata listen on the other part. For the traffic replay I’m using tcpreplay 3.4.4 from the Ubuntu archive. As I have a lot of pcaps to process I intend to use the –topspeed option to keep runtimes as low as possible. This will result in…
I’ve been spending the holidays to upgrade some of my own servers. One of them is the Sguil server I use. Until now it ran Debian Squeeze. On Debian Squeeze you could use tcl8.3, which has threading disabled. For Sguil tcl threading needs to be disabled: ERROR: This version of tcl was compile with threading enabled. Sguil is NOT compatible with threading. This is a compile time option in…
With the holidays approaching and the 1.4.7 and 2.0beta2 releases out, I thought it was a good moment for some reflection on how development is going. I feel things are going very well. It’s great to work with a group that approaches this project from different angles. OISF has budget have people work on overall features, quality and support. Next to that, our consortium supporters help…
I have revoked my old gpg keys 0BA788C7 and F5DFF229. They are superseded by 926856B4. You can grab that from http://pgp.mit.edu . Reason for the update is to get rid of the 1024 bit keys. I have no reason to believe they were compromised :)
Last week I’ve added some more profiling options to Suricata. It’s part of the current git master. It’s enabled only when --enable-profiling and then through the suricata.yaml: profiling: # per keyword profiling keywords: enabled: yes filename: keyword_perf.log append: yes This will output a table similar to below:…
Next month I will be attending Hack.lu . The entire Suricata team will be present as well. We’ll be doing several meetings, including a training day on the 24th of October. If you are close and interested in Suricata, please consider joining us. The training is free and does not require you to pay for the conference.
Ran into a problem with autotools today, thought I’d share my solution. First, the error only happened on an old system: $ bash autogen . sh Found libtoolize Remember to add ` AC_PROG_LIBTOOL ' to `configure.ac' . libtoolize: ` config . guess ' exists: use `--force' to overwrite libtoolize: ` config . sub ' exists: use `--force' to overwrite libtoolize: ` ltmain . sh ' exists: use `--force'…
This morning I added flowint lua functions for incrementing and decrementing flowints. From the commit : Add flowint lua functions for incrementing and decrementing flowints. First use creates the var and inits to 0. So a call: a = ScFlowintIncr(0) Results in a == 1. If the var reached UINT_MAX (2^32), it’s not further incremented. If the var reaches 0 it’s not decremented further.…
A few days ago I wrote about my Emerging Threats sponsored work to support flowvars from Lua scripts in Suricata. Today, I updated that support. Flowvar ‘sets’ are now real time. This was needed to fix some issues where a script was invoked multiple times in single rule, which can happen with some buffers, like HTTP headers. Also, I implemented flowint support. Flowints in Suricata are…
When processing the TCP 3 way handshake (3whs), Suricata’s TCP stream engine will closely follow the setup of a TCP connection to make sure the rest of the session can be tracked and reassembled properly. Retransmissions of SYN/ACKs are silently accepted, unless they are different somehow. If the SEQ or ACK values are different they are considered wrong and events are set. The stream events…
Funded by Emerging Threats, I’ve been working on giving the lua scripts access to flowvars. Currently only “flowvars” are done, “flowints” will be next. Please review the code at: https://github.com/inliniac/suricata/tree/dev-lua-flowvar Pcre based flowvar capturing is done in a post-match fashion. If the rule containing the “capture” matches, the var is…
Major 1.4 update. https://suricata.io/2013/03/08/suricata-1-4-1-released/
I just released a new Vuurmuur version: 0.8rc1. The first release candidate for the 0.8 series. This release improves IPv6 support a lot. The wizard is now also fully functional. Try “vuurmuur_conf –wizard”. Improved IPv6 support: #115 Improved Debian packages, switching to nflog as default for logging. Fix connection viewer not showing accounting on newer systems. #141 Amd64…
So with 1.4 out the door we have a new stable. However, we’re keeping 1.3 around for a few more months to give everyone the chance to plan updating to 1.4. Of course, we think 1.4 is a lot better than anything we released before, so we do recommend updating as soon as you can. Continued support for 1.3 means we’ll do more releases to fix critical issues. We’ll probably include…
About 5 months after 1.3 came out we’ve released 1.4 , and we’ve been quite busy. Eric Leblond’s post here has all the stats and graphs. There are three big new features: unix socket , ip reputation and luajit . For each of these the same is true: it’s usesable now, but it’s the potential that we’re most excited about. Over the next months we’ll be…
First stable release in the 1.4 branch, right in time for the holidays. Enjoy! https://suricata.io/2013/03/08/suricata-1-4-1-released/
Lots of activity on the IPv6 front lately. There was a talk on a conference on bypassing IDS using IPv6 tricks. Also a new scan tool (Topera) claimed to scan a host while staying below the radar of an IDS was released. To start with the latter, even though Suricata doesn’t have a dedicated port scan detector, the tool’s traffic lights up like a Christmas tree. The trick it pulls is to…
I just made Suricata 1.4rc1 available with some pretty exciting features: unix socket mode and IP reputation. Unix socket First of all, Eric Leblond’s work on the Unix socket was merged. The unix socket work consists of two parts. The unix socket protocol implementation and a new runmode. The protocol implementation is based on JSON messages over unix socket. Eric will be fully documenting…
Disclaimer: this work was sponsored by Emerging Threats Pro . One thing we’ve been talking about for many years at OISF is IP Reputation. The basic idea is that many organizations have information about specific IP-addresses. This information may be that a host is infected, acts as a spam relay or many other things. We’ve always thought it might be useful to apply this info to the IDS…
We just released Suricata 1.3.3 which contains some important accuracy fixes. Also, it should be much more robust against out of memory conditions. For those of you running Suricata in IPS mode, this is important as well. We found that rules that have the drop or reject actions, were not playing well with thresholding. So upgrading is highly recommended! Code changes are not too big, largest…
The Dutch security site security.nl has interviewed me about the Suricata IDS project . The two part (Dutch language) article can be found here and part two here . Thanks to Joran Polak of security.nl for giving me the opportunity to tell something about this project!
I recently found out that Fedora includes Vuurmuur in it’s repositories. Since Suricata is also included, I figured I would do a quick write up on how to setup a Fedora IPS. While writing it turned more into a real “howto”, so I decided to submit it to Howtoforge. It can be found here one HowtoForge . Vuurmuur on Fedora is at the 0.7 version, which is still the current stable.…
Today, a day after 1.3.2 , we’ve released 1.4beta2 . While 1.3.2 is an important update for those running 1.3.1 or lower, today’s release is where things get exciting. A lot of things were improved and added. Let me show some numbers first. The 1.4beta2 release is a pretty big update over 1.4beta1 as it touches over 5k lines of code: 234 files changed, 5033 insertions(+), 3759…
Today we released Suricata 1.3.2. Not a big update, but there are some important fixes in the stream engine, fast_pattern:chop handling, HTTP multipart parsing and the flow keyword with “nostream”. As the diff stat output shows, it’s a rather light maintenance update over 1.3.1: [sourcecode] ChangeLog | 12 ++ libhtp/configure.ac | 2 +- libhtp/htp.pc.in | 2 +- libhtp/htp/htp.h | 2…
After an exciting week of meeting and working with the team around the RAID conference, time for another lua update. The keyword supports an interesting set of buffers now: packet payload http.uri http.uri.raw http.request_line http.request_headers http.request_headers.raw http.request_cookie http.request_user_agent http.request_body http.response_headers http.response_headers.raw…
Today I decided to look into the potential performance of the luajit keyword a bit. It’s important to know if this can perform at reasonable speeds so that we can actually use it in real deployments. Even if we can’t the feature may still be appealing though, for offline pcap analysis. So far, the results are rather encouraging. First, I added 2 buffers today: http.uri, which contains…
Today I improved the lua jit support in Suricata further. The scripts will now need to express their “needs” through an “init” function in the script that is called only at Suricata startup. The “init” function fills a lua table. This will allow the user to indicate what buffers the script needs to inspect. The script will then only be invoked when these buffers…
The first test release for the new Suricata 1.4 branch as just been released. Some really exciting stuff was added. Let me highlight some of it: AF_PACKET IPS mode: Eric Leblond has been working on extending the passive AF_PACKET support to support IPS as well. Eric has documented the new feature on his blog . TLS logging and certificate storage: created by contributor Jean-Paul Roliers under…