Threat Hunting Lab (Part II) : Sending PfSense Netflow data to Elastic Stack

29th March 2020   |   by hilo21

Over prepare, then go with the flow

Regina Brett

Description

In this part of these blog series we will try to see how can we integrate Netflow with Elastic Stack for increasing visibility. We will be using Netflow data from our PfSense firewall.

Personally, I believe that Netflow data doesn’t bring much to the table when it comes to information security from a Detection-Prevention perspective but it adds much more context to your security operations and gives you a better visibility on your inbound/outbound traffic in general.

In the last blog, Part I, we were able to setup an Elastic Stack all-in-one VM machine.

Starting Kibana
First glance at Kibana UI

Setting up Softflowd on PfSense and Logstash:

Step 1 : Install Softflowd package on PfSense :

On PfSense web interface, go to System / Package Manager / Available Packages then look for softflowd and install it :

Installing softflowd on pfsense

Step 2 : Configure SoftFlowd

On the Services / softflowd panel, configure the softflowd’s parameters as it suites you.

Softflowd settings

For me, I will be forwarding all netflow data to my ElasticSIEM VM at 10.10.10.129 on port 2055 from my WAN and LAN interfaces using Netflow version 9 :

Configuring Softflowd to forward data to ElasticSIEM

Step 3 : Configure Logstash with Netflow module

As we know by now, we’ve already set up an All-in-one ElasticSIEM VM that has Elasticsearch, Kibana and Logstash. For this part of the configuration, we will be using Logstash’s Netflow module to create a netflow index and generate Kibana dashboards automatically. However, using this module won’t allow us to make NetFlow data useful on the Elastic SIEM Module since Logstash 7.2.1 is not compliant with ECS (Elastic Common Schema) yet as described in the next figure :

Logstash

You can use Filebeat’s netflow module since it is compatible with the ECS for a better visibility on your SIEM App. But for simplicity reasons, I won’t be using that because I am not planning on using Netflow data that much in my Labs.

Before starting Netflow module make sure that kibana and elasticsearch are running.

We can use a one-liner command line to run Logstash with the Netflow module by entering this command from the logstash folder installation:

[elastic@elasticsiem logstash]$ bin/logstash --modules netflow --setup -M netflow.var.input.udp.port=2055 -M netflow.var.elasticsearch.hosts="localhost:9200" -M netflow.var.elasticsearch.ssl.enabled=false -M netflow.var.kibana.host="10.10.1.129:80" -M netflow.var.kibana.scheme=http -M netflow.var.kibana.ssl.enabled=false -M netflow.var.kibana.ssl.verification_mode=disable

Lets break it down:

  • bin/logstash --modules netflow : option spins up a Netflow-aware Logstash pipeline for ingestion.
  • -- setup : The --setup option creates a netflow-* index pattern in Elasticsearch and imports Kibana dashboards and visualizations. Running --setup is a one-time setup step. Omit this option for subsequent runs of the module to avoid overwriting existing Kibana dashboards.
  • -M netflow.var.input.udp.port=2055 : Specifies the port logstash instance will be listening on.
  • -M netflow.var.elasticsearch.hosts="localhost:9200" : Specifies the Elasticsearch database host for Logstash.
  • -M netflow.var.elasticsearch.ssl.enabled=false : For this Lab, we have an Elasticsearch instance with no required SSL certificate validation so we have to specify that SSL is not enabled.
  • -M netflow.var.kibana.host="10.10.1.129:80" : Specifying Kibana’s host IP & port where we will be uploading Netflow dashboards.
  • -M netflow.var.kibana.scheme=http : Specify what scheme we will be using to forward those dashboard to Kibana.
  • -M netflow.var.kibana.ssl.enabled=false : We have Kibana accessible with HTTP.
  • -M netflow.var.kibana.ssl.verification_mode=disable : Disable verification mode otherwise you will get errors related to this matter.
Reference : https://www.elastic.co/guide/en/logstash/7.2/netflow-module.html
Netflow Module started successfully .

Going back to Kibana you will notice that the netflow-* index has been created :

Note: It makes a while to forward netflow logs from PfSense to ElasticSIEM instance so give it some time.

netflow index created

Also, a bunch of useful dashboards :

Kibana dashboards created and added to Kibana automatically.

Go to the Discovery panel and you will notice that you are receiving Netflow Logs:

Here are some interesting Dashboards that were loaded automatically with the Netflow Module into Kibana :

Hope this blog article was useful for you, see you next time with a Part III explaining how to setup ZEEK/BRO with Elastic STACK.

References :

  • https://github.com/solvaholic/solvaholic.github.io/wiki/Netflow-with-pfSense-and-ELK.
  • https://docs.netgate.com/pfsense/en/latest/monitoring/exporting-netflow-with-softflowd.html#configuring-and-launching-softflowd
  • https://www.elastic.co/guide/en/logstash/7.x/netflow-module.html

Leave Your Comment