Threat Hunting Lab (Part II) : Sending PfSense Netflow data to Elastic Stack
29th March 2020 | by hilo21
Over prepare, then go with the flowRegina Brett
In this part of these blog series we will try to see how can we integrate Netflow with Elastic Stack for increasing visibility. We will be using Netflow data from our PfSense firewall.
Personally, I believe that Netflow data doesn’t bring much to the table when it comes to information security from a Detection-Prevention perspective but it adds much more context to your security operations and gives you a better visibility on your inbound/outbound traffic in general.
In the last blog, Part I, we were able to setup an Elastic Stack all-in-one VM machine.
Setting up Softflowd on PfSense and Logstash:
Step 1 : Install Softflowd package on PfSense :
On PfSense web interface, go to System / Package Manager / Available Packages then look for softflowd and install it :
Step 2 : Configure SoftFlowd
On the Services / softflowd panel, configure the softflowd’s parameters as it suites you.
For me, I will be forwarding all netflow data to my ElasticSIEM VM at 10.10.10.129 on port 2055 from my WAN and LAN interfaces using Netflow version 9 :
Step 3 : Configure Logstash with Netflow module
As we know by now, we’ve already set up an All-in-one ElasticSIEM VM that has Elasticsearch, Kibana and Logstash. For this part of the configuration, we will be using Logstash’s Netflow module to create a netflow index and generate Kibana dashboards automatically. However, using this module won’t allow us to make NetFlow data useful on the Elastic SIEM Module since Logstash 7.2.1 is not compliant with ECS (Elastic Common Schema) yet as described in the next figure :
You can use Filebeat’s netflow module since it is compatible with the ECS for a better visibility on your SIEM App. But for simplicity reasons, I won’t be using that because I am not planning on using Netflow data that much in my Labs.
Before starting Netflow module make sure that kibana and elasticsearch are running.
We can use a one-liner command line to run Logstash with the Netflow module by entering this command from the logstash folder installation:
[elastic@elasticsiem logstash]$ bin/logstash --modules netflow --setup -M netflow.var.input.udp.port=2055 -M netflow.var.elasticsearch.hosts="localhost:9200" -M netflow.var.elasticsearch.ssl.enabled=false -M netflow.var.kibana.host="10.10.1.129:80" -M netflow.var.kibana.scheme=http -M netflow.var.kibana.ssl.enabled=false -M netflow.var.kibana.ssl.verification_mode=disable
Lets break it down:
bin/logstash --modules netflow: option spins up a Netflow-aware Logstash pipeline for ingestion.
-- setup: The
--setupoption creates a
netflow-*index pattern in Elasticsearch and imports Kibana dashboards and visualizations. Running
--setupis a one-time setup step. Omit this option for subsequent runs of the module to avoid overwriting existing Kibana dashboards.
-M netflow.var.input.udp.port=2055: Specifies the port logstash instance will be listening on.
-M netflow.var.elasticsearch.hosts="localhost:9200": Specifies the Elasticsearch database host for Logstash.
-M netflow.var.elasticsearch.ssl.enabled=false: For this Lab, we have an Elasticsearch instance with no required SSL certificate validation so we have to specify that SSL is not enabled.
-M netflow.var.kibana.host="10.10.1.129:80": Specifying Kibana’s host IP & port where we will be uploading Netflow dashboards.
-M netflow.var.kibana.scheme=http: Specify what scheme we will be using to forward those dashboard to Kibana.
-M netflow.var.kibana.ssl.enabled=false: We have Kibana accessible with HTTP.
-M netflow.var.kibana.ssl.verification_mode=disable: Disable verification mode otherwise you will get errors related to this matter.
Reference : https://www.elastic.co/guide/en/logstash/7.2/netflow-module.html
Going back to Kibana you will notice that the
netflow-* index has been created :
Note: It makes a while to forward netflow logs from PfSense to ElasticSIEM instance so give it some time.
Also, a bunch of useful dashboards :
Go to the Discovery panel and you will notice that you are receiving Netflow Logs:
Here are some interesting Dashboards that were loaded automatically with the Netflow Module into Kibana :
Hope this blog article was useful for you, see you next time with a Part III explaining how to setup ZEEK/BRO with Elastic STACK.