Suricata Won't Auto Start
-
Hi,
Using 2.5.2-RC amd64 and Suricata Package 6.0.0_10
Suricata wont start if I reboot the device. If I start suricata on each interface manually after the reboot, suricata starts fine.
The log says:
5/7/2021 -- 23:25:29 - <Notice> -- This is Suricata version 5.0.6 RELEASE running in SYSTEM mode 5/7/2021 -- 23:25:29 - <Info> -- CPUs/cores online: 4 5/7/2021 -- 23:25:29 - <Info> -- HTTP memcap: 67108864 5/7/2021 -- 23:25:29 - <Notice> -- using flow hash instead of active packets 5/7/2021 -- 23:25:29 - <Error> -- [ERRCODE: SC_ERR_ADDRESS_ENGINE_GENERIC(89)] - failed to parse address "Array" 5/7/2021 -- 23:25:29 - <Error> -- [ERRCODE: SC_ERR_INVALID_YAML_CONF_ENTRY(139)] - failed to parse address var "HOME_NET" with value "[9.9.9.9/32, 10.128.128.0/24, 10.255.255.254/32, 127.0.0.1/32, 149.112.112.112/32, 192.168.1.1/32, 192.168.1.4/32, 192.168.2.0/24, 192.168.3.0/24, ::1/128, Array, fe80::2e0:67ff:fe26:3f6a/128, fe80::2e0:67ff:fe26:3f68/128, fe80::2e0:67ff:fe26:3f69/128]". Please check it's syntax 5/7/2021 -- 23:25:29 - <Error> -- [ERRCODE: SC_ERR_INVALID_YAML_CONF_ENTRY(139)] - basic address vars test failed. Please check /usr/local/etc/suricata/suricata_27056_igb1/suricata.yaml for errorsI can see that the word "Array" is being put in the array of IP addresses for the HOME_NET definition in the suricata.yaml file.
If I try and delete the erroneous "Array" word, pfsense just replaces it on reboot and Suricata refuses to start..
-
@propercactus said in Suricata Won't Auto Start:
Hi,
Using 2.5.2-RC amd64 and Suricata Package 6.0.0_10
Suricata wont start if I reboot the device. If I start suricata on each interface manually after the reboot, suricata starts fine.
The log says:
5/7/2021 -- 23:25:29 - <Notice> -- This is Suricata version 5.0.6 RELEASE running in SYSTEM mode 5/7/2021 -- 23:25:29 - <Info> -- CPUs/cores online: 4 5/7/2021 -- 23:25:29 - <Info> -- HTTP memcap: 67108864 5/7/2021 -- 23:25:29 - <Notice> -- using flow hash instead of active packets 5/7/2021 -- 23:25:29 - <Error> -- [ERRCODE: SC_ERR_ADDRESS_ENGINE_GENERIC(89)] - failed to parse address "Array" 5/7/2021 -- 23:25:29 - <Error> -- [ERRCODE: SC_ERR_INVALID_YAML_CONF_ENTRY(139)] - failed to parse address var "HOME_NET" with value "[9.9.9.9/32, 10.128.128.0/24, 10.255.255.254/32, 127.0.0.1/32, 149.112.112.112/32, 192.168.1.1/32, 192.168.1.4/32, 192.168.2.0/24, 192.168.3.0/24, ::1/128, Array, fe80::2e0:67ff:fe26:3f6a/128, fe80::2e0:67ff:fe26:3f68/128, fe80::2e0:67ff:fe26:3f69/128]". Please check it's syntax 5/7/2021 -- 23:25:29 - <Error> -- [ERRCODE: SC_ERR_INVALID_YAML_CONF_ENTRY(139)] - basic address vars test failed. Please check /usr/local/etc/suricata/suricata_27056_igb1/suricata.yaml for errorsI can see that the word "Array" is being put in the array of IP addresses for the HOME_NET definition in the suricata.yaml file.
If I try and delete the erroneous "Array" word, pfsense just replaces it on reboot and Suricata refuses to start..
You likely have an interface that is not coming up fully BEFORE Suricata is started by pfSense during a boot up. Thus the address info for that interface comes back corrupted or non-existent when Suricata asks pfSense for it. Never heard that reported before, but it seems to be the most likely case. Do you maybe have some kind of VPN interface included in the Suricata configuration?
You need to examine all of your defined Suricata interfaces and see where during the boot sequence each is initialized by pfSense. Look at the sequence of events in the pfSense system log to see what's happening and the timeline for it.
-
@bmeeks said in Suricata Won't Auto Start:
Do you maybe have some kind of VPN interface included in the Suricata configuration?
Heya nope, I only have LAN and OPT1 configured and they are both physical ports
-
@propercactus said in Suricata Won't Auto Start:
@bmeeks said in Suricata Won't Auto Start:
Do you maybe have some kind of VPN interface included in the Suricata configuration?
Heya nope, I only have LAN and OPT1 configured and they are both physical ports
Well, my bet is one of them, for some reason, is not fully coming up before Suricata does. Thus Suricata gets bad or unexpected info when querying pfSense for the information. That's how the default HOME_NET list is created.
You can manually create a HOME_NET list using the PASS LIST tab, and then assign your custom list on the INTERFACE SETTINGS tab. That would hard-code the values and thus Suricata would not go out upon startup and ask pfSense for the interface values.
-
-
@bmeeks said in Suricata Won't Auto Start:
You can manually create a HOME_NET list using the PASS LIST tab, and then assign your custom list on the INTERFACE SETTINGS tab. That would hard-code the values and thus Suricata would not go out upon startup and ask pfSense for the interface values.
I think I will have to do that
-
@propercactus said in Suricata Won't Auto Start:
@bmeeks said in Suricata Won't Auto Start:
You can manually create a HOME_NET list using the PASS LIST tab, and then assign your custom list on the INTERFACE SETTINGS tab. That would hard-code the values and thus Suricata would not go out upon startup and ask pfSense for the interface values.
I think I will have to do that
That should work. Can you share what IP address you believe would go where the bogus "Array" entry is located? Is it perhaps an IPv6 address for either the LAN or OP1 interface?
-
@bmeeks said in Suricata Won't Auto Start:
Is it perhaps an IPv6 address for either the LAN or OP1 interface?
I disabled ipv6 so I don't even know why it's getting ipv6 addresses at all!
-
Except maybe I didn't... hmmmm
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: RED options=8100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER> ether 00:e0:67:26:3f:68 inet6 fe80::2e0:67ff:fe26:3f68%igb0 prefixlen 64 scopeid 0x1 inet 192.168.1.4 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: GREEN options=8100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER> ether 00:e0:67:26:3f:69 inet6 fe80::2e0:67ff:fe26:3f69%igb1 prefixlen 64 scopeid 0x2 inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: BLUE options=8100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER> ether 00:e0:67:26:3f:6a inet6 fe80::2e0:67ff:fe26:3f6a%igb2 prefixlen 64 scopeid 0x3 inet 192.168.3.1 netmask 0xffffff00 broadcast 192.168.3.255 media: Ethernet autoselect status: no carrier nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> igb3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:e0:67:26:3f:6b media: Ethernet autoselect status: no carrier nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> enc0: flags=41<UP,RUNNING> metric 0 mtu 1536 groups: enc nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 inet 127.0.0.1 netmask 0xff000000 inet 10.255.255.254 netmask 0xffffffff groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> pflog0: flags=100<PROMISC> metric 0 mtu 33160 groups: pflog pfsync0: flags=0<> metric 0 mtu 1500 groups: pfsync -
@propercactus said in Suricata Won't Auto Start:
@bmeeks said in Suricata Won't Auto Start:
Is it perhaps an IPv6 address for either the LAN or OP1 interface?
I disabled ipv6 so I don't even know why it's getting ipv6 addresses at all!
Those are the default Link-Local addresses the operating system adds by default.
For some reason, pfSense is returning an unexpected value when Suricata queries it for interface IP address info during startup. The most likely scenario, in my mind, is that the particular interface has not finished "coming up" at the time Suricata asks pfSense about it. Of course that's just an educated guess. Looking at the HOME_NET list shown in the log, do you see an IP that is missing? The default HOME_NET should include all the locally-attached interface subnets, except on the WAN it will only include the firewall's public WAN IP (and not the whole WAN subnet you are connected into on your ISP's side).
-
@bmeeks said in Suricata Won't Auto Start:
@propercactus said in Suricata Won't Auto Start:
@bmeeks said in Suricata Won't Auto Start:
Is it perhaps an IPv6 address for either the LAN or OP1 interface?
I disabled ipv6 so I don't even know why it's getting ipv6 addresses at all!
Those are the default Link-Local addresses the operating system adds by default.
For some reason, pfSense is returning an unexpected value when Suricata queries it for interface IP address info during startup. The most likely scenario, in my mind, is that the particular interface has not finished "coming up" at the time Suricata asks pfSense about it. Of course that's just an educated guess. Looking at the HOME_NET list shown in the log, do you see an IP that is missing? The default HOME_NET should include all the locally-attached interface subnets, except on the WAN it will only include the firewall's public WAN IP (and not the whole WAN subnet you are connected into on your ISP's side).
I reckon it's the wireguard tun_wg0 because on reboot ther routes to the gateway don't get set properly either
-
@propercactus said in Suricata Won't Auto Start:
I reckon it's the wireguard tun_wg0 because on reboot ther routes to the gateway don't get set properly either
I would agree. I'm not a wireguard user, but if that tunnel is not completely up at the time Suricata is starting, then pfSense could very well return unexpected information to Suricata when queried about it.
-
@propercactus Suricata + WireGuard is something that needs testing, I personally haven't tested it.
WireGuard interfaces are created as early as possible (there is an earlyshellcmd that kicks this off, and it's pushed to the very top of the earlyshellcmd list just to be extra certain it's ran first).
If I'm understanding correctly, Suricata isn't starting even though it isn't configured to monitor tun_wg0?
-
@bmeeks yea that was it, when I removed the wireguard S2S it works now on reboot. Gunna have to revert to IPsec S2S I think.
@theonemcdonald said in Suricata Won't Auto Start:
If I'm understanding correctly, Suricata isn't starting even though it isn't configured to monitor tun_wg0?
Exactly yes, it's trying to get an ipv6 address from tun_wg0 to add to HOME_NET and it sticks a dummy value called "Array" in the list instead which breaks the Suricata config.
-
Suricata, when starting or regenerating the
suricata.yamlconfiguration file for an interface, will ask pfSense for a list of all local interfaces so that it can generate a default HOME_NET variable that includes the IP subnets for all the locally-attached interfaces. As I mentioned before, for the WAN side, it will only include the actual public WAN IP (so a /32 subnet mask). For the other local interfaces, it will ask pfSense for the mask info. -
@ProperCactus what if you as a test assign an IPv6 address to your tun_wg0 interface. Just so it doesn't return an empty array?
-
@theonemcdonald that might do it coz currently it's this:
tun_wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420 options=80000<LINKSTATE> inet 10.128.128.1 netmask 0xffffff00 groups: wg WireGuard nd6 options=101<PERFORMNUD,NO_DAD>I will try with the ipv6 but first i'm trying to understand what I do gateway wise as it doesn't seem right to create static routes out a gateway that is not connected to the tunnel like you suggest?
-
@theonemcdonald
tun_wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1500 description: WREGRD options=80000<LINKSTATE> inet 10.128.128.1 netmask 0xffffffff inet6 fd1f:c547:3e98:b32f:: prefixlen 64 groups: wg WireGuard nd6 options=101<PERFORMNUD,NO_DAD>Gave it a local link ipv6 but the issue persists.
If I delete the tun_wg0 the issue goes away.
