pfSense and Wireshark issue
-
I installed a pfSense 2.6.0 VM in a VMware test environment. I set it up with a WAN and LAN, the WAN is on one Distributed Port Group on it's own VLAN and the LAN is a different Distributed Port Group and a different VLAN. I have DHCP setup on the pfSense LAN so that any machine on that Distributed Port Group/VLAN will get a DHCP address. I have 13 Kali Linux VM's that are on that Distributed Port Group and are getting a DHCP address. I can ping each machine from each other, no problem. The issue is when I try to do a Wireshark packet capture. It only works if the VM is on the same host as the pfSense VM. I can capture all traffic from any VM that has a DHCP address from the pfSense on any VM that is on the same host as the pfSense VM. If I try to capture traffic on a VM not on the same host as the pfSense VM I get nothing. I looked at the firewall to see if the rule was the issue, it's an any to any rule so all traffic should be let through. The Distributed Port Group is set up to accept promiscuous mode, MAC address changes, and forged transmits. I read somewhere that those had to be switched to Accept for it to work right. I've done quite a bit of research all day and I've found basically nothing. Anyone ever had this happen before or does anyone have a clue what the issue could be?
-
Where and how are you doing the pcap? What are you expecting to see there exactly?
-
@stephenw10 I'm running the pcap from any one of the 13 Kali VM's. I'm running a ping to the gateway from another Kali VM on a different host in the same cluster. I'm expecting to capture the ping from that machine that's pinging the gateway on the one that's running the pcap.
-
@buzzhussman said in pfSense and Wireshark issue:
I'm running a ping to the gateway from another Kali VM on a different host in the same cluster. I'm expecting to capture the ping from that machine that's pinging the gateway on the one that's running the pcap.
What would this have to do with pfsense?
And why would you think machine A on network X, would see pings from machine B to pfsense on network X..
If you want to see the pings from B to pfsense, then sniff on B or pfsense.
That ping from B to pfsense would be unicast to pfsense - why would you think some other box even if on the same network would see that traffic? Is this traffic running through a HUB?? That this machine A is also attached to, some sort of bridge? Have you setup a span port for machine A so it sees other traffic?
-
@johnpoz said in pfSense and Wireshark issue:
@buzzhussman said in pfSense and Wireshark issue:
I'm running a ping to the gateway from another Kali VM on a different host in the same cluster. I'm expecting to capture the ping from that machine that's pinging the gateway on the one that's running the pcap.
What would this have to do with pfsense?
And why would you think machine A on network X, would see pings from machine B to pfsense on network X..
If you want to see the pings from B to pfsense, then sniff on B or pfsense.
That ping from B to pfsense would be unicast to pfsense - why would you think some other box even if on the same network would see that traffic? Is this traffic running through a HUB?? That this machine A is also attached to, some sort of bridge? Have you setup a span port for machine A so it sees other traffic?
Lemme break down a few scenarios. I will run a packet capture on the pfSense itself to eliminate any 3rd party issues with Wireshark.
First scenario: In vCenter I have one VM called KALI1 on ESXi Host A and the pfSense VM is also on ESXi Host A. I have another VM called KALI2 that is on ESXi Host B. If I ping KALI 2 to KALI1 (or vice versa) and I run a packet capture on the pfSense I can see the ICMP request and reply.
Second scenario: I have another VM called KALI3 on ESXi host C. If I ping KALI3 to KALI2 (on Host B) and run a packet capture on the pfSense (still on Host A) I do not see the ICMP request and reply. But if I ping KALI3 to KALI1 (still on Host A), I will see the ICMP request and reply.
Is this some kind of limitation with the pfSense that in order for it to see traffic from one VM to another they all need to be on the same host?
-
@buzzhussman said in pfSense and Wireshark issue:
VM to another they all need to be on the same host?
That would have zero to do with it.. Do you have some sort of bridge setup in pfsense? If I have a network lets call it 192.168.0.0/24.. with A 192.168.0.10 and B 192.168.0.20 setup and pfsense is C 192.168.0.1 the gateway to get off the network.. Pfsense in no way shape or form would see traffic between A and B..
The only way that happens is if pfsense is the bridge and A and B are on opposite sides of the bridge..
-
@johnpoz said in pfSense and Wireshark issue:
@buzzhussman said in pfSense and Wireshark issue:
VM to another they all need to be on the same host?
Do you have some sort of bridge setup in pfsense?
There are no bridges set up. It's pretty barebones as it's only set up in a test environment. I have a WAN, LAN, DHCP, and the default Any to Any IPv4 Firewall rule set up.
-
@buzzhussman well your virtual switch in your esxi then is exposing pfsense interface to the traffic. Which would never happen in a normal network.
Been a while since I played with esxi, but I do recall this mode
-
@johnpoz said in pfSense and Wireshark issue:
@buzzhussman well your virtual switch in your esxi then is exposing pfsense interface to the traffic. Which would never happen in a normal network.
Been a while since I played with esxi, but I do recall this mode
Yeah, I did all that already. So basically I've come to the conclusion that it's not vCenter/Hosts or the pfSense VM. Since the pfSense VM will only sniff the host that it's on and not all hosts it's supposed to with promiscuous mode on I have a hunch that it's the physical switches. I found out that they were recently updated from some really old code to the latest code available so it was a pretty big update jump. Something probably got borked during the update somehow. I don't like getting my fingers bit by the network admins so I try to make sure it's nothing else before pointing the finger at their equipment.
Thank you for the assistance!
-
@buzzhussman said in pfSense and Wireshark issue:
not all hosts it's supposed to with promiscuous mode
Huh? Clearly stated on the link - its only for that specific hosts traffic
"promiscuous mode causes it to receive all frames passed on the virtual switch on that host only"
-
@johnpoz said in pfSense and Wireshark issue:
@buzzhussman said in pfSense and Wireshark issue:
not all hosts it's supposed to with promiscuous mode
Huh? Clearly stated on the link - its only for that specific hosts traffic
"promiscuous mode causes it to receive all frames passed on the virtual switch on that host only"
Yep, I get that. But as I stated in the original post I'm using Distributed Port Groups (Distributed Switches), not standard vSwitches or Port Groups which reside at the host level and only apply to that particular host. Distributed Switches and Distributed Port Groups reside at the vCenter level and apply to all hosts defined in that Distributed Switch/Port Group. When you change a setting on the Distributed Switch/Port Group, like promiscuous mode, it applies to all hosts defined in them and it's supposed to work as explained but it's not and something else entirely is causing it to fail.
-
@buzzhussman but where does that say that traffic on host A would be seen by some box on host B.
I could guess its possible that traffic coming in from the real network on host A from some vm on host B might be seen by all devices on the vswitch on host A.. But then again that only might happen for traffic that is local to the vswitch on host A..
If you want to see traffic from some VM on host B talking to pfsense on host A - why do you not just sniff on pfsense itself?
