Pfsense with LAN adresse that is not set by GUI/Setup
-
Sorry for jumping to a conclusion, I'll try and answer as best as I can….
After checking the reported ARP table, I logged into my my managed switch.
From there I could see both MAC (XX:7b:2 and XX:7b:0) coming from the same physical interface.
I also know for a fact that the pfsense is the only one connected to that physical port."How did you look"....
Well, here I'm open for suggestion on where too look. ifconfig/netstat/sockstat would be my preferred way.
I do not know pfsense that well, so claiming that there is no sign... was maybe a bit too quick...Reinstall is a bold claim, but I would go for a "no import/export of current config"
-
-
[2.0.1-RELEASE][root@pfsnse]/tank(54): arp -a
…... bunch of clients ....
? (10.x.x.9) at zz:zz:zz:zz:zz:26 on bce2 expires in 402 seconds [ethernet]
pfsense (10.x.x.1) at aa:bb:cc:dd:7b:00 on bce2 permanent [ethernet]
…... bunch of clients ....
? (172.x.x.1) at aa:bb:cc:dd:db:b2 on bce1 permanent [ethernet]
…... WAN ......
DHCP-lease-name (xx.xx.xx.xx) at aa:bb:cc:dd:db:b0 on bce0 permanent [ethernet]Removed unwanted info from ARP output… no sign of 10.0.1.25
My test LAN PC is 10.x.x.9, and that's where I can see 10.0.1.25.Logged into my switch, cleared ARP tabel.. no sign of 10.0.1.25
Started a ping from 10.x.x.9 ---> 10.0.1.25... and then it pops up in the ARP table again... -
Started a ping from 10.x.x.9 –-> 10.0.1.25... and then it pops up in the ARP table again...
Switch ARP table? From same port?
-
Correct… on the switch, same port as the the pfsense box.
-
It seems a coincidence that your LAN IP is 10.0.1.1/25 and that the "25" in the netmask happens to pop up at the end of 10.0.1.25 also.
You could see if 10.0.1.25 is mentioned anywhere in your config - either use Diagnostics, Edit File end bring up /cf/conf/config.xml (or wherever yours is), or from the command line:
grep 10.0.1.25 /cf/conf/config.xml -
Are you bridging some other NIC to LAN and have that .25 IP on it (or it's set to DHCP)? Something very close to the same MAC would almost always be two built-in NICs in the same system, and bridging would be the only way you could have two MACs from the same system on a single switch port.
-
It seems a coincidence that your LAN IP is 10.0.1.1/25 and that the "25" in the netmask happens to pop up at the end of 10.0.1.25 also.
You could see if 10.0.1.25 is mentioned anywhere in your config - either use Diagnostics, Edit File end bring up /cf/conf/config.xml (or wherever yours is), or from the command line:
grep 10.0.1.25 /cf/conf/config.xmlGood trick.. (I'll put a note in my pfsense cheat-sheet).. but no luck… There is no entry in the config of 10.0.1.25
@cmb:
Are you bridging some other NIC to LAN and have that .25 IP on it (or it's set to DHCP)? Something very close to the same MAC would almost always be two built-in NICs in the same system, and bridging would be the only way you could have two MACs from the same system on a single switch port.
Also a valid explanation, but as far as I know, there should be no bridging on this pfsense box. If I had some kind of bridging setup, wouldn't I see that in
ifconfig?? And btw, the .25 is outside of the DHCP range… Anywhere else I could check? -
Not giving up here… did some ICMP ping testing with tcpdump on pfsense box..
The test against 10.0.1.1, looks normal, you can see both request and reply, but for the 10.0.1.25
You can only see the request....[2.0.1-RELEASE][root@pfsense.local]/root(42): tcpdump -ibce2 -vv -n | grep ICMP tcpdump: listening on bce2, link-type EN10MB (Ethernet), capture size 96 bytes 11:41:38.963495 IP (tos 0x0, ttl 64, id 43294, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.10 > 10.0.1.1: ICMP echo request, id 54649, seq 0, length 64 11:41:38.963510 IP (tos 0x0, ttl 64, id 58355, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.1 > 10.0.1.10: ICMP echo reply, id 54649, seq 0, length 64 11:41:39.955940 IP (tos 0x0, ttl 64, id 43297, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.10 > 10.0.1.1: ICMP echo request, id 54649, seq 1, length 64 11:41:39.955956 IP (tos 0x0, ttl 64, id 4144, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.1 > 10.0.1.10: ICMP echo reply, id 54649, seq 1, length 64 11:41:40.948931 IP (tos 0x0, ttl 64, id 43299, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.10 > 10.0.1.1: ICMP echo request, id 54649, seq 2, length 64 11:41:40.948947 IP (tos 0x0, ttl 64, id 52480, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.1 > 10.0.1.10: ICMP echo reply, id 54649, seq 2, length 64 11:41:40.949827 IP (tos 0x0, ttl 64, id 43303, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.10 > 10.0.1.25: ICMP echo request, id 54905, seq 0, length 64 11:41:41.942910 IP (tos 0x0, ttl 64, id 43306, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.10 > 10.0.1.25: ICMP echo request, id 54905, seq 1, length 64 11:41:42.935894 IP (tos 0x0, ttl 64, id 43309, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.10 > 10.0.1.25: ICMP echo request, id 54905, seq 2, length 64From the test machine:
PING 10.0.1.1 (10.0.1.1): 56 data bytes 64 bytes from 10.0.1.1: icmp_seq=0 ttl=64 time=0.055 ms 64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.057 ms 64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.057 ms --- 10.0.1.1 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.055/0.056/0.057/0.001 ms PING 10.0.1.25 (10.0.1.25): 56 data bytes 64 bytes from 10.0.1.25: icmp_seq=0 ttl=64 time=0.408 ms 64 bytes from 10.0.1.25: icmp_seq=1 ttl=64 time=0.387 ms 64 bytes from 10.0.1.25: icmp_seq=2 ttl=64 time=0.403 ms --- 10.0.1.25 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.387/0.399/0.408/0.009 ms -
Why don't you just look on pfsense?? ifconfig would show you the macs of all the interfaces and what IPs on are them.
Also the response times of your pings points to something other than pfsense
so you ping .1 and get .05ms
when you ping .25 you get .5msThis is quite a dif if same machine answering the pings. .5 ms is clearly lan response times. .05 would seem to be just pinging itself if you ask me not lan type response times.
Is pfsense on a VM?
-
Why don't you just look on pfsense?? ifconfig would show you the macs of all the interfaces and what IPs on are them.
The easiest solution might be the best, but not in this case. If you look at my responds to wallabybob, I listed a set of tools that I've used to check for existent.
Also the response times of your pings points to something other than pfsense
so you ping .1 and get .05ms
when you ping .25 you get .5msThis is quite a dif if same machine answering the pings. .5 ms is clearly lan response times. .05 would seem to be just pinging itself if you ask me not lan type response times.
Is pfsense on a VM?
I agree, it looks like 0.05 would be more like pinging itself, rather then ping another machine on the LAN.. but in fact it's not.. they are all physical machines, no VM.
-
I don't see how you could be pinging across a switch with different ports and 2 different cables and be getting .055 ms
What switch and network cards do you have??? Across a switch your going to see around .5 ms not .05 ms
If I ping myself on my linux box I can see under .100 ms, but anything else on the switch and your look at .400 to .600ms. I just don't buy it that your pinging across a switch and getting .055 ms RTT?
So why can you not show us ifconfig off your pfsense box? Hide any public IPs
–[2.1-BETA0][root@pfsense.local.lan]/root(4): ifconfig
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:02
inet 192.168.1.253 netmask 0xffffff00 broadcast 192.168.1.255
inet6 fe80::250:56ff:fe00:2%em0 prefixlen 64 scopeid 0x1
inet6 2001:470:xx:xx::1 prefixlen 64
nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:01
inet6 fe80::250:56ff:fe00:1%em1 prefixlen 64 scopeid 0x2
inet 24.13.xx.xx netmask 0xfffff800 broadcast 255.255.255.255
nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
vmx3f0: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
options=403bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso>ether 00:0c:29:1e:18:90
media: Ethernet 10Gbase-T (autoselect)
status: no carrier
vmx3f1: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
options=403bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso>ether 00:0c:29:1e:18:9a
media: Ethernet 10Gbase-T (autoselect)
status: no carrier
plip0: flags=8810 <pointopoint,simplex,multicast>metric 0 mtu 1500
pflog0: flags=100 <promisc>metric 0 mtu 33200
enc0: flags=0<> metric 0 mtu 1536
pfsync0: flags=0<> metric 0 mtu 1460
syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
nd6 options=3 <performnud,accept_rtadv>ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
options=80000 <linkstate>inet6 fe80::250:56ff:fe00:2%ovpns1 prefixlen 64 scopeid 0xb
inet 10.0.200.1 –> 10.0.200.2 netmask 0xffffffff
nd6 options=3 <performnud,accept_rtadv>Opened by PID 7348
gif0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
tunnel inet 24.13.xx.xx --> 216.66.77.230
inet6 fe80::250:56ff:fe00:2%gif0 prefixlen 64 scopeid 0xa
inet6 2001:470:xx:xx::2 prefixlen 64
nd6 options=3 <performnud,accept_rtadv>options=1 <accept_rev_ethip_ver>This would for FACT Show you want IPs your box could answer on and what MAC address.</accept_rev_ethip_ver></performnud,accept_rtadv></up,pointopoint,running,multicast></performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></pointopoint,simplex,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso></broadcast,simplex,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso></broadcast,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast> -
As I said, it look strange that I wold get a sub 0.0x respond… I did the same test from another machine and got a more "normal" result.
So to put this machine out of the loop, I turned it off....If it would be a simpel answer, that the ip and interface would be shown by ifconfig, I would not be asking question on this forum...
But here you go, my complete ifconfig.....[2.0.1-RELEASE][root@pfsense.local]/root(1): ifconfig bce0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=c00bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ="" m,vlan_hwtso,linkstate="">ether 00:10:18:b8:db:b0 inet6 fe80::210:18ff:feb8:dbb0%bce0 prefixlen 64 scopeid 0x1 inet xxx.xxx.xxx.xxx netmask 0xffffff80 broadcast 255.255.255.255 nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>) status: active bce1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=c00bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ="" m,vlan_hwtso,linkstate="">ether 00:10:18:b8:db:b2 inet 172.16.0.1 netmask 0xffffffe0 broadcast 172.16.0.31 inet6 fe80::210:18ff:feb8:dbb2%bce1 prefixlen 64 scopeid 0x2 nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>) status: active bce2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=c00bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ="" m,vlan_hwtso,linkstate="">ether bc:30:5b:e5:7b:00 inet 10.0.1.1 netmask 0xffffff80 broadcast 10.0.1.127 inet6 fe80::be30:5bff:fee5:7b00%bce2 prefixlen 64 scopeid 0x3 nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>) status: active bce3: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500 options=c01bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ="" m,tso4,vlan_hwtso,linkstate="">ether bc:30:5b:e5:7b:01 media: Ethernet autoselect (none) status: no carrier pflog0: flags=100 <promisc>metric 0 mtu 33664 pfsync0: flags=0<> metric 0 mtu 1460 syncpeer: 224.0.0.240 maxupd: 128 syncok: 1 enc0: flags=0<> metric 0 mtu 1536 lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384 options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8 nd6 options=3 <performnud,accept_rtadv>ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500 options=80000 <linkstate>inet6 fe80::210:18ff:feb8:dbb0%ovpns1 prefixlen 64 scopeid 0x9 inet 10.10.10.1 --> 10.10.10.2 netmask 0xffffffff nd6 options=3 <performnud,accept_rtadv>Opened by PID 2136</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ></broadcast,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsu ></up,broadcast,running,simplex,multicast> -
A picture is worth more then 1000 words?
If the text on some of the elements is to small, let me know and I'll tell you what it says…
I cleared my ARP table on the switch, and then did a mapping on everything it can see... (happy that there is not much traffic early in the morning..)
There is only one switch, so where you find more MACs behind one physical port is because they are on a wireless network. (The AP is on port GE19)
-
So here's the thing - you don't show that MAC, you don't show that IP. In doing a sniff you see the ping come in - but you don't see it go out.
If you DON'T see it go OUT, but you see an answer from the client that sent the ping. How do you think its pfsense answering?
As to .055 ms looking strange?? How about impossible?
Add -e to your tcpdump so we can see mac, you sure you don't have some sort of mirror/span port setup on your switch? As to why pfsense to would see those packets, but clearly it did not answer them. And even if it did - how would it be possible it did in .055 ms, when you ping its normal IP its take .500 ms??
So yeah that is really really odd - so that is a dual port nic, is it possible there is some sort of load balancing/teaming going on where it created a VIP and mac and freebsd just can not show this because of lack of software from broadcom? Is this .25 in your dhcp scope?
-
As to .055 ms looking strange?? How about impossible?
Let's forget this one for now (the machine that I got the result from is turned off, and should be out of the loop).
We might take this behavior up on another tread if your up for it… :)Guess I'm no hard core tcpdumper.. I see that I should have used the -e earlier... but here are the results..
[2.0.1-RELEASE][root@pfsense.local]/root(3): tcpdump -ibce2 -vv -n -e | grep ICMP tcpdump: listening on bce2, link-type EN10MB (Ethernet), capture size 96 bytes 15:22:23.352700 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 47054, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 45884, seq 32, length 64 15:22:24.353708 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 15857, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 45884, seq 33, length 64 15:22:25.354754 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 12483, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 45884, seq 34, length 64 15:22:26.356435 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 65454, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 45884, seq 35, length 64Regarding span/mirror, I thought of setting up when I got the switch, but never got around to configure it. Checked right now STP was enable but not in use, disables it now, no differance.. othere places I've checked..
Port and VLAN Mirroring - None
Link Aggregation - None
STP Status & Global Settings - DisabledSo yeah that is really really odd - so that is a dual port nic, is it possible there is some sort of load balancing/teaming going on where it created a VIP and mac and freebsd just can not show this because of lack of software from broadcom? Is this .25 in your dhcp scope?
I have no idea… .25 is outside of my DHCP scope.. that's all I know.. and I did run some load balancing software (varnish/mod_security ++ other 3rd party extension) but they have all been removed... the only one I have left, is a export for OpenVPN clients..
Open for suggestions...
-
So can understand why you see the request.. if your switch shows that :02 mac on the port bce2 is connected too. But clearly its not sending out a reply.
But your saying 10.0.1.9 is seeing the response? And on .9 do a tcpdump, it shows the response coming from that :02 mac? Then why did you not see it on your pfsense dump??
Very very strange issue yes - but if pfsense is sending out the bce2 port – shouldn't you see it via the tcpdump??
-
Never though of check the respond, just did… and yes I do get the respond from the :2 MAC...
How'ever I do get a bad chsum on all the request.....Mac-mini:~ root# tcpdump -ien0 -vv -n -e | grep ICMP tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes 18:17:43.480336 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 34074, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->df6d)!) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 3901, seq 6, length 64 18:17:43.480737 bc:30:5b:e5:7b:02 > c8:2a:14:36:3b:26, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 47612, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.25 > 10.0.1.9: ICMP echo reply, id 3901, seq 6, length 64 18:17:44.481574 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 55579, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->8b6c)!) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 3901, seq 7, length 64 18:17:44.481982 bc:30:5b:e5:7b:02 > c8:2a:14:36:3b:26, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 47613, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.25 > 10.0.1.9: ICMP echo reply, id 3901, seq 7, length 64 18:17:45.482753 c8:2a:14:36:3b:26 > bc:30:5b:e5:7b:02, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 21348, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->1124)!) 10.0.1.9 > 10.0.1.25: ICMP echo request, id 3901, seq 8, length 64 18:17:45.483179 bc:30:5b:e5:7b:02 > c8:2a:14:36:3b:26, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 47614, offset 0, flags [none], proto ICMP (1), length 84) 10.0.1.25 > 10.0.1.9: ICMP echo reply, id 3901, seq 8, length 64And yeah. I do agree, my thinking would also say that if the pfsense is answering it should show it on the tcpdump…
-
very very odd - only thing I can think of is card is putting it on the wire directly from a virtual mac that is outside the OS. Where it is getting the IP is strange as well - unless you had set it on the card at some point with some broadcom software or firmware you can access on the card.
What is the specific model number of the card - is there a way to flush is firmware settings?
-
How'ever I do get a bad chsum on all the request…..
If the software is using the hardware to generate IP checksums on transmit then tcpdump won't necessarily see a correct IP checksum on the transmit frames.
