Can't get additional interface to work
-
Ok, I have been using pfSense for my normal routing duties for a while, with good success. In addition to my "normal" LAN, I have a completely segregated network that is specifically for IP security cameras and the computer that handles the video footage. I added an extra network card to my pfSense box for the sole purpose of running an NTP server to keep the cameras time synced. I added a rule to allow traffic, but the NTP has never really worked. As I've investigated, I've realized that I can't ping the security LAN interface in the router and get a response, and that's probably the problem.
I know it's probably something stupid, but I've not been able to locate the source of my stupidity yet. Any help would be appreciated!
-
Is the rule you added to pass traffic from the new subnet limited to TCP-only? Try protocol any there.
-
The rule is set to allow any, not just TCP.
-
Post a network map that includes the subnets in use along with the firewall rules on both interfaces.
-
It's a pretty simple setup. All of the normal devices are on 192.168.1.xxx, and pfSense is set to 192.168.1.1 for that interface. DHCP is used, with most devices having static mappings. No issues on this interface.
All of the security devices are on 10.10.10.xxx, with the pfSense interface on that network segment at 10.10.10.100, and all of the devices have IPs statically assigned.
The only firewall rule on the security interface is a rule to pass all traffic (any protocol). There's also a rule to block any traffic from that interface to WAN, but I've had it disabled while I troubleshoot.
The computer the footage goes to has network cards for both LANs, but they aren't bridged.
There's a 24 port managed switch with separate port based VLANs to segment the traffic between the two networks.
That's all there is to it.
-
The computer the footage goes to has network cards for both LANs, but they aren't bridged.
That can easily create asymmetric routing situations you will need to understand and account for.
You will probably have to draw up what you have there, and where all the hosts and servers in play are in relation to the firewall.
-
I guess I don't understand…..pfSense isn't really doing any "routing tasks" on the Security LAN. All of this was up and running fine before I installed pfSense, and I added the network card to the pfSense box for the specific purpose of having the cameras hit it to sync the time.
-
List your interfaces, addresses, subnets and their default gateways of clients and servers. at least one example on each interface. include both interfaces on the server.
Some routers allow bad network design by default.
Not going to be able to help without a better picture of your network.
-
List your interfaces, addresses, subnets and their default gateways of clients and servers. at least one example on each interface. include both interfaces on the server.
Some routers allow bad network design by default.
Not going to be able to help without a better picture of your network.
WAN address public or private IP space?
Doesn't conflict does it?
-
Much depends on the default gateway of the system with 2 NICs in it and whether that system is talking through the router, on a local subnet, or both.
Some routers will allow hairpinning of traffic in and out the same interface by default. pfSense does not allow it by default.
Need more information.
-
Attached are the ipconfig output of the computer with the dual interfaces, the capture of the interfaces on pfSense, and a crude network map.
EDIT: Where the network map says that "various" clients are connected, each client is of course plugged into the switch with its own cable…there aren't switches downstream or anything.
-
"The computer the footage goes to has network cards for both LANs, but they aren't bridged."
Why???
Why do you have your PC with 2 default gateways?
And where exactly can you not ping 10.10.10.100? From your pc?? And what are the rules on your security interface on pfsense? You mention any any.. but is it for only tcp or something?
Can your PC see mac of pfsense lan interface? If not then you got some sort of connectivity issue. Wrong setup on your switch would be first guess.
-
Can maybe fix your problem if the actual, specific issue you are really seeing is communicated.
Only one of your interfaces should have a default gateway, as has been noticed by @johnpoz.
If it were me I would take one of those interfaces out of the server and route traffic through the firewall from one interface or the other to the server. Doesn't really matter which way. Probably the one that involves the least traffic. Like if the cameras are constantly streaming to the server and PCs only connect once in a while to view footage, I would remove the 192.168.1.10 interface.
-
"The computer the footage goes to has network cards for both LANs, but they aren't bridged."
Why???
Because I want to keep the networks entirely segregated.
Why do you have your PC with 2 default gateways?
I don't actually want the Security LAN to be able to get to WAN….so the Security LAN is just routed to pfSense for the default gateway on that interface. I'll remove the gateway from that interface and see if that makes a difference.....
And where exactly can you not ping 10.10.10.100? From your pc??
Correct. The PC with the two network cards cannot ping the pfSense box on the 10.10.10.100 interface.
And what are the rules on your security interface on pfsense? You mention any any.. but is it for only tcp or something?
I simply did a copy and paste of the default pfSense rule for allowing LAN traffic that is added in the default configuration for the 192.168.1.XXX interface. It's set to allow any protocol. Clearly, traffic is being allowed since the cameras are communicating fine back to the PC where the footage is stored.
Can your PC see mac of pfsense lan interface? If not then you got some sort of connectivity issue. Wrong setup on your switch would be first guess.
Yes. No problems there. The 192.168.1.1 interface works fine.
-
Can maybe fix your problem if the actual, specific issue you are really seeing is communicated.
I wish I had more information to give you. Essentially, as far as the rest of the network is concerned, the 10.10.10.100 interface on pfSense does not exist. The lone computer on that netwrok segment can't see it, and neither can any of the cameras when I try to point them to that IP as the NTP server.
If it were me I would take one of those interfaces out of the server and route traffic through the firewall from one interface or the other to the server. Doesn't really matter which way. Probably the one that involves the least traffic. Like if the cameras are constantly streaming to the server and PCs only connect once in a while to view footage, I would remove the 192.168.1.10 interface.
I need both interfaces in that machine because it handles more duties than just the cameras. That computer is, however, the only PC physically connected to the 10.10.10.XXX network segment (besides the firewall, of course). The only purpose I need pfSense to serve on that network segment is as an NTP server.
-
PC (192.168.1.100) needs to ping 10.10.10.100
PCs default gateway is 192.168.1.1 so it sends the traffic to pfSense
pfSense sends traffic to 10.10.10.100 sourced from 192.168.1.100
10.10.10.100 has a route for 192.168.1.0/24 so it sends the reply directly to 192.168.1.100, creating asymmetry. Should work fine for ping in that case.
However the problem might be that the server at 10.10.10.100 has a firewall enabled and is rejecting traffic from 192.168.1.0/24 on that interface.
If you want to segment and isolate two networks, pretty much the last thing you want to do is put a host with nics on both sides that isn't a firewall.
-
Sounds like either your layer 2 is hosed or your firewall rules are wrong. Nothing really fits what you are describing.
If you have an interface with the address 10.10.10.1 on pfSense, you need a firewall rule on the 192.168.1.1 passing traffic (including ICMP) from 192.168.1.0/24 to 10.10.10.1. It can also be an any rule as long as the traffic matches. The default LAN rules will pass it.
If that is in place and you still cannot ping 10.10.10.1 from hosts one 192.168.1.0/24 AND the default gateway of those hosts is pfSense, then traffic is not routing in a sane manner and something else is wrong somewhere.
For hosts on 10.10.10.0/24 to be able to ping that interface, the rules have to be on that interface.
https://doc.pfsense.org/index.php/Firewall_Rule_Basics
https://doc.pfsense.org/index.php/Firewall_Rule_Troubleshooting
-
I'll willfully profess my ignorance here. I put this setup in place before pfSense was even a part of the network. The purpose in segmenting the network this way was to make absolutely certain that no one logging onto the "normal" LAN would have access to the cameras. There's probably a simpler way to accomplish this. The computer with both NICs is locked away, so there is no physical access to it. I usually get on it by remote using RealVNC.
Regardless of me being able to ping from the computer, the cameras, which have no connection to 192.168.1.XXX at all, still can't connect to the 10.10.10.100 interface for NTP. Cameras have static IPs, and mask of 255.255.255.0, and a default gateway of the 10.10.10.100, the pfSense interface.
This is the sole rule in place on the 10.10.10.100 interface:
-
I also thought I might try to change the IP of the 10.10.10.100 interface to one on the 192.168.1.XXX subnet that was free and see if I could ping that, just to make sure there's not some physical problem with the NIC, but pfSense doesn't like that…..
-
That's because it is unsound to have the same subnet on two different interfaces. Nothing about what pfSense does or doesn't like. That's basic IP networking.
The rule you posted is for connections from the 10.10.10.0/24 network and has ZERO to do with what is accessible from the 192.168.1.0/24 network. That is governed by rules on the other interface. The ones that connections go INTO. All covered in the links above.
Try to ping 10.10.10.1 from the LAN side then go to Status > System Logs, Firewall and filter on destination 10.10.10.1
Anything show up there?
Click the red x if you see blocks. That will tell you which rule blocked it.
-
I don't need that interface to be accessible from the 192.168.1.XXX network. I understand what you're saying about the ping, but it still doesn't explain why the cameras, which have no connection to 192.168.1.0/24 can't get on. So setting up a rule to make sure that traffic passes from 192.168.1.0/24 through to 10.10.10.100 isn't something I really want to put in place, and it doesn't help me troubleshoot my core issue.
Unfortunately, the cameras don't have a diagnostic feature to ping an IP. At first I thought it was just that my NTP setup was hosed, but I don't think that's the case. In any event, here's the LAN rule for 192.168.1.0/24:
-
Try to ping 10.10.10.1 from the LAN side then go to Status > System Logs, Firewall and filter on destination 10.10.10.1
Anything show up there?
Click the red x if you see blocks. That will tell you which rule blocked it.
Nope, nothing to do with that ping request, anyway….
-
Do the cameras have a default gateway set? Is it the pfSense interface?
Does a laptop on 10.10.10.0/24 get a DHCP address? Can it ping 10.10.10.100? Get out to the internet? Can it ping 10.10.10.101? Access that server?
Did you (or someone else) turn on manual outbound NAT? If so did you add outbound NAT for source 10.10.10.0/24? (This will have zero to do with pinging 10.10.10.100 from that subnet)
Your problem is virtually 100% not pfSense with those rules in place on those interface. Unless there are some cockamamie rules on the Floating tab or a captive portal you haven't said anything about or IPsec traffic selectors that match or something else that might suck up the traffic)
-
Do the cameras have a default gateway set? Is it the pfSense interface?
Yes, it is the 10.10.10.100 interface.
Does a laptop on 10.10.10.0/24 get a DHCP address? Can it ping 10.10.10.100? Get out to the internet? Can it ping 10.10.10.101? Access that server?
DHCP isn't running on that interface, but I'll assign a laptop a free static address and try that in a bit.
Did you (or someone else) turn on manual outbound NAT? If so did you add outbound NAT for source 10.10.10.0/24? (This will have zero to do with pinging 10.10.10.100 from that subnet)
Negative.
Your problem is virtually 100% not pfSense with those rules in place on those interface. Unless there are some cockamamie rules on the Floating tab or a captive portal you haven't said anything about or IPsec traffic selectors that match or something else that might suck up the traffic)
Shouldn't be any of that crazy mess going on. Most of what I use the firewall for is controlling access my kids' devices have at bed time and such.
-
Beginning to suspect johnpoz was right and your VLAN 2 is not set up correctly at all.
Wrong setup on your switch would be first guess.
-
Beginning to suspect johnpoz was right and your VLAN 2 is not set up correctly at all.
Wrong setup on your switch would be first guess.
It's just a simple port based VLAN. That was honestly my first thought….that I had bungled that and plugged the 10.10.10.100 patch cable into the other side of the switch, so that it was segregated. It's a 24 port switch, half of which are PoE for the cameras. So I have ports 1-12 for the security network, and ports 13-24 for the regular LAN. I have verified that pfSense is connected to the correct group of ports.
I connected a laptop with a statically assigned IP address of 10.10.10.9, mask of 255.255.255.0, and gateway of 10.10.10.100 (the interface in question). Same result. I can ping cameras, I can ping the server at 10.10.10.101, but I can't hit the pfSense interface at 10.10.10.100. I can't ping 8.8.8.8 either, but that's not a surprise since I can't get to the firewall either....
-
When I have a chance, I think I will enable DHCP on that interface, and then plug a laptop directly into the patch cable and see what happens.
-
If you cannot ping 10.10.10.100 from a PC on that network on, say, 10.10.10.150/24, DHCP will not work either.
This isn't a guessing game. If that doesn't work, your switching (layer 2) is hosed.
-
If you cannot ping 10.10.10.100 from a PC on that network on, say, 10.10.10.150/24, DHCP will not work either.
This isn't a guessing game. If that doesn't work, your switching (layer 2) is hosed.
That's why I'm going to just connect directly to the NIC with a patch cable, and take the switch out of the equation.
If it does, then the next question is to why switching works fine on that switch between cameras and the server, and pfSense seems to be the odd man out.
-
And it looks live I've stumbled upon the problem, just trying enable DHCP.
For some reason, the interface address was set to 10.10.10.100**/32**. I only realized that when I went to enable DHCP, and the available range was as the attached picture. No idea why it was set this way. Changing the IP address to 10.10.10.100/24 allowed the DHCP range to open up, and now I can ping the interface.
-
That'll do it. good deal.
-
I figured it was probably something stupid…fat fingering the mask was a sure way to get there.
Thanks for all of your help!
