Cannot connect/ping from internal to pfSense and vice versa
-
You could also use it in combination, where some vlans/networks are routed via switch and others pfsense.
But it does seem odd that you have svi set on every vlan, if its just going to be L2. Normally the only svi setup when doing just L2 is management IP. And only reason default gateway would need to be set is for the switch itself to be managed from other than the local management vlan, or for it to talk to stuff that is outside its management vlan for say ntp or grab updates or something, remote stuff to talk to it for say snmp, etc.
While you can put as many svis on the switch as you want - really comes down to where your clients on these vlans point to for their gateway.. Is it pfsense IP in the vlan, or the switches IP on that vlan.
Setting a svi in each vlans could be required to run dhcp server on the switch, your going to want to make sure it hands out the pfsense IP as the gateway for its dhcp clients, etc.
if your going to use the switch as the router - also keep in mind if the switch is going to do the routing for all or some of the vlans. then you really need to setup a transit vlan between and the vlans would not be setup on pfsense for the vlans the switch is going to route.
edit: BTW the use of 1.1.1.1 on your switch for anything is bad idea.. Yeah cisco use to love to do that.. But now 1.1.1.1 is valid on the internet.. So you really should move away from such a practice.
-
@marvosa thanks again! So say if we let 3750 to be internal routing device for internal subnets, I should keep ip routing running on my 3750, I remember last night when I remove "ip routing" from 3750, my Cisco 2504 WLC stopped working and my Cisco 3700 CAPWAP APs lost connections. So to avoid too many config changes, maybe better to keep ip routing on 3750.
In that case, should I add static routes for the internal subnets on pfSense(under system>routing)? Currently only one route being added automatically there for pfSense WAN port, which is getting DHCP IP from ISP(Comcast) cable modem.
Also, the port g1/0/48 on 3750 connecting to pfSense LAN(em1) port, I set it as trunk and encapsulation is 802.1q, allow all vlans to pass through. hope that is correct.
No idea what else I missed?
-
@icq9988 said in Cannot connect/ping from internal to pfSense and vice versa:
I set it as trunk and encapsulation is 802.1q, allow all vlans to pass through. hope that is correct.
No its not if your going to route on your switch, the only vlan that would connect pfsense to the switch if its going to do the routing is a transit network. That can be tagged or untagged - but pfsense would not have any of the other vlans setup on it.
But yes you would have to setup up routing on pfsense for the downstream networks, and you would have to adjust your rules on the now transit vlan to allow for the downstream networks.
And if you are not using automatic outbound nat, then you need to make sure there are correct outbound nat rules for downstream networks.
If auto is used for outbound, then when you create the routes to get to these downstream networks pfsense would auto adjust your outbound nats to allow for those.
edit:
Here is a drawing Derelict did back in the day that is great info on how setup L2 or L3 on pfsense depending
-
@johnpoz Thanks! Does that mean if I run routing internally on my Cisco 3750, I should remove all VLANs from pfSense, only keep WAN interface and LAN interface, and assign an IP to pfSense lan interface(say 10.30.54.1) and connect to an access port(with the same vlan that 10.30.54.0/24, no trunking needed)?
Then after that I need add route on pfSense under system -> routing:
10.30.54.0/24 next hop to 10.30.54.254(vlan1 ip on 3750)
10.30.58.0/24 next hop to 10.30.58.254(vlan700 ip on 3750)
10.30.60.0/24 next hop to 10.30.60.254(vlan1000 ip on 3750)
10.30.52.0/24 next hop to 10.30.52.254(vlan100 ip on 3750)So I guess I still need the default route: "ip route 0.0.0.0 0.0.0.0 10.30.54.1 " on my 3750? (10.30.54.1 IP assigned to pfSense LAN-em1 port)
-
@icq9988 said in Cannot connect/ping from internal to pfSense and vice versa:
I should remove all VLANs from pfSense, only keep WAN interface and LAN interface
Yeah if your downstream is going to do all the routing for your internal networks.. Pfsense has no use for interface in that network/vlan or needs to about them or tags because it would never see the tags.
But if your going to use your lan as your transit - keep in mind that no other devices go on this network..
A transit network is any network that connects routers.. Hosts do not belong on transit networks. If you do place hosts on transit network your going to run into asymmetrical traffic unless you specifically route on that host, or do nat downstream, etc.
edit:
Then after that I need add route on pfSense under system -> routing: 10.30.54.0/24 next hop to 10.30.54.254(vlan1 ip on 3750) 10.30.58.0/24 next hop to 10.30.58.254(vlan700 ip on 3750) 10.30.60.0/24 next hop to 10.30.60.254(vlan1000 ip on 3750) 10.30.52.0/24 next hop to 10.30.52.254(vlan100 ip on 3750)No you wouldn't do it that way.. You would use just your lan as your transit.. The routes to all those networks would use the switches IP in the transit (lan) as the gateway to get to that network. Its all laid out in the drawing I posted.
edit2: If your pfsense is more than capable of handing the intervlan traffic as the speeds you need I would look to getting away from the switch doing the routing. When you do the routing downstream you loose any ability for pfsense to firewall between your vlans.. Other than breaking up your broadcast domains via your vlans - switches rarely have easy or robust way of doing filtering between vlans.. While if pfsense is doing the routing - you have simple easy to use and configure firewall that allows you to control the traffic between your vlans to only the stuff you want to allow.
And can almost promise you the dhcp server in pfsense is easier to configure and get info from and set options than the dhcp server on your switch.. Which if your vlans are all downstream you can not leverage pfsense dhcp server and easy to use interface.
-
@icq9988
Ok, if the 3750 is the core router and doing inter-vlan routing, I'd do this:-
Remove the VLANs from PFsense
-
Create a transit network between the LAN interface on PFsense and the 3750 (G1/0/48)
-
Give the LAN interface on PFsense an IP in the transit network
-
Re-configure 3750-1/0/48 as a routed port and give it an IP in the transit network
-
Create static routes on PFsense for each subnet pointed at the IP assigned to the routed port on 3750-1/0/48
-
The switch will need an SVI for each VLAN, but I'm guessing this is already done
-
Remove the "ip default-gateway 10.30.54.1" line on the 3750
-
Modify the default route on the 3750 so it points at the IP of your LAN interface on PFsense
-
Disable VLAN1 on the 3750 as it's no longer needed
-
Disable the loopback interface unless it's being used for something specific
At this point, you'll need to determine what's going to handle DHCP and DNS. You mentioned using the switch for DHCP, but personally, I'd either spin up something like AD or Infoblox and point your helper addresses at one of those... I wouldn't use the switch.
Also, note that your DHCP server will need to hand out the SVI as the default gateway... not PFsense
-
-
@marvosa My 3750 is LANBASED so int g1/0/48 cannot be assigned IP address, but I guess use "switchport mode access" "switchport access vlan 100" , it should work. Assuming I removed vlan 1, and use 10.30.54.1 for pfSense lan port IP and 10.30.54.254 as SVI for vlan 100 on 3750.
Thanks so much Marvosa and Johnpoz, I will try both way.
If this not work out, I will try plan B: remove all routing or default-gateway from 3750, and config TRUNK bewteen 3750-pfSense, and add VLANs back to pfSense to let it working as routing device instead of 3750, and also config DHCP in pfSense ... ...
-
@icq9988 said in Cannot connect/ping from internal to pfSense and vice versa:
My 3750 is LANBASED
Bah... update that license! ;)
If not, with LAN BASE, in theory, you "should" be able to keep G1/0/48 as a trunk, create a transit VLAN at both ends, point your static routes on PFsense towards the SVI of the transit VLAN on the 3750, and then point the default route on the 3750 towards the IP assigned to the transit VLAN on PFsense.
Otherwise, you may want to go back to an L2 approach and have PFsense do the inter-vlan routing... in which case you can use PFsense as your DHCP server.
-
@marvosa @johnpoz this is to update:
I took the approach with Cisco 3750 doing the routing, removed all VLAN from pfSense, but still have to add a gateway with lan port on pfsense, and add static route on this gateway. lan port ip is 10.30.54.1, Onr cisco 3750, VLAN 100 SVI has IP 10.30.54.254.
After that, I found the vlan with the same subnet(10.30.54.0/24) on pfSense lan port work fine. I can go internet and everything looks alright. I have removed "ip default-gateway", and has "ip route 0.0.0.0 0.0.0.0 10.30.54.1".
However, for other vlans, they still not working. From Cisco 3750, if I do this "ping 10.30.54.1 source 10.30.60.254" (10.30.60.254.is vlan 1000's SVI ip address on Cisco 3750), it will failed to ping. That is why other vlans cannot go internet. I guess that's because the 3750 is not a true L3 device and cannot assign IP address on physical port so make it hard to routing as we expected.SO for now I just removed all other vlans and make it work for the purpose. As my pfsense hardware is too old(maybe more than 10 years), I will go ahead to get a new hardware, and install with newest pfsense, and implement as making Cisco 3750 pure L2 device and trunking to pfsense, to let pdSense do L3 routing and traffic control, As johnpoz suggested.
Again thanks a lot for your helps!!!
-
Your hardware age is not an issue... Routing is routing..
-
@johnpoz I'm not 100% sure why routing not working at this case.
I did check "show ip route" and the default route marked as "*s" which told me it's has been put in routing table. But for this switch still not working.
Interesting thing is, if I do ping test from pfSense, I can ping anywhere inside, even other VLANs.The hardwre age is about the pfSense upgrade thing, I have not been able to upgrade since ver 2.4.3_1.
-
If you can ping the downstream devices. But you can not ping pfsense from the devices. What are the rules on your transit interface (lan?)
by default source of lan net would be only source, so no downstream networks would not be able to ping pfsense, or go to the internet..
I clearly went over all of that..
As to your version of pfsense - again.. Doesn't matter this so basic... You could be running pfsense version 1 and it would work..
-
@johnpoz for pfSense version, I just want to be up to date. Sorry about the confusion, that is another thing unrelated to routing stuff.
Last night when I tested. if I PING anywhere from Cisco 3750, it works, because by default Cisco 3750 uses 10.30.54.254 as the source. However, if I added source ip(from other vlan) with ping. It will fail. Basically, I did that ping test with source IP because I found only 10.30.54.0(which has the same subnet with pfsense LAN 10.30.54.1) can go internet.
-
@icq9988 said in Cannot connect/ping from internal to pfSense and vice versa:
I found only 10.30.54.0(which has the same subnet with pfsense LAN 10.30.54.1) can go internet.
And again what are the rules on this interface in pfsense? Out of the box no other networks would not have access..
Yes it pretty vital to keep your firewall current version for many many reasons. But something as simple basic connectivity is not one of them..
-
@johnpoz since I removed all VLANs from pfSense, only rule I added is to allow any from LAN_net to any.
-
@icq9988 said in Cannot connect/ping from internal to pfSense and vice versa:
from LAN_net to any.
And are these downstream networks in lan net? No they have different IP range.. So no they are not going to work..
I refer to my previous post where I went over all of this. And even gave you a picture that includes all the pertinent details needed
https://forum.netgate.com/post/998526
-
@johnpoz Sorry I missed that details. I will stop by tonight to install another Cisco Aeronet WAP, will try for that.
Again appreciate for your help on this :)
-
@icq9988 Yes, you will need an any/any (or defined alias/any) on the LAN interface assuming that's where 10.30.54.1 is addressed.
If the other VLANs still aren't communicating once you've added the firewall rule(s) to the LAN, post the running-config, "sh ip route" from the switch, and the routing table from PFsense.
