HELP! I need an extra set of eyes.

srt86

Hello all!
I’m a bit new to PFSENSE and I’m having an issue with no internet access on each vlan of my layer 3 switch. I have several vlans and have static routing pointed to their network address as follows 10.0.0.0/8, 192.168.0.0/16 and 172.16.0.0/12. The gateway is set for 10.0.2.1 which is on the WAN vlan and the firewall is 10.0.2.2. This is one of the issues I am having on network 10.0.50.0 I can ping 10.0.50.1 from the firewalls diagnostics page but when I try to ping 10.0.50.1 on the host I receive destination net unreachable. I also cannot ping the host from the diagnostics page of the firewall which was 10.0.50.22. Looking at the firewall log I have udp 10.0.50.22:534948.8.8.8:53 NO_TRAFFIC:SINGLE . Any ideas how I can get this working? The layer 3 switch has DNS set to google to keep things simple.Thanks you very much for your help ahead of time!!

jahonix

That must be a really, really big company you're working for:
/8 : 16.777.214 hosts
/12: 01.048.574 hosts
/16: 00.065.534 hosts

That's nearly 18 million hosts on your router. How do you cope with the broadcasts alone those devices generate? And most of them in only one subnet. Impressive!

Draw a diagram of your network (logical layout will do). No clue from verbal description alone.
http://packetpushers.net/how-to-draw-clear-l3-logical-network-diagrams/

Hint: usually a /24 will do just fine, extend to a /22 max on a single segment to keep it manageable and secure. Divide & conquer is key.

srt86

The host of the laptop at the bottom is 10.0.50.22 . I have a decent sized range of addresses on different vlans. I am not very good with drawing diagrams lol.

http://www.image-share.com/ijpg-3388-29.html

http://www.image-share.com/ijpg-3388-44.html

heper

it appears you L3 switch is only routing, but not doing NAT? correct?

by default pfsense will only allow traffic with source=lan_subnet. you'd need to change that to "any", because your lan_subnet is just a transit network for all the networks behind the L3 switch.

johnpoz

"The gateway is set for 10.0.2.1 which is on the WAN vlan and the firewall is 10.0.2.2."

Where is this set on your L3 switch? Lets be clear are you using this as a ROUTER or is just a L3 switch you bought and really only using it as L2??

Your using the complete rfc1918 space with your routes, which clearly would overlap your transit network if using your L3 switch as router… When you call out that its a L3, this really points to being a ROUTER and not just switch.. If your routing at the switch, then pfsense gives 2 shits about vlans. It would just have a transit network to your L3 switch (Router).

All the networks gateways would be on the downstream router, and sure pfsense would have routes setup to those.. See 1st pic attached.

That is a typical setup with a downstream router (l3 switch) So there is a transit network, that does not overlap any of your downstream networks. Then you can have as many say 10 networks you want. And you can route with simple summary 10.0.0.0/8

So pfsense would need a gateway setup on this lan/transit connection. Not set on the interface, juts a gateway create and not set to default. You would then add your static routes using this gateway.

Now on your outbound nats you would need to adjust so it nats these networks to your pfsense wan(internet) and you would need allow on the lan/transit firewall tab rules for these networks on how you want them to access the internet, be it any any or restrictive, etc.

All of the devices in these downstream networks would use the downstream routers IP in these vlans.. 0.1, 1.1, 2.1 etc.. This router would have default route pointing to pfsense 172.16.0.1

With the switch being use as just layer 2, then all the networks end up on pfsense as vlans.. With a trunk port to pfsense, and vlans setup with all the vlans you setup on your switch And they point to the vlan IPs on pfsense as their gateways.

So if you want help we need some details.. So breakout the crayons and napkin if you must and draw up your network.. Snap a pic of that with your phone.. But from your current info I am not sure if your actually routing, clearly doesn't seem like your using a transit if you are. Or if your just calling it a L3 switch and using L2. Clearly your networks and or routes you created seem to overlap, etc..

L3routertypical.png_thumb

typicalvlans.png_thumb

srt86

@heper:

it appears you L3 switch is only routing, but not doing NAT? correct?

by default pfsense will only allow traffic with source=lan_subnet. you'd need to change that to "any", because your lan_subnet is just a transit network for all the networks behind the L3 switch.

Correct, The layer 3 switch is handling dhcp on some vlans and acl mostly between vlans.I tired any any yesterday but maybe I needed to reboot? I will try again today and see if it works. Also the pfsense box is on standard setup for LAN and sub interfaced the lagg group for the chelsio t520.

johnpoz

So its routing all your vlans… Then it should be connected to pfsense with a TRANSIT network..

srt86

@johnpoz:

So its routing all your vlans… Then it should be connected to pfsense with a TRANSIT network..

I am still able to ping the gateway address or 10.0.50.1 on the switch. Traffic gets from the inside to the firewall it just doesn't seem to be getting through the firewall. Do you think it could be more likely a NAT issue?

johnpoz

Are you natting these downstream networks? If not then YEAH.. But again.. Where is your transit network… You say pfsense has an address of 10.0.50, this is inside what you say you created a stated route of 10/8

That screams problem right there!!