Pfsense and l3 switch and dmz

johnpoz

@chinchun late to the game here, but your drawing is a bit noisy.. A logical drawing would be much easier and faster to take in.

For example really don't care what devices are in vlans - that some ubuntu lts 20 is in vlan whatever, or what vlan your plex server is in, etc.

while this might be great for you.. In troubleshooting routing - all that is needed is logical diagram.

What specific network is your transit network? And is your R7800 routing? You have 2 gateways setup on it? one to the sg500 and another to pfsense?

Why are you complicating it with all the L3 stuff, why not just do layer 2 on your AP and SG500 and let pfsense handle all the routing?

chinchun

@stephenw10
Let me make something clear first,

vlan 110 is defined in my switch sg500, vlan interface is 10.1.10.2, it does not have an interface in pfsense;
dmz is directly connected to one of the sg5100 firewall (ix2); interface address is 10.254.254.1, 10.254.254.254 is an vm,

And for your questions:

When I say I can ping 10.254.254.1, that is form outside of my home network using wireguard vpn connected to pfsense;
10.1.10.10 indeed is an vm; gateway is 10.1.10.2 (switch vlan110's interface address)
I did what you said, but I can not see any states in pfsense, so I guess traffic form 10.1.10.10 does not reach pfsense?

Here is a simple topology, and my switch, ESXI, pfsense related settings, hope is helps.

Because my mother language is not english, I am not sure if I understand you perfectly, and made myself clear, if there is any misunderstandings, please point it out to me. Many thanks!

chinchun

@johnpoz
Sorry for the mess, and for your questions:

transit network is 10.10.10.0/30;
R7800 does not do routing, just act as ap, and it only have one gateway which is pfsense's lan address
I did try to let pfsense handle all the routing (which is working perfactly), but sg5100 only have gigabit ethernet ports, so I want the sg500 handle all the inter vlan routing instead;

Basicly I have two set of vlans, one is vlan 1xx, the other is vlan 2xx, vlan 1xx need linespeed inter vlan routing, so I defined them with my l3 switch, vlan 2xx don't need inter vlan routing, so I defined them in pfsense for better control.

johnpoz

@chinchun said in Pfsense and l3 switch and dmz:

10.1.10.10 indeed is an vm; gateway is 10.1.10.2 (switch vlan110's interface address)

I did what you said, but I can not see any states in pfsense, so I guess traffic form 10.1.10.10 does not reach pfsense?

What are the rules on 10.10.10.1 (ix3) interface in pfsense. If your going to want your downstream networks to go to other networks on attached to pfsense, or the internet - rules would have to be setup on this interface to allow that.

If you do not have a rule to allow the traffic from say 10.1.10.10 then no a state would never be created.

chinchun

@johnpoz
Here it is

chinchun

@johnpoz
Don't know if I did it the right way.

johnpoz

@chinchun Your throwing the traffic from your downstream networks into a gateway, your LoadBalance.

Yeah that is policy routing, if you want these downstream networks to go to another network(s) off of pfsense you would have to allow for that in the rules.. I doubt your gateway your forcing the traffic to can get there ;)

https://docs.netgate.com/pfsense/en/latest/multiwan/policy-route.html#bypassing-policy-routing

stephenw10

OK, you need a rule on SG500 to allow traffic from 10.1.10.1/24 to DMZ without a gateway set.
Otherwise that traffic is forced via the load-balance gateway group and cannot reach the DMZ.

See: https://docs.netgate.com/pfsense/en/latest/multiwan/policy-route.html#bypassing-policy-routing

Steve

Edit: Snap!

johnpoz

@stephenw10 beat you too it Steve ;) hehehe -- jinx, you owe me a beer!

chinchun

@johnpoz
Thanks for the apply!
I will read it first then try it out. Not an expert of pfsense, just an architect who love these things, still learning

chinchun

@stephenw10
And many thanks to you too, bro!
Almost 3 am here, night!

johnpoz

@chinchun here to help, you seem to have a pretty nice network setup..

And I wouldn't mind having a sg500, I have some sg300s

I don't do any routing on mine (other than lab for helping here if needed) mine are in L3 mode but only use L2 on them.

My sg300 is getting a bit long in the tooth, and is eol here next year I think.. I have my eye out for replacement.. Love to have something that has multiple gig interfaces 1/2.5/5/10 in the 24 port range.. And at a great price hehehe..

The cisco smb line with 24ports some with poe, and with multigig would be sweet! But this unicorn switch is not on the market that I can find - at least not at the price willing to pay..

chinchun

@johnpoz
Thank you! Cost me some time to figure it out.
I'm also looking for some l3 fanless with 10g ports switches, but unfortunately so far I did't find any that both cheap and poe capable and have enough ports.

johnpoz

@chinchun was just looking at the newer sg350's that have multigig and poe.. But the prices currently are just insane for home use ;)

No freaking way could get that past the budget committee (wife) hehehe

Maybe in a year or so they might be more home budget friendly.. I picked up my sg300-28 new for like 200..

chinchun

@johnpoz
No budget for me to get brand new these stuffs
My most equipments are used, except for the T630. It's getting harder and harder to get a cisco in my area. So I'm considering change the sg500 with a ICX7150-C12P(for l3 switching and poe) and a C2960L-24TQ(for access).
But don't know the compatibility between Ruckus and Cisco