Subnet problem?
-
Short run down!
We are currently using Fortinet. But with the costs of annual service I am looking into alternatives. I really like pfsense but I have been having the same problem now for a week.
Running Microsoft DHCP services.
example
pfSense firewall on 10.35.10.234,255.255.255.0
host computer on 10.35.60.85,255.255.255.0,10.35.60.254When host computer is on 10 network all is good. but on 60 traffic does not make its way back unless i put the 60 network interface directly on pfsense
Basically traffic goes (out host)(out wan)(back in wan)but stops there
I have been searching all over and I am stuck.
-
huh? If you want pfsense to route between your 2 networks there then yes it would need an interface in both of the networks - or would need a transit connection to other router..
If devices do not point to pfsense as gateway then yeah more than likely your going to run into asymmetrical problems.
Draw up your network and what your wanting to do exactly and we can discuss. At a loss to what dhcp on MS has to do with the price of tea in china or routing traffic.. Unless your handing out the wrong gateway in your dhcp scopes?
-
@johnpoz
I was just saying that pfSense is not handling dhcplast resort is "ip route 0.0.0.0 0.0.0.0 10.35.10.235" but then I have this in our cisco router to test with 1 client
route-map pfSense permit 10
match ip address 10
set ip default next-hop 10.35.10.234interface Vlan60
ip policy route-map pfSenseaccess-list 10 permit 10.35.60.85
-
That doesn't help me visualize your network... Other than suggesting you have a downstream router on that 35.10 network. Which yeah if your clients point that is going to be asymmetrical.
If you want to test on one client you would need a route on that HOST to say hey to get to 10.35.60 use the pfsense IP.
-
@johnpoz
Hmmm, I've learned so much about networking trying to get this to work but I'm a Sys Admin not a Net admin. I'm not even sure I could help you help me :(I'll just keep plugging away and searching google
thanks
-
Figured it out! On my Internal network in pfsense i changed it to a 10.0.0.0/8 mask.
all is good in the world now!
-
OH MY GAWD! NO!!!
That is NOT going to fix anything...
Do you NOT have a crayon and a napkin you can use to draw up your network..
So you just want all your devices on 1 flat network? How many devices do you have?
-
@fratopolis said in Subnet problem?:
Figured it out! On my Internal network in pfsense i changed it to a 10.0.0.0/8 mask.
all is good in the world now!
Going to a 10.0.0.0/8 subnet is not a solution. Post a network map, so we know what we're dealing with.
-
@johnpoz said in Subnet problem?:
OH MY GAWD! NO!!!
That is NOT going to fix anything...
Do you NOT have a crayon and a napkin you can use to draw up your network..
So you just want all your devices on 1 flat network? How many devices do you have?
Before I go all out and break out my crayons what is wrong with this.
Basically it is saying if traffic has a destination on 10.35.x.x 10.20.x.x 10.120.x.x etc it should be routed through the LAN interface so long as it passes the pfsense rules and suricata. Then our Cisco C6832-X-LE will route the traffic to the subnet it needs to go to. We have over 11000 Chromebooks, over 3000 Windows, just under 400 printers and bout 1000 phones.
It is not a flat network as pfSense is not handling routing, vlans, etc of any sort. Instead of creating an interface for each this just says if bound for 10.x.x.x route through Internal network.
-
our current Fortinet firewall is actually the same way
-
@fratopolis said in Subnet problem?:
@johnpoz said in Subnet problem?:
OH MY GAWD! NO!!!
That is NOT going to fix anything...
Do you NOT have a crayon and a napkin you can use to draw up your network..
So you just want all your devices on 1 flat network? How many devices do you have?
Before I go all out and break out my crayons what is wrong with this.
Basically it is saying if traffic has a destination on 10.35.x.x 10.20.x.x 10.120.x.x etc it should be routed through the LAN interface so long as it passes the pfsense rules and suricata. Then our Cisco C6832-X-LE will route the traffic to the subnet it needs to go to. We have over 11000 Chromebooks, over 3000 Windows, just under 400 printers and bout 1000 phones.
It is not a flat network as pfSense is not handling routing, vlans, etc of any sort. Instead of creating an interface for each this just says if bound for 10.x.x.x route through Internal network.
It's not that it won't work, it's just a bad design on multiple levels. I can almost guarantee the decision to assign the entire 10.0.0.0/8 to one interface will be revisited down the road.
-
@marvosa said in Subnet problem?:
@fratopolis said in Subnet problem?:
@johnpoz said in Subnet problem?:
OH MY GAWD! NO!!!
That is NOT going to fix anything...
Do you NOT have a crayon and a napkin you can use to draw up your network..
So you just want all your devices on 1 flat network? How many devices do you have?
Before I go all out and break out my crayons what is wrong with this.
Basically it is saying if traffic has a destination on 10.35.x.x 10.20.x.x 10.120.x.x etc it should be routed through the LAN interface so long as it passes the pfsense rules and suricata. Then our Cisco C6832-X-LE will route the traffic to the subnet it needs to go to. We have over 11000 Chromebooks, over 3000 Windows, just under 400 printers and bout 1000 phones.
It is not a flat network as pfSense is not handling routing, vlans, etc of any sort. Instead of creating an interface for each this just says if bound for 10.x.x.x route through Internal network.
It's not that it won't work, it's just a bad design on multiple levels. I can almost guarantee the decision to assign the entire 10.0.0.0/8 to one interface will be revisited down the road.
(22 different sites and multiple vlans at each site)
Well it is either doing it this way or around 100 network interfaces to add to pfsense and as you can see it was apparently how Fortinet was setup by the vendor. But all traffic is supposed to move internal anyway that does not get blocked so unless there is an actual reason why its not good then I think I'll stick with this way.I am all about doing things a better way and listening to someone smarter but if the best reason to do something different is "it's just a bad design on multiple levels" without an actual reason then I do not know how I am supposed to respond.
I'll draw up a simple network map during my xmas break and post it here.
-
@fratopolis said in Subnet problem?:
(22 different sites and multiple vlans at each site)
Well it is either doing it this way or around 100 network interfaces to add to pfsense and as you can see it was apparently how Fortinet was setup by the vendor. But all traffic is supposed to move internal anyway that does not get blocked so unless there is an actual reason why its not good then I think I'll stick with this way.
I am all about doing things a better way and listening to someone smarter but if the best reason to do something different is "it's just a bad design on multiple levels" without an actual reason then I do not know how I am supposed to respond.
I'll draw up a simple network map during my xmas break and post it here.Several of us would be more than happy to offer opinions and best practice ideas, but to offer anything targeted, we need a better understanding of your network, which is why most people have left things mostly high level and vague. So, if you're looking for more helpful feedback, you'll need to share more details... e.g. posting a network map and providing more details on how things are connected.
To put things in perspective, at work we have 32,000+ people spread over ~200 sites... several of which include large hospitals and there isn't one segment of the network that is larger than a /22 (most are /24). Design was mentioned because it sounds like it's possible that your network could benefit from an improved design. When following best practices in design, I can't think of any scenario where an interface on a firewall would need to have an 8 bit mask.
-
Ok I did it. If you need more on this god help me then lmao. There are many more vlans set up but we would need and entire wall to map it out completely. But this is the idea of it.
pfSense addon running are:
Squid Proxy Server
SquidGuard Proxy filter
DNS Resolver
softflowd
Suricata
-
So your 10.35.10 should be a TRANSIT network servers should not be on this network. Or just create a new transit network to connect pfsense to your cisco. Say 172.16.0.0/30 or /29
So 10.37.1 is also a transit that connects your different sites cisco routers?
So is the plan to get rid of the cisco as well, or just use pfsense as new edge router?
So you have 18 routers in each site? That seems a bit excessive.. But without seeing the whole thing its hard to say how best to fix it.. Guess nobody ever heard of cisco 3 layer model? ;) Core, Distribution, Access?
-
@johnpoz
LOL no 1 Router per site with many switches. I just didnt map the switches.Sorry forgot about this - Technically the transit that connects the routers at the sites is 10.254.1 and the transit within the sites switches is what I listed
example these are all mostly Cisco C3850-24XS routers and C2960X-48FPD switches
10.254.1.35 and at this site switches start at 10.35.1.10 with router being 10.35.1.254
10.254.1.37 and at this site switches start at 10.37.1.10 with router being 10.37.1.254
10.254.1.102 and at this site switches start at 10.102.1.10 with router being 10.102.1.254
10.254.1.104 and at this site switches start at 10.104.1.10 with router being 10.104.1.254
etc.i just access them by the switch transit on the router itself. Also if at any point you peeps think I'm without hope lol let me know. I can't learn any better than making mistakes myself lmao. My boss/friend calls me his biggest asset as well as his biggest liability for a reason.
Also I am learning terminology as I go if you can't tell. Didnt even know what transit meant here until you used it a couple times and I figured out what you meant and wanted.
**As I said, I am not a net admin (but having fun learning it).
My sysadmin skills are no longer growing(I have come up with every idea under the sun and implemented it so I am moving on as they say)I have no formal edu here as I graduated with a BS in Business Admin but had a passion since I took my first computer apart the day I got it on xmas in '97. Needless to say mom wasn't happy her $2k present was ripped apart.**
-
Replace both Ciscos and the Fortinet with two pfSense boxes. Done!
This network can be simplified, and by doing so, troubleshooting will be a LOT easier.
Run pfSense in a basic, default mode FIRST to get your routing working properly. Once you've done that, add stuff. If you cannot get the routing to work and you install add-ons, it makes troubleshooting that much more difficult.
-
@tim-mcmanus said in Subnet problem?:
Replace both Ciscos and the Fortinet with two pfSense boxes. Done!
This network can be simplified, and by doing so, troubleshooting will be a LOT easier.
Run pfSense in a basic, default mode FIRST to get your routing working properly. Once you've done that, add stuff. If you cannot get the routing to work and you install add-ons, it makes troubleshooting that much more difficult.
Sorry, there are not just two Cisco routers, but many many more than that, and this is a production environment so replacing them would mean bukku overtime during off hours which I can guarantee will not be approved. That sort of undertaking would have to be during summertime being I work for a school district.
-
Based on your drawing, my guess is at least one, if not multiple, things were happening in the beginning:
-
There was no return route on PFsense for the 10.35.60.0/24 network
-
The router at site 1 was not routing traffic properly to PFsense. You would need to configure PBR on the router at site 1, but considering traffic is flowing now, this appears to be done and working.
-
If there was a return route on PFsense, it may be pointed to the wrong IP.
-
There was no outbound NAT configured for traffic sourced from 10.35.60.0/24
PFsense will only NAT traffic from interfaces that it knows about, so when you had your LAN interface configured as 10.35.10.234/24, it was only NATing traffic sourced from 10.35.10.0/24 and nothing else. In order to get to the internet from your test host, you would've had to switch to Hybrid Outbound NAT mode and manually configure an Outbound NAT mapping for 10.35.60.0/24.
So, as soon as you gave your LAN interface a /8 mask, two things happened:
-
PFsense auto-generated an outbound NAT for traffic sourced from 10.0.0.0/8, which obviously includes IP's from the 10.35.60.0/24 network.
-
A route for 10.0.0.0/8 was added to the routing table, so now PFsense knows where to send return traffic
Your current setup will continue to work as is, but IMO there may be a cleaner approach. I would configure an unused, much smaller subnet as your transit network between PFsense and your Cisco router @ site 1 and then run OSPF as a routing protocol throughout the entire network. You could then get as granular as necessary with your NATing and firewall rules by creating aliases for allowing and/or denying internet access for certain subnets. Personally, I'd do all the same stuff even if you keep your Fortinet. Doing so will avoid possible routing issues down the road from things like site-to-site tunnels where the remote end has subnets in the 10.0.0.0/8 IP space.
@johnpoz said in Subnet problem?:
If devices do not point to pfsense as gateway then yeah more than likely your going to run into asymmetrical problems.
Not necessarily, It depends on the design. In this instance, his endpoints will be using the IP of each subinterface on the router or the SVI of an L3 switch stack as the default gateway.
@tim-mcmanus said in Subnet problem?:
Replace both Ciscos and the Fortinet with two pfSense boxes. Done!
This network can be simplified, and by doing so, troubleshooting will be a LOT easier.I agree with the K.I.S.S. principle, however, replacing the edge router (and others) at each site with PFsense would not simplify his network. All sites are connected via private circuits with internal addressing, so outside of special use cases, there is no need for a firewall at each site. Yes, PFsense can be configured as a router only, but IMO why add the complexity? His high-level drawing is fairly common among org's with multiple sites that are routed thru a main branch for internet. Not to mention, PFsense itself was not actually the issue, so IMO adding more PFsense boxes would not have helped him troubleshoot any more than he already had for this particular problem.
@Fratopolis Thank you for posting the map. Things became much clearer after seeing it.
-
-
@marvosa said in Subnet problem?:
Not necessarily, It depends on the design. In this instance, his endpoints will be using the IP of each subinterface on the router or the SVI of an L3 switch stack as the default gateway.
Yeah if there is a transit network connecting the L3 then no you wouldn't be asymmetrical.. But if he is going to try and leverage that 35.10 as a transit then yeah its asymmetrical.
Ya don't put hosts on a transit... Not sure how that network came to be.. But sorry while /8 might be a work around.. Its not the correct way to fix those problems.. He needs actual transit to connect pfsense to the downstream.
