Broadcast storm
-
Ok so use policy based routing..
What is this broadcast storm your saying happens?
-
Ok so use policy based routing..
What is this broadcast storm your saying happens?
That's what I'm doing and after I create the firewall rule for policy based routing a broadcast storm occurs. It is like a cable is connected from and to the same switch, loop.
-
Dude draw up your connections.. You mention vlans.. And what is the broadcast storm.. Where is is coming from, what is the broadcasts your seeing?
"and all these network are on one vlan,"
So your trying to run different L3 over the same L2 ??? Your saying pfsense wan and lan plug into the same switch on the same vlan? Well yeah that is a freaking loop!!
-
what i make of it:
-DMZ is in an isolated subnet, no way to get to/from it
-fortigate & asa share the same ip adress
-all your gateways are inside the same subnet (= no go)as @johnpoz said: need more info
-
Also looks like two of your gateways have the same IP address. You mention your gateways have a route (I'm assuming static) to your LAN, but does your LAN have a static route back? Did you manually assign each client their own gateway? I'm assuming these are hooked up to different ISP'S or is this a completely isolated network? So your static routes are using /32? Can we see your routing table? Like others network looks a little weird, but I'm sure their is a reason you have it wired this way? Do you have a dynamic routing protocol turned on? This can cause a broadcast storm if you have multiple paths to the same network.
-
Dude draw up your connections.. You mention vlans.. And what is the broadcast storm.. Where is is coming from, what is the broadcasts your seeing?
"and all these network are on one vlan,"
So your trying to run different L3 over the same L2 ??? Your saying pfsense wan and lan plug into the same switch on the same vlan? Well yeah that is a freaking loop!!
A router shouldn't forward broadcasts between interfaces. That's what the interface bridging is for. I can't see where the broadcast storm is from.
Here is a new picture with the connections. Just to be clear, I know it's not best practice to put the interfaces in the same vlan but that was just for this lab environment.
I'll reproduce it again in the lab environment and debug where the broadcast is from, not sure if it's layer 2 or 3 broacast. If it's a layer 3 broadcast it might be explainable. But still weird it only happens after I change the gateway for the default rule.
-
what i make of it:
-DMZ is in an isolated subnet, no way to get to/from it
-fortigate & asa share the same ip adress
-all your gateways are inside the same subnet (= no go)as @johnpoz said: need more info
You're right, I messed up the drawing a bit. (Reality and lab mixed in this drawing haha)
About the gateway thing… why not have multiple gateways inside same subnet?new drawing:
-
Also looks like two of your gateways have the same IP address. You mention your gateways have a route (I'm assuming static) to your LAN, but does your LAN have a static route back? Did you manually assign each client their own gateway? I'm assuming these are hooked up to different ISP'S or is this a completely isolated network? So your static routes are using /32? Can we see your routing table? Like others network looks a little weird, but I'm sure their is a reason you have it wired this way? Do you have a dynamic routing protocol turned on? This can cause a broadcast storm if you have multiple paths to the same network.
Yes, drawing was messed up, added a new one in the post above. LAN doesn't need a static route back to the dmz, because it is in the direct connected network from the pfsense.
It is not a seperate ISP, all is one ISP, but we have a routed subnet so the external firewalls (fortigate, sophos cisco) have an external (routable) address and an internal address for the DMZ.
I didn't turn on anything like dynamic routing protocol, where is this located?Edit:
if you mean with dynamic routing = RIP then no, it's not turned on. -
Dude that is such a BAD idea be it lab or not… Your running different layer 3 over the same layer 2.. And why are you using a /16 network that is just BAD practice as well.
Why not just get another switch?
Show how you have your gateways setup in pfsense..
-
As it is set up now, you don't have any separation between LAN and DMZ because they are on the same switch effectively nullifying any firewalling you have on the pfSense.
-
Dude that is such a BAD idea be it lab or not… Your running different layer 3 over the same layer 2.. And why are you using a /16 network that is just BAD practice as well.
Why not just get another switch?
Show how you have your gateways setup in pfsense..
Haha I know it's not a good idea, but a broadcast storm shouldn't happend either. Who / what says that /16 is bad practice? It is in RFC for private network ip range: https://tools.ietf.org/html/rfc1918 it is very common. We need it for the internal address space, we have a lot of servers.
It actually is 802.1q capable switch, but wanted to setup the lab fast so didn't configure it.
Currently I'm not able to reach the lab enviroment. I'll show gateway setup as soon as I can.
-
@kpa:
As it is set up now, you don't have any separation between LAN and DMZ because they are on the same switch effectively nullifying any firewalling you have on the pfSense.
Yes, but that is not the purpose of this post. And what is currently looks like even separting the switches do not have any effect if the broadcasts are being forwarded. But that is just speculation, I have to debug more to see what broadcasts are being forwarded.
-
/16 has 65K hosts… You have that many servers?? That is one freaking huge broadcast domain.. Nobody would ever have that many hosts on the same broadcast domain.. So 10 in the rfc1918 space a /8 -- you think its good idea to use a /8 mask on your interfaces.. You don't think that might have issues with overlap somewhere?
It is bad practice to use an unrealistically large mask because of pure laziness yes.. Do you need that in a rfc somewhere to know its a bad idea?? And bad practice?
-
/16 has 65K hosts… You have that many servers?? That is one freaking huge broadcast domain.. Nobody would ever have that many hosts on the same broadcast domain.. So 10 in the rfc1918 space a /8 -- you think its good idea to use a /8 mask on your interfaces.. You don't think that might have issues with overlap somewhere?
It is bad practice to use an unrealistically large mask because of pure laziness yes.. Do you need that in a rfc somewhere to know its a bad idea?? And bad practice?
True, current network is around 500 ip adresses. Thats also why im setting this up because we are going to split the subnets. It's not that I do not agree but was just curious if there is any best practice for subnet sizes.
-
i doubt there are any "official" rules, but imho, it's best to limit to /23 or /22 (thats around 512/1024 available ip's)
I have a /22 on a public wifi hotspot because i don't wish to run the same SSID on multiple vlans to split it up / it's not an ideal situation, but for sake of simplicity i keep it like that -
Best practice would be correct size ;) That allow for room for growth.. and its good idea if possible to leave adjacent networks open so that you could expand even more or bring up a new subnet in the same logical block, for lots of reasons one of which is the ability to summary route, etc..
If you have 500 servers then a /23 would be most likely enough.. But doesn't leave a lot of room.. Do you really have all your servers on same subnet now?? These servers all do the same thing for the same people? Quite often servers would be broken up into their own subnets based upon location/function/dept etc. etc.. 500 on the same broadcast domain seems high to me to be honest..
You sure wouldn't put servers that serve public or other parts of the network on same subnet as say your database servers or print servers or AD servers, etc.. So that can firewall traffic.. Sure I could see if small location with a handful of servers, and nothing to the public might be easier to just put everything on one network..
But a location that has 500 servers, unless your talking 500 of the same thing I don't see those being on the same network anyway..
-
Best practice would be correct size ;) That allow for room for growth.. and its good idea if possible to leave adjacent networks open so that you could expand even more or bring up a new subnet in the same logical block, for lots of reasons one of which is the ability to summary route, etc..
If you have 500 servers then a /23 would be most likely enough.. But doesn't leave a lot of room.. Do you really have all your servers on same subnet now?? These servers all do the same thing for the same people? Quite often servers would be broken up into their own subnets based upon location/function/dept etc. etc.. 500 on the same broadcast domain seems high to me to be honest..
You sure wouldn't put servers that serve public or other parts of the network on same subnet as say your database servers or print servers or AD servers, etc.. So that can firewall traffic.. Sure I could see if small location with a handful of servers, and nothing to the public might be easier to just put everything on one network..
But a location that has 500 servers, unless your talking 500 of the same thing I don't see those being on the same network anyway..
Yes, 500 servers in one broadcast domain. That's why im going to split it up now before it's to late haha . It's very easy to split it up because there are a lot of different servers.
-
Ok, I did some more research. I found out that the storm is a L3 storm, a netbios storm.
I did a wireshark capture, and this is the line that keeps coming back:
source 172.16.1.1 destination 172.16.255.255 protocol NBNS info name query NB ISATAP<00>This is an example, but the info part changes according to other names.
I found this post that looks like a similar issue:
https://forum.pfsense.org/index.php?topic=95379.0Is it normal behavior to forward netbios request between interfaces? If so, what's the use of it?
Edit:
The equivelant of cisco is "no ip directed-broadcast" is this possible with pfsense? And so not, isn't pfsense vulnerable for a smurf attack? -
And the weird thing, if I turn of the gateway in the default lan to * rule this is not happening.
-
I am not aware of pfsense forwarding directed broadcast or broadcast traffic.. But maybe since you say that ANYTHING send it to this gateway… But if you didn't have those 2 different layer 3 on the same layer 2, then you wouldn't have an issue would you??
