Subnetting /32 vs /24

[fireix]

I will start with a new /24 public static range of IPv4 and I'm looking for some insight.

Most customers (99%) will have a single public IP for their webserver. The only local interaction is against common backup-server and monitoring. For simplicity, assume transparent pfsense (bridge-setup).

Simplified picture - in this order as seen from the external Internet:
My ISP in data center -> pfsense FW -> Switch -> Servers (30 of them)

My main goal for this exercise is to reduce broadcast noise/possible misconfiguration and keep maintenance down (not operate with a set of both local and remote IPs). Each server has their own software fw in addition to the main central pfSense (for instance to be able to track repeated logins/blocking and local software that only server knows about).

Will having each server configured as IP-address/32 vs IP-address/24 reduce any broadcast traffic - when it is all connected to same switch before their gateway (the pfSense FW) and I have not enabled/configured any VLAN? Or will it be equal? Will this change at all if going with /30? I assume that unless I do VLAN and port-limitations, I will not get any benefit neither on security (isolation from bad hosts on the same switch) nor limit broadcast traffic (to avoid overloading network(/swich and reduce servers listening to other servers).

[JKnott]

@fireix

You need at least a /31. A /32 won't do much, as you need at least 2 addresses for anything to work. As for broadcasts, etc., are these on separate networks? If not, it won't make much difference.

[johnpoz]

@fireix if your goal is limiting broadcast and or multicast between your different servers.. I would do this at the switch level, be it private vlans to just isolate them. Or with ACLs to block the multicast or whatever traffic.

You could also just setup specific vlans on the switch for each server, ie 30 of them

[fireix]

@JKnott said in Subnetting /32 vs /24:

@fireix

You need at least a /31. A /32 won't do much, as you need at least 2 addresses for anything to work. As for broadcasts, etc., are these on separate networks? If not, it won't make much difference.

What do you mean? It will not work if I configure it as /32? It only really needs to communicate with the GW at the pfSense and then out on the internet, but if it will not work to configure it as /32, then no sense no :) It is on same switch, the only divider is if possible this subnetting of the /24 into smaller networks.

[fireix]

@johnpoz said in Subnetting /32 vs /24:

@fireix if your goal is limiting broadcast and or multicast between your different servers.. I would do this at the switch level, be it private vlans to just isolate them. Or with ACLs to block the multicast or whatever traffic.

You could also just setup specific vlans on the switch for each server, ie 30 of them

So a pure subnetting by dividing the network into /32 or /31 (from the /24 method today where every server is on same network) will not improve anything unless I configure VLANs/ports on the switch, correct?

[johnpoz]

@fireix said in Subnetting /32 vs /24:

configure it as /32? It only really needs to communicate with the GW

And why would it talk to the gateway.. There is no network with a /32 - it just has the IP.. To be honest a /32 isn't really a valid interface mask, its valid as say a firewall rule, or even a route.. But I wouldn't put it on an interface. As a loopback maybe.

Lets say it does, why would that stop broadcast.. A broadcast is to 255.255.255.255, doesn't matter what the source IP network is. Same with multicast, those are to different networks completely.. Even if not sent to all FFs for mac, and sent to a multicast mac, the switch since its multicast mac would send it to all ports, and it would be seen by all devices on the switch.

Even if you used say a /31 or /30 even.. this isn't going to stop a broadcast.. It could limit your directed broadcast network, ie for example 192.168.0.0/24 the directed broadcast would go to 192.168.0.255, but it would still be sent to the all FFs address at L2..

If your goal is to limit broadcast and multicast from being seen by other devices on the same switch, either vlan them out, use private vlans (which is just really simple way to not actually create vlans for each device) or create ACLs on the switch to block broadcast/multicast

What mask you use for the network be it a /30 or a /16 really isn't going to have anything to do with broadcast or multicast.

a Broadcast sent to all FFs or multicast mac is at layer 2, your setting of the L3 ip range doesn't really have anything to do with that.

Other options is to limit or reduce the amount of broadcast/multicast traffic being sent by the client.. Windows is horrible for example on the multicast it will send.. It can be reduced by turning of specific things like SSDP, etc. But if your goal is reduced layer 2 traffic that is seen by other clients on the same switch - that is done at the switch level, not at the IP network you assign to a specific host connected to the switch.

edit:
Arp is another thing that will be sent to all FFs - so even if the device was only arping for its gw, which with a /32 is outside its "network" and not a common setup for a gatway to be outside the network the interface is on. It would be sent to all FFs mac and the switch would send it to all ports that are in the L2 vlan it saw the arp come in on..

[JKnott]

@fireix said in Subnetting /32 vs /24:

What do you mean? It will not work if I configure it as /32? It only really needs to communicate with the GW

You have it right there. The gateway also needs an address, so that's 2, which requires a /31 or shorter mask.

[JKnott]

@johnpoz said in Subnetting /32 vs /24:

And why would it talk to the gateway.. There is no network with a /32 - it just has the IP.. To be honest a /32 isn't really a valid interface mask, its valid as say a firewall rule, or even a route.. But I wouldn't put it on an interface. As a loopback maybe.

A /32 can be an interface address, just like with a /128 on IPv6. My global unique WAN address has a /128. However, it's a bit different with IPv6 in that link local addresses are often used for routing and not the global WAN address. Can IPv4 route, without the gw and device being in the same subnet? I'm thinking back to when I had a dial in connection to my home computer, back when I was running Red Hat. I had to configure an address for both ends of the connection and then use proxy arp to work through it.

To route with either IPv4 or IPv6, you need the next hop. This is typically an IP address, though could be just the interface with point to point links. With IPv4, how do you reach the gw on the LAN, without it having an address within the same subnet? If the gw has an address within the subnet, it's just a matter of doing an arp request to get the MAC and then forwarding the frame using that MAC. If it's not, it's going to try forwarding through the gw, even though it doesn't know how to reach it. With IPv6, you just do a router solicitation, which returns the link local address and MAC. You then use the MAC as before.

This is something that confuses some people about IPv6. They wonder why they only have a /128 for the global WAN address, if they even have one at all. The fact is they don't need one, as routing is done via the next hop link local address and the global WAN address is just another address on the destination network. That it happens to be on the WAN interface is irrelevant. It's also why if you're not assigned a WAN address, you can make your own by using a address from within your prefix and assigning it to the WAN interface.

[johnpoz]

@JKnott What I think he was thinking of doing was putting the /24 on pfsense so say 1.2.3.1/24 and then making his servers 1.2.3.2/32, 1.2.3.3/32, 1.2.3.4/32 etc..

Or even making pfsense 1.2.3.1/32 ?

But you don't filter layer 2 things with layer 3.. None of the traffic he mentions filtering broadcast and multicast makes any difference what the layer 3 address of the device interface is. Its either going to 255.255.255.255 with all FFs as the mac, or its going to some multicast address 224/4 or 239/8 which will go to a multicast mac, and have zero to do with whatever IP or mask the devices interface might have on it.

So even if he uses a /31 or /30 - that not going to stop broadcast and multicast from being seen by the other devices on the switch... The only way to do that is isolation at layer 2.

edit:
But you bring up a good point about IPv6 - unless he has some way of completely utterly disabling that, which is unlikely at the link-local level - there would be also be multicast traffic from that being seen by all devices on the switch in the same vlan.

edit2: if me I would just setup the /24 he has.. Then setup private vlan so all the devices are isolated from each other and can only talk to the pfsense IP.. Any broadcast or multicast traffic they send should only ever go out the pfsense port on the switch. This port would be the uplink port or promiscuous port.. And as long as the ports he connects his servers too as private nothing else connected would see any of the broadcast/multicast traffic from other devices.

If he has customers with need of multiple devices connected, he can set those up as community ports where broadcast/multicast would be seen by all ports in that community.

@fireix if your not familiar with private vlans, which I assume with this sort of question in the first place

https://en.wikipedia.org/wiki/Private_VLAN

[fireix]

@johnpoz said in Subnetting /32 vs /24:

What I think he was thinking of doing was putting the /24 on pfsense so say 1.2.3.1/24 and then making his servers 1.2.3.2/32, 1.2.3.3/32, 1.2.3.4/32 etc..

100 % correct. pfSense is GW with .1/24 today.

I have run the same config with IP-address 1.2.3.2/24, 1.2.3.3/24 etc on each servers for 15 years and it works of course. Lot of broadcast traffic when sniffing the traffic. But since I do get a new range now (still /24) and could actually start from scratch, I was wondering if adjusting it to be /32 on each server as @johnpoz mention (1.2.3.2/32, 1.2.3.3/32 etc on each server) would help isolate a bit. And the answer looks like no, unless I do some more work with vlan as mentioned. I'll read up on that.

All ipv6 devices on the network is disabled, so that's not an issue.

[johnpoz]

@fireix yeah not a fan of broadcast and multicast that serves no purpose.. In the big picture its not a lot of traffic or anything.. But its still noise if you ask me - if your not going to be using it..

Arp, dhcp - ok those make sense for broadcast - hey who has ip xyz, hey is there a dhcp server out there I need an IP.. And this only sent when something is needed, want to talk to specific IP - you need to know the mac, etc.. But its not arping every freaking 10 seconds, etc.. And dhcp hey only when need an IP or want to renew its current one..

SSDP and UPnP etc - just noise, because those protocols are not used on my network for anything.. Devices love to send that shit out.. All the time.. Plex sends out ssdp ever freaking 10 seconds and there is no way to disable it.. I finally just used an ACL at the port my nas is connected with that plex runs on blocking it from entering the network.. So it can send it down the wire to the switch - but switch doesn't send it anywhere else..

Its not that was a lot of traffic or anything - it was more was tired of seeing it when sniffing on my network ;) hehehe

You throw a bunch of windows machines on the same network - and there is going to be quite a bit of noise!!

Apple devices are noisy little shits too ;) All of that noise should be able to be disabled.. And not just multicast - dns noise also drives me nuts

lb._dns-sd._udp.0.2.168.192.in-addr.arpa
lb._dns-sd._udp.home.arpa

Not doing anything with dns-sd - stop asking for it!! Should be able to turn that off if not using it.. But can not find any way to do so - wife and I both have iphones, and an ipad - all noise senders!!

[fireix]

@johnpoz I have read up on Private VLAN and the benefits there. It started kind of interesting, but I hit a brick wall I think.

Supermicro blade switch seems to do private vlan using protected-term and bit different from general Private VLAN terms, but that is fine I guess once I found out. With one single command, every port will be isolated and can only communicate with the Unprotected Port (in this case Uplink port) - so that is great:

SMIS# configure term
SMIS(config)# interface range fx 0/1-48
SMIS(config-if)# switchport protected
SMIS(config-if)# exit

So this looked good to stop inter-communication at that switch level, but what would stop the units from communicate through the Uplink switch and back? :) I assume the answer is with configuring VLAN upstream also, either/both in the Upstream switch and/or the pfSense at final stage?

5.14 Private Edge VLAN / Protected Ports
The Private Edge VLAN or also called Protected Ports feature helps to isolate the traffic among the same
VLAN ports. A protected port cannot forward any traffic to another protected port on the switch even if
they are in the same VLAN.

[johnpoz]

@fireix how would they talk back? Pfsense isn't going to route the traffic, they would never send the traffic to pfsense to get to something on their own network anyway.

If you were running reverse proxy on pfsense - you could prob access other box website it was running.

[fireix]

@johnpoz Okey, lets keep Pfsense out of the picture for now to simplify it. Let's say a device on the protected port/protected switch sends out for instance a DHCP broadcast (as someone fires up a DHCP server for fun) everywhere (just to take one example). This would now be blocked on the protected switch-level, no inter-communication possible on that switch. Then this is sent upwards to the Upstream switch... When I write this down now, I think I understand what you say.. it wouldn't go back into the protected switch ;) But the DHCP broadcast would at least be given to all servers on the Uplink switch unless I also do some protection there at least?

I know that DHCP can be turned off using policy or a checkbox on the port, so just an example of inter-port communication.

[JonathanLee]

Research broadcast domains and what separates them. This was a big Cisco Netacad test topic years ago. Switches filter by layer 2 48 bit MAC addresses so it has a valid arp table and knows what MAC address per port. When a system arps out to find something that is your broadcast domain, routers stop broadcast domains much like a demarcation point ends the wan connection and marks where the private networks begin. It will not matter what cider notation you use it will matter only really matters the size of an arp table, with today’s network equipment broadcast storms are a thing of the past. Spanning tree and layer 3 switches yes layer 3 switches make issues you are asking about kind of outdated. I would use a subnet calculator and see how many devices you need account for future growth and choose that subnet. /24 or what ever that may be.

/24 you get 254 useable hosts
/32 you get 1 useable hosts
/14 you get 262,142 useable hosts

Doesn’t mean it will arp out to hosts not configured it will poll only what is in that arp table for that section of the network. With ipv6 it’s even more simplified.

[johnpoz]

@fireix so you have a daisy chained switches? If you create a private vlan on switch X, you should be creating that protected vlan on all switches. That vlan is on.

[fireix]

@johnpoz I have tried to draw this in Lucidchart. My plan is to get help to configure it if I can't do it myself, but does this make sense? :) Let me know what I should do to make drawing even more clear.

All this gear is in one single rack with only spf+ ports/switches. I haven't used MLAG (that my BladeSwitches support) and real stacking switches (FS stackable) before now, so I hope this will make the network far better than relying on STP and slow failover time. It is mainly each blade in the chassis I want to isolate from eachother, they don't need anything local other than backup-server and monitoring.

I haven't choosen what of the three types of bladeswitches I will choose. There is both one Layer2 and one Layer3. I assume Layer3 would give me most options later...

[coxhaus]

@fireix Just remember VLANs were designed to limit broadcast traffic. The next thing is switching is faster than routing. And last, layer 3 switches are faster than routers. I think of this when I design networks.