Max Mac Count



  • I'm asking since i want to connect large netorks in lan , trafic is minimal or not a issue , rather the large number of devices.



  • @blackbrayn:

    I'm asking since i want to connect large netorks in lan , trafic is minimal or not a issue , rather the large number of devices.

    There is no particular max MAC count. In practice, the maximum will depend on the type of NICs, the amount of memory available to pfSense and traffic patterns.

    How large is "large"? 10s? 100s?



  • i'm talking about serios numbers 8k+ , as far as i know there is a limit in the kernel .



  • ps: all devices come on separate vlans on the same nic, and i need to create multiple virtual interfaces (svi).

    Very curios about similar setups/large numer of hosts behind pfsense.


  • Netgate Administrator

    Isn't there a limit of the number of vlans per trunk of 4096?



  • @stephenw10:

    Isn't there a limit of the number of vlans per trunk of 4096?

    I believe allowed VLAN ids are in the range 1 to 4095 inclusive.



  • Yeah, aside from MAC addresses, I would also worry about switches and pfSense, itself, handling that many VLANs.  Assuming you mean each individual device gets it's own individual VLAN, or are groups of devices segmented on to a smaller number of VLANs, like, say, 4+ devices on a VLAN?  Something to keep the number of VLANs at least slightly sane.  I know of some switches that might be able to handle VLANs in the higher numerical address numbers, but not exactly a large number of concurrent VLANs on a single switch.  And you might not notice that until you try (although I'm sure it's buried in a white paper somewhere, or it might even be an extra license, which is always fun.)

    In any case, considering you're using (or planning to use) VLANs to segment that traffic already, it seems like these devices may not be needing to talk to each-other, so it might even benefit you to run a couple of downstream pfSense routers to reduce the MAC address and VLAN load on both your switches and pfSense.  I've known a few places that do this to reduce the internal network chatter from the LAN interface of their WAN facing router (Yes, with pfSense, it's for a 400+ person LAN party, lots of internal traffic.)



  • Maybe QinQ could help you on this, but why to separate 8k+ clients to own vlans



  • not a vlan number problem , each vlan has it's own network like /24 , /23 , /22 the problem is just total mac count .

    bump.



  • @blackbrayn:

    bump.

    What do you mean? You still have an unanswered issue?

    As has been already stated, you can't get 8K devices on a distinct VLAN per device on a single NIC. The limitation is the 12 bit VLAN id field in the frame.



  • 8k is the Total mac count , meaning this 8k is split among the diferent vlans. Switches and routers have a TOTAL mac adress limit .



  • check this out , this is the real problem like stated here : http://www.vyatta.org/forum/viewtopic.php?p=53428



  • I can't find any evidence of a limit on the FreeBSD arp table size, but that may just be an indication I don't know where to look.



  • If I recall correctly, pfsense is limited only the amount of ram you have installed. and continuying the same remembering line, one mac entry is one kB of ram, so 8MB would handle 8k mac-addresses.

    But this was only that if I remember correctly.

    Comparison between Switch and pfSense is not reasonable anyway. Those are different kind of beasts for it's own purpose.



  • FreeBSD uses the routing table for ARP entry storage, as such it's limited by the routing table size. The routing table doesn't have a fixed limit. You'll eventually run out of kernel memory, but we're talking very large numbers, into the millions. There are many production systems that have 400,000+ routes (full BGP feed), which is basically equivalent to having a small number of routes and 400,000+ ARP cache entries. So I don't think you'll have any issues unless you have millions of directly-connected devices, which would probably be a really poor network design and probably generate enough traffic it would melt down any firewall you can get.



  • so you're saying i don't need to modify the "net.ipv4.neigh.default.gc_thresh3=8192" to a value of my needs? 8192 is arbitrary , i chosed this large number.



  • @blackbrayn:

    so you're saying i don't need to modify the "net.ipv4.neigh.default.gc_thresh3=8192" to a value of my needs? 8192 is arbitrary , i chosed this large number.

    There is no such tunable on pfSense.

    pfSense is based on FreeBSD, not Linux (like Vyatta you mentioned, and most other router/firewall distros), so the base system is very different "under the hood".



  • So the question remains , is there a limit on the arp entries after all? till now nobody can say for sure , a definitive answer would be good , since i plan to deploy a install and the sucess is based on pfsense suporting such a high number or arp entries.



  • @blackbrayn:

    Switches and routers have a TOTAL mac adress limit .

    Many of them have much less memory than the pfSense minimum.

    @blackbrayn:

    So the question remains , is there a limit on the arp entries after all? till now nobody can say for sure , a definitive answer would be good

    I doubt you will get a more definitive answer than
    @cmb:

    FreeBSD uses the routing table for ARP entry storage, as such it's limited by the routing table size. The routing table doesn't have a fixed limit. You'll eventually run out of kernel memory, but we're talking very large numbers, into the millions. There are many production systems that have 400,000+ routes (full BGP feed), which is basically equivalent to having a small number of routes and 400,000+ ARP cache entries. So I don't think you'll have any issues unless you have millions of directly-connected devices, which would probably be a really poor network design and probably generate enough traffic it would melt down any firewall you can get.

    FreeBSD has a number of memory "pools" that are used to allocate memory for different purposes. The sizes of these pools depends on the memory in the system. Hence the routing table might be able to grow to a significantly larger size on a system with 2GB RAM than it can on a system with 256MB RAM.


  • Netgate Administrator

    Here's a second opinion from a few years ago:
    http://freebsd.1045724.n5.nabble.com/Maximum-ARP-Entries-td4017394.html

    Steve


Log in to reply