LAGG Traffic going missing - Not THAT Simple :)

evilgoat76

Hi All, this one is driving me nuts.

Have had a few glitches with this VM so debating burning it and starting over, first time I've had an 'out of box' setup play up, BUT…

PFsense 2.3.4 in a ESXi 6 VM

PF has 4 interfaces. em0 is a passthrough at hardware level as are em2 and em3
em0 is "wan", my internal lan on 192.168.99.1
em1 is the rack's lan, the PF manages this, dhcp on 192.168.224.1/24, all works hapilly
em2 and emg are in a LAGG and connected via fibre to a VDSL2 DSLAM. There is no DHCP on this interface and PPPOE server. Network is 192.168.225.1/24

So
Modems fire up, sync, PPPOE wakes up, packets fly, everything handshakes and the world is happy. Clients can browse, phone works, all great

HOWEVER
Some client devices need to appear on the LAGG as individual devices. SO the modems are jammed in bridge mode and the client is given a static IP address. This is where it all breaks

The clients cant ping, do DNS resolution or anything. TCPdump on the DSLAM shows data coming through correctly but no reply. EG a ping I see the ICMP request go out, nothing comes back. On the PF a tcpdump on LAGG0 shows the same story. If I fire up another modem with PPPOE I can see the PPPOE traffic to and from it and the second modem works.

Bizarreley, ARP is working. EG if I flush the ARP table and then ping 192.168.225.1 from the client, an arp request is sent, a reply is received and the ARP table is correct. I can see this in TCPdump.

Nothing in the firewall logs pointing to traffic being dropped.

If I try to ping accross the PF, eg something on the LAN, something on my LAN or the outside world the packets never make it to the destination interface.

Any suggestions?

evilgoat76

resolved. LAGG0 was set to 192.168.225.1/32 for some odd reason.

johnpoz

most likely because that is what an IPv4 address defaults when you set it, and prob forgot to change it. I have noticed more than just a few threads because user forgets to set the mask and it default to /32 for ipv4 and /128 for ipv6

I have put in a feature request to change that, but sure there is way more important things for them to work on ;)

evilgoat76

I was about to say I never have, but thats not true as I always do the inital setup through the webUI or First Use Wizard.

If its a reocurring issue shouldnt it perhaps default to /24 which I can see being a more useful default?

This whole setup is a huge learning curve for me, its way past the usual setup firwall, openvpn, job done…

johnpoz

The reason I put in the feature request.. No you wouldn't see it if just setting up via wizard, etc.. It comes into play when adding opt interfaces, etc. The mask is a drop down list - has to default to somewhere ;) Defaulting to the end of the list seems to be just the way it normally happens.

I do concur that the most common mask used as default might eliminate such new user mistakes.. But when it comes down too it, user has really nobody to blame but themselves - when setting up a static IP on an interface they for sure should be checking and understanding what mask they want to set on the IP…

Part of the problem is just users that don't really understand.. Just take a gander at how many posts you see where users have set the default mask on say 10/8 or 192.168/16 etc.. ;) And trying to make it kind of idiot/mistake proof while also allowing for more complex sort of setups, etc.