Microsoft NLB and Pfsense version 2.2.4 issue

chriva

Hi to all,
I need help for a quite strange problem.
I have a 2 physical nodes pfsense 2.2.4 cluster (64 bit version), just upgraded from 2.2.3.
I have lagg with some vlan configured between internal interfaces.
On one of those interface there is a Microsoft NLB cluster, that needs to be accessed from the others interfaces with microsoft integrated authentication
After adding the system tunable net.link.ether.inet.allow_multicast=1 the NLB cluster is available (ping works) and 50% of the clients connect to the NLB iis correctly.
The problem seems to be linked to the PC originating the request, and no to OS version or subnet or user but. (we have some win7 and win8 on subnet A and subnet B woking and some others not working).
NLB 192.168.205.8
FW 192.168.205.254, 192.168.202.254, 192.168.203.254
FWP 192.168.205.250 (active node phisical interface)
Good Client 192.168.203.46
Bad Client 192.168.202.244

I inspected the traffic with 2 symultaneous captures at a time (one NLB interface , the other on the client interface).
The capture begins the same, and after evolves differently, as below

common part capture
…
client → NLB http get /resource/
NLB → client http/1.1 401 unauthorized
client → NLB http get /resource/ http/1.1 NTLMSSP_NEGOTIATE
...
NLB facing interface, for the bad client
…
NLB → client http/1.1 401 unauthorized , NTLMSSP_CHALLENGE (text/html)***
(0.000010 sec delay and no other packet between)
FWP → NLB icmp destination unreachable (host unreachable) (NLB → client)
client → NLB http tcp retrasmission get /resource/ http/1,1, NTLMSSP_NEGOTIATE
NLB → client tcp dup ack of ***

client facing interface, for the bad client
…
client → NLB http tcp retransmission get /resource/ http/1.1 NTLMSSP_NEGOTIATE
NLB → client tcp previous segment not captured

NLB facing interface, for the good client
…
NLB → client http/1.1 401 unauthorized , NTLMSSP_CHALLENGE (test/html)
client → NLB http get /resource/ http/1,1 NTLMSSP_AUTH , user: domain\user
…

client facing interface, for the good client
…
NLB → client http/1.1 401 unauthorized , NTLMSSP_CHALLENGE (text/html)
client → NLB http get /resource/ http/1,1 NTLMSSP_AUTH , user: domain\user
…

I've found that pfsense is systematically dropping the packet indicated ***
No other strange packet are present in the cature (no arp request).
The same results with a single windows server in the NLB.
I also tried adding the bad client mac address statically to the pfsense, with not luck.
Has anyone an idea about this issue?

Thanks,
Chris

jimp

https://doc.pfsense.org/index.php/Upgrade_Guide#Microsoft_Load_Balancing_.2F_Open_Mesh_Traffic

chriva

Hi jimp,
thanks for the hint,
but I already set the value in the system tunables: some clients can connect correctly from the other networks.
Other suggestions?

jimp

Nope, that's the only thing on the firewall that will care about the way MS does load balancing.

chriva

Hi jimp,
I know this and all was smooth until version 2.2.3 (with net.link.ether.inet.allow_multicast=1 )
The problem begun when we upgraded to 2.2.4.
The network trace shows a systematical drop for a specific packet (for an already open connection) from the NLB interface to the client interface.

Hoping someone can help,
Chris

jimp

There weren't any changes that I can see that would have affected anything of that nature between 2.2.3 and 2.2.4.

You can try a 2.2.5 snapshot from https://snapshots.pfsense.org/ to see if it's any better though.

chriva

I,
after a deep dive in packet analisys an sniffing i found out that the problem was due to large packets with a strange (0.06 sec or greater) delay.
Those packet disappears without any warning when hitting client interface.
I finally found a workaround
with a standard rule on client interface

client --> NLB:80 with advanced features
state type = none

Bye,
Chris