[CARP/VIPs] Problem accessing servers

ptitvert

Hello,
I am quite new in the pfSense world, and I am not a network specialist…

I have created a network schema to show what I have build.
I wanted to have some VPN redundancy in order to allow our users to connect from different part of the world.

When they connect to the vpn01 (on the VLAN 500) everything works, and the users can access all the servers in VLAN 100.

When they connect to the vpn01 (on the VLAN 510), then the VLAN 1000, 500 and 510 are working, but not the VLAN 100, which is quite a problem...

I have created in the both pfSense servers (Master2 and Backup2) 2 VIPs as gateway for both VLAN 1000 and 100.

I have tried many things, and I have quite a problem to troubleshoot the issue in my design OR in my pfSense configuration...
I have used quite a lot of tcpdump, and it seems that a ping from the server 10.2.1.2 (vpn02) reach the server 10.100.0.20 and when it comes back then the packet are lost somewhere in space...
From a firewall point of view, it seems that all packet are passed correctly, until it goes to server pfSense FW 2


18:09:33.509178 IP 10.2.1.2.58738 > 10.100.0.20.27017: Flags [P.], seq 474:632, ack 1270, win 1444, options [nop,nop,TS val 435338063 ecr 881923519], length 158
18:09:33.528206 IP 10.100.0.20.27017 > 10.2.1.2.58738: Flags [P.], seq 1270:1693, ack 632, win 1452, options [nop,nop,TS val 881924024 ecr 435338063], length 423

I see going it to the interface with the ip 10.2.1.1


18:09:31.508867 IP 10.2.1.2.58738 > 10.100.0.20.27017: Flags [.], ack 1270, win 1444, options [nop,nop,TS val 435337563 ecr 881923519], length 0
18:11:53.007406 IP 10.100.0.20.27017 > 10.2.1.2.58738: Flags [P.], seq 424:847, ack 316, win 1452, options [nop,nop,TS val 881958894 ecr 435372933], length 423

And in the VPN02 (ip: 10.2.1.2) I get that:


18:16:09.787331 IP 10.2.1.2.58738 > 10.100.0.20.27017: Flags [.], ack 424, win 1444, options [nop,nop,TS val 435437132 ecr 882023088], length 0
18:16:10.359822 IP 10.2.1.2.59092 > 10.100.0.20.27017: Flags [s], seq 2340577243, win 29200, options [mss 1460,sackOK,TS val 435437276 ecr 0,nop,wscale 7], length 0
18:16:11.787244 IP 10.2.1.2.58738 > 10.100.0.20.27017: Flags [P.], seq 158:316, ack 424, win 1444, options [nop,nop,TS val 435437632 ecr 882023088], length 158
18:16:11.806357 IP 10.100.0.20.27017 > 10.2.1.2.58738: Flags [P.], seq 424:847, ack 316, win 1452, options [nop,nop,TS val 882023593 ecr 435437632], length 423
18:16:11.806374 IP 10.2.1.2.58738 > 10.100.0.20.27017: Flags [.], ack 847, win 1444, options [nop,nop,TS val 435437637 ecr 882023593], length 0
18:16:13.294391 IP 10.2.1.2.59270 > 10.100.0.20.27017: Flags [s], seq 3643486226, win 29200, options [mss 1460,sackOK,TS val 435438009 ecr 0,nop,wscale 7], length 0
18:16:13.807158 IP 10.2.1.2.58738 > 10.100.0.20.27017: Flags [P.], seq 316:474, ack 847, win 1444, options [nop,nop,TS val 435438137 ecr 882023593], length 158
18:16:13.826300 IP 10.100.0.20.27017 > 10.2.1.2.58738: Flags [P.], seq 847:1270, ack 474, win 1452, options [nop,nop,TS val 882024098 ecr 435438137], length 423

At the moment, I have not firewall in the host 10.2.1.2... no tcp wrapper, or something that blocks the network flow.

Could you help me to understand what could be my problem?

Thank you very for your help!

Alex
![simpleSchema.png](/public/_imported_attachments_/1/simpleSchema.png)
![simpleSchema.png_thumb](/public/_imported_attachments_/1/simpleSchema.png_thumb)[/s][/s]

awebster

Sounds like a routing problem.
Suggest you use traceroute to try to figure out where packets are getting lost.
Please also post routing tables for each piece of equipment, that will help shed some light on the issue.

Derelict

You don't configure a CARP cluster like that. You configure everything on the primary and duplicate it on the secondary, preferably by letting XMLRPC sync do the duplication work. The secondary does nothing until a failover event occurs.

Best source of info:

https://portal.pfsense.org/docs/book/highavailability/multi-wan-with-ha.html

See sig for a link to get access to the book cheap.