NAT issues with multi-WAN

[Briantist]

GruensFroeschli, please tell me you haven't forgotten about me!  :'(

Anyone have any ideas? Any experiences with this?

[GruensFroeschli]

No i havent forgotten about you.
In fact i'm working on it right now ;)

But i think i've run into another problem with NAT.
Trying to reproduce it right now >_<

Will probably write back later today. (if i dont go crazy)

[GruensFroeschli]

Ok i think i tried everything.
But i havent been able to reproduce your problem.

Here is what i did:

		(WAN - public IP)
		ADSL-modem/router
		(LAN - 192.168.20.1/29)
		   |
		   |
192.168.20.4/29	   |
test-client-----switch--------------------------(WAN - 192.168.20.5/29)
		   |				pfSense2
		   |				(LAN - 192.168.40.1/24)
		   |				   |
		   |				   |
	(WAN - 192.168.20.6/29)			   |
	pfSense1 (OPT1 - 192.168.40.2/24)--------switch----------test-client/server
	(LAN - 10.0.0.1/24)					192.168.40.200/24
		   |
		   |
		server
		10.0.0.10/24

I can access the server from the 192.168.20.4 test-client as expected if i connect to 192.168.20.6.
I can access the server as well if i connect to 192.168.20.5

What you described is, that if the gateway on OPT1 is set you can no longer access the server from (in my case) the 192.168.40.x/24 range.
This worked for me.

I'm not sure what the problem in your case could be >_<

[GruensFroeschli]

Schnittstelle: 192.168.40.200 –- 0x3
Physikalische Adresse . . . . . . : 00-03-25-09-91-19  <<–- test-client/server
  Internetadresse      Physikal. Adresse    Typ
  192.168.40.1          00-02-44-8f-03-ae    dynamisch    <<–- Gateway
  192.168.40.2          00-0d-b9-05-67-25    dynamisch <<–- OPT-interface
@OK:

22:54:31.781241 00:03:25:09:91:19 > 00:0d:b9:05:67:25, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 56430, offset 0, flags [DF], proto: TCP (6), length: 48) 192.168.40.200.1596 > 192.168.40.2.80: S, cksum 0x30be (correct), 1713509472:1713509472(0) win 65535

@STRANGE:

22:54:31.782478 00:0d:b9:05:67:25 > 00:02:44:8f:03:ae, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 127, id 31917, offset 0, flags [DF], proto: TCP (6), length: 48) 192.168.40.2.80 > 192.168.40.200.1596: S, cksum 0x7865 (correct), 1117681065:1117681065(0) ack 1713509473 win 65535

@OK:

22:54:31.782905 00:03:25:09:91:19 > 00:0d:b9:05:67:25, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 56431, offset 0, flags [DF], proto: TCP (6), length: 40) 192.168.40.200.1596 > 192.168.40.2.80: ., cksum 0xa461 (correct), 1:1(0) ack 1 win 65535

@OK:

22:54:31.819768 00:03:25:09:91:19 > 00:0d:b9:05:67:25, ethertype IPv4 (0x0800), length 591: (tos 0x0, ttl 128, id 56432, offset 0, flags [DF], proto: TCP (6), length: 577) 192.168.40.200.1596 > 192.168.40.2.80: P, cksum 0x6236 (correct), 1:538(537) ack 1 win 65535

@STRANGE:

22:54:31.822462 00:0d:b9:05:67:25 > 00:02:44:8f:03:ae, ethertype IPv4 (0x0800), length 299: (tos 0x0, ttl 127, id 4765, offset 0, flags [DF], proto: TCP (6), length: 285) 192.168.40.2.80 > 192.168.40.200.1596: P, cksum 0xc9bd (correct), 1:246(245) ack 538 win 64998

@OK:

22:54:31.945686 00:03:25:09:91:19 > 00:0d:b9:05:67:25, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 56435, offset 0, flags [DF], proto: TCP (6), length: 40) 192.168.40.200.1596 > 192.168.40.2.80: ., cksum 0xa248 (correct), 538:538(0) ack 246 win 65290

Obviously i can reproduce that traffic is being sent to the wrong MAC.

But i'm wondering why it's working here…..

[GruensFroeschli]

Ok i went at it with wireshark again. (see attachment)
The downloaded content is the page on http://psymia.mine.nu
Everything seems to be in order…. (see frame 6)

But now i'm wondering why the capture from pfSense itself differs from the capture with wireshark.

??? ??? ???

test_capture.pcap.txt

[Briantist]

GruensFroeschli, thanks so much for putting time into this. I'm sorry I haven't been able to do anything with it; I had to move suddenly, and things have been really crazy. I have had no time whatsoever to devote to this, and it might be a while before I can try again. I will resume working on this problem though, and I'll let you know how it turns out. Thanks again!

[eri--]

Can any of you describe what the issue is, if any, so i can give a looka t it?

[Briantist]

ermal, I'm confused. What information do you need that is not in the thread? I think we've been really descriptive.