MUTLI WAN & NAT ??? [SOLVED - Reply#10]
-
Ok will get you some screenshots ,np.
Ok so I have just tried
(WAN)
SERVER 1 (Port80) –-------------> --------------> WAN1
SWITCH -----------> PFSENSE
SERVER 2 (Port22) ---------------> --------------> WAN2
(OPT1)what I tried in my first post was
(WAN)
SERVER 1 (Port80) ---------------> --------------> WAN1
SWITCH -----------> PFSENSE
SERVER 1 (Port80) ---------------> --------------> WAN2
(OPT1)ALSO
(WAN)
SERVER 1 (Port80) ---------------> --------------> WAN1
SWITCH -----------> PFSENSE
SERVER 1 (Port22) ---------------> --------------> WAN2
(OPT1)What I have noticed , I think it's a problem wit NAT on the OPT1 interface... It doesn't matter if I use the same LAN IP or a differnet one, the port wont open on the OPT1 interface...and yes I have the services running ;)
-
right now I have my Firewall: NAT: Outbound to:
Automatic outbound NAT rule generation
(IPsec passthrough included)
and there are 4 DNS servers there, I just removed 3 of them for the screenshot.
-
So I have gone out and purchased another NIC so now I have tried
LAN2 .167 (port22) .167
–-------------> --------> ------------->WAN
SERVER 1 SWITCH PFSENSE
---------------> --------> ------------->WAN2 (OPT1)
LAN1 .168 (port80) .168So I have done that above , 2 complete separate NIC cards for 2 different LAN segments and have even set the gateways separate LAN1 gets WAN GW , LAN2 gets WAN2 GW
The port still wont open?!?
here is my firewall logs when I try to check the port from say http://www.yougetsignal.com/tools/open-ports/
-
Ok so I have done the exact same setup as above . I am not sure why I didn't come to be before.. I clicked the Act of the blocked rule of port 22, there was a 1 scrub on all fragment reassemble block and a block all.. I used the easy rule pass and i disabled scrub and I am not getting firewall block logs for port 22 now when I check to see if it is open.. ( the port still shows closed when I check it ) when I check say port 9191 that is open on WAN1 and not WAN2 , I get the block rule and thats perfect, if i check the port against WAN1 , its open and all is fine.
On my default route page how can I change the default route gateway to the MultiWAN group I have made, Right now it it set for the WAN gateway only…even thou LAN traffic is LB via multiWAN gw rule on LAN... I'm a little confused now..
-
OK! So I have " Fixed " it.. The issue was just as I figured in my last post.. on the Diagnostics: Routing tables page, The Default route was the " WAN PORT" IP.. I changed my default gateway to my OPT1 Interface and now the default route ip shows the OPT1 IP. Now I am able to reach any port I open from both WAN and OPT1 Ip's …. This is clearly a bug in PFSense, There should be a way to change the default route to a gateway group..this would solve the problem I think.. :) Well my issue is solved correct workaround or not... it's a fix for now..
-
When i change the default gateway to OPT1, then everything i opend on WAN1 dont work anymore…
-
Hey Marv, Here is my full config. I even have ICMP setup on this just so you can check from an external source if ur even able to ping the ip. It's all working very well for me now. nslookup on the domain resolves both ip's and load balancing is working great.
a link to the image for full size
http://m37offroading.ca/fullconfig.png -
Thanks for ur Time.
But i doesnt work for me ;/
I have no floating rules…The Rule is linked?? Yours not, what is this?
My Pfsense Version is 2.0.2-Release i386
I just want to have access to 8059 from the outside, over the FritzboxWAN ;(
Without Failover and only Fritzbox connectet it works fine ;( -
If i Set the Fritzbox Wan to default it works, but then every Portforwarding on the Unitymedia WAN is unable to reach :/
-
If I do this it works, with default on WAN…(pic)
I cant forward ports on WAN, i can do verthing, it never works...
Iam freaking out :D
-
Well I see a couple things wrong,
your Gateway group, Set both to Tier 1,
For the Firewall: NAT: Port Forward Page, On the configuration of the port 8059 go the the Filter rule association and make that None, Also NAT Refection , Enabled ( if you want to be able to access it from internal also)
Firewall: Rules, See mine above, See how I have rules in ever NIC and a Floating DNS failover rule, Include all these rules.
Leave your gateways leave as no default as it is in the screenshot.
This should work, I have successfully re-installed 2.0.1 -> upgraded to 2.0.2 and did my rules again and it all works perfect with no trouble. If you need more help PM me your email and we can get you up and working.
The Fix is above thou , that works and tested twice on 2 installs. :)
-
okay, thank you.
I will try this tommorow.
Ist just Failover not LoadBalancing, so Tier 1 and 2 is correct , or? -
No that is not correct, Please marv, follow my screenshots very carefully and match everything the same as your config. It will work ! :)
-
Ty for your Time, but I have to say that you are wrong ;/
Both on Tier1 is LoadBalancing, i got then two IP, when i checked my Ip in the internet.
I installed Pfsene again with CD now (therefore i did this over memstick.img).
Now it works, why ever oO ::)
The only things i did after the newinstall was. Setting the Gateways on Tier 1 &2, named it Failover, Add a (one) Nat-Rule- with assiocated Firewallrule. And add in the "lan to any" rule, the Failover Gateway.Works.
But thank you for help so far
-
OK! So I have " Fixed " it.. The issue was just as I figured in my last post.. on the Diagnostics: Routing tables page, The Default route was the " WAN PORT" IP.. I changed my default gateway to my OPT1 Interface and now the default route ip shows the OPT1 IP. Now I am able to reach any port I open from both WAN and OPT1 Ip's …. This is clearly a bug in PFSense, There should be a way to change the default route to a gateway group..this would solve the problem I think.. :) Well my issue is solved correct workaround or not... it's a fix for now..
Ok I figured out, what is different to my others installations, i swapped the WAN Interface in this Installation.
If the 2WAN is the one with the higher priority in the Gateway Group all is ok, is the first WAN-Interface with higher Prioty nothing works anymore.
Thats truly a bug. -
Marv! I am glad you figured it out… This is very much a bug and I have noticed the same thing... I was not 100% Sure that was what was fixing it the first time I got it working.. Doing that the second time It did not work..then I got my config that I have posted in reply 10 and it works great, with a re-install.. I just tried it AGAIN after you said this because I know for a FACT I had to do the same exact thing.
-
I don't have enough experience with multi-WAN to be able to just spot the issue, but I do know of one potential issue I had seen in the rules someone mentioned. As far as I know, "Pass" on port forwards bypasses the firewall rules, likely including the ones specifying which gateway to use. Firewall rules should be used instead of directly using "Pass" on port forwards if you have any special firewall rules with gateway settings. The same may apply to limiters or traffic shaping. Depending on how your rules are set up, you may need to duplicate some of the settings to the linked rules for your port forwards (I'm not certain of this, though).
-
Indeed, using "pass" on rdr will not give them the proper reply-to behavior to work with multi-wan.
Use a linked rule instead.
And if your WAN is a static IP type, make sure you have the gateway selected on the interface config so that the firewall will treat the interface as a WAN (this happens automatically for DHCP/PPPoE/etc) and put reply-to on the firewall rules.
-
Indeed, using "pass" on rdr will not give them the proper reply-to behavior to work with multi-wan.
Use a linked rule instead.
And if your WAN is a static IP type, make sure you have the gateway selected on the interface config so that the firewall will treat the interface as a WAN (this happens automatically for DHCP/PPPoE/etc) and put reply-to on the firewall rules.
Yeah, did that :)
Fact is, that i got problems with the WAN1 as Default.
Lets say WAN1 is UM(the internetprovider)
Lets say WAN2(opt1) is 1und1.
If UM is default on WAN1 it doesnt work.
If i swap the assignement and UM is on WAN2 everything works well, with exact the same rules. -
That still suggests your rules do not get reply-to for some reason. As above, the usual suspects are:
- Using "pass" on port forwards (rdr pass does not support reply-to, so that can't ever work, not something we can change)
- Not selecting a gateway on the Interface page for both WAN and WAN2 if using a static IP (this will make the firewall fail to fully treat the interface as a WAN, and skip reply-to)
- Checking the box in the advanced options to disable reply-to (Which, as the option implies, removes reply-to from rules)
Post the full contents of config.xml (you can remove keys/passwords) and a copy of /tmp/rules.debug.
If you choose to obscure IP addresses, leave enough to distinguish them (e.g. last octet), and do not remove them entirely.
-
That still suggests your rules do not get reply-to for some reason. As above, the usual suspects are:
- Using "pass" on port forwards (rdr pass does not support reply-to, so that can't ever work, not something we can change)
i used pass and no RDR ( The checkbox is unchecked) - Not selecting a gateway on the Interface page for both WAN and WAN2 if using a static IP (this will make the firewall fail to fully treat the interface as a WAN, and skip reply-to)
I have set a gateway for the static and none for the dhcp (will choose it on his own) - Checking the box in the advanced options to disable reply-to (Which, as the option implies, removes reply-to from rules)
No, xD. With Wan2 as default it works…
Post the full contents of config.xml (you can remove keys/passwords) and a copy of /tmp/rules.debug.
If you choose to obscure IP addresses, leave enough to distinguish them (e.g. last octet), and do not remove them entirely.
I will it post later, not at home right at the moment
- Using "pass" on port forwards (rdr pass does not support reply-to, so that can't ever work, not something we can change)
-
If you used "pass" on the port forward, it will not work with any non-default route WAN, and it can't work with "pass".
Change that to "add associated firewall rule" and then save/apply the port forward. Do that for every port forward you have that uses 'pass'
-
If you used "pass" on the port forward, it will not work with any non-default route WAN, and it can't work with "pass".
Change that to "add associated firewall rule" and then save/apply the port forward. Do that for every port forward you have that uses 'pass'
Sry I was not accurate enough. I used pass in the Firewall rule, not block and not reject.
BTW: add associated firewall rule is the default setting, when you create a NAT rule.
-
I'll still need to see the config.xml and /tmp/rules.debug. A screenshot of that doesn't tell me enough about what it's doing in the background.
-
Hey Jim, Ok I fully understand and now agree with you.; I just got in from a 14 hour shift and have been dealing with recovery of failed modules and stacks we had go down lastnight. I am very tired and do not think I can put my full 100% into this right at this moment. I will grab a copy of my current config, but just to note this is a working copy. I would like to get a 'not' working copy to you also. I would love to do the comparison by my self also, The more I think about it without actually doing it is just racking my brain and have also thought about what you said and you are correct, it very well could be a PFSense configuration issue. From a networking perspective I do know that default route could of been causing the issue. But enough guessing , I will upload a working config today to get you started and I cant promise tomorrow night when I get in from work since I'm sure I will still be swamped at work. But when I get up Monday, I will completely re-install from 2.0.1 -> upgrade to 2.0.2 and cause my same issue.
and on a note of multiple users, I have people that I know in person and I have read 1 more post on another site. It was no help, but I will try and find that again for you.
-
Here is my raw config.xml & /tmp/rules.debug
http://m37offroading.ca/PFSENSE/config.xml
http://m37offroading.ca/PFSENSE/rules.debug
I have also uploaded them on the redmine post. I will get you copies of the failed config.xml and rules.debug by Monday.
-
FYI- You only have the port forward in there once. It should be in there twice (once for each WAN). The WAN rules look OK though.
rdr on nfe0 proto tcp from any to 24.212.178.88 port $Cloud -> $Server pass in log quick on $THOMSONWAN reply-to ( re0 198.84.211.65 ) proto tcp from any to $Server port $Cloud flags S/SA keep state label "USER_RULE: Allow Server to Cloud on ThomsonWAN" pass in log quick on $MOTOROLAWAN reply-to ( nfe0 24.212.178.65 ) proto tcp from any to $Server port $Cloud flags S/SA keep state label "USER_RULE: Allow Server to Cloud on MotorolaWAN"
-
I saw port forwards for each. However, was it your intention to have your port forwards use two IP addresses for the redirect target IP? (10.10.10.20 and 10.10.10.30) I think using an alias with two (or more) addresses in it like that will make the forwarding cycle through the addresses in the alias on each connection attempt, using a different one each time.
-
? How could it only be in there once if NAT is working right now on both IP's, and yes. That example that server runs quad NIC's , the 2 there are used for public traffic.
-
So here we go.
This is a RAR with a Working Config and a RAR with a non working config.I send the password over PM to relevant People, because i dont know that i delete all passwords (or change them in the file to FFF/111)
http://www.marv21.de/RouterConfig.rar
-
? How could it only be in there once if NAT is working right now on both IP's, and yes. That example that server runs quad NIC's , the 2 there are used for public traffic.
Hmm I must have overlooked it, but now the rules.debug file is 404 so I can't check it again. Though Efonne is right about using multiple IPs in the target of a port forward. It will do round-robin connection alternating, it can't forward the same port to both systems and work for both at the same time.
And if you really want to do load balancing, use the load balancer, not a port forward, since a port forward with two entries would not be smart enough to remove a down host from the pool when it's not in use so half the connections would just fail. -
So here we go.
This is a RAR with a Working Config and a RAR with a non working config.I send the password over PM to relevant People, because i dont know that i delete all passwords (or change them in the file to FFF/111)
http://www.marv21.de/RouterConfig.rar
In your "don't work" folder the config.xml has two GW_WAN entries, but otherwise I don't see anything there that looks obviously out of place upon first glance. The rules have reply-to, so the default gateway shouldn't matter for items hitting the port forward. Though one of your WANs is getting double NAT, so that doesn't help, but it shouldn't interfere. USB nics (your ue0) are crappy in general but shouldn't affect it either.
I see you completely reconfigured the two NICs between those two configs, swapping them between "wan" and "opt1". So as it currently is, does the port forward still only work on one WAN? Or does it work on both now?
-
With the "work not" config only the default Gateway works ( in this config the UM WAN ( the dynamic)).
AT all iam fine, i swaped the assignment of the WAN1 and WAN2 (em0 and ue0) and everything is ok.
Maybe i forgot to delete a Wan entrie (if i swaped the assignment there a two dynamic Gateways for UM), but i correct that, but i doenst help.
-
Hey, Sorry not sure why it was deleted. It's back on same version. Ok I understand to use load balance if I was going to load balance against 2 web servers to a single nat ip.. but this is not what I am doing or my problem at that.
my problem is that the multi wan wont nat , even if I have 1 ip of the servers , so forget " dual server ips ". I still have issues with it allowing nat out on the port. this should work…correct ?
-
Hey, Sorry not sure why it was deleted. It's back on same version. Ok I understand to use load balance if I was going to load balance against 2 web servers to a single nat ip.. but this is not what I am doing or my problem at that.
my problem is that the multi wan wont nat , even if I have 1 ip of the servers , so forget " dual server ips ". I still have issues with it allowing nat out on the port. this should work…correct ?
BAD Question, but… Are u trieing it from the outside of your LAN?
-
of course ;)