Directing outbound traffic

[pfsense1921]

@viragomann
That option for the source was not available. Also, when I choose Hybrid with auto it made rules to allow everything everywhere so I made it all manual. I got more to work, but still have one strange issue. I am temporarily going to post all my details. (There is nothing on these networks yet, but don't think I am sharing any security details.)

So, From the diagram linked to below, LAN_Email <--> WAN10 I can ping everywhere, ping www.google.com success. But I can not browse anywhere from any pc. (I can move that PC to LAN_office and LAN_web and everything works. )

To me this says I have some setting wrong in pfsense. The link below has many screenshots of my pfsense config. I am sure I am missing something simple. I have reset states and rebooted several times. Thanks for your help, I just can not find the issue.

http://salesleads.live/zup/network.pdf

[viragomann]

@pfsense1921
The outbound NAT rules don't allow anything, they only do S-NAT (masquerading) on traffic that is allowed by your firewall rules.

When you run it in hybrid mode, you can add custom rules, while you keep the automatic rules. Custom rules which match to the same traffic overrides the automatically generated.

From the diagram linked to below

Maybe you want to hide your WAN IPs.

LAN_Email <--> WAN10 I can ping everywhere, ping www.google.com success. But I can not browse anywhere from any pc. (I can move that PC to LAN_office and LAN_web and everything works. )

You need to allow DNS access for this devices. The rules on the interface only allow HTTP and HTTPS.

You have also to obey this when you add a policy routing rule to allow any, but your devices are configured to use pfSense or any other internal DNS. @mcury mentioned this already above.

I set up a floating rule to aim this with a single rule on all interfaces.

[pfsense1921]

@viragomann
Thank You (Plan to remove the file soon)

I will add the DNS now, the other LANs work without the DNS listed

LAN_Office <--> WAN9 works no DNS listed
LAN_Web <--> WAN11 works no DNS listed

LAN_Email <--> WAN10 Not working no DNS listed.

Added DNS
LAN_Email <--> WAN10 Not working with DNS listed.
Here was how I added DNS:

[mcury]

@pfsense1921 said in Directing outbound traffic:

@viragomann
Thank You (Plan to remove the file soon)

I will add the DNS now, the other LANs work without the DNS listed

LAN_Office <--> WAN9 works no DNS listed
LAN_Web <--> WAN11 works no DNS listed

LAN_Email <--> WAN10 Not working no DNS listed.

Added DNS
LAN_Email <--> WAN10 Not working with DNS listed.
Here was how I added DNS:

DNS/53 should be TCP/UDP, 99,99% of times it will use UDP, but sometimes it can use TCP.

[pfsense1921]

@pfsense1921
The LAN_Office and LAN_Web can ping everywhere and browse internet

The LAN_Email can ping everywhere but Not browse internet

Also disabled the offloading:

[pfsense1921]

@mcury
Thanks. Made the change, still did not work. Here is how it is now:

[mcury]

Open a prompt MS DOS in a machine from the LAN_Email network.
Type nslookup press enter, then google.com and press enter and confirm if DNS is working.

[pfsense1921]

@mcury
It looks like it works, (Does "ping www.google.com" provide a similar test of dns?)

I can move the PC from lan_email to either lan_web or lan_office and browsing works.
And vice versa, no browsing on lan_email. I feel like deleting everything and starting over which would probably be faster, but now I want to learn what the problem is.

Thanks for the help.

[pfsense1921]

@pfsense1921

Very Odd this is. I can not pin point where the problem is.

IF, I only connect one WAN to pfsense, the corresponding LAN works.
I can do this with any WAN, All three LAN work, but only one at a time.
(Tested with all 3 static IP, and moved them around)

IF I connect two WAN, the two corresponding LANs work perfect.
Again, I can do this with any combo of two WAN

Everything seems to be working properly. IF your corresponding WAN is not connected, you can Not ping outside your LAN.

Now, when I connect the third LAN everything goes to crap. I loose one LAN and sometimes two LAN.

How would I prove/find if the problem is with pfsense OR ATT. I have 5 public static IP addresses, I am only using 3. Those tests can prove that both systems are working, neither can prove which one is having the problem. Something is crapping when all 3 LAN are connected.

[viragomann]

@pfsense1921
Your basic mistake is, that you have configured multiple WAN interfaces, which are all within the same subnet.
Routing doesn’t work this way.
You have to add all WAN IPs to a single interface.

[mcury]

@pfsense1921 said in Directing outbound traffic:

It looks like it works, (Does "ping www.google.com" provide a similar test of dns?)

Yes, its the same thing..

@viragomann said in Directing outbound traffic:

Your basic mistake is, that you have configured multiple WAN interfaces, which are all within the same subnet.
Routing doesn’t work this way.
You have to add all WAN IPs to a single interface.

I would be on it, I would like to know how he was able to configure three interfaces in the same subnet, I suppose this is not possible, he is probably using a /32 mask ?

[viragomann]

@mcury
Yes, pfSense will throw overlapping networks errors, when doing this.
With a /32 musk it's possible. But I'm not sure if you can add the same gateway to multiple interface, even if one has the /29 mask.

[mcury]

@viragomann said in Directing outbound traffic:

But I'm not sure if you can add the same gateway to multiple interface, even if one has the /29 mask.

He should be using only one WAN interface, getting the entire /29 subnet and using virtual IPs..

[viragomann]

@mcury said in Directing outbound traffic:

He should be using only one WAN interface, getting the entire /29 subnet and using virtual IPs..

Yes, since he has only a single WAN gateway, that's the only option.

Routing is based on gateways, hence with the same gateway on multiple WAN interfaces any upstream packet can go out on any WAN interface and the outbound NAT rules will not work.

[pfsense1921]

@viragomann @mcury

Hello,

Based upon what was said previously, In Theory:
Would the M400 with pfsense work like I am trying with 3 different ISP Providers?

I apologize that my lack of understanding is making this a painful discussion for you. This is obviously far from a standard 1 WAN / 1LAN setup.
Thanks for trying to help me.

So, AFAIK, on my ATT service I have 4 different static Public IP addresses, I can route traffic to the public internet, independently on each port.
I am only using 3.

I am trying to do this:

Not This:

Attached below is my ATT modem.
Each one of the four yellow ports have individual public IP addresses.
Imagine as 4 different ISP, I am only trying to use 3.
I have verified that each port works like its own separate ISP.
(One test, I connected a PC directly to the port and "Whatsmyip" all said the public IP address for that port)

This last paragraph is me crying for help with the questions flying around inside my head. The best way for me to learn is to understand how to get from where I am to where you are. So I am sharing where I am

I do understand that my public IP addresses are on the same subnet, and they all use the same public gateway ( .14) - for the 4 IP addresses. (the addresses are public and not .14)
Prior Statement that I must use only one WAN because those IP addresses use the same One Gateway. Does that mean that every ISP with a public address would require its own separate gateway? And the gateway can not be shared with other ISPs? Soo, If two neighbors were using the same ISP Provider with public addresses on the same gateway, they would have to route thru the same WAN port on a firewall and not be able to use two different firewalls? Why is that different than what I am trying to do?

I apologize if you had to read that last paragraph twice. Thanks for your help!

[mcury]

@pfsense1921 Try this:
Connect just one cable from ARRIS LAN1 to pfsense WAN0
If things are as I think they are, you will receive an IP like this: X.X.X.X/29 with the gateway X.X.X.14/29 as you stated.
Do not connect other ports from ARRIS to pfsense.

Then create virtual IP, selecting IP Alias, WAN, Single address, put the IP and MASK accordingly, do not repeat the same IP that your pfsense WAN has, put the next one on the sequence.

Then change your NAT rule to hibrid, create a NAT OUTBOUND like this:
Interface WAN
Address Family: IPv4
Protocol any
Source network: GROUP B of computers: Destination: ANY
Destination any
Translation: Select the virtual IP address you created above

Then test and come here to tell us if it worked.

I suppose this video is perfect for you:
Youtube Video

Edit: One observation here is that in the video he is doing a portforward, you will do Outbound ok? So that step is different.

[viragomann]

@pfsense1921 said in Directing outbound traffic:

So, AFAIK, on my ATT service I have 4 different static Public IP addresses, I can route traffic to the public internet, independently on each port.

Agree, this should be possible.

I am trying to do this:
I.am.trying.to.do.jpg
Not This:
Not.This.png

There is logically no different at all. All hte traffic has to pass the common gateway.

Each one of the four yellow ports have individual public IP addresses.

Are you sure? Almost these boxes have a simple switch inside, so that each IP is available on any port.

I have verified that each port works like its own separate ISP.
(One test, I connected a PC directly to the port and "Whatsmyip" all said the public IP address for that port)

Set the IP manually on the PC and try it out.

If that's really the case, you need either a switch behind where you connect all ports and pfSense, or you bridge all interfaces on pfSense.

Prior Statement that I must use only one WAN because those IP addresses use the same One Gateway. Does that mean that every ISP with a public address would require its own separate gateway? And the gateway can not be shared with other ISPs? Soo, If two neighbors were using the same ISP Provider with public addresses on the same gateway, they would have to route thru the same WAN port on a firewall and not be able to use two different firewalls?

No, putting customers in a common subnet is rarely used here, but I've already used such a setup from a cable provider.
The customer have not to use a shared firewall, they have only to share the ISPs gateway in this case. But the gateway is outside of your home. There is no problem with that.

You can imagine this like your LAN devices are using your routers LAN IP as gateway. You configure all for your LAN subnet, e.g. a /24, which also the gateway is inside and all devices can communicate with the gateway independently. Only if the want to talk with another LAN device (the neighbor) they need not to pass the gateway, but can go directly.

However, most ISPs here provide PPP connections to customers with a single WAN IP instead of subneting.

[pfsense1921]

@mcury
Love Lawrence!

[pfsense1921]

@pfsense1921
I got it working the way I want.

There is a difference between the two drawings, I do not understand your statement with the gateway and how it relates to the drawings. One drawing (Not) shows one common public IP address, The other shows 3 independent public IP addresses.

to an observer on the internet:

The Not drawing: you can conclude that all 3 LAN pass thru one point managed by a PRIVATE entity.

The WANT drawing: you can NOT conclude that all 3 LAN pass thru one point managed by a private entity. Only conclusion is that the 3 public addresses use the same gateway. That Could be 3 different private entities (neighbors with same ISP provider) OR could be the same.

That is different.

Depending on how far away or closely you look at something depends if it looks the same or not. From a distance two items may look the same. The closer we get we will find differences, with anything.

Lawerence was talking about one WAN port and making Virtual IP addresses (to be used in a "similar" fashion). I am talking about Three REAL public IP addresses, not virtual IP addresses, that again is different.

If you were saying there is logically no difference in their connection to the internet, I do agree.

[mcury]

@pfsense1921 said in Directing outbound traffic:

I got it working the way I want.

Thats is nice

@pfsense1921 said in Directing outbound traffic:

Lawerence was talking about one WAN port and making Virtual IP addresses (to be used in a "similar" fashion). I am talking about Three REAL public IP addresses, not virtual IP addresses, that again is different.

He was receiving a few public IPs in his WAN, so he created the virtual IPs so he could use them.
In that situation, pfsense WAN gets only one, so he needs to create the virtual IPs to assign them by NAT.

But if its working, leave like that..