Internet sharing to multiple subnets (with NAT)

rexis

Greetings.

Based on this document I decided to put this topic here:
http://doc.pfsense.org/index.php/Network_Address_Translation

As titled, I have one internet connection and would like to share it to multiple subnets. With the following details:

some subnets are located at different site connected via lease lines, therefore it is not possible to have an interface at every subnets.
transparent internet connection is required as some are wifi users, hence setting proxy in web browsers is not an option.

Here is my set up:

( internet ) –--| modem |-----to wan port-----| PfSense Main LAN=10.0.0.1 |----> ... (local network)

The PfSense 1, 2 and 3 will have captive portal running and provide connections to users from their subnet.
(btw, if my poor artwork failed to explain what I am trying to do, please notify me!)

I imagined that from PfSense Main I would have a NAT 1:1 rule to direct the internet traffic to the respective downstream PfSense, and perhaps PfSense 123 will have some sort of NAT rules to direct internet traffic back to Pfsense Main. But I am still struggling on what is the actual rules to be added, anyone can shed me some light on this?

Millions of thank you in advanced. And have a nice day.

Rexis

podilarius

What is your subnet mask at each location? How are they getting internet now? If you are using pfSense with captive portal, then those are doing NAT for you into you main network so you just have to set the gateway of the captive portal to the main pfSense FW. This creates a double NAT which could possibly cause problems. You could setup proxy arp on the LAN interface of the main FW and put that in the different subnets then create a main captive portal and necessary rules (Advanced Outbound NAT and Access rules) to handle the traffic from those locations. If you are trying to keep it all separate, then you should be good with what you are doing.

Where is the breakdown in communication? pfSense is a statefull firewall, so if a user connects through a different subnet a state is opened from your first level pfSense and another state is created on the Main. Are your captive portals routing only or are they also have firewalling enabled?

rexis

Hello,

The subnet mask is 255.255.255.0 for all network.

Currently I have a main line and a commercial firewall is doing 1:1 NAT (public IP to local IP) to the WAN address of every PfSense servers in the network.

Noted, am learning about Proxy ARP at the moment….

What is your subnet mask at each location?

How are they getting internet now?

If you are using pfSense with captive portal, then those are doing NAT for you into you main network so you just have to set the gateway of the captive portal to the main pfSense FW. This creates a double NAT which could possibly cause problems. You could setup proxy arp on the LAN interface of the main FW and put that in the different subnets then create a main captive portal and necessary rules (Advanced Outbound NAT and Access rules) to handle the traffic from those locations. If you are trying to keep it all separate, then you should be good with what you are doing.

Where is the breakdown in communication? pfSense is a statefull firewall, so if a user connects through a different subnet a state is opened from your first level pfSense and another state is created on the Main. Are your captive portals routing only or are they also have firewalling enabled?

rexis

some update,

It seem like PARP is not applicable to my situation too, if I understand correctly(pardon me for sleeping thru networking classes!), my problem is actually that my network default gateway is not routing my internet traffic to the direction I wanted. As following scenario:

Subnet A (proxy server sitting here with direct internet connection)
Subnet B (Internet user network which utilized the proxy server by setting them in browser)
Subnet C (Wifi user which required transparent internet access, PfSense box sitting here)*
(All subnets can ping each other, but only A has direct internet access)
(I do not have access to the core switch)

My problem is at the Subnet C where the users are expecting transparent internet connection, as setting upstream proxy in PfSense Squid Proxy(transparent mode) will only take care of http traffic(port 80), and render other traffic such as https (port 443) unavailable.

I can't just make the proxy server from Subnet A as the gateway of Subnet C, as it is logically impossible.

Currently my work around is NAT a public IP from the main fw in Subnet A to Subnet C, but that would consume one of my public IP address pool.

The problem arises when we need to expand to several more Wifi subnets (say, Subnet D, E, and F), and I am out of public IP.

….................................

My solution(after days of battled research) is utilizing OpenVPN of PfSense. Based on all the guides in PfSense Doc/forum, OpenVPN is being used to connect different sites via the internet cloud. I imagined that LAN is also a kind of cloud, I should be able to OVPN through the LAN cloud too. The main feature that OVPN catches my attention is that it is able to force all traffic through the OVPN tunnel(redirect-gateway def1).

So my expected setting (which proven workable in a test environment) is as following:

Subnet A: PfSense OVPN server
Wifi Subnet C, D, E, F: 1 PfSense OVPN clients in each subnet

(All PfSense OVPN clients will create a tunnel to PfSense OVPN server, internet traffic will route through the tunnel which eventually reaches OVPN server located in Subnet A.)

So far, I have not seen any similar configuration as described above, so this probe me another question, is this an efficient set up!? Am I solving an easy problem with a complicated method? Any comments on this?

Thank you podilarius for your prompt reply and hints, without it I would never figured out what is Advanced Outbound NAT nor it has anything to do with OpenVPN!

(ps: perhaps this thread is not about NAT afterall)

joako

It sounds as if you have everything under control.

My problem is at the Subnet C where the users are expecting transparent internet connection, as setting upstream proxy in PfSense Squid Proxy(transparent mode) will only take care of http traffic(port 80), and render other traffic such as https (port 443) unavailable.

You can not transparent proxy HTTPS. Everything is working as it should.

rexis

@joako:

It sounds as if you have everything under control.

You can not transparent proxy HTTPS. Everything is working as it should.

Yes, things are working better than I expected. The next thing is to do some stress test and see how much load the main server can take.

Noted that, but users only worry about not getting their gmail checked : /

podilarius

Rexis, you are going to find that using OpenVPN works, but you are going to have a internet speed issue once you get to about about 30Mbits/s. VPN in general are slower because of encryption/decryption that has to be done.

I think that this can be done using routing and rules. You should be able to set the default gateway to your main pfSense or Firewall. For your firewalls behind the main, you are going to have an IP in both the local LAN and the remote LAN. You are just going to have to be creative with your rules to make sure you are allowing the right kind of traffic.

Perhaps I don't understand the network setup. Can you put together a visio (or similiar) diagram and post it in here? Or perhaps a better description of the network setup. I think I have had something similar setup before.

rexis

podilarius,

The extra overhead is a concern, anyway, most likely each VPN tunnel will be shaped to say 1-2 Mbps(its just free Wifi anyway) so it will be way below 30Mbits/s, the internet line(that's gonna be added and shared) I am talking about here is just 10 Mbits/s, so obviously we would like to keep it way below 30Mbits/s.

That might not because of you don't understand, but it is me who not quite understood my company's network! Am not the guy who set it up… Basically, based on my limited understanding my current network is something like:

=========================start=============================

Subnet X: Public IP Subnet 5x.x.x.x (Internet)
    |
    | 1\. Proxy Server A
    | 2\. Some commercial firewall F
    |
Subnet 1: Server VLAN 10.0.1.0/24, Default Gw 10.0.1.254
Subnet 2: Staff Dept a VLAN 10.0.2.0/24, Default Gw 10.0.2.254
Subnet 3: Wifi VLAN 10.0.3.0/24, Default Gw 10.0.3.254, PfSense box J sitting here

|| (Metro-e connection)

Subnet 4: Branch Staff VLAN 10.0.4.0/24, Default Gw 10.0.4.254
Subnet 5: Wifi VLAN 10.0.5.0/24, Default Gw 10.0.5.254, PfSense box K sitting here

PfSense J
WAN IP: 10.0.3.1/24, Gw 10.0.3.254; LAN/WIFI IP: 10.0.99.1/24 connected to wifi routers

PfSense K
WAN IP: 10.0.5.1/24, Gw 10.0.5.254; LAN/WIFI IP: 10.0.100.1/24 connected to wifi routers

Commercial Firewall F
NAT: 5x.x.x.1 to 10.0.3.1
NAT: 5x.x.x.2 to 10.0.5.1

=========================continued=============================

Note:

Proxy Server A have two NIC, one at Subnet X, another one at Subnet 1, it have a default gateway on Subnet X(and out to internet), and a routing rule to route all 10.0.0.0/16 traffic via 10.0.1.254.
All subnets are unable to ping each other except Subnet 1(aka all subnets only able to ping Subnet 1), we get out of the internet by adding Proxy Server A in browser.

Added:

We are thinking of installing a 10 Mbps broadband with dynamic ip, and share the connections to all the PfSense boxes. The reason of doing so is to free up some public IP and bandwidth from the original line as we just downgraded it for cost saving.
It is not possible to have a physical NIC in both Subnet 4 and Subnet 1, different geographical location, or it is some fw tricks?

==========================end============================

Hope I didn't complicate the scenario! Did I described it better? O.o

Happy seasonal greetings.

podilarius

Okay, so all the *.254 are in the commercial firewall?
The pfSense boxes are only for the wireless?

If it is a true metro-e … it is just one big switch (or patch cable) ... you could even have it in VLAN1 if you like ... I like to keep locations separate, but that is just me. Intresting thing is that if you are trying to get from subnet 5 to subnet 4, you have to go out then in the metro-e (if I gather your network setup correctly). Perhaps you could clarify a bit more with what devices hold what addresses.
Think of this an exercise in getting to now your network. Like this perhaps:

internet
|
|

|VLANX:50.x.x.1 | VLANX:50.x.x.2
ProxyA Comercial FW
|VLAN1:10.0.1.1/24 | VLAN1:10.0.1.254/24 (Default GW)

|VLAN1:10.0.1.253/24
Core Router
|VLAN2:10.0.2.254/24
|VLAN3:10.0.3.254/24
|VLAN4:10.0.4.254/24

VLAN5:10.0.5.254/24
\ VLAN3:10.0.3.1/24 (GW 10.0.3.254)
Cisco Router pfsense
<-or->
VLAN4 ======== Wifi Net
VLAN5
Metro-E

/ VLAN4 \ VLAN5
Cisco Router pfsense WiFi
| |VLAN100
Network Wireless Network
.
.
.

And so on.

then you could provide details per device on what network IPs and subnets there are. You might not have a core router and the VLANs are in the commercial FW. Just have to adjust based on your setup. Visio is your friend here. Change the values to what is correct for each node in your system. Before we can make recommendations or you make some changes, you are going have to know how traffic flows in your network setup.

Internet sharing to multiple subnets (with NAT)

Subnet A: PfSense OVPN server Wifi Subnet C, D, E, F: 1 PfSense OVPN clients in each subnet

(All PfSense OVPN clients will create a tunnel to PfSense OVPN server, internet traffic will route through the tunnel which eventually reaches OVPN server located in Subnet A.)

internet | |

|VLANX:50.x.x.1 | VLANX:50.x.x.2 ProxyA Comercial FW |VLAN1:10.0.1.1/24 | VLAN1:10.0.1.254/24 (Default GW)

Subnet A: PfSense OVPN server
Wifi Subnet C, D, E, F: 1 PfSense OVPN clients in each subnet

internet
|
|

|VLANX:50.x.x.1 | VLANX:50.x.x.2
ProxyA Comercial FW
|VLAN1:10.0.1.1/24 | VLAN1:10.0.1.254/24 (Default GW)