Dedicated Business Fiber Internet and Netgate

ltechnology

Dedicated business internet access is a fairly popular product these days. Most vendors use an underlying carrier, mostly ATT for the fiber loops, and the vendor provides service on both ends for access. ATT typically terminates their loop with a NID, providing an ethernet interface for CPE connections - although I have seen instances where a fiber bulkhead was installed. The end provider normally offers and recommends they supply a router to provide the interface from the fiber loop to an ethernet connection with the customer IP(s). These routers are typically Cisco or comparable routers, and take the ATT network assigned address and routes them to a single or multiple IPs assigned for customer use on the output ethernet on the router. The customer then connects their firewall, router and/or network equipment to this connection.

I was surprised to find that there is little or not discussion or documentation to integrate this process within Pfsense/Netgate, especially with a higher end configuration that has some horsepower.

So using a post from a prior post on this forum, the vendor provides the following information for configuration on the intermediate router:
IP Address Block
IP Address: 12.xxx.xxx.128/29

WAN Information
CR Serial IP Address: 32.xxx.xx.226/30
AR Serial IP Address: 32.xxx.xx.225/30
WAN IP Address: 32.xxx.xx.224/30

LAN Information
LAN IP Gateway: 12.xxx.xx.129
First Network Assignable: 12.xxx.xx.130
Usable IP Range: 12.xxx.xx.130 - 12.xxx.xx.134
Broadcast IP Address: 12.xxx.xx.135
Subnet Mask: 255.255.255.248

The intention would then be with this intermediate router in place with the above routing, the pfsense/Netgate unit would take 12.xx.xx.190 as the WAN and then provide a LAN interface, something like 192.168.10.1 with DHCP, etc.

My question is why can't you take that segment services by the intermediate router and incorporate it in the Netgate with the hope of avoiding a double NAT situation, or now that the OS supports multiple virtual instances - create two instances with one to provide the intermediate function and the 2nd to provide the basic firewall configuration. I have seen information on the virtual abilities but have yet to explore them.

Can anyone provide some thoughts, guidelines or suggestions?

Thanks
R

bp81

@ltechnology said in Dedicated Business Fiber Internet and Netgate:

My question is why can't you take that segment services by the intermediate router and incorporate it in the Netgate with the hope of avoiding a double NAT situation...

Just looking at the configuration, I don't see how you could get into a double NAT situation, even if you tried to. We have a very similar configuration (AT&T MIS fiber with intermediate Cisco router terminating the fiber and giving us a cat6 handoff to a pfSense router, exactly as you describe) at our HQ and we don't have a double NAT running.

The set of usable IP addresses that AT&T gives you, they refer to as "LAN" addresses (your listed usable range here is 12.xxx.xx.130 - 12.xxx.xx.134. They gave us something very similar). Just because AT&T calls them "LAN" addresses doesn't mean they're running a NAT there. They aren't. I think they call it LAN because it's the network segment of addresses your company gets to use and they are "private" to your company in the sense that only your company gets to use them.

Effectively, what AT&T calls "LAN" is really a WAN address block from the perspective of your pfSense router.

Simply statically assign the IP address you want to use for the WAN interface on your router from the one of the addresses in the "LAN" block AT&T gave you, and set your gateway correctly and you should be good to go. No double NAT involved.

ltechnology

Thanks so much for the comment and insight.

So - I would assign 12.xx.xx.130 for the Wan port, and use the 32.xxx.xxx.xxx port for the remote gateway?

Not clear how the translation from the Serial IP to 'Lan' IP takes place if I don't do it on the Netgate.

Sorry - it should be pretty clear but every time I think it through and play on the test unit, I end up down the rabbit hole.

Thanks.

bp81

@ltechnology said in Dedicated Business Fiber Internet and Netgate:

Thanks so much for the comment and insight.

So - I would assign 12.xx.xx.130 for the Wan port, and use the 32.xxx.xxx.xxx port for the remote gateway?

Not clear how the translation from the Serial IP to 'Lan' IP takes place if I don't do it on the Netgate.

Sorry - it should be pretty clear but every time I think it through and play on the test unit, I end up down the rabbit hole.

Thanks.

I tried to find one of our old AT&T order confirmations to compare to, but couldn't find one, so I am going on my recollection here. Grain of salt, but I think I'm right.

No, I don't believe you would use 32.xxx.xx.224/30 as your gateway. The "WAN Information" AT&T provides, if I remember correctly, will be applied to the router they'll put on your premise. We have such a setup and we never configured anything on that router other than bolting into our rack, making connections, and powering it on.

In other words, the "WAN Information" is, for your purposes, a red herring. You probably don't even need to know any of it. They gave us that kind of information as well and we had no need for it. It might matter if you wanted to have your own router in place of the AT&T provided router that accepts the fiber optic connection and hands off as cat6 to you, but we never had a need for anything like that here. Personally I was fine with AT&T managing that device for me, as I feel I have enough to manage already.

Given the info that AT&T has provided, it appears they've given you /29 CIDR block of addresses, so your CIDR block is 12.xxx.xx.128/29 and contains addresses ending in .128 through .135. The first and last addresses in the block (.128 and .135) aren't available for use because they fulfill special functions. .129 won't be available because that is going to be your gateway IP address (as a side note: almost always, the gateway IP address of a wan connection will be inside the same subnet / CIDR block as the router's WAN IP address. If you see a configuration where that's not the case, it's a red flag and is a misconfiguration 99% of the time, barring a few special edge cases).

That leaves you .130 through .134, as your information from AT&T specifies in your Usable IP Range information.

With that information in hand, here is the configuration I would attempt on pfSense:

First, create your gateway in System -> Routing -> Gateways.

Click Add to create a new gateway.
Select your WAN interface in the "Interface" dropdown.
Address Family: IPv4
Name: add a reasonable name. ATT or whatever you like.
Gateway: 12.xxx.xx.129 (this is the LAN IP Gateway Address in the info AT&T provided)
Setup Gateway monitoring if you want to use it, skip if you don't.
Enter a Description if you want. Optional field I believe, and has no effect on function.
Save and Apply Changes.

Now configure your WAN interface:

Check the box to enable the inferface
Enter a Description for the interface. Probably WAN, but whatever you like is fine.
IPv4 Configuration Type: Static IPv4
IPv6 Configuration Type: None
Under Static IPv4 Configuration section, set IPv4 Address to: 12.xxx.xx.130/29
IPv4 Upstream Gateway: Select the Gateway you just created (12.xxx.xx.129)
Recommended that you checkmark the boxes to "block private networks and loopback addresses" and "block bogon networks."
Save and apply changes.

If this is as I remembered it, your internet connection should be live now.

If you are very interested in how I think the WAN IP stuff works with AT&T's equipment, read on:

First, let me describe how this stuff is usually physically wired. It will typically look like this:

pfSense WAN ---> AT&T Cisco Ethernet Port
AT&T Cisco Fiber Optic Port ---> Fiber cable to demark box's "inward" facing interface
Fiber Demark Box's "outward" facing interface ---> whatever is out there on the utility pole

AT&T Cisco Ethernet Port is your gateway and is 12.xxx.xx.129/29
I think, maybe, the WAN IP Address is the address of the AT&T Cisco Router's Fiber Optic Port. I think that the AR Serial IP Address is the IP address of the inward facing fiber interface on the demark box that is connected to the Cisco Router's Fiber Optic Port.

I think the CR Serial IP Address is the IP address of the outward facing fiber port on the demark box that is connected to whatever is outside the building on the utility pole.

Both the Cisco router and the fiber demark box will likely have their own routing tables to correctly direct traffic, just as your pfSense router does. That's how you don't have a double NAT here, just ordinary routing tables.

ltechnology

Thanks for the added information.
I think what I did not communicate properly is that I don't want to put a 2nd piece of equipment (or use their provided router) as I have in the past. I have traditionally opted to purchase a Cisco router, configure and put this in place as the intermediate piece instead of giving ATT any extra money.

What I am hoping to do is integrate the functionality and routing of this Cisco piece into the front end of the Negate, eliminating the hardware.

I think that one of two options might work:

Set up the WAN port with the serial IP, and set the LAN IP (from ATT) to a virtual IP. Route the virtual IP to the LAN on configured on the Netgate.
Utilize the virtual layers now available, set up 1 virtual to handle the ATT portion, and then have it hand off to the 2nd virtual which would be my normal Netgate firewall./router configuration, using the 1st virtual as my WAN interface.

I know they are using a Fortinet Fortigate router for the provided equipment now and doing this all in that unit - so it should be workable on the Netgate as well. Strangely noone has done this - or has taken the time to post and share their efforts.

Thanks