Need help routing real IPs!



  • I've been fighting with this for a while and I think PfSense is the solution, but I need some assistance.
    So, the story is I have an actiontec m1000 dsl modem and a /29 subnet of real routable IPs from my provider. Due to a firmware flaw in the m1000 I have to put it in transparent bridged mode and use a router behind it in order for everything to work. Basically the M1000 doesn't completely disable its firewall when in  unnumbered mode.

    So, here's my question and the part I need help with.
    I'll be using 2 wan and 2 lan connections. The DSL and one of teh lan connections is completely isolated from the other wan and 2nd lan. Basically I have a couple servers and a dsl line, then my home internet and our personal computers.

    I've run this setup before and it works, but I've never done this routing public IP addresses.

    I placed the modem in transparent bridge, got the pfsense a gateway address 67.x.x.94/29, and now I need to figure out how to make systems on the lan talk to the pfsense box. I gave my laptop an ip of 67.x.x.90/29 and a gateway address of the pfsense box, but that didn't work.

    Next I bridged the lan and wan connections, giving the lan connection on the pf box 67.x.x.93/29
    Laptop still can't talk to the pfsense box, either with the 93, or the 94 IP as gateway.
    PF box can ping the world though, that works. Just need to get the lan to talk to the pfsense box, and the pfsense box to pass things along to the world.

    I think if I disabled nat on the box things would work, but since I also need this box to be a router/firewall for my home network, I can't disable nat cause that is a system wide setting.

    I hope I've made the situation clear. I've googled all about how to route public IPs and even bought the O'Reilly TCP/IP Network Administration book, but dudes, I'm stumped.

    I would gladly answer any additional questions you might need to help solve my problem.
    Any help would be greatly appreciated.

    Thanks in advance!



  • The coffee shot ain't working yet so a diagram would be nice :)

    But if I'm correct in that you have 1 dsl line, I would use virtual ip with port forward or 1:1 nat



  • Maybe you need to clarify first:
    Does this /29 subnet get routed by your ISP to the public IP you have on your WAN?

    Since you have a public /29 subnet there are multiple approaches:

    1: You bridge the OPT1, on which your public IP's are used, to it's WAN.

    2: You create virtual IP's on your WAN, use private addresses in your OPT1 and just forward the traffic you need from the VIP's to your private IP's.

    3: You route your public IP's. though with only a /29 you would waste one of your 6 IP's and you'd be left with only 5.
    (This only works of you have another public IP on WAN that's not within this /29

    The first approach is maybe the best if you want the public IP's directly on you servers, and your subnet does NOT get routed to a public address on your WAN. Downside is you use one of your public IP's up for the WAN.
    (You still can create NAT-forwardings from the WAN-address to computers in your personal LAN but that's probably not what you want.)

    The second approach lets you use all the IP's out of your /29 subnet –> you can forward ports from all 6 IP's.
    The downside is that your servers use private IP's which migh crete problems for certain setups.

    The third approach would be imo the most clean approach. But it only works if your ISP routes your /29 to another public IP you have on the WAN. You disable NAT for your server-subnet and just route the IP's :)



  • Sorry, I threw that post up right before bed hoping to let it get seen overnight.

    Diagram attached.

    So, the dsl on the wan connection origionally tells me to use "unnumbered mode" on the modem.
    this qwest pdf details what they want me to do. http://www.qwest.com/internethelp/modems/m1000/pdf/16304_M1000_MultiStatIP_PPPoA_VIP.pdf

    The reason I can't follow their outline is this:
    In unnumbered mode, the client computers have a public static IP. But, the modem is still firewalling some ports. For instance, it requires me to enable port forwarding for port 80, but I want to disable the firewall for these public IP addresses. Or at least not have to use the M1000's port forwarding. It only lets port 80 be forwarded to a single IP, but I have several IPs all that need port 80.

    I can't use 1:1 nat or anything because I want the servers to have public IPs.

    To GruensFroeschli. I think you've got the right idea about what I want to do. I wouldn't mind if I had to waste one of the IPs, I just want it to work. I do want public IPs on y servers, but I don't know what the ISP does as far as routing. I think with their setup the modem does all that transparently. If their modem would just dmz all the IP addresses the way its supposed to I wouldn't even need a PFsense box. ugh!
    Options 1 and 3 both sound fine, I just don't know which I need. Maybe if you look at the PDF you can figure out what qwest is doing and help me pick an option?

    I have a document detailing my IP addressing. Here are the specifics:

    Reserved Network 67.x.x.88
    User-assignable 67.x.x.89
    User-assignable 67.x.x.90
    User-assignable 67.x.x.91
    User-assignable 67.x.x.92
    User-assignable 67.x.x.93
    Reserved Gateway 67.x.x.94
    Reserved Broadcast 67.x.x.95
    Subnet Mask 255.255.255.248

    To use your new IP addresses:
    Now that you've leased a block of static IP addresses, you will need to configure the TCP/IP
    stack on each individual computer with one of the user-assignable static IP addresses, the
    subnet mask, gateway and domain name server. You will also need to configure your DSL
    modem/router if you have Qwest Broadband™.

    One thing I noticed:
    When configured correctly in unnumbered mode, the modem is using the gateway address of 67.x.x.94, and is routing to an entirely different subnet. It's like qwest sends all traffic for my IPs to the gateway address and expects that gateway device to do the routing for the remaining IP addresses… maybe?

    Thanks for your help, I look forward to your responses.




  • Found some additional detail that might be useful.

    I connected the DSL line as a transparent bridge to the pfsense box and did pppoe auth on the wan interface.
    It has the following stats:
    IP address: 67.x.x.94
    Subnet mask: 255.255.255.255
    Gateway: 207.225.84.212

    From pfsense I can ping google.

    I bridged the lan and wan interfaces and gave the lan interface the same IP as the wan, 67.x.x.94, then disabled outbound nat, enabled filtering bridge, and gave my laptop on the lan interface one of the 5 free IPs, 67.x.x.90.
    I setup a rule to pass all lan traffic to the wan, and all wan traffic to the lan, and, IT WORKS!

    I don't know why, but it works. It seems like having the same IP address on the lan and wan interfaces would break something, even though they have different subnet masks. I'm not complaining though.

    I do wonder, is this a correct configuration? It seems like I just stumbled upon something that worked, but I would like to make sure I have it setup "right"

    I would like to be able to still use nat for the comcast connection and personal computers, IE: opt 1 and opt 2.
    I don't know how to write advanced outbound nat rules, but it seems like it should be easy enough, right?
    When I enabled automatic nat rule generation, inbound still works fine, but outbound everything gets nat'ed to the wan IP of x.94. So I just need to write rules for the opt_wan/comcast interface, right?

    I won't go much further incase I need to make some major change. I still think having the wan and lan IPs the same doesn't seem right.

    Thanks for letting me think aloud, and I look forward to your responses!



  • You kind of mixed the different possible solutions.

    If you bridge two interfaces, the one being bridged should have no IP.
    –> Bridge LAN to WAN --> LAN without IP.

    If you bridge two interface together no NAT will occur because no traffic will be sent to pfSense.
    Your clients/servers should have as their gateway the next hop behind pfSense (your ISP).
    The IP of pfSense on WAN is purely for managment purposes.

    Perhaps you should read the howto's for a bridging solution.
    http://forum.pfsense.org/index.php/topic,7001.0.html

    If you enable automatic nat rule generation and your clients get NATed they definitly have the wrong gateway set.

    If you want an additional interface on which you can add private computers you need the NAT.
    Otherwise your private subnet wont have an IP from which they can be NATed to the internet.



  • So the clients/servers would use a gateway of Gateway: 207.225.84.212, would I need to change the subnet mask on the clients too? the PFsense wan is using /32.

    There was a checkbox to enable bridging, it said by enabling the check box, bridged packets were processed by PFsense. When I unchecked this, the clients/servers lost connectivity; would this be resolved by them using the ISP gateway?

    Also, you said by not using the pfsense box as a gateway, these IPs wouldn't be subject to nat. Does that mean I can re-enable automatic nat rule generation? I need this for my opt1 and opt2 interfaces.

    Thank you for being so patient. I know I'm kinda asking for things to be spelled out, but it makes things a lot easier to understand since this is my first time working with such a complex setup.



  • I just tried re-bridging the lan to the wan and not giving it an ip and it complained and required me to enter an IP.
    It kinda seems like bridging isn't working. When I uncheck Enable filtering bridge, everything stops working.
    Giving the client/servers a gateway address of the ISP, everything stopped working. Only way for me to get internet connectivity again was to give the lan and wan interfaces the same IP and give the client/servers a gateway of that IP, x.94.



  • I think the problem is that your WAN is PPPoE –> you get a /32 mask on WAN.
    Now if you bridge your LAN, which has public IPs in it, they are not able to reach the gateway because of the /32 >_<

    I only used a bridging setup where the WAN had a /27 or so and i could set the gateway of my clients to the next hop.
    Maybe this is just the strangeness of PPPoE...

    You wrote before:

    I bridged the lan and wan interfaces and gave the lan interface the same IP as the wan, 67.x.x.94, then disabled outbound nat, enabled filtering bridge, and gave my laptop on the lan interface one of the 5 free IPs, 67.x.x.90.
    I setup a rule to pass all lan traffic to the wan, and all wan traffic to the lan, and, IT WORKS!

    Since this seems to work: use it :)
    But i'm wondering. You set the gateway of the client to 67.x.x.94, after disabling NAT completly, right? (enable "Advanced outbound NAT" and delete all rules)

    Now if you access something on the internet. Can you check what the server you access sees as source?

    Also you might want to remove the allow any-WAN to any-LAN rule since now you open your server completly to the internet.
    Only add rules that allow access to the services you're running (or you completly defeat the purpose of the firewall :D)

    Try to set the LAN-IP not to the same as the WAN-IP.
    Just take a private IP that wont have any impact on your network.
    Does that work?



  • @GruensFroeschli:

    Now if you access something on the internet. Can you check what the server you access sees as source?

    http://whatsmyip.org will tell you.



  • I would like to use it as it is since it works, just needed a reality check to make sure what I'm doing isn't something waiting to explode in failure sometime down the road.

    Yes, I disabled nat (advanced outbound and deleted the rule) and it sees the request as coming from the client IP x.90. When I enable nat using the other radio button and do the same test, the request comes from the wan IP of my pfsense box. So it seems to work!

    I will disable the all-wan to all-lan once everything is up and running. Just wanted to eliminate that as a point of failure.
    Honestly, I don't especially care if I have a firewall since it's my understanding that debian linux is "secure by default" by which I mean, its only open ports are ones I open by installing apache and so on.

    The only reason I'm using PFsense for the servers right now is that I needed a platform to route these IP addresses and the DSL modem wasn't cutting it. If I could get this same setup out of a little dlink router, I would actually kinda prefer that for the lower power usage. But I'm not gonna mess with it now that it works, I think. Besides, once I get my private lan routed through the pfsense box and it sees some more utilization i won't feel so bad about using such big hardware for a simple purpose.



  • Hehe.
    If you want smaller hardware: http://pcengines.ch –> ALIX
    With that you can route/filter (almost) at wirespeed.
    The power consumption is around 5-10 W.

    But i think your setup should work without an imminent explosion ;)
    It's just a bit strange because of the /32 WAN.
    Now you just need to check if users can access your public IP's from the internet.


Locked