TFTP Proxy over double NAT?



  • This is my setup:

    WAN1<–>pfSense1<-(LINK)------------(LINK)->pfSense2<-->WAN2
                      |                                                    |
                    LAN1                                              LAN2

    I want to be able to access a TFTP server on LAN1 from LAN2. I have everything setup correctly so LAN1 PCs can see and access LAN2 PCs without issues and viceversa. I think that the problem is that I have a double NAT situation here.

    LAN1 is in the range 192.168.0.0/24
    LAN2 is in the range 192.168.1.0/24
    The LINK interface is on the range 10.254.254.0/30 (1 IP for each pfSense)

    So any packet going from LAN1 to LAN2 first gets NAT'ed to 10.254.254.0/30 and then to 192.168.1.0/24.

    I have NAT and routing already configured and working. But I cannot make the TFTP work! I tried enabling the TFTP Proxy Helper on both boxes, in different configurations, but I couldn't make it work.

    Is this even possible?? Any special port or firewall rules settings? What interfaces should I set the TFTP proxy to listen to on each pfSense??

    Thanks!!!!

    Cheers!



  • I've been reading a bit about how the TFTP protocol works… Since the reply port is randomized by the server, I can't see how to handle the 2nd NAT without a specially designed module...

    Does anyone know if this can be achieved just by a proper configuration of existing tools???

    Thanks again



  • You shouldn't be using any NAT internally… This is where routing comes into play. If you have routing setup correctly what should happen is this:

    192.168.0.7 requests a packet to be sent to 192.168.1.7. The PC knows that ip is not on it's network, so it sends the packet to the first PFSense box. That box should have a static route setup to indicate that it needs to pass that traffic to PFSense box 2. Then the 2nd box goes I have an interface on 192.168.1.X and passes that packet to the pc on that network.

    I'm definitely not up on my routing but you should have a static route that reads something like this:

    NFE0 = WAN
    NFE1 = LAN 192.168.0.1
    NFE2 = LINK 10.254.254.1

    192.168.1.0/24 dev nfe1 proto kernel  scope link  src 10.254.254.2

    So use this command on PFSense1:
    route add -net 192.168.1.0/24 10.254.254.2

    a netstat -nr should look something like this:

    Internet:
    Destination         Gateway            Flags    Refs      Use    Netif    Expire
    default               ISP gateway                                    NFE0
    192.168.0.0/24   Link#2                                               NFE1
    192.168.1.0/24   10.72.254.2                                        NFE2
    192.168.0.1        Link#2                                               lo0
    10.254.254.0/30 Link#3                                               NFE2
    10.254.254.1      Link#3                                               lo0
    127.0.0.1            Link#4                                               lo0

    You want to tell your PFSense1 box that if a packet comes from interface 1 heading to an address on 192.168.1.X subnet, it needs to send it to a different gateway, in this case PFSense2. That box will know which interface to send the packet out of. Any other packets will still go out the default gateway of your ISP.

    The same thing needs to be done on PFSense2 except in reverse order

    NFE0 = WAN
    NFE1 = 192.168.1.0
    NFE2 = LINK 10.254.254.1

    192.168.0.0/24 dev nfe1 proto kernel  scope link  src 10.254.254.1

    So use this command on PFSense2:
    route add -net 192.168.0.0/24 10.254.254.1

    Internet:
    Destination         Gateway            Flags    Refs      Use    Netif    Expire
    default               ISP gateway                                    NFE0
    192.168.1.0/24   Link#2                                               NFE1
    192.168.0.0/24   10.72.254.1                                        NFE2
    192.168.1.1        Link#2                                               lo0
    10.254.254.0/30 Link#3                                               NFE2
    10.254.254.2      Link#3                                               lo0
    127.0.0.1            Link#4                                               lo0

    I'm not 100% sure if my syntax is correct so I apologize if it is.



  • Thanks for the reply! In fact, I do have everything setup exactly as you say. I have static routes on each pfSense that indicates that traffic destined to the other subnet should be routed through the other pfSense, and everything works perfectly. I can even control which WAN gets the traffic routed to on each subnet.

    You are right. I made a mistake when I explained in my first post. This is not NAT'ting, it is just basic routing. I do not have any kind of internal port forwading rule or anything alike. The only NAT'ting is towards each WAN, and does not play a role in my TFTP problem.

    So, how is TFTP supposed to work in this scenario???

    Thanks again!



  • I'm guessing you're using it in a way that a client is booting up and looking for a TFTP Server. It sounds like you want something similar to a DHCP relay. You'll want to have your TFTP Server on Box 1 listening on both 192.168.0.X and 10.254.254.X. The TFTP proxy would reside on PFSense 2 listening on the LAN and pick up any traffic it sees on 192.168.1.X and then forward that to 10.254.254.X of the IP of PFSense1. That server should be able to return the reply to the client saying here are the files you want.

    The ports they actually communicate on between each other are not important since they are not being firewalled. The main thing is that you need to have the server setup to listen on port 69 for the original request. So what would happen is client A sends a request to PFSense2 on port 69. The proxy setup on Server two takes that request and passes it onto PFSense1 on port 69. That server takes the request and contacts the client to setup the connection between the server and the client.

    I'm not exactly sure what you're using for TFTP proxy but it sounds like there is an open-bsd proxy program with some syntax like this:

    To make use of the proxy, pf.conf(5) needs the following rules.  The
        anchor is mandatory.  Adjust the rule as needed for your configuration.

    anchor "tftp-proxy/*"
              pass in quick on $int_if proto udp from $lan to any port tftp
                  rdr-to 127.0.0.1 port 6969

    you would want that to say rdr-to 10.72.254.X port 69 since that is where PFSense1 will be listening for connections. It's also possible to just send it to 192.168.0.X if you can't make the TFTP server listen on both interfaces at the same time.

    You only want to have the TFTP proxy on PFSense2 and have the actual TFTP Server on PFSense1.



  • I should have explained the whole thing since the beginning  :P

    I have an already running and configured TFTP server on one subnet that I use for PXE booting, so I can install Windows, run MemTest, etc, directly by booting from the network. My preferred solution would be to have that TFTP server accesible from the other subnet.

    Plan B scenario would be to have pfSense itself act as a TFTP server, but I would rather avoid that since the Windows netinstall is quite picky when it comes to the TFTP server (considering the whole installation system runs on Linux). For example, I had to manually patch and compile the regular "atftpd" server to allow case-insensitive lookups, etc. I am not handy at all with FreeBSD, I will end up learning of course, but for consistency I would like to use the existing TFTP server that is already configured.

    In fact, I completely messed up this topic. I don't really have any NAT internally (I was really tired when I wrote the first topic  :P ), so now I don't think I need to use the TFTP Proxy. This is just regular routing. I don't have any restricting rules on the firewall, I don't understand why it does not work right away…

    Before anyone asks, both pfSenses are correctly configured to pass the "next-server" and "filename" over DHCP.

    If anyone come up with an idea, it will be much appreciated!

    Thanks!



  • Okay, so when you try to do a boot what happens on the client?

    Does it sit at the network boot screen and  show DHCP…..*  and not do anything? Say no PXE server found, exiting?

    Or does it get the DHCP lease then sit there at TFTP...... and then say something like TFTP open timeout?

    I'm guessing it's doing the 2nd one and that's where the issue is coming from. If you can try I would take your TFTP Server and put it on the other lan temporarly and just make sure that works first. I did a little testing with wireshark and the traffic is pretty simple. The client is going to try to contact the TFTP server on port 69, then that server replies from a random port back to the client on port 69. If you are familiar with wireshark maybe you could put that on the lan and see if you see the client trying to contact your TFTP server, and if the server is attempting to respond. There's a bit of arp traffic but that might not be related.

    Also... do you have any type of IP tables on the linux TFTP server? It's possible that your server doesn't allow connections from the other lan, or the TFTP server daemon has a spot in the config to say what networks can connect to it. I know in Samba I had to make a change to allow access when I moved the server's subnet. I would guess you can ping the TFTP server from the client on the other lan?



  • Sorry I didn't answer before! I was able to sort this out. In fact I was a little messed up with the config. I did have some sort of double NAT internally, indeed. After cleaning up the outbound NAT rules and setting everything as "just routing" like you mentioned, it is working perfectly.

    Thanks!!



  • Glad you got it working!  ;D


Log in to reply