How to route between 2 LANs?



  • I have installed PfSense on my old PC with 3 NIC's.
    1 NIC is used for my WAN
    2 NIC is used for LAN network 192.168.0.0
    3 NIC is used for LAN network 192.168.123.0 (and all traffic is routed through my VPN provider)

    I have a Plex server behind my NIC 3 192.168.123.20 and this is not reachable behind any PC on NIC 2 LAN 192.168.0.0

    How can I allow traffic to pass between these interfaces?



  • At LAN interface access is allowed to anywhere by default. There is an appropriate rule placed in firewall > rules > LAN.
    On additional interfaces you've added the allow rule must be set up manually.

    I guess, your pfSense router interface is not the default gateway at the host you try to reach.



  • @viragomann:

    At LAN interface access is allowed to anywhere by default. There is an appropriate rule placed in firewall > rules > LAN.
    On additional interfaces you've added the allow rule must be set up manually.

    I guess, your pfSense router interface is not the default gateway at the host you try to reach.

    Pfsense is my default gateway

    I followed this complete guide
    https://airvpn.org/topic/11245-how-to-set-up-pfsense-21-for-airvpn/

    I have my ISP modem in bridge mode connected to NIC 1 of PfSense
    NIC2 is connected to a 10 port 1G unmanaged switch
    NIC3 is connected to my Plex server PC (VPN traffic)

    what firewall rules do I need to add for traffic to be passed from NIC 2 to NIC3










  • I see, your "ALLOW LAN OUTBOUND" rule at LAN interface directs any traffic to WAN gateway.

    So add an additional allow rule for the Plex server above of this one.
    Set the protocol as you need, source = LAN net, destination = Plex server



  • @viragomann:

    I see, your "ALLOW LAN OUTBOUND" rule at LAN interface directs any traffic to WAN gateway.

    So add an additional allow rule for the Plex server above of this one.
    Set the protocol as you need, source = LAN net, destination = Plex server

    I tried what you suggested, unless I entered it wrong, it didn't work. I think the issue is plex needs to be auto discovered by the clients (xbox, ipad etc) They don't know what IP it is on the local LAN.




  • Are your Plex server and clients signed in to your Plex.TV account? Plex uses Plex.TV for discovery if they aren't on the same subnet.

    Also the second last default IPv4 allow rule at the end of your firewall rules on the LAN interface doesn't achieve anything as the block all rule just above overrides it.



  • @kesawi:

    Are your Plex server and clients signed in to your Plex.TV account? Plex uses Plex.TV for discovery if they aren't on the same subnet.

    Also the second last default IPv4 allow rule at the end of your firewall rules on the LAN interface doesn't achieve anything as the block all rule just above overrides it.

    yes my clients are logged in, which is strange. At the very least my client might not see the auto discovery but should see the plex as a remote online server (this is not ideal as it will try to stream over internet, but should work at least)

    for example, my iPhone connected to my home LAN NIC#2 192.168.0.0 network cannot connect to plex (when it should at least see it as a remote server, it should act like i'm not at home but it doesn't) If I disconnect the wifi and use my LTE cellular, then it connects to my plex server as a remote server

    very strange, could PfSense be blocking something else here?



  • Do your clients connect if you manually specify the IP address of the Plex server? Have you tried disabling secure connection s on the Plex server as this can often cause issues?



  • @kesawi:

    Do your clients connect if you manually specify the IP address of the Plex server? Have you tried disabling secure connection s on the Plex server as this can often cause issues?

    no they don't connect if I manually specify ip
    I tried disabling secure connections but no change

    what is strange is even if the clients on network 192.168.0.0 cannot auto discovery my plex server on network 192.168.123.0 , they should at least connect to plex.tv and see my server as a remote server (like when viewing from home) and they don't. So this is leading me to believe that PfSense is somehow blocking that connecting. Do I need a firewall rule to allow an outbound connection on network 192.168.0.0?



  • The list of ports Plex requires for access is given at https://support.plex.tv/hc/en-us/articles/201543147-What-network-ports-do-I-need-to-allow-through-my-firewall-.

    Perhaps Plex needs one of these ports open to the from your AIRVPN_LAN to your LAN in order to initiate a connection?

    Is there a particular reason you have the hosts which require access to the VPN on a separate LAN segment? If it is to purely contain all of their traffic to the VPN then this can be easily achieved with a few firewalls rules from your LAN network and simplify things for you.



  • @kesawi:

    The list of ports Plex requires for access is given at https://support.plex.tv/hc/en-us/articles/201543147-What-network-ports-do-I-need-to-allow-through-my-firewall-.

    Perhaps Plex needs one of these ports open to the from your AIRVPN_LAN to your LAN in order to initiate a connection?

    Is there a particular reason you have the hosts which require access to the VPN on a separate LAN segment? If it is to purely contain all of their traffic to the VPN then this can be easily achieved with a few firewalls rules from your LAN network and simplify things for you.

    I want my hosts to use my internet straight through my ISP (no VPN) I only want my server (which has plex, my webserver and torrents) routing through the VPN

    But why can I not even ping from network to network?

    From 192.168.123.20 I can ping its gateway 192.168.123.1 and I can ping the gateway of the other network 192.168.0.1 but I cannot ping 192.168.0.2 or anything else on that network, why?

    How can I create a rule for example on my LAN (192.168.0.0) interface to pass any traffic with the below ports directly to my VPN_LAN interface (192.168.123.0)
    UDP: 32410, 32412, 32413, 32414 (for current GDM Plex network discovery)
    and then create the same rule for the opposite



  • To simplify things you could put your server in the LAN subnet and create the following rules to all direct server traffic through the VPN:

    Action: Pass
    Interface: LAN
    TCP/IP Version: IPv4
    Protocol: Any
    Source: Server_IP
    Destination: any
    Port Range: any
    Gateway: AirVPN_WAN
    
    Action: Reject
    Interface: LAN
    TCP/IP Version: IPv6
    Protocol: Any
    Source: Server_IP
    Destination: Not LAN net
    Port Range: any
    

    The following two floating rules would also be required to stop server traffic from using the WAN interface if your VPN goes down (ie a VPN kill switch):

    Action: Match
    Interface: LAN
    Direction: in
    TCP/IP Version: IPv4 + IPv6
    Protocol: Any
    Source: Server_IP
    Destination: any
    Advanced Options: Mark packets matching this rule: NO_WAN_EGRESS
    
    Action: Reject
    Interface: WAN
    Quick: Enabled
    Direction: out
    TCP/IP Version: IPv4 + IPv6
    Protocol: Any
    Source: Any
    Destination: any
    Advanced Options: Match packet on a mark: NO_WAN_EGRESS
    

    EDIT: Corrected errors in second rule above.

    If you have squid proxy running with a transparent proxy then you will need to specify the server's IP address in the Bypass Proxy for These Source IPs option. You can also add the following to the Custom ACLS (Before Auth) box in the Advanced features (this only works if you have a static IP for your VPN interface:

    ##Send specific source hosts via VPN
    acl src_to_vpn src <server_ip>
    tcp_outgoing_address <airvpnwanip> src_to_vpn</airvpnwanip></server_ip>
    

    If you still want to keep the server in a separate subnet then create the following rule for your needs and place it before your rule which directs traffic to your VPN gateway:

    Action: Pass
    Interface: AIRVPN_LAN
    TCP/IP Version: IPv4
    Protocol: UDP
    Source: 192.168.123.20
    Destination: LAN Net
    Port Range: 32410 - 32414
    

    You shouldn't need a rule on your LAN as you already have one allowing all traffic from the LAN to the AIRVPN_LAN.

    Finally, monitor the firewall logs on both pfSense and your server to see if traffic is getting blocked. You can also use packet capture under the diagnostics menu in pfSense to check what traffic is being sent.



  • thanks for the detailed instructions, I will try them when I have some time after the holidays and report back



  • I kept my plex server on the separate subnet (192.168.120.20 which is routing through my VPN)
    I entered in the rule for the UDP ports as you described but still my plex server is unreachable
    below are the packaet captures from my LAN subnet and they all show as tcp protocol not udp

    11:17:00.511338 IP 192.168.0.109.55972 > 192.168.123.20.32400: tcp 0
    11:17:01.644173 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:02.647225 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:03.682973 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:04.698628 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:05.705162 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:06.706325 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:08.716646 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:12.727664 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:20.739385 IP 192.168.0.109.56016 > 192.168.123.20.32400: tcp 0
    11:17:21.985613 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:21.987733 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:22.987385 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:22.990385 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:23.995935 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:23.999922 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:24.996856 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:25.002709 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:26.001396 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:26.006495 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:27.006533 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:27.013279 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:29.017479 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:29.024224 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:33.032870 IP 192.168.0.109.56032 > 192.168.123.20.32400: tcp 0
    11:17:33.038242 IP 192.168.0.109.56033 > 192.168.123.20.54473: tcp 0
    11:17:36.940327 IP 192.168.0.109.56040 > 192.168.123.20.54473: tcp 0
    11:17:36.947198 IP 192.168.0.109.56042 > 192.168.123.20.32400: tcp 0
    11:17:37.945613 IP 192.168.0.109.56040 > 192.168.123.20.54473: tcp 0
    11:17:37.952858 IP 192.168.0.109.56042 > 192.168.123.20.32400: tcp 0
    11:17:38.948649 IP 192.168.0.109.56040 > 192.168.123.20.54473: tcp 0
    11:17:38.954769 IP 192.168.0.109.56042 > 192.168.123.20.32400: tcp 0
    11:17:39.258217 IP 192.168.0.109.56045 > 192.168.123.20.54473: tcp 0
    11:17:39.264338 IP 192.168.0.109.56046 > 192.168.123.20.32400: tcp 0
    11:17:40.264172 IP 192.168.0.109.56045 > 192.168.123.20.54473: tcp 0
    11:17:40.267625 IP 192.168.0.109.56046 > 192.168.123.20.32400: tcp 0
    11:17:40.635659 IP 192.168.0.109.56048 > 192.168.123.20.32400: tcp 0
    11:17:40.635912 IP 192.168.0.109.56049 > 192.168.123.20.54473: tcp 0
    11:17:41.260667 IP 192.168.0.109.56053 > 192.168.123.20.54473: tcp 0
    11:17:41.265165 IP 192.168.0.109.56054 > 192.168.123.20.32400: tcp 0
    11:17:41.805472 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:41.814381 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:42.808899 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:42.815504 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:43.811189 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:43.818790 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:44.813588 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:44.823211 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:45.812995 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:45.824242 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:46.814782 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:46.827160 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:48.820746 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:48.831234 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0
    11:17:52.823397 IP 192.168.0.109.56057 > 192.168.123.20.54473: tcp 0
    11:17:52.834011 IP 192.168.0.109.56058 > 192.168.123.20.32400: tcp 0

    When I check the AIRVPN_LAN packet capture, I see nothing from IP 192.168.0.109

    my other question is, even if my LAN cannot communicate with my AIRVPn_LAN network, why can my LAN plex client not see my plex server like a remote server (like I was at my friends house etc)



  • Can you please post a screenshot of your current firewall rules for your LAN and AIRVPN_LAN interfaces so that we can see the changes you've made since your original post?

    Also when posting logs or masses of text output, it's better to format text as CODE (the # button above box where you enter the text for your post).

    Plex clients initiate the connection to the Plex server over TCP port 32400, and this is what the packet capture shows. The LAN firewall rules your posted earlier allow any IPv4 traffic to 192.168.123.20 so this should be getting through to the Plex server. However, subsequent changes you've made could have impacted this, which is why I'd like to see your current ruleset.

    For the rules you created for the Plex UDP ports, are they on the AIRVPN_LAN interface to allow traffic from the Plex server back to your LAN subnet?

    A couple of things to try:

    • Check the firewall settings on the Plex server itself to see whether it is allowing incoming connections, particularly from the IP range of your other subnet

    • Run a packet capture on the AIRVPN_LAN to see what traffic is going to/from your Plex server

    • Run a packet capture on your Plex server to see if it is actually sending or receiving any traffic

    • Enabled logging on your Plex firewall rules to check if they are being triggered (you will want to enable logging, test and then disable logging to avoid flooding your system logs)



  • finally got it t work, I have my server on NIC#2 connected to VPN and everything else connected to NIC#1 straight to my ISP

    see attached screenshots (8.png and 9.png)

    last thing I noticed after getting my plex resolved is I am not able to access my webpage from the internet. I use XAMP port 8080 , and before using PfSense I had my store router set to port forward port 80 to my LAN IP 192.168.0.20 port 8080 and it worked fine.
    See screenshot (10.png) I have firewall rule for WAN to forward and destination port 80 (http) packets to AIRVPN_LAN IP 192.168.123.20 (my server) port 8080
    but it doesn't work, I keep getting my PfSense login page when I try my domain from the internet







  • LAYER 8 Netgate

    First, Destination Address in your port forwards should not be any, but the appropriate interface address or VIP.

    Second, pfSense is listening on port 80. Set it to HTTPS only and disable the port 80 redirect.

    ![Screen Shot 2016-01-05 at 4.19.32 PM.png](/public/imported_attachments/1/Screen Shot 2016-01-05 at 4.19.32 PM.png)
    ![Screen Shot 2016-01-05 at 4.19.32 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2016-01-05 at 4.19.32 PM.png_thumb)



  • Still doesn't work, but at least I'm no longer getting the Pfsense login.





  • LAYER 8 Netgate

    Nobody said anything about setting a source port.



  • removed source ports, still not reaching my server


  • LAYER 8 Netgate

    Post again. This stuff just works.

    Look at EVERYTHING on this list:

    https://doc.pfsense.org/index.php/Port_Forward_Troubleshooting



  • I don't have any other issues port forwarding as you can see my Plex and utorrent port forwards are working fine
    I have no firewall on my server and windows firewall is turned off.
    localhost:8080 displays my webpage so my server is running fine



  • LAYER 8 Netgate

    OK don't listen. Not my network to fix.



  • Do you have a corresponding firewall rule on your WAN? If you selected the appropriate filter rule association when you created the port forward, one should have been created automatically.

    Action: Pass
    Interface: WAN
    TCP/IP Version: IPv4
    Source: Any
    Port: Any
    Destination: 192.168.123.20
    

    Have you tried using the squid reverse proxy instead of a port forward?



  • I checked and see screenshot, it appears I do have the rule. I don't have squid installed



  • LAYER 8 Netgate

    Also on the list is detailing packet captures.  I would packet capture on LAN limiting to ip address 192.168.123.20 on that LAN interface, test a connection from outside, and see what you see. I'll bet you see the connection SYN going out LAN and nothing coming back.



  • I tried packet capture and one other step.

    Packet capture, see attached, showed nothing except the pfsense connection , nothing on port 80 or 8080

    I then shutdown the pfsense and removed it from my network. I connected back my old router and fired it up. I tested my website http://threebeesandme.com and it was displayed with no issues. (this was using my cell phone with wifi turned off, cell network only) so that proved godaddy is directing the requests to the correct IP and my server is responding correctly, the culprit is pfsense but I am lost here.




  • Weird, tried again and it's working now from the internet (www.threebeesandme.com)
    But doesn't work from LAN 192.168.0.1/24 or from VPN_LAN 192.169.123.0/24 very strange



  • LAYER 8 Netgate

    It is not very strange. It is completely expected. Implement split DNS so your internal hosts connect to an internal IP address.



  • Actually I think it is very strange that from the public internet I could not reach my web server before I did a packet capture. Then directly after a packet capture I can, when absolutely no changes were made on pfsense. That is the strange part I am referring too

    How exactly do I implement split DNS while insuring the 192.168.123.0/24 clients do not leak dns on the other 192.168.0.0/24 subnet?


  • LAYER 8 Netgate

    Why do you care if DNS leaks to the other subnet? WTF are you worried about exactly?

    Say you have external, global DNS that has an A record of 65.65.65.65 for www.mycoolsite.com.  Your internal DNS has an A record of 192.168.123.20 for www.mycoolsite.com.

    You want ALL internal (Not NAT) hosts to get 192.168.123.20 when they ask for the address of www.mycoolsite.com. You want all external hosts to get 65.65.65.65.

    Whether or not the users on 192.168.0.0/24 can access the services on 192.168.123.20 is handled by firewall rules on the 192.168.0.0/24 interface, not DNS.

    If you REALLY want to make DNS answers different for clients on 192.168.0.0/24 and 192.168.123.0/24 you are probably looking at BIND and views. I, personally, would use a VM for that, not the BIND package, but people tend to contract a brain virus that makes them try to make pfSense do absolutely everything.

    If you stop blaming pfSense you might get your network configured properly. This stuff just works when you do it right. It doesn't pass traffic one minute but not another just because it feels like it - something was changed.


Log in to reply