Can't connect to client network?

cyruspy

Hi, I'm testing a 3 site connection. In my setup, S1 is client for S2 and S2 is client for S3. The thing is I can ping from LAN1 to LAN2, from LAN2 to LAN3, from LAN3 to LAN2 but not from LAN2 to LAN1. There are no messages in in the firewall logs and fw2 has a route defined for LAN1, any ideas?

GruensFroeschli

Comming from this thread:
http://forum.pfsense.org/index.php/topic,23780.0/topicseen.html

Can you post a few more information?
I assume you're using a PSK setup.
Did you add all the needed routes in the custom options field? Or where did you configure them?

cyruspy

Hi!, according to the recommendation from the book for site-to-site setups, i'm using shared keys. This is what i've done so far:

Site1
        LAN: 192.168.1.1
        WAN [link]: 10.10.1.1
Site2
        LAN: 192.168.2.1
        OPT1 [link]: 10.10.1.2
        WAN [link]: 10.20.1.2
Site3
        LAN: 192.168.3.1
        OPT1 [link]: 10.20.1.3
        WAN

[code]#
# S1
#
- Configurar WAN
        Type: Static
        IP address: 10.10.1.1
        Gateway:
        Uncheck: Block Private Networks
        Uncheck: Block Bogon Networks
- Setup LAN
- Setup VPN
        VPN --> OpenVPN --> Client --> add
                * Server address: 10.10.1.2
                * Interface IP: 172.31.54.0/30
                * Remote network: 192.168.2.0/24
                * Shared key: (paste)
                * Description: Link to site 2
                save --> Apply changes
- Disable NAT
        Firewall --> NAT --> Outbound
                Manual outbound --> save --> Apply changes
                # Delete the rule listed --> Apply changes[/code]

[code]#
# S2
#
- Setup WAN
        Type: Static
        IP address: 10.20.1.2/24
        Gateway: 10.20.1.254 ?
        Uncheck: Block Private Networks
        Uncheck: Block Bogon Networks
- Setup LAN
- Add OPT1, setup, enable
        Type: Static
        IP address: 10.10.1.2/24
        Gateway:
        Check: Disable userland ftp helper
- Setup VPN to Site 1
        VPN --> openVPN --> add
                * Address Pool: 172.31.54.0/30
                * Remote network: 192.168.1.0/24
                * Shared key:  (paste)
                * Description: Link to site 1
                save --> Apply changes
- Setup VPN to Site 3
        VPN --> OpenVPN --> Client -->  add tunnel
                * Server Address: 10.20.1.3
                * Interface IP: 172.31.55.0/30
                * Remote network: 192.168.3.0/24
                * Shared key: (paste)
                * Custom options: lport 1195
                * Description: Link a sucursal 3
                save --> Apply changes

- Disable NAT
        Firewall --> NAT --> Outbound
                Manual outbound --> save --> Apply changes
                # Delete the listed rule --> Apply changes

- Allow VPN to be set
        Firewall --> Rules --> OPT1 --> Add
                Protocol: UDP
                Source 10.10.1.1
                Destination: OPT1 address
                Destination port: OpenVPN
                Descripcion[/code]

[code]
#
# S3
#

- Setup WAN
        Type: Static
        IP address: Public IP?
        Gateway: ?
- Setup LAN
- Add OPT1, setup, enable
        Type: Static
        IP address: 10.10.1.3/24
        Gateway:
        Check: Disable userland ftp helper
- Setup VPN to Site 3
        VPN --> OpenVPN --> add
                * Address pool: 172.31.55.0/30
                * Remote network: 192.168.2.0/24
                * Shared key: (paste)
                * Description:
                save --> Apply changes
- Allow VPN to be setup
        Firewall --> Rules --> OPT1 --> Add
                Protocol: UDP
                Source 10.20.1.2
                Destination: OPT1 address
                Destination port: OpenVPN
                Descripcion[/code]

I just wanted to setup basic conectivity like this first:

Site1 <--> Site2 (works only Site1 --> Site2)
Site2 <--> Site3 (works)

The idea is to route everything to internet connection at Site3, where I'll be using transparent squid.[/link][/link][/link][/link]

cyruspy

This are the routes created from the above setup. I haven't add any static route yet.

FW1

I see that now the route for LAN2 is missing and apparently the VPN is not working. I'm checking this right now…

FW2

FW3

GruensFroeschli

You cannot ping from LAN2 to LAN1 because on FW1 you obviously dont have any routing table entries to answer back.
Are you sure that you have filled in the "remote network" field there?

In the GUI you have the field: "Remote network".
This allows you to add a single network which gets routed to the other side of the tunnel.
Behind the scene this creates in the config file something along the line of:
route 192.168.1.0 255.255.255.0

You can add in the custom options field as many additional such route's as you want. (Limited by the routing table).

If you want to redirect everything you can go at this like this:

route IP_of_other_side_of_link 255.255.255.255;
route 0.0.0.0 128.0.0.0;
route 128.0.0.0 128.0.0.0;

The first route makes sure that the other side is always reachable.
The second and third rule create a kind of "new default gateway" without killing the original default gateway.
These routes are added dynamically when the link comes up.
When the link dies they get removed from the routing table.

So basically in your setup:
S1:
Create these 3 rules with the /32 route pointing to S2

S2:
Create the same 3 rules with the /32 pointing to S3
Also create here a rule for the 192.168.1.0/24 subnet. (route 192.168.1.0 255.255.255.0)

S3:
Here you dont need the "redirect everything" rules.
Just two rules for S1 and S2
–>
route 192.168.1.0 255.255.255.0;
route 192.168.2.0 255.255.255.0;

cyruspy

"Fixed" FW1 with a reboot:

I can ping again from Site1 to Site2, the other way still doesn't work

GruensFroeschli

Did you disable automatic rule-generation for the VPN?
If so: Did you assign the interface with as IP "none"?
Also did you create rules actually allowing ICMP (not just tcp/udp)?

If you assign the interfaces.
It's a good idea to add to the OpenVPN config the line:
dev tunXXX
where XXX is a number you can define.
If you dont do this the tunnels will come up with a non-deterministic number.
Making it hard to assign and create rules.
If you fix them to a tun you can be sure you always have the same tunnel on the same tun.

cyruspy

I didn't disable automatic rule-generation, at least not explicitly. How can I check that?
Edit: Found the option in System –> Advanced, it's not disabled

From what I understand, I should add this options:

FW1 as client
route 192.168.2.1 255.255.255.255;
route 0.0.0.0 128.0.0.0;
route 128.0.0.0 128.0.0.1;

FW2 as client
route 192.168.3.1 255.255.255.255;
route 0.0.0.0 128.0.0.0;
route 128.0.0.0 128.0.0.1;

FW3 as server
route 192.168.1.1 255.255.255.255;
route 192.168.2.1 255.255.255.255;

Is this correct?

After those changes:

Ping LAN1 --> LAN2: works
Ping LAN2 --> LAN1: doesn't work
Ping LAN2 --> LAN3: works
Ping LAN3 --> LAN2: doesn't work (that's new)
Ping LAN1 --> LAN3: doesn't work
Ping LAN3 --> LAN1: Works!!! (that's new)

Edit: "Ping LAN3 --> LAN2" works again, without modifications ???

cyruspy

Routes after above changes

Site1

Site2

Site3

GruensFroeschli

Ugh. It's getting hard to do this all in my head ^^"

Sorry i messed up your routing table :D I just noticed i wrote an error before.
You need to redirect everything 0.0.0.0/1 and 128.0.0.0/1
/1 is 127.0.0.0 and not 127.0.0.1 >_<

The /32 (255.255.255.255) routes are not really needed.
They just are there to ensure you really send traffic to the other side of the tunnel even if you have a router in between.
They dont hurt but for this setup i think you can safely leave them away.

Are you sure that 1 and 2 are client and only 3 is the server?

Dont you mean
Link1-2: 1 is client, 2 is server
Link2-3: 2 is client, 3 is server
?

I'll try to write up a list of how your custom routes should look like:
Your link-pool is according to http://forum.pfsense.org/index.php/topic,23854.msg122952.html#msg122952

S1-client:
route 0.0.0.0 127.0.0.0;
route 128.0.0.0 127.0.0.0;
dev tun12;

S2-server:
route 192.168.1.0 255.255.255.0;
dev tun21;

S2-client:
route 0.0.0.0 127.0.0.0;
route 128.0.0.0 127.0.0.0;
dev tun23;

S3-server:
route 192.168.2.0 255.255.255.0;
route 192.168.3.0 255.255.255.0;
dev tun32;

If you deactivated automatic rule generation you need to assign the new tun interfaces (firewall –> assign).
After assigning them you need to activate it on it's config page.
Put as IP "none" and leave the gateway field empty.
Then create firewall rules on the new interface.

cyruspy

Thanks for being patient!.

Are you sure that 1 and 2 are client and only 3 is the server?

I meant I only setup the client parameters for fw2, as the route to LAN1 should be setup using the remote network parameter.
This is correct:

Dont you mean
Link1-2: 1 is client, 2 is server
Link2-3: 2 is client, 3 is server?


I'll try to write up a list of how your custom routes should look like:

Fixed the configuration as suggested.


If you deactivated automatic rule generation you need to assign the new tun interfaces (firewall --> assign).

I haven't disable the automatic rule generation, should I?

This are the routes now:

FW1

FW2

FW3

This doesn't work:

Ping LAN1 –> LAN3
Ping LAN2 --> LAN1

GruensFroeschli

Well if you don't, you don't have the option to create firewall rules for the OpenVPN interface.

So yes i would activate that, but later.
For now we need to get routing up and running correctly.

How exactly are you testing if your tunnels are up?
Did you connect clients in the LAN subnet?
Or are you pinging from the pfSenses directly?

cyruspy

I have as client:

LAN1: Linux VM
LAN2: Windows VM
LAN3: Linux VM

Tested from LAN clients and also from FW in each site

GruensFroeschli

Sorry i only just saw your screenshots.
I wrote an error again.
I saw in your first screenshots that you had as subnet for the overwriting subnet /80000001
This was wrong. I didn't realize that it was because of 127.0.0.1.
Now with 128.0.0.0 it's 7f000000 which is just as bad…
It should just be /1 (or 80000000). So in the end it really has to be 127.0.0.0
Sorry.

cyruspy

That would be something like this?

S1-client:
route 0.0.0.0 127.0.0.0;
route 127.0.0.0 127.0.0.0;
dev tun12;

S2-server:
route 192.168.1.0 255.255.255.0;
dev tun21;

S2-client:
route 0.0.0.0 127.0.0.0;
route 127.0.0.0 127.0.0.0;
dev tun23;

S3-server:
route 192.168.1.0 255.255.255.0;
route 192.168.2.0 255.255.255.0;
dev tun32;

GruensFroeschli