Routing Problem OpenVPN/Gateway/Rule

taunusstein.net

Hello,

there is a very strange routing problem with pfSense 2.1-RELEASE with following Scenario:

An OpenVPN tunnel routes a network (assigned by RIPE) from a server to pfSense, where pfSense is client.
OpenVPN local interface was assigned with an interface configuration, so pfSense can ping OpenVPN server and vice versa. Server routes RIPE subnet to pfSense OpenVPN IP address.
On pfSense a gateway (non default) was configured on that interface, so interfaces can be selected in firewall rules. Rules on a vlan interface are using that interface.

Following config items are used:

<openvpn-client><vpnid>2</vpnid>
<protocol>UDP</protocol>
<dev_mode>tap</dev_mode>
<ipaddr><interface>opt1</interface>
<local_port><server_addr>x.x.x.1</server_addr>
<server_port>31001</server_port>
<resolve_retry><proxy_addr><proxy_port><proxy_authtype>none</proxy_authtype>
<proxy_user><proxy_passwd><mode>p2p_shared_key</mode>
<shared_key>***</shared_key>
<crypto>AES-128-CBC</crypto>
<engine>cryptodev</engine>
<tunnel_network>10.0.0.20/30</tunnel_network>
<tunnel_networkv6><remote_network><remote_networkv6><use_shaper><compression>yes</compression>
<passtos></passtos></use_shaper></remote_networkv6></remote_network></tunnel_networkv6></proxy_passwd></proxy_user></proxy_port></proxy_addr></resolve_retry></local_port></ipaddr></openvpn-client>

<opt5><if>ovpnc2</if>
<enable><spoofmac><ipaddr>10.0.0.22</ipaddr>
<subnet>30</subnet></spoofmac></enable></opt5>

<gateway_item><interface>opt5</interface>
<gateway>10.0.0.21</gateway>
<name>ovpnc2GW</name>
<weight>1</weight>
<ipprotocol>inet</ipprotocol>
<interval><monitor_disable></monitor_disable></interval></gateway_item>

<rule><id><type>pass</type>
<interface>opt6</interface>
<ipprotocol>inet</ipprotocol>
<tag><tagged><max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype>
<os><source>
<any><destination><any></any></destination>
<descr><gateway>ovpnc2GW</gateway></descr></any></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule>

Test from local server to Internet, works as expected, all IP packets are routed via OpenVPN interface:

[root@webserver ~]# ping -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8 ) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=50 time=11.7 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=50 time=9.81 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=50 time=9.55 ms

–- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2013ms
rtt min/avg/max/mdev = 9.552/10.371/11.746/0.984 ms
[root@webserver ~]#

Test from Internet to local server, works not, reply packets are still routed by WAN interface, where WAN interface is em2:

[2.1-RELEASE][root@pfsense.lan]/root(67): tcpdump -ni em2 icmp and host x.x.x.226
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em2, link-type EN10MB (Ethernet), capture size 96 bytes
11:53:57.065045 IP x.x.x.226 > x.135.164.119: ICMP echo reply, id 38727, seq 1, length 64
11:53:58.063965 IP x.x.x.226 > x.135.164.119: ICMP echo reply, id 38727, seq 2, length 64
11:53:59.064510 IP x.x.x.226 > x.135.164.119: ICMP echo reply, id 38727, seq 3, length 64
^C
3 packets captured
189 packets received by filter
0 packets dropped by kernel

Why does pfSense route reply packets other than originating packets?

Update: Same on 2.1.1-PRERELEASE (i386) built on Fri Feb 21 11:18:18 EST 2014 FreeBSD 8.3-RELEASE-p14

bennyc

Not following entirely with your description… a drawing could help a lot here.
pfSense usually just does what you configure it should do. What rules did you configure? (hint: for policy based routing & OpenVPN, use the floating rules)