How do I curl though an OpenVPN interface from a script.

[guardian]

I put this on the back burner because I'm stuck, but I'm still looking for a solution. Any assistance would be much appreciated.

[Gertjan]

Use curl like this :

curl -v --interface em3 https://www.test-domaine.fr/phpinfo.php

where --interface is the interface name , em3 in my case. To be replaced by 'ovpnc1 ' if that's your OpenVPN client interface name.

You wind up having a lot of details - as the -v option means

curl --help | grep 'verbose'
     --trace-time    Add time stamps to trace/verbose output
 -v, --verbose       Make the operation more talkative

[guardian]

@Gertjan Thanks for the reply... I'm wondering if the issue is a routing problem.

Note that the system is working as intended, I have 2 VLANs that use policy routing to send all traffic out the VPN.

All other VPNs go directly out the WAN directly. The VPNs function normally, the gateway pinger works fine. What confuses me is that there are no rules in the interface tab and I get this messge in the system log

Jun 19 03:44:56 --------- php-fpm[78492]: /rc.start_packages: GATEWAYS: Group -------- did not have any gateways up on tier 1!

This is nothing new, I've been getting this message for years!), but everything works except that I can't curl over the VPN. I assume the reason why I don't need any rules on gateway interface is that those rules must be for unsolicited (no open connection) incomming connections from the VPN.

Here is my output from the tests:

Test run on WAN Interface

curl -v --interface em0 http://wttr.in
*   Trying 5.9.243.187:80...
* TCP_NODELAY set
* Local Interface em0 is ip xxx.xxx.xxx.xxx using address family 2
* Local port: 0
* Connected to wttr.in (5.9.243.187) port 80 (#0)
> GET / HTTP/1.1
> Host: wttr.in
> User-Agent: curl/7.67.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.10.3
< Date: Fri, 19 Jun 2020 06:25:15 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 8940
< Connection: keep-alive
< Access-Control-Allow-Origin: *
< 
Weather report: ... Output as expected

Test run on ovpnc1

curl -v --interface ovpnc1 http://wttr.in
*   Trying 5.9.243.187:80...
* TCP_NODELAY set
* Local Interface ovpnc1 is ip 10.28.10.10 using address family 2
* Local port: 0
* connect to 5.9.243.187 port 80 failed: Operation timed out
* Failed to connect to wttr.in port 80: Operation timed out
* Closing connection 0
curl: (28) Failed to connect to wttr.in port 80: Operation timed out

Any suggestions are much appreciated.

[guardian]

@Gertjan said in How do I curl though an OpenVPN interface from a script.:

You want select the interface that curl uses ?
If so, click.

I looked at this and I think that I've followed the directions, but it still doesn't work; Any idea what I am doing wrong?

This works (no interface specified)

curl -4  http://www.google.com

This hangs and times out:

curl -v -4 --interface ovpnc1 http://www.google.com
*   Trying 172.217.164.196:80...
* TCP_NODELAY set
* Local Interface ovpnc1 is ip 10.52.10.10 using address family 2
* Local port: 0
* connect to 172.217.164.196 port 80 failed: Operation timed out
* Failed to connect to www.google.com port 80: Operation timed out
* Closing connection 0
curl: (28) Failed to connect to www.google.com port 80: Operation timed out

This is the OpenVPN interface:

ifconfig ovpnc1
ovpnc1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
	options=80000<LINKSTATE>
	inet6 fe80::228:1aff:fee0:1004%ovpnc1 prefixlen 64 scopeid 0x27
	inet 10.52.10.10 --> 10.52.10.9 netmask 0xffffffff
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	groups: tun openvpn
	Opened by PID 40301

So what am I doing wrong?

[netblues]

@guardian In order for this to work, the other end of the vpn must support natting to the internet.

Here is the exact same command

/root: curl -v -4 --interface ovpnc1 http://www.google.com

• Trying 172.217.22.4:80...
• TCP_NODELAY set
• Local Interface ovpnc1 is ip 192.168.127.5 using address family 2
• Local port: 0
• Connected to www.google.com (172.217.22.4) port 80 (#0)

GET / HTTP/1.1
Host: www.google.com
User-Agent: curl/7.67.0
Accept: /

• Mark bundle as not supporting multiuse
  < HTTP/1.1 200 OK
  < Date: Thu, 23 Jul 2020 09:25:51 GMT
  < Expires: -1
  < Cache-Control: private, max-age=0
  < Content-Type: text/html; charset=ISO-8859-1
  < P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
  < Server: gws
  < X-XSS-Protection: 0
  < X-Frame-Options: SAMEORIGIN
  < Set-Cookie: 1P_JAR=2020-07-23-09; expires=Sat, 22-Aug-2020 09:25:51 GMT; path=/; domain=.google.com; Secure
  < Set-Cookie: NID=204=VGdCUajKT8B5UcexExeV2Km1ye5cs22px0VMjGalBW2Y7qhYMYuw4ty83dvPKnan6gXumpG4fqDlpnl7_2dEFVPE2SZvrNSDambfGuA5YOhLau9C65DK_nBgeZmVgtvD1t5XGwWdvy6sBGNbSz6k-NUvnCSzNF8cc2kGpl61Nyg; expires=Fri, 22-Jan-2021 09:25:51 GMT; path=/; domain=.google.com; HttpOnly
  < Accept-Ranges: none
  < Vary: Accept-Encoding
  < Transfer-Encoding: chunked
  <

[guardian]

@Gertjan said in How do I curl though an OpenVPN interface from a script.:

You want select the interface that curl uses ?
If so, click.

@Gertjan I tried everything that I could see in the link you mentioned, but it didn't work as you can see from the post above.

Any idea how I can troubleshoot?

[netblues]

@guardian Your curl command is fine. You have routing/natting issues.

[guardian]

@netblues said in How do I curl though an OpenVPN interface from a script.:

@guardian Your curl command is fine. You have routing/natting issues.

Can someone give me some steps to troubleshoot this?

It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.

Any advice would be much appreciated.

[Gertjan]

@guardian said in How do I curl though an OpenVPN interface from a script.:

It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.

Your 'curl' is using that rule 'that matches all traffic' and that firewall rule is probably NOT logging.
That why you see no logs.
Be careful : activating logging for the default pass all rule will log huge numbers of log lines at lightning speed.

[netblues]

I doubt you will find anything on pf logs.
The thing is, that when you select the vpn client interface, you send your request inside the tunnel to the device at the other end.

If you curl a service running on the host on the other end of the tunnel, then it would work.
If however you ask for e.g. google, then the device at the other end of the tunnel has to do nat forward the request to google , get the reply and translate it back to you.

So what is at the other end of the tunnel interface you are curling into?

[guardian]

Thanks @Gertjan @netblues The tunnel is a public VPN service with minimal filtering. I'm pretty sure it has to do with rules/routing regarding the curl command. My resaon for saying this is that I have a guest wifi network that is policy routed out that interface, and the same command works (except it doesn't have the --interface parameter).

If we assume the prblem is local, what would be my next step(s) to troubleshoot?

@Gertjan said in How do I curl though an OpenVPN interface from a script.:

@guardian said in How do I curl though an OpenVPN interface from a script.:

It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.

Your 'curl' is using that rule 'that matches all traffic' and that firewall rule is probably NOT logging.
That why you see no logs.
Be careful : activating logging for the default pass all rule will log huge numbers of log lines at lightning speed.

curl -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -v --interface ovpnc1 'http://ifconfig.me/ip'
@netblues said in How do I curl though an OpenVPN interface from a script.:

I doubt you will find anything on pf logs.
The thing is, that when you select the vpn client interface, you send your request inside the tunnel to the device at the other end.

If you curl a service running on the host on the other end of the tunnel, then it would work.
If however you ask for e.g. google, then the device at the other end of the tunnel has to do nat forward the request to google , get the reply and translate it back to you.

So what is at the other end of the tunnel interface you are curling into?

[netblues]

@guardian I do have this exact setup in place
so here is the result
/root: curl -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -v --interface ovpnc1 'http://ifconfig.me/ip'

• Trying 216.239.32.21:80...
• TCP_NODELAY set
• Local Interface ovpnc1 is ip 192.168.127.5 using address family 2
• Local port: 0
• Connected to ifconfig.me (216.239.32.21) port 80 (#0)

GET /ip HTTP/1.1
Host: ifconfig.me
Accept: /
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

• Mark bundle as not supporting multiuse
  < HTTP/1.1 200 OK
  < Date: Sat, 25 Jul 2020 18:43:32 GMT
  < Content-Type: text/plain; charset=utf-8
  < Content-Length: 14
  < Access-Control-Allow-Origin: *
  < Via: 1.1 google
  <
• Connection #0 to host ifconfig.me left intact
  other.end.wan.ip

Please post your opevpn firewall rules/and or any rules on specific client interface (if exists)

[throwaway29]

I have been having this EXACT same problem for the past year. I haven't been able to figure out why the pfsense machine won't curl out the interface using the VPN.

I suspect this is an NAT Outbound issue... but nothing I do there has fixed it so far. I have manual rules setup for my Outbound NAT.

This whole issue prevents my script running on pfsense using curl to utilize my VPN. It's very annoying. For a while I simply used the pull routes option from the VPN and then my script worked but everything then went out the VPN from my shell that wasn't specifically setup otherwise. I had DNS going out the VPN so much though that I eventually reverted and decided to stick with the more secure crippled version.

[backL]

This post is deleted!