How do I curl though an OpenVPN interface from a script.
-
You want select the interface that curl uses ?
If so, click. -
@Gertjan said in How do I curl though an OpenVPN interface from a script.:
You want select the interface that curl uses ?
If so, click.Thanks @Gertjan for the reply, unfortunately it doesn't seem to be working. Here is the output from my tests:
-------------------------------------------------------------------------------- OpenVPN interface info - Tested/Working -------------------------------------------------------------------------------- # ifconfig ovpnc1 ovpnc1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> inet6 fe80::228:1aff:fee0:1004%ovpnc1 prefixlen 64 scopeid 0x27 inet 10.26.10.6 --> 10.26.10.5 netmask 0xffffffff nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: tun openvpn Opened by PID 63812 ------------------------------------------------------------------------------- Working with no interface parameter and When WAN interface explicitly specified. ------------------------------------------------------------------------------- # curl http://ifconfig.me 99.254.xxx.xxx # curl --interface em0 http://ifconfig.me 99.254.xxx.xxx ------------------------------------------------------------------------------- Does not work when OpenVPN interface explicitly specified ^C = Process hangs and ^C used to Kill after 30 seconds - 1 minute ------------------------------------------------------------------------------- # curl --interface ovpnc1 http://ifconfig.me ^C # curl --interface 10.26.10.6 http://ifconfig.me ^C # curl --interface 10.26.10.5 http://ifconfig.me curl: (45) bind failed with errno 49: Can't assign requested address ------------------------------------------------------------------------------- Verbose test -Ping shows IP address of ifconfig.me successfully resolved ------------------------------------------------------------------------------- # ping ifconfig.me PING ifconfig.me (216.239.32.21): 56 data bytes 64 bytes from 216.239.32.21: icmp_seq=0 ttl=56 time=9.799 ms ^C # curl -v --interface ovpnc1 http://ifconfig.me * Expire in 0 ms for 6 (transfer 0x803a94000) * Expire in 1 ms for 1 (transfer 0x803a94000) * Expire in 0 ms for 1 (transfer 0x803a94000) ------------------------------------------------Many similar lines removed * Expire in 200 ms for 1 (transfer 0x803a94000) * Expire in 200 ms for 1 (transfer 0x803a94000) * Trying 216.239.32.21... * TCP_NODELAY set * Local Interface ovpnc1 is ip 10.26.10.6 using address family 2 * Local port: 0 * Expire in 149659 ms for 3 (transfer 0x803a94000) * Expire in 200 ms for 4 (transfer 0x803a94000) ^C -
@guardian Can anyon offer any suggestions as to why curl won't route through the VPN interface? I checked out the ideas in the links reference, but as can be seen by my tests, it didn't work.
Are there any firewall rules that need to be changed?
-
@guardian said in How do I curl though an OpenVPN interface from a script.:
Are there any firewall rules that need to be changed?
Not the default ones.
Did you create ones yourself ? In that case, maybe. Can't tell.Something like
curl --interface ovpnc1 http://ifconfig.mewill use port 80 by default (to be checked) and is using TCP.
That's is exactly what a browsers uses. -
@Gertjan said in How do I curl though an OpenVPN interface from a script.:
curl --interface ovpnc1 http://ifconfig.me
@Gertjan thanks for the reply -- that is exactly what I tried, but for some reason it doesn't work... the command just hangs. Any idea how to troubleshoot?
-
@Gertjan said in How do I curl though an OpenVPN interface from a script.:
You want select the interface that curl uses ?
If so, click.I checked this out... the test results are posted above.
Any other ideas on possible steps to take to break down/troubleshoot what is going on wpi;d be very much appreciated.
-
I put this on the back burner because I'm stuck, but I'm still looking for a solution. Any assistance would be much appreciated.
-
Use curl like this :
curl -v --interface em3 https://www.test-domaine.fr/phpinfo.phpwhere --interface is the interface name , em3 in my case. To be replaced by 'ovpnc1 ' if that's your OpenVPN client interface name.
You wind up having a lot of details - as the -v option means
curl --help | grep 'verbose' --trace-time Add time stamps to trace/verbose output -v, --verbose Make the operation more talkative -
@Gertjan Thanks for the reply... I'm wondering if the issue is a routing problem.
Note that the system is working as intended, I have 2 VLANs that use policy routing to send all traffic out the VPN.
All other VPNs go directly out the WAN directly. The VPNs function normally, the gateway pinger works fine. What confuses me is that there are no rules in the interface tab and I get this messge in the system log
Jun 19 03:44:56 --------- php-fpm[78492]: /rc.start_packages: GATEWAYS: Group -------- did not have any gateways up on tier 1!This is nothing new, I've been getting this message for years!), but everything works except that I can't curl over the VPN. I assume the reason why I don't need any rules on gateway interface is that those rules must be for unsolicited (no open connection) incomming connections from the VPN.
Here is my output from the tests:
Test run on WAN Interface
curl -v --interface em0 http://wttr.in * Trying 5.9.243.187:80... * TCP_NODELAY set * Local Interface em0 is ip xxx.xxx.xxx.xxx using address family 2 * Local port: 0 * Connected to wttr.in (5.9.243.187) port 80 (#0) > GET / HTTP/1.1 > Host: wttr.in > User-Agent: curl/7.67.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Server: nginx/1.10.3 < Date: Fri, 19 Jun 2020 06:25:15 GMT < Content-Type: text/plain; charset=utf-8 < Content-Length: 8940 < Connection: keep-alive < Access-Control-Allow-Origin: * < Weather report: ... Output as expectedTest run on ovpnc1
curl -v --interface ovpnc1 http://wttr.in * Trying 5.9.243.187:80... * TCP_NODELAY set * Local Interface ovpnc1 is ip 10.28.10.10 using address family 2 * Local port: 0 * connect to 5.9.243.187 port 80 failed: Operation timed out * Failed to connect to wttr.in port 80: Operation timed out * Closing connection 0 curl: (28) Failed to connect to wttr.in port 80: Operation timed outAny suggestions are much appreciated.
-
@Gertjan said in How do I curl though an OpenVPN interface from a script.:
You want select the interface that curl uses ?
If so, click.I looked at this and I think that I've followed the directions, but it still doesn't work; Any idea what I am doing wrong?
This works (no interface specified)
curl -4 http://www.google.comThis hangs and times out:
curl -v -4 --interface ovpnc1 http://www.google.com * Trying 172.217.164.196:80... * TCP_NODELAY set * Local Interface ovpnc1 is ip 10.52.10.10 using address family 2 * Local port: 0 * connect to 172.217.164.196 port 80 failed: Operation timed out * Failed to connect to www.google.com port 80: Operation timed out * Closing connection 0 curl: (28) Failed to connect to www.google.com port 80: Operation timed outThis is the OpenVPN interface:
ifconfig ovpnc1 ovpnc1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> inet6 fe80::228:1aff:fee0:1004%ovpnc1 prefixlen 64 scopeid 0x27 inet 10.52.10.10 --> 10.52.10.9 netmask 0xffffffff nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: tun openvpn Opened by PID 40301So what am I doing wrong?
-
@guardian In order for this to work, the other end of the vpn must support natting to the internet.
Here is the exact same command
/root: curl -v -4 --interface ovpnc1 http://www.google.com
- Trying 172.217.22.4:80...
- TCP_NODELAY set
- Local Interface ovpnc1 is ip 192.168.127.5 using address family 2
- Local port: 0
- Connected to www.google.com (172.217.22.4) port 80 (#0)
GET / HTTP/1.1
Host: www.google.com
User-Agent: curl/7.67.0
Accept: /- Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 23 Jul 2020 09:25:51 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< Server: gws
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: 1P_JAR=2020-07-23-09; expires=Sat, 22-Aug-2020 09:25:51 GMT; path=/; domain=.google.com; Secure
< Set-Cookie: NID=204=VGdCUajKT8B5UcexExeV2Km1ye5cs22px0VMjGalBW2Y7qhYMYuw4ty83dvPKnan6gXumpG4fqDlpnl7_2dEFVPE2SZvrNSDambfGuA5YOhLau9C65DK_nBgeZmVgtvD1t5XGwWdvy6sBGNbSz6k-NUvnCSzNF8cc2kGpl61Nyg; expires=Fri, 22-Jan-2021 09:25:51 GMT; path=/; domain=.google.com; HttpOnly
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
<
-
@Gertjan said in How do I curl though an OpenVPN interface from a script.:
You want select the interface that curl uses ?
If so, click.@Gertjan I tried everything that I could see in the link you mentioned, but it didn't work as you can see from the post above.
Any idea how I can troubleshoot?
-
@guardian Your curl command is fine. You have routing/natting issues.
-
@netblues said in How do I curl though an OpenVPN interface from a script.:
@guardian Your curl command is fine. You have routing/natting issues.
Can someone give me some steps to troubleshoot this?
It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.
Any advice would be much appreciated.
-
@guardian said in How do I curl though an OpenVPN interface from a script.:
It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.
Your 'curl' is using that rule 'that matches all traffic' and that firewall rule is probably NOT logging.
That why you see no logs.
Be careful : activating logging for the default pass all rule will log huge numbers of log lines at lightning speed. -
I doubt you will find anything on pf logs.
The thing is, that when you select the vpn client interface, you send your request inside the tunnel to the device at the other end.If you curl a service running on the host on the other end of the tunnel, then it would work.
If however you ask for e.g. google, then the device at the other end of the tunnel has to do nat forward the request to google , get the reply and translate it back to you.So what is at the other end of the tunnel interface you are curling into?
-
Thanks @Gertjan @netblues The tunnel is a public VPN service with minimal filtering. I'm pretty sure it has to do with rules/routing regarding the curl command. My resaon for saying this is that I have a guest wifi network that is policy routed out that interface, and the same command works (except it doesn't have the --interface parameter).
If we assume the prblem is local, what would be my next step(s) to troubleshoot?
@Gertjan said in How do I curl though an OpenVPN interface from a script.:
@guardian said in How do I curl though an OpenVPN interface from a script.:
It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.
Your 'curl' is using that rule 'that matches all traffic' and that firewall rule is probably NOT logging.
That why you see no logs.
Be careful : activating logging for the default pass all rule will log huge numbers of log lines at lightning speed.curl -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -v --interface ovpnc1 'http://ifconfig.me/ip'
@netblues said in How do I curl though an OpenVPN interface from a script.:I doubt you will find anything on pf logs.
The thing is, that when you select the vpn client interface, you send your request inside the tunnel to the device at the other end.If you curl a service running on the host on the other end of the tunnel, then it would work.
If however you ask for e.g. google, then the device at the other end of the tunnel has to do nat forward the request to google , get the reply and translate it back to you.So what is at the other end of the tunnel interface you are curling into?
-
@guardian I do have this exact setup in place
so here is the result
/root: curl -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -v --interface ovpnc1 'http://ifconfig.me/ip'- Trying 216.239.32.21:80...
- TCP_NODELAY set
- Local Interface ovpnc1 is ip 192.168.127.5 using address family 2
- Local port: 0
- Connected to ifconfig.me (216.239.32.21) port 80 (#0)
GET /ip HTTP/1.1
Host: ifconfig.me
Accept: /
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0- Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Sat, 25 Jul 2020 18:43:32 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 14
< Access-Control-Allow-Origin: *
< Via: 1.1 google
< - Connection #0 to host ifconfig.me left intact
other.end.wan.ip
Please post your opevpn firewall rules/and or any rules on specific client interface (if exists)
-
I have been having this EXACT same problem for the past year. I haven't been able to figure out why the pfsense machine won't curl out the interface using the VPN.
I suspect this is an NAT Outbound issue... but nothing I do there has fixed it so far. I have manual rules setup for my Outbound NAT.
This whole issue prevents my script running on pfsense using curl to utilize my VPN. It's very annoying. For a while I simply used the pull routes option from the VPN and then my script worked but everything then went out the VPN from my shell that wasn't specifically setup otherwise. I had DNS going out the VPN so much though that I eventually reverted and decided to stick with the more secure crippled version.
-
This post is deleted!
