• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

How do I curl though an OpenVPN interface from a script.

Scheduled Pinned Locked Moved OpenVPN
21 Posts 5 Posters 8.8k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • G
    guardian Rebel Alliance
    last edited by Apr 24, 2020, 9:52 PM

    How do I curl though an OpenVPN interface from a script? So far I can only get curl to go out through the WAN.

    If you find my post useful, please give it a thumbs up!
    pfSense 2.7.2-RELEASE

    1 Reply Last reply Reply Quote 0
    • G
      Gertjan
      last edited by Apr 25, 2020, 11:22 AM

      You want select the interface that curl uses ?
      If so, click.

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      G 3 Replies Last reply Jun 5, 2020, 4:09 AM Reply Quote 0
      • G
        guardian Rebel Alliance
        last edited by guardian Apr 27, 2020, 1:25 AM Apr 27, 2020, 1:23 AM

        @Gertjan said in How do I curl though an OpenVPN interface from a script.:

        You want select the interface that curl uses ?
        If so, click.

        Thanks @Gertjan for the reply, unfortunately it doesn't seem to be working. Here is the output from my tests:

        --------------------------------------------------------------------------------
        OpenVPN interface info - Tested/Working 
        --------------------------------------------------------------------------------
        
        # ifconfig ovpnc1
        ovpnc1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
                options=80000<LINKSTATE>
                inet6 fe80::228:1aff:fee0:1004%ovpnc1 prefixlen 64 scopeid 0x27
                inet 10.26.10.6 --> 10.26.10.5 netmask 0xffffffff
                nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                groups: tun openvpn
                Opened by PID 63812
        
        -------------------------------------------------------------------------------
        Working with no interface parameter 
        and 
        When WAN interface explicitly specified.
        -------------------------------------------------------------------------------
        # curl  http://ifconfig.me
        99.254.xxx.xxx
        
        # curl --interface em0 http://ifconfig.me
        99.254.xxx.xxx
        
        -------------------------------------------------------------------------------
        Does not work when OpenVPN interface explicitly specified
        ^C = Process hangs and ^C used to Kill after 30 seconds - 1 minute
        -------------------------------------------------------------------------------
        
        # curl --interface ovpnc1 http://ifconfig.me
        ^C
        
        # curl --interface 10.26.10.6 http://ifconfig.me
        ^C
        
        # curl --interface 10.26.10.5 http://ifconfig.me
        curl: (45) bind failed with errno 49: Can't assign requested address
        -------------------------------------------------------------------------------
        Verbose test 
        -Ping shows IP address of ifconfig.me successfully resolved
        -------------------------------------------------------------------------------
        # ping ifconfig.me
        PING ifconfig.me (216.239.32.21): 56 data bytes
        64 bytes from 216.239.32.21: icmp_seq=0 ttl=56 time=9.799 ms
        ^C
        # curl -v --interface ovpnc1 http://ifconfig.me
        * Expire in 0 ms for 6 (transfer 0x803a94000)
        * Expire in 1 ms for 1 (transfer 0x803a94000)
        * Expire in 0 ms for 1 (transfer 0x803a94000)
        ------------------------------------------------Many similar lines removed
        * Expire in 200 ms for 1 (transfer 0x803a94000)
        * Expire in 200 ms for 1 (transfer 0x803a94000)
        *   Trying 216.239.32.21...
        * TCP_NODELAY set
        * Local Interface ovpnc1 is ip 10.26.10.6 using address family 2
        * Local port: 0
        * Expire in 149659 ms for 3 (transfer 0x803a94000)
        * Expire in 200 ms for 4 (transfer 0x803a94000)
        ^C
        

        If you find my post useful, please give it a thumbs up!
        pfSense 2.7.2-RELEASE

        G 1 Reply Last reply May 1, 2020, 8:48 PM Reply Quote 0
        • G
          guardian Rebel Alliance @guardian
          last edited by May 1, 2020, 8:48 PM

          @guardian Can anyon offer any suggestions as to why curl won't route through the VPN interface? I checked out the ideas in the links reference, but as can be seen by my tests, it didn't work.

          Are there any firewall rules that need to be changed?

          If you find my post useful, please give it a thumbs up!
          pfSense 2.7.2-RELEASE

          G 1 Reply Last reply May 2, 2020, 10:29 AM Reply Quote 0
          • G
            Gertjan @guardian
            last edited by May 2, 2020, 10:29 AM

            @guardian said in How do I curl though an OpenVPN interface from a script.:

            Are there any firewall rules that need to be changed?

            Not the default ones.
            Did you create ones yourself ? In that case, maybe. Can't tell.

            Something like

            curl --interface ovpnc1 http://ifconfig.me
            

            will use port 80 by default (to be checked) and is using TCP.
            That's is exactly what a browsers uses.

            No "help me" PM's please. Use the forum, the community will thank you.
            Edit : and where are the logs ??

            G 1 Reply Last reply May 7, 2020, 10:09 AM Reply Quote 0
            • G
              guardian Rebel Alliance @Gertjan
              last edited by May 7, 2020, 10:09 AM

              @Gertjan said in How do I curl though an OpenVPN interface from a script.:

              curl --interface ovpnc1 http://ifconfig.me

              @Gertjan thanks for the reply -- that is exactly what I tried, but for some reason it doesn't work... the command just hangs. Any idea how to troubleshoot?

              If you find my post useful, please give it a thumbs up!
              pfSense 2.7.2-RELEASE

              G 1 Reply Last reply Jun 19, 2020, 2:00 AM Reply Quote 0
              • G
                guardian Rebel Alliance @Gertjan
                last edited by Jun 5, 2020, 4:09 AM

                @Gertjan said in How do I curl though an OpenVPN interface from a script.:

                You want select the interface that curl uses ?
                If so, click.

                I checked this out... the test results are posted above.

                Any other ideas on possible steps to take to break down/troubleshoot what is going on wpi;d be very much appreciated.

                If you find my post useful, please give it a thumbs up!
                pfSense 2.7.2-RELEASE

                1 Reply Last reply Reply Quote 0
                • G
                  guardian Rebel Alliance @guardian
                  last edited by Jun 19, 2020, 2:00 AM

                  I put this on the back burner because I'm stuck, but I'm still looking for a solution. Any assistance would be much appreciated.

                  If you find my post useful, please give it a thumbs up!
                  pfSense 2.7.2-RELEASE

                  1 Reply Last reply Reply Quote 0
                  • G
                    Gertjan
                    last edited by Jun 19, 2020, 6:01 AM

                    Use curl like this :

                    curl -v --interface em3 https://www.test-domaine.fr/phpinfo.php
                    

                    where --interface is the interface name , em3 in my case. To be replaced by 'ovpnc1 ' if that's your OpenVPN client interface name.

                    You wind up having a lot of details - as the -v option means

                    curl --help | grep 'verbose'
                         --trace-time    Add time stamps to trace/verbose output
                     -v, --verbose       Make the operation more talkative
                    

                    No "help me" PM's please. Use the forum, the community will thank you.
                    Edit : and where are the logs ??

                    G 1 Reply Last reply Jun 19, 2020, 9:46 AM Reply Quote 0
                    • G
                      guardian Rebel Alliance @Gertjan
                      last edited by Jun 19, 2020, 9:46 AM

                      @Gertjan Thanks for the reply... I'm wondering if the issue is a routing problem.

                      Note that the system is working as intended, I have 2 VLANs that use policy routing to send all traffic out the VPN.

                      All other VPNs go directly out the WAN directly. The VPNs function normally, the gateway pinger works fine. What confuses me is that there are no rules in the interface tab and I get this messge in the system log

                      Jun 19 03:44:56 --------- php-fpm[78492]: /rc.start_packages: GATEWAYS: Group -------- did not have any gateways up on tier 1!
                      

                      This is nothing new, I've been getting this message for years!), but everything works except that I can't curl over the VPN. I assume the reason why I don't need any rules on gateway interface is that those rules must be for unsolicited (no open connection) incomming connections from the VPN.

                      Here is my output from the tests:

                      Test run on WAN Interface

                      curl -v --interface em0 http://wttr.in
                      *   Trying 5.9.243.187:80...
                      * TCP_NODELAY set
                      * Local Interface em0 is ip xxx.xxx.xxx.xxx using address family 2
                      * Local port: 0
                      * Connected to wttr.in (5.9.243.187) port 80 (#0)
                      > GET / HTTP/1.1
                      > Host: wttr.in
                      > User-Agent: curl/7.67.0
                      > Accept: */*
                      > 
                      * Mark bundle as not supporting multiuse
                      < HTTP/1.1 200 OK
                      < Server: nginx/1.10.3
                      < Date: Fri, 19 Jun 2020 06:25:15 GMT
                      < Content-Type: text/plain; charset=utf-8
                      < Content-Length: 8940
                      < Connection: keep-alive
                      < Access-Control-Allow-Origin: *
                      < 
                      Weather report: ... Output as expected
                      

                      Test run on ovpnc1

                      curl -v --interface ovpnc1 http://wttr.in
                      *   Trying 5.9.243.187:80...
                      * TCP_NODELAY set
                      * Local Interface ovpnc1 is ip 10.28.10.10 using address family 2
                      * Local port: 0
                      * connect to 5.9.243.187 port 80 failed: Operation timed out
                      * Failed to connect to wttr.in port 80: Operation timed out
                      * Closing connection 0
                      curl: (28) Failed to connect to wttr.in port 80: Operation timed out
                      

                      Any suggestions are much appreciated.

                      If you find my post useful, please give it a thumbs up!
                      pfSense 2.7.2-RELEASE

                      1 Reply Last reply Reply Quote 0
                      • G
                        guardian Rebel Alliance @Gertjan
                        last edited by guardian Jul 23, 2020, 9:07 AM Jul 23, 2020, 9:06 AM

                        @Gertjan said in How do I curl though an OpenVPN interface from a script.:

                        You want select the interface that curl uses ?
                        If so, click.

                        I looked at this and I think that I've followed the directions, but it still doesn't work; Any idea what I am doing wrong?

                        This works (no interface specified)

                        curl -4  http://www.google.com
                        

                        This hangs and times out:

                        curl -v -4 --interface ovpnc1 http://www.google.com
                        *   Trying 172.217.164.196:80...
                        * TCP_NODELAY set
                        * Local Interface ovpnc1 is ip 10.52.10.10 using address family 2
                        * Local port: 0
                        * connect to 172.217.164.196 port 80 failed: Operation timed out
                        * Failed to connect to www.google.com port 80: Operation timed out
                        * Closing connection 0
                        curl: (28) Failed to connect to www.google.com port 80: Operation timed out
                        

                        This is the OpenVPN interface:

                        ifconfig ovpnc1
                        ovpnc1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
                        	options=80000<LINKSTATE>
                        	inet6 fe80::228:1aff:fee0:1004%ovpnc1 prefixlen 64 scopeid 0x27
                        	inet 10.52.10.10 --> 10.52.10.9 netmask 0xffffffff
                        	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                        	groups: tun openvpn
                        	Opened by PID 40301
                        

                        So what am I doing wrong?

                        If you find my post useful, please give it a thumbs up!
                        pfSense 2.7.2-RELEASE

                        N 1 Reply Last reply Jul 23, 2020, 9:29 AM Reply Quote 0
                        • N
                          netblues @guardian
                          last edited by Jul 23, 2020, 9:29 AM

                          @guardian In order for this to work, the other end of the vpn must support natting to the internet.

                          Here is the exact same command

                          /root: curl -v -4 --interface ovpnc1 http://www.google.com

                          • Trying 172.217.22.4:80...
                          • TCP_NODELAY set
                          • Local Interface ovpnc1 is ip 192.168.127.5 using address family 2
                          • Local port: 0
                          • Connected to www.google.com (172.217.22.4) port 80 (#0)

                          GET / HTTP/1.1
                          Host: www.google.com
                          User-Agent: curl/7.67.0
                          Accept: /

                          • Mark bundle as not supporting multiuse
                            < HTTP/1.1 200 OK
                            < Date: Thu, 23 Jul 2020 09:25:51 GMT
                            < Expires: -1
                            < Cache-Control: private, max-age=0
                            < Content-Type: text/html; charset=ISO-8859-1
                            < P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
                            < Server: gws
                            < X-XSS-Protection: 0
                            < X-Frame-Options: SAMEORIGIN
                            < Set-Cookie: 1P_JAR=2020-07-23-09; expires=Sat, 22-Aug-2020 09:25:51 GMT; path=/; domain=.google.com; Secure
                            < Set-Cookie: NID=204=VGdCUajKT8B5UcexExeV2Km1ye5cs22px0VMjGalBW2Y7qhYMYuw4ty83dvPKnan6gXumpG4fqDlpnl7_2dEFVPE2SZvrNSDambfGuA5YOhLau9C65DK_nBgeZmVgtvD1t5XGwWdvy6sBGNbSz6k-NUvnCSzNF8cc2kGpl61Nyg; expires=Fri, 22-Jan-2021 09:25:51 GMT; path=/; domain=.google.com; HttpOnly
                            < Accept-Ranges: none
                            < Vary: Accept-Encoding
                            < Transfer-Encoding: chunked
                            <
                          1 Reply Last reply Reply Quote 0
                          • G
                            guardian Rebel Alliance @Gertjan
                            last edited by Jul 24, 2020, 7:35 AM

                            @Gertjan said in How do I curl though an OpenVPN interface from a script.:

                            You want select the interface that curl uses ?
                            If so, click.

                            @Gertjan I tried everything that I could see in the link you mentioned, but it didn't work as you can see from the post above.

                            Any idea how I can troubleshoot?

                            If you find my post useful, please give it a thumbs up!
                            pfSense 2.7.2-RELEASE

                            N 1 Reply Last reply Jul 24, 2020, 8:22 AM Reply Quote 0
                            • N
                              netblues @guardian
                              last edited by Jul 24, 2020, 8:22 AM

                              @guardian Your curl command is fine. You have routing/natting issues.

                              G 1 Reply Last reply Jul 25, 2020, 8:17 AM Reply Quote 0
                              • G
                                guardian Rebel Alliance @netblues
                                last edited by Jul 25, 2020, 8:17 AM

                                @netblues said in How do I curl though an OpenVPN interface from a script.:

                                @guardian Your curl command is fine. You have routing/natting issues.

                                Can someone give me some steps to troubleshoot this?

                                It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.

                                Any advice would be much appreciated.

                                If you find my post useful, please give it a thumbs up!
                                pfSense 2.7.2-RELEASE

                                GertjanG 1 Reply Last reply Jul 25, 2020, 8:41 AM Reply Quote 0
                                • GertjanG
                                  Gertjan @guardian
                                  last edited by Jul 25, 2020, 8:41 AM

                                  @guardian said in How do I curl though an OpenVPN interface from a script.:

                                  It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.

                                  Your 'curl' is using that rule 'that matches all traffic' and that firewall rule is probably NOT logging.
                                  That why you see no logs.
                                  Be careful : activating logging for the default pass all rule will log huge numbers of log lines at lightning speed.

                                  No "help me" PM's please. Use the forum, the community will thank you.
                                  Edit : and where are the logs ??

                                  N 1 Reply Last reply Jul 25, 2020, 9:08 AM Reply Quote 0
                                  • N
                                    netblues @Gertjan
                                    last edited by Jul 25, 2020, 9:08 AM

                                    I doubt you will find anything on pf logs.
                                    The thing is, that when you select the vpn client interface, you send your request inside the tunnel to the device at the other end.

                                    If you curl a service running on the host on the other end of the tunnel, then it would work.
                                    If however you ask for e.g. google, then the device at the other end of the tunnel has to do nat forward the request to google , get the reply and translate it back to you.

                                    So what is at the other end of the tunnel interface you are curling into?

                                    G 1 Reply Last reply Jul 25, 2020, 5:44 PM Reply Quote 1
                                    • G
                                      guardian Rebel Alliance @netblues
                                      last edited by Jul 25, 2020, 5:44 PM

                                      Thanks @Gertjan @netblues The tunnel is a public VPN service with minimal filtering. I'm pretty sure it has to do with rules/routing regarding the curl command. My resaon for saying this is that I have a guest wifi network that is policy routed out that interface, and the same command works (except it doesn't have the --interface parameter).

                                      If we assume the prblem is local, what would be my next step(s) to troubleshoot?

                                      @Gertjan said in How do I curl though an OpenVPN interface from a script.:

                                      @guardian said in How do I curl though an OpenVPN interface from a script.:

                                      It doesn't seem to generate any firewall log entries and I have a an allow all IPv4 on the OpenVPN firewall rules tab.

                                      Your 'curl' is using that rule 'that matches all traffic' and that firewall rule is probably NOT logging.
                                      That why you see no logs.
                                      Be careful : activating logging for the default pass all rule will log huge numbers of log lines at lightning speed.

                                      curl -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -v --interface ovpnc1 'http://ifconfig.me/ip'
                                      @netblues said in How do I curl though an OpenVPN interface from a script.:

                                      I doubt you will find anything on pf logs.
                                      The thing is, that when you select the vpn client interface, you send your request inside the tunnel to the device at the other end.

                                      If you curl a service running on the host on the other end of the tunnel, then it would work.
                                      If however you ask for e.g. google, then the device at the other end of the tunnel has to do nat forward the request to google , get the reply and translate it back to you.

                                      So what is at the other end of the tunnel interface you are curling into?

                                      If you find my post useful, please give it a thumbs up!
                                      pfSense 2.7.2-RELEASE

                                      N 1 Reply Last reply Jul 25, 2020, 6:49 PM Reply Quote 0
                                      • N
                                        netblues @guardian
                                        last edited by Jul 25, 2020, 6:49 PM

                                        @guardian I do have this exact setup in place
                                        so here is the result
                                        /root: curl -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0' -v --interface ovpnc1 'http://ifconfig.me/ip'

                                        • Trying 216.239.32.21:80...
                                        • TCP_NODELAY set
                                        • Local Interface ovpnc1 is ip 192.168.127.5 using address family 2
                                        • Local port: 0
                                        • Connected to ifconfig.me (216.239.32.21) port 80 (#0)

                                        GET /ip HTTP/1.1
                                        Host: ifconfig.me
                                        Accept: /
                                        User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

                                        • Mark bundle as not supporting multiuse
                                          < HTTP/1.1 200 OK
                                          < Date: Sat, 25 Jul 2020 18:43:32 GMT
                                          < Content-Type: text/plain; charset=utf-8
                                          < Content-Length: 14
                                          < Access-Control-Allow-Origin: *
                                          < Via: 1.1 google
                                          <
                                        • Connection #0 to host ifconfig.me left intact
                                          other.end.wan.ip

                                        Please post your opevpn firewall rules/and or any rules on specific client interface (if exists)

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          throwaway29
                                          last edited by Aug 17, 2020, 7:55 AM

                                          I have been having this EXACT same problem for the past year. I haven't been able to figure out why the pfsense machine won't curl out the interface using the VPN.

                                          I suspect this is an NAT Outbound issue... but nothing I do there has fixed it so far. I have manual rules setup for my Outbound NAT.

                                          This whole issue prevents my script running on pfsense using curl to utilize my VPN. It's very annoying. For a while I simply used the pull routes option from the VPN and then my script worked but everything then went out the VPN from my shell that wasn't specifically setup otherwise. I had DNS going out the VPN so much though that I eventually reverted and decided to stick with the more secure crippled version.

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            [[user:consent.lead]]
                                            [[user:consent.not_received]]