Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Testing some good Regex expressions for use with Squid Proxy and custom spliced URL lists.

    Cache/Proxy
    2
    5
    495
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JonathanLee
      JonathanLee last edited by JonathanLee

      Hello fellow Netgate Community,

      I put together some small regular expressions you can use if you want they are working great for me.

      This is for use with Roblox and its use with multiple t and c and apis servers for rbxcdn.com and the need to splice them. I got tired of entering multiple urls for urls that change one digit or have t or a c in them.

      Screenshot 2023-02-05 at 5.04.02 PM.png
      I have a custom URL spliced file,

      acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.nobump"

      Screenshot 2023-02-05 at 4.49.41 PM.png
      (use with custom file)

      Roblox:
      ^(apis|((t|c)[0-9])).rbxcdn.com

      Disney Plus and the multiple bamgrid.com usage:

      ^(disney.(content|connections)).edge.bamgrid.com

      Android smartphone push notifications use while on wifi:

      ^((alt)([0-9])(-))?mtalk.google.com

      ^(((clients)[0-9])|accounts).google.(com|us)

      ^(crl|ocsp).pki.google.com

      Use of these regular expressions clears up a lot of entries where you need to splice and not intercept.

      JonathanLee 1 Reply Last reply Reply Quote 1
      • JonathanLee
        JonathanLee @JonathanLee last edited by

        @jonathanlee
        Screenshot 2023-02-05 at 5.10.24 PM.png

        if you want, test out your own.

        M 1 Reply Last reply Reply Quote 0
        • M
          michmoor @JonathanLee last edited by

          @jonathanlee good job!

          Firewall: NetGate 6100/7100U, Palo Alto
          Routing: Juniper MX204 , Arista 7050X3
          Switching: Juniper EX/QFX. Arista 7050SX
          Wireless: Unifi, Aruba IAP

          JonathanLee 1 Reply Last reply Reply Quote 1
          • JonathanLee
            JonathanLee @michmoor last edited by JonathanLee

            @michmoor thanks let me know if you have any others 😊.

            Here is another one that includes more for mtalk.
            ^((alt[0-9]-mtalk.)|(mtalk.)|(mtalk-(staging|dev).))google.com
            works for both alt0-9 and mtalk and mtalk-staging mtalk-dev

            With reflection on BigO notation or speed it takes to process the URL this could be better or worse with respect to just separating the other beginning terms outside of the alt[0-9] terms within this regular expression. Again, that website also provides a time and steps to process counter to help with that. Does it reflect the speed within pfsense? That is something I do not know as some firewalls come with multi cores alongside multiple CPUs.

            JonathanLee 1 Reply Last reply Reply Quote 0
            • JonathanLee
              JonathanLee @JonathanLee last edited by

              @jonathanlee

              Screenshot 2023-02-10 at 6.32.55 PM.png

              I adapted this for testing and set it to stare all because of this statement on their website, "The following configuration obtains SNI by parsing TLS Client Hello (due to a matching peek rule at step1) and then either splices bank connections OR stares at the TLS Server Hello (due to a matching stare rule) and bumps non-bank connections (due to the default bump-after-stare rule)."

              It has a default bump after stare rule, so bump step 3 is not needed
              I am thinking. This also seemed to speed up everything.

              Ref:
              https://wiki.squid-cache.org/Features/SslPeekAndSplice

              1 Reply Last reply Reply Quote 0
              • First post
                Last post