Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    What's the trick to matching on DSCP? [Answer: Statefulness]

    Scheduled Pinned Locked Moved Traffic Shaping
    10 Posts 4 Posters 2.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ?
      A Former User
      last edited by

      UPDATE: I totally misunderstood what was happening here.  The initial SSH packet didn't have the DSCP marking I wanted and of course, once the initial packet goes thru the state is setup and then DSCP isn't examined again.  For any other newbies reading this, pfSense ONLY CHECKS THE DSCP OF THE INITIAL STATE SETUP PACKET! So if you setup a state with 0x00 and THEN start sending traffic marked as EF, well that EF rule is never going to match.  This is just how stateful firewalls work.  Similarly, if your first packet is EF but every packet past that isn't, well every packet passing will be subject to your EF rule.
      End Update…

      Hi,

      I'm trying to match on DSCP so that my scp file transfers don't nuke my upstream, but I want my normal SSH traffic to be interactive.

      I have the following floating rules:

      I match anything with a Destination Port of 22 to be in my "High" queue.  This works great.

      Then I setup another rule further down to match Destination Port of 22, but only if the destination host was the IP I want to copy files to, to put traffic in my Bulk queue.  This worked great too, but it means that all SSH/SCP traffic to this host goes into my "Bulk" queue.  Not the best because I want my regular SSH to be in my High queue, but SCP traffic to be in Bulk.

      So I thought I'll get clever and match on the fact that SCP sends traffic marked with ToS 0x8.

      So I edited my second rule I described above to only send traffic into Bulk queue if it's marked with CS1.  But it doesn't work, when I apply the rule and then look at "pftop -v queue -s1" I see all the Bulk counters at zero.

      My first thought is "I'm not sending traffic marked properly" but looking at my vtnet1 interface (LAN interface that the host is connected to) I see the following dumps:

      
18:04:51.864681 IP (tos 0x8, ttl 56, id 57637, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.51488: Flags [F.], cksum 0x2841 (correct), seq 2836, ack 91674, win 1477, options [nop,nop,TS val 4186855853 ecr 3455906274], length 0
18:04:51.864884 IP (tos 0x8, ttl 64, id 65275, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.51488 > 205.233.254.38.ssh: Flags [.], cksum 0x2cc4 (correct), seq 91674, ack 2837, win 296, options [nop,nop,TS val 3455906300 ecr 4186855853], length 0

      So why won't my pfSense match on it?  If I remove the DSCP marking rule, it works again.

      Are there limitations of DSCP matching I'm not aware of, such as not working if src/dst is selected, or a port is selected, or can it not be used in a floating rule?
      Or am I doing something else boneheaded?

      I have tried all the usual things of ensuring that there's no sessions already established etc.  But there isn't, these are are SCP sessions using TCP and as soon the SCP is finished the session is torn down.  As soon as I remove the DSCP marking rule, I start seeing traffic in Bulk again when I restart my test, so I'm sure my test is valid.

      PfSense version 2.4.2-RELEASE-p1 (amd64) running in a Proxmox Virtual Machine.

      1 Reply Last reply Reply Quote 0
      • DerelictD
        Derelict LAYER 8 Netgate
        last edited by

        Hmm. What kind of floating rules? How about you post them?

        Chattanooga, Tennessee, USA
        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
        Do Not Chat For Help! NO_WAN_EGRESS(TM)

        1 Reply Last reply Reply Quote 0
        • ?
          A Former User
          last edited by

          Here's the rules from /tmp/rules.debug that work correctly:

          
anchor "userrules/*"
pass  on {  pppoe0  } inet proto { tcp udp }  from any to $2Talk tracker 1517387427 keep state  queue (Priority)  label "USER_RULE: Voice Traffic"
pass inet proto udp  from any to any port 53 tracker 1517388091 keep state  queue (VeryHigh)  label "USER_RULE: DNS Traffic"
pass inet proto tcp  from any to any port 22 tracker 1517390346 flags S/SA keep state  queue (High)  label "USER_RULE: SSH Traffic"
pass inet proto tcp  from any to any port 5000 >< 5003 tracker 1519355311 flags S/SA keep state  queue (High)  label "USER_RULE: IceCast Traffic"
pass inet proto tcp  from any to any port 80 tracker 1517388036 flags S/SA keep state  queue (Medium)  label "USER_RULE: Web Traffic"
pass inet proto tcp  from any to any port 443 tracker 1517388051 flags S/SA keep state  queue (Medium)  label "USER_RULE: Web Traffic"
pass inet proto tcp  from any to any port 8080 tracker 1517388891 flags S/SA keep state  queue (Medium)  label "USER_RULE: Netflix"
pass inet proto tcp  from any to $mu tracker 1517389952 flags S/SA keep state  queue (Medium)  label "USER_RULE: Prophecy External Services"
pass inet proto udp  from any to any port 1194 tracker 1517390097 keep state  queue (Medium)  label "USER_RULE: OpenVPN Traffic"
pass inet proto tcp  from any to $animal port 22 tracker 1519336221 flags S/SA keep state  queue (Bulk)  label "USER_RULE: Bulk SCP Traffic"
pass inet proto tcp  from any to $micro port 9100 >< 9104 tracker 1519340474 flags S/SA keep state  queue (Bulk)  label "USER_RULE: Bacula Traffic"

          See the section to last rule there?  It matches traffic to animal on port 22 and puts it in Bulk.

          Here's the rule now altered where I look for DSCP CS1, but the rule is never matched:

          
anchor "userrules/*"
pass  on {  pppoe0  } inet proto { tcp udp }  from any to $2Talk tracker 1517387427 keep state  queue (Priority)  label "USER_RULE: Voice Traffic"
pass inet proto udp  from any to any port 53 tracker 1517388091 keep state  queue (VeryHigh)  label "USER_RULE: DNS Traffic"
pass inet proto tcp  from any to any port 22 tracker 1517390346 flags S/SA keep state  queue (High)  label "USER_RULE: SSH Traffic"
pass inet proto tcp  from any to any port 5000 >< 5003 tracker 1519355311 flags S/SA keep state  queue (High)  label "USER_RULE: IceCast Traffic"
pass inet proto tcp  from any to any port 80 tracker 1517388036 flags S/SA keep state  queue (Medium)  label "USER_RULE: Web Traffic"
pass inet proto tcp  from any to any port 443 tracker 1517388051 flags S/SA keep state  queue (Medium)  label "USER_RULE: Web Traffic"
pass inet proto tcp  from any to any port 8080 tracker 1517388891 flags S/SA keep state  queue (Medium)  label "USER_RULE: Netflix"
pass inet proto tcp  from any to $mu tracker 1517389952 flags S/SA keep state  queue (Medium)  label "USER_RULE: Prophecy External Services"
pass inet proto udp  from any to any port 1194 tracker 1517390097 keep state  queue (Medium)  label "USER_RULE: OpenVPN Traffic"
pass inet proto tcp  from any to $animal port 22  dscp "8" tracker 1519336221 flags S/SA keep state  queue (Bulk)  label "USER_RULE: Bulk SCP Traffic"
pass inet proto tcp  from any to $micro port 9100 >< 9104 tracker 1519340474 flags S/SA keep state  queue (Bulk)  label "USER_RULE: Bacula Traffic"

          Again, it's the second to last rule, just this time it also has that match for DSCP in there.

          Thanks!

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            Hmm. I really don't like all those pass rules. For instance:

            pass inet proto tcp  from any to any port 80 tracker 1517388036 flags S/SA keep state  queue (Medium)  label "USER_RULE: Web Traffic"
            pass inet proto tcp  from any to any port 443 tracker 1517388051 flags S/SA keep state  queue (Medium)  label "USER_RULE: Web Traffic"
            pass inet proto tcp  from any to any port 8080 tracker 1517388891 flags S/SA keep state  queue (Medium)  label "USER_RULE: Netflix"

            All of those rules pass all of that traffic into WAN to any destination because they are on any interface in any direction. They are not quick but they will apply into WAN unless explicitly blocked later.

            I would use match rules to get the traffic into queues.

            If you are actually trying to shape inbound connections, I would set those queues on the WAN rules that pass the traffic inbound. Not on floating rules on any interface in any direction.

            As to the matching of the traffic, I would look at the actual states created. Start an scp session to $animal. Replace $animal with the actual address here:

            pfctl -vvss | grep -A 3 $animal

            That will show the rule that actually created the state.

            You can then look at the rule set.

            Example:

            lagg0.27 tcp 172.22.65.40:443 <- 172.21.17.223:53862      ESTABLISHED:ESTABLISHED
              [4028622099 + 63839]  [2827568173 + 41808]
              age 36:54:41, expires in 23:59:52, 49468:87001 pkts, 7928748:83412443 bytes, rule 430
              id: 030000005b237f67 creatorid: a50e3ea2

            pfctl -vvsr | grep '^@__430__'

            @430(1461729127) pass in quick on lagg0.27 inet from 172.21.17.192/26 to any flags S/SA keep state label "USER_RULE"

            Be sure that is the floating rule you want. If not figure out why the rule that is creating the state is matching instead.

            It looks like what you have should be working. Have to figure out why it isn't.

            I don't have time to lab this right now. Sorry.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • ?
              A Former User
              last edited by

              Thank you for such a detailed reply, it's really given me a bunch of stuff to work on and try!
              I will spend a couple of days myself looking into it, then reply back hopefully with the answer.

              Thanks again for the details, really really appreciate it.

              Tim

              1 Reply Last reply Reply Quote 0
              • ?
                A Former User
                last edited by

                Ok so I think I've figured out my problem here.

                This is my rule I'm testing with:

                @53(1519336221) match log inet from any to animal:1dscp 0x38 label "USER_RULE: Bulk SCP Traffic" queue qBulk

                A ping from a host like so:

                tim@micro:~$ ping -Q 56 animal
PING animal.muppetz.com (205.233.254.38) 56(84) bytes of data.
64 bytes from 205.233.254.38: icmp_seq=1 ttl=56 time=10.7 ms
64 bytes from 205.233.254.38: icmp_seq=2 ttl=56 time=13.4 ms

                Gives me this:

                Mar 19 21:04:15 trogdor filterlog: 53,,,1519336221,vtnet1,match,unkn(%u),in,4,0x38,,64,30858,0,none,1,icmp,84,192.168.0.5,205.233.254.38,request,6614,164
Mar 19 21:04:15 trogdor filterlog: 53,,,1519336221,pppoe0,match,unkn(%u),out,4,0x38,,63,20843,0,none,1,icmp,84,202.137.243.17,205.233.254.38,request,65431,164

                So great, it appears to be working!

                But it doesn't work for a SSH session like this:

                tim@micro:~$ ssh -o IPQoS=0x38 animal

                Nothing is logged.

                But if I look at a FULL TCPDump of that ssh session (my snippet above was half way through the session, doh) we see:

                [2.4.2-RELEASE][admin@trogdor.muppetz.com]/tmp: tcpdump -vv -i vtnet1 tcp port 22 and not host 192.168.0.1
tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:07:00.386060 IP (tos 0x0, ttl 64, id 25351, offset 0, flags [DF], proto TCP (6), length 60)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [s], cksum 0xeb9e (correct), seq 1729564940, win 29200, options [mss 1460,sackOK,TS val 3716706942 ecr 0,nop,wscale 7], length 0
21:07:00.399712 IP (tos 0x0, ttl 56, id 5708, offset 0, flags [none], proto TCP (6), length 60)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [S.], cksum 0x44f5 (correct), seq 1232304240, ack 1729564941, win 28960, options [mss 1460,sackOK,TS val 993831528 ecr 3716706942,nop,wscale 7], length 0
21:07:00.400428 IP (tos 0x0, ttl 64, id 25352, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xe3ef (correct), seq 1, ack 1, win 229, options [nop,nop,TS val 3716706955 ecr 993831528], length 0
21:07:00.400495 IP (tos 0x0, ttl 64, id 25353, offset 0, flags [DF], proto TCP (6), length 92)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0x4a1d (correct), seq 1:41, ack 1, win 229, options [nop,nop,TS val 3716706956 ecr 993831528], length 40
21:07:00.411999 IP (tos 0x0, ttl 56, id 21861, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xe3bc (correct), seq 1, ack 41, win 227, options [nop,nop,TS val 993831540 ecr 3716706956], length 0
21:07:00.448110 IP (tos 0x0, ttl 56, id 49564, offset 0, flags [none], proto TCP (6), length 91)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x4cd1 (correct), seq 1:40, ack 41, win 227, options [nop,nop,TS val 993831577 ecr 3716706956], length 39
21:07:00.448332 IP (tos 0x0, ttl 64, id 25354, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xe33e (correct), seq 41, ack 40, win 229, options [nop,nop,TS val 3716707004 ecr 993831577], length 0
21:07:00.449125 IP (tos 0x0, ttl 64, id 25355, offset 0, flags [DF], proto TCP (6), length 1484)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0x25d8 (correct), seq 41:1473, ack 40, win 229, options [nop,nop,TS val 3716707005 ecr 993831577], length 1432
21:07:00.459741 IP (tos 0x0, ttl 56, id 29788, offset 0, flags [none], proto TCP (6), length 1132)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x60ad (correct), seq 40:1120, ack 41, win 227, options [nop,nop,TS val 993831588 ecr 3716707004], length 1080
21:07:00.459813 IP (tos 0x0, ttl 56, id 15286, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xd94e (correct), seq 1120, ack 1473, win 249, options [nop,nop,TS val 993831588 ecr 3716707005], length 0
21:07:00.464736 IP (tos 0x0, ttl 64, id 25356, offset 0, flags [DF], proto TCP (6), length 100)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0xd33b (correct), seq 1473:1521, ack 1120, win 245, options [nop,nop,TS val 3716707020 ecr 993831588], length 48
21:07:00.475503 IP (tos 0x0, ttl 56, id 15502, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xd8ff (correct), seq 1120, ack 1521, win 249, options [nop,nop,TS val 993831604 ecr 3716707020], length 0
21:07:00.492397 IP (tos 0x0, ttl 56, id 41985, offset 0, flags [none], proto TCP (6), length 472)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xada7 (correct), seq 1120:1540, ack 1521, win 249, options [nop,nop,TS val 993831621 ecr 3716707020], length 420
21:07:00.496197 IP (tos 0x0, ttl 64, id 25357, offset 0, flags [DF], proto TCP (6), length 68)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0xcce4 (correct), seq 1521:1537, ack 1540, win 262, options [nop,nop,TS val 3716707052 ecr 993831621], length 16
21:07:00.554333 IP (tos 0x0, ttl 56, id 64760, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xd6dc (correct), seq 1540, ack 1537, win 249, options [nop,nop,TS val 993831683 ecr 3716707052], length 0
21:07:00.554610 IP (tos 0x0, ttl 64, id 25358, offset 0, flags [DF], proto TCP (6), length 96)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0x05ad (correct), seq 1537:1581, ack 1540, win 262, options [nop,nop,TS val 3716707110 ecr 993831683], length 44
21:07:00.567629 IP (tos 0x0, ttl 56, id 39300, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xd669 (correct), seq 1540, ack 1581, win 249, options [nop,nop,TS val 993831696 ecr 3716707110], length 0
21:07:00.567715 IP (tos 0x0, ttl 56, id 2597, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x1115 (correct), seq 1540:1584, ack 1581, win 249, options [nop,nop,TS val 993831696 ecr 3716707110], length 44
21:07:00.567999 IP (tos 0x0, ttl 64, id 25359, offset 0, flags [DF], proto TCP (6), length 112)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0x6da5 (correct), seq 1581:1641, ack 1584, win 262, options [nop,nop,TS val 3716707124 ecr 993831696], length 60
21:07:00.592754 IP (tos 0x0, ttl 56, id 11919, offset 0, flags [none], proto TCP (6), length 104)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xb3a2 (correct), seq 1584:1636, ack 1641, win 249, options [nop,nop,TS val 993831721 ecr 3716707124], length 52
21:07:00.593095 IP (tos 0x0, ttl 64, id 25360, offset 0, flags [DF], proto TCP (6), length 416)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0xfaaf (correct), seq 1641:2005, ack 1636, win 262, options [nop,nop,TS val 3716707149 ecr 993831721], length 364
21:07:00.605595 IP (tos 0x0, ttl 56, id 34723, offset 0, flags [none], proto TCP (6), length 376)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x95f8 (correct), seq 1636:1960, ack 2005, win 272, options [nop,nop,TS val 993831734 ecr 3716707149], length 324
21:07:00.607715 IP (tos 0x0, ttl 64, id 25361, offset 0, flags [DF], proto TCP (6), length 688)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0xbdf2 (correct), seq 2005:2641, ack 1960, win 279, options [nop,nop,TS val 3716707163 ecr 993831734], length 636
21:07:00.623142 IP (tos 0x0, ttl 56, id 5685, offset 0, flags [none], proto TCP (6), length 80)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x0f49 (correct), seq 1960:1988, ack 2641, win 294, options [nop,nop,TS val 993831752 ecr 3716707163], length 28
21:07:00.623517 IP (tos 0x0, ttl 64, id 25362, offset 0, flags [DF], proto TCP (6), length 164)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0x70ca (correct), seq 2641:2753, ack 1988, win 279, options [nop,nop,TS val 3716707179 ecr 993831752], length 112
21:07:00.684087 IP (tos 0x0, ttl 56, id 23914, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xcf2e (correct), seq 1988, ack 2753, win 294, options [nop,nop,TS val 993831813 ecr 3716707179], length 0
21:07:00.706876 IP (tos 0x0, ttl 56, id 4891, offset 0, flags [none], proto TCP (6), length 552)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xbc98 (correct), seq 1988:2488, ack 2753, win 294, options [nop,nop,TS val 993831835 ecr 3716707179], length 500
21:07:00.749092 IP (tos 0x0, ttl 64, id 25363, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xcca4 (correct), seq 2753, ack 2488, win 296, options [nop,nop,TS val 3716707305 ecr 993831835], length 0
21:07:00.759668 IP (tos 0x0, ttl 56, id 58862, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xa49f (correct), seq 2488:2532, ack 2753, win 294, options [nop,nop,TS val 993831888 ecr 3716707305], length 44
21:07:00.760229 IP (tos 0x0, ttl 64, id 25364, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xcc39 (correct), seq 2753, ack 2532, win 296, options [nop,nop,TS val 3716707315 ecr 993831888], length 0
21:07:00.760298 IP (tos 0x38, ttl 64, id 25365, offset 0, flags [DF], proto TCP (6), length 512)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [P.], cksum 0x3d43 (correct), seq 2753:3213, ack 2532, win 296, options [nop,nop,TS val 3716707316 ecr 993831888], length 460
21:07:00.771680 IP (tos 0x0, ttl 56, id 56191, offset 0, flags [none], proto TCP (6), length 52)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [.], cksum 0xca4c (correct), seq 2532, ack 3213, win 316, options [nop,nop,TS val 993831900 ecr 3716707316], length 0
21:07:00.777534 IP (tos 0x10, ttl 56, id 52774, offset 0, flags [none], proto TCP (6), length 160)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x8456 (correct), seq 2532:2640, ack 3213, win 316, options [nop,nop,TS val 993831906 ecr 3716707316], length 108
21:07:00.781437 IP (tos 0x10, ttl 56, id 23766, offset 0, flags [none], proto TCP (6), length 160)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xcf77 (correct), seq 2640:2748, ack 3213, win 316, options [nop,nop,TS val 993831908 ecr 3716707316], length 108
21:07:00.781531 IP (tos 0x10, ttl 56, id 44659, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x49c1 (correct), seq 2748:2784, ack 3213, win 316, options [nop,nop,TS val 993831909 ecr 3716707316], length 36
21:07:00.781605 IP (tos 0x10, ttl 56, id 11619, offset 0, flags [none], proto TCP (6), length 168)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x4b67 (correct), seq 2784:2900, ack 3213, win 316, options [nop,nop,TS val 993831909 ecr 3716707316], length 116
21:07:00.781653 IP (tos 0x10, ttl 56, id 9059, offset 0, flags [none], proto TCP (6), length 160)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x5cd7 (correct), seq 2900:3008, ack 3213, win 316, options [nop,nop,TS val 993831910 ecr 3716707316], length 108
21:07:00.781843 IP (tos 0x10, ttl 56, id 59429, offset 0, flags [none], proto TCP (6), length 136)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xd536 (correct), seq 3008:3092, ack 3213, win 316, options [nop,nop,TS val 993831910 ecr 3716707316], length 84
21:07:00.782269 IP (tos 0x10, ttl 56, id 46271, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x76bd (correct), seq 3092:3128, ack 3213, win 316, options [nop,nop,TS val 993831911 ecr 3716707316], length 36
21:07:00.782291 IP (tos 0x38, ttl 64, id 25366, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc814 (correct), seq 3213, ack 3092, win 296, options [nop,nop,TS val 3716707338 ecr 993831906], length 0
21:07:00.782814 IP (tos 0x10, ttl 56, id 39464, offset 0, flags [none], proto TCP (6), length 160)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x53e6 (correct), seq 3128:3236, ack 3213, win 316, options [nop,nop,TS val 993831911 ecr 3716707316], length 108
21:07:00.783021 IP (tos 0x38, ttl 64, id 25367, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc77e (correct), seq 3213, ack 3236, win 296, options [nop,nop,TS val 3716707339 ecr 993831911], length 0
21:07:00.783318 IP (tos 0x10, ttl 56, id 15367, offset 0, flags [none], proto TCP (6), length 120)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x3453 (correct), seq 3236:3304, ack 3213, win 316, options [nop,nop,TS val 993831912 ecr 3716707316], length 68
21:07:00.783610 IP (tos 0x10, ttl 56, id 45046, offset 0, flags [none], proto TCP (6), length 112)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x1605 (correct), seq 3304:3364, ack 3213, win 316, options [nop,nop,TS val 993831912 ecr 3716707316], length 60
21:07:00.784279 IP (tos 0x38, ttl 64, id 25368, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc6fc (correct), seq 3213, ack 3364, win 296, options [nop,nop,TS val 3716707340 ecr 993831912], length 0
21:07:00.784620 IP (tos 0x10, ttl 56, id 33344, offset 0, flags [none], proto TCP (6), length 152)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x0066 (correct), seq 3364:3464, ack 3213, win 316, options [nop,nop,TS val 993831913 ecr 3716707316], length 100
21:07:00.829154 IP (tos 0x38, ttl 64, id 25369, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc66a (correct), seq 3213, ack 3464, win 296, options [nop,nop,TS val 3716707385 ecr 993831913], length 0
21:07:00.845140 IP (tos 0x10, ttl 56, id 16624, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x6067 (correct), seq 3464:3500, ack 3213, win 316, options [nop,nop,TS val 993831974 ecr 3716707385], length 36
21:07:00.845411 IP (tos 0x38, ttl 64, id 25370, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc5f9 (correct), seq 3213, ack 3500, win 296, options [nop,nop,TS val 3716707401 ecr 993831974], length 0
21:07:00.845595 IP (tos 0x10, ttl 56, id 42749, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x844f (correct), seq 3500:3536, ack 3213, win 316, options [nop,nop,TS val 993831974 ecr 3716707385], length 36
21:07:00.846012 IP (tos 0x10, ttl 56, id 57000, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x97a4 (correct), seq 3536:3572, ack 3213, win 316, options [nop,nop,TS val 993831975 ecr 3716707385], length 36
21:07:00.846317 IP (tos 0x38, ttl 64, id 25371, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc5d4 (correct), seq 3213, ack 3536, win 296, options [nop,nop,TS val 3716707402 ecr 993831974], length 0
21:07:00.846391 IP (tos 0x10, ttl 56, id 27603, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x6536 (correct), seq 3572:3608, ack 3213, win 316, options [nop,nop,TS val 993831975 ecr 3716707385], length 36
21:07:00.846677 IP (tos 0x38, ttl 64, id 25372, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc5af (correct), seq 3213, ack 3572, win 296, options [nop,nop,TS val 3716707402 ecr 993831975], length 0
21:07:00.846759 IP (tos 0x10, ttl 56, id 20197, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xec69 (correct), seq 3608:3644, ack 3213, win 316, options [nop,nop,TS val 993831975 ecr 3716707385], length 36
21:07:00.846809 IP (tos 0x38, ttl 64, id 25373, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc58b (correct), seq 3213, ack 3608, win 296, options [nop,nop,TS val 3716707402 ecr 993831975], length 0
21:07:00.847276 IP (tos 0x10, ttl 56, id 12605, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xc7d2 (correct), seq 3644:3680, ack 3213, win 316, options [nop,nop,TS val 993831976 ecr 3716707385], length 36
21:07:00.847500 IP (tos 0x38, ttl 64, id 25374, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc542 (correct), seq 3213, ack 3680, win 296, options [nop,nop,TS val 3716707403 ecr 993831975], length 0
21:07:00.847797 IP (tos 0x10, ttl 56, id 22490, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xb19c (correct), seq 3680:3716, ack 3213, win 316, options [nop,nop,TS val 993831976 ecr 3716707385], length 36
21:07:00.848028 IP (tos 0x38, ttl 64, id 25375, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc51c (correct), seq 3213, ack 3716, win 296, options [nop,nop,TS val 3716707404 ecr 993831976], length 0
21:07:00.848576 IP (tos 0x10, ttl 56, id 52770, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xa70e (correct), seq 3716:3760, ack 3213, win 316, options [nop,nop,TS val 993831977 ecr 3716707385], length 44
21:07:00.848814 IP (tos 0x38, ttl 64, id 25376, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc4ef (correct), seq 3213, ack 3760, win 296, options [nop,nop,TS val 3716707404 ecr 993831977], length 0
21:07:00.849341 IP (tos 0x10, ttl 56, id 12364, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x5f05 (correct), seq 3760:3804, ack 3213, win 316, options [nop,nop,TS val 993831978 ecr 3716707385], length 44
21:07:00.849576 IP (tos 0x38, ttl 64, id 25377, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc4c1 (correct), seq 3213, ack 3804, win 296, options [nop,nop,TS val 3716707405 ecr 993831978], length 0
21:07:00.854643 IP (tos 0x10, ttl 56, id 2324, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xaa40 (correct), seq 3804:3848, ack 3213, win 316, options [nop,nop,TS val 993831983 ecr 3716707385], length 44
21:07:00.854932 IP (tos 0x38, ttl 64, id 25378, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc48b (correct), seq 3213, ack 3848, win 296, options [nop,nop,TS val 3716707410 ecr 993831983], length 0
21:07:01.018397 IP (tos 0x10, ttl 56, id 56956, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xeca6 (correct), seq 3848:3884, ack 3213, win 316, options [nop,nop,TS val 993832147 ecr 3716707410], length 36
21:07:01.018671 IP (tos 0x10, ttl 56, id 5466, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x359e (correct), seq 3884:3920, ack 3213, win 316, options [nop,nop,TS val 993832147 ecr 3716707410], length 36
21:07:01.018821 IP (tos 0x38, ttl 64, id 25379, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc31f (correct), seq 3213, ack 3884, win 296, options [nop,nop,TS val 3716707574 ecr 993832147], length 0
21:07:01.019127 IP (tos 0x38, ttl 64, id 25380, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc2fa (correct), seq 3213, ack 3920, win 296, options [nop,nop,TS val 3716707575 ecr 993832147], length 0
21:07:01.019193 IP (tos 0x10, ttl 56, id 41745, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xd170 (correct), seq 3920:3956, ack 3213, win 316, options [nop,nop,TS val 993832147 ecr 3716707410], length 36
21:07:01.019280 IP (tos 0x10, ttl 56, id 43459, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x8e61 (correct), seq 3956:3992, ack 3213, win 316, options [nop,nop,TS val 993832148 ecr 3716707410], length 36
21:07:01.019481 IP (tos 0x10, ttl 56, id 9999, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x2fc5 (correct), seq 3992:4028, ack 3213, win 316, options [nop,nop,TS val 993832148 ecr 3716707410], length 36
21:07:01.019607 IP (tos 0x10, ttl 56, id 61745, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x2c31 (correct), seq 4028:4064, ack 3213, win 316, options [nop,nop,TS val 993832148 ecr 3716707410], length 36
21:07:01.020121 IP (tos 0x10, ttl 56, id 2260, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x362d (correct), seq 4064:4100, ack 3213, win 316, options [nop,nop,TS val 993832149 ecr 3716707410], length 36
21:07:01.020212 IP (tos 0x10, ttl 56, id 31065, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xb43a (correct), seq 4100:4136, ack 3213, win 316, options [nop,nop,TS val 993832149 ecr 3716707410], length 36
21:07:01.020227 IP (tos 0x38, ttl 64, id 25381, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc269 (correct), seq 3213, ack 4064, win 296, options [nop,nop,TS val 3716707576 ecr 993832147], length 0
21:07:01.020647 IP (tos 0x10, ttl 56, id 14436, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x12cf (correct), seq 4136:4172, ack 3213, win 316, options [nop,nop,TS val 993832149 ecr 3716707410], length 36
21:07:01.020850 IP (tos 0x10, ttl 56, id 19036, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x081f (correct), seq 4172:4208, ack 3213, win 316, options [nop,nop,TS val 993832150 ecr 3716707410], length 36
21:07:01.021102 IP (tos 0x10, ttl 56, id 6602, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xb4fb (correct), seq 4208:4244, ack 3213, win 316, options [nop,nop,TS val 993832150 ecr 3716707410], length 36
21:07:01.021397 IP (tos 0x10, ttl 56, id 61588, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x1b05 (correct), seq 4244:4280, ack 3213, win 316, options [nop,nop,TS val 993832150 ecr 3716707410], length 36
21:07:01.021445 IP (tos 0x38, ttl 64, id 25382, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc1b2 (correct), seq 3213, ack 4244, win 296, options [nop,nop,TS val 3716707577 ecr 993832149], length 0
21:07:01.021597 IP (tos 0x10, ttl 56, id 58619, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xf678 (correct), seq 4280:4316, ack 3213, win 316, options [nop,nop,TS val 993832150 ecr 3716707410], length 36
21:07:01.022107 IP (tos 0x10, ttl 56, id 60730, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xe02f (correct), seq 4316:4352, ack 3213, win 316, options [nop,nop,TS val 993832151 ecr 3716707410], length 36
21:07:01.022530 IP (tos 0x38, ttl 64, id 25383, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc144 (correct), seq 3213, ack 4352, win 296, options [nop,nop,TS val 3716707578 ecr 993832150], length 0
21:07:01.022670 IP (tos 0x10, ttl 56, id 6693, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x9458 (correct), seq 4352:4388, ack 3213, win 316, options [nop,nop,TS val 993832151 ecr 3716707410], length 36
21:07:01.022830 IP (tos 0x38, ttl 64, id 25384, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc11f (correct), seq 3213, ack 4388, win 296, options [nop,nop,TS val 3716707578 ecr 993832151], length 0
21:07:01.022900 IP (tos 0x10, ttl 56, id 26204, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xed23 (correct), seq 4388:4424, ack 3213, win 316, options [nop,nop,TS val 993832151 ecr 3716707410], length 36
21:07:01.023081 IP (tos 0x38, ttl 64, id 25385, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc0fa (correct), seq 3213, ack 4424, win 296, options [nop,nop,TS val 3716707579 ecr 993832151], length 0
21:07:01.023281 IP (tos 0x10, ttl 56, id 1963, offset 0, flags [none], proto TCP (6), length 88)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x4482 (correct), seq 4424:4460, ack 3213, win 316, options [nop,nop,TS val 993832152 ecr 3716707410], length 36
21:07:01.023400 IP (tos 0x10, ttl 56, id 21540, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0xa278 (correct), seq 4460:4504, ack 3213, win 316, options [nop,nop,TS val 993832152 ecr 3716707410], length 44
21:07:01.023591 IP (tos 0x38, ttl 64, id 25386, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc0d5 (correct), seq 3213, ack 4460, win 296, options [nop,nop,TS val 3716707579 ecr 993832152], length 0
21:07:01.023694 IP (tos 0x38, ttl 64, id 25387, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc0a9 (correct), seq 3213, ack 4504, win 296, options [nop,nop,TS val 3716707579 ecr 993832152], length 0
21:07:01.023867 IP (tos 0x10, ttl 56, id 2383, offset 0, flags [none], proto TCP (6), length 96)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x15a7 (correct), seq 4504:4548, ack 3213, win 316, options [nop,nop,TS val 993832153 ecr 3716707410], length 44
21:07:01.024008 IP (tos 0x38, ttl 64, id 25388, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xc07b (correct), seq 3213, ack 4548, win 296, options [nop,nop,TS val 3716707580 ecr 993832153], length 0
21:07:01.024906 IP (tos 0x10, ttl 56, id 56468, offset 0, flags [none], proto TCP (6), length 192)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x2310 (correct), seq 4548:4688, ack 3213, win 316, options [nop,nop,TS val 993832154 ecr 3716707410], length 140
21:07:01.025024 IP (tos 0x38, ttl 64, id 25389, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xbfdc (correct), seq 3213, ack 4688, win 313, options [nop,nop,TS val 3716707581 ecr 993832154], length 0
21:07:01.026515 IP (tos 0x10, ttl 56, id 19106, offset 0, flags [none], proto TCP (6), length 120)
    205.233.254.38.ssh > micro.muppetz.com.38226: Flags [P.], cksum 0x1d17 (correct), seq 4688:4756, ack 3213, win 316, options [nop,nop,TS val 993832155 ecr 3716707410], length 68
21:07:01.026848 IP (tos 0x38, ttl 64, id 25390, offset 0, flags [DF], proto TCP (6), length 52)
    micro.muppetz.com.38226 > 205.233.254.38.ssh: Flags [.], cksum 0xbf96 (correct), seq 3213, ack 4756, win 313, options [nop,nop,TS val 3716707582 ecr 993832155], length 0

I see that there is [b]SOME [/b]packets marked with DSCP 0x38, but not the first few!
Thus I think my problem here is that [i]because the check for traffic happens only when the session is established[/i], it's impossible to match this traffic at all.

The only way to make this work would be a IPTables rule on my Linux box to FORCE all traffic to be marked with 0x38.

Does that make sense?

There's no way to make pfSense say "Match ANY traffic, established session or not" is there?[/s]
```</animal:1>
                1 Reply Last reply Reply Quote 0
                • ?
                  A Former User
                  last edited by

                  And having read this redmine, I feel like a right wally.

                  Sorry everyone I had to learn at your expense! I'm a doofus!

                  S 1 Reply Last reply Reply Quote 1
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    @muppet:

                    There's no way to make pfSense say "Match ANY traffic, established session or not" is there?

                    You can do that by using a rule set to state type 'None'. It will not create a state so each packet is assessed. Obviously it's then non-stateful so TCP packets are passed out of order etc. Also since it doesn't create a state you need rules to pass the reply traffic and avoid hitting any default rules that might be there that would create a state. It gets complex!

                    I wouldn't recommend doing that but if you really need it I have used that in the past.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • S
                      scurrier @A Former User
                      last edited by

                      @muppet Thank you so much for following up on the answer. I encountered the same thing. I'm using corporate VPN from home and the VoIP packets are (shamefully) routed through VPN when it's connected. The VPN laudably sets the Expedited Forwarding DSCP tag for the VoIP packets, but I could see in the pfSense Queue Status that the queue wasn't seeing them. This totally explains it and, darn!, this is a real bummer that the queuing is per-state and not per-packet.

                      ? 1 Reply Last reply Reply Quote 0
                      • ?
                        A Former User @scurrier
                        last edited by

                        @scurrier I'm glad that I was able to help! I spent so much time doing my head in on this one hehe, glad to have saved someone else some of the headache.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.