Traffic Shapper Queue Issue

[Narkhelek]

I'm having issues getting VPN return traffic matched to a traffic shaper queue. Here is the scenario:
I'm using a VPN service on a laptop and I have configured my pfSense to match that traffic and put it in to outbound or inbound queues. I have no problem matching the traffic outbound and getting it in to it's queue but I'm not able to get the return traffic in to it's queue. I've probably tried dozen or so different ways to skin this and no matter what I try I'm not able to get the return queue traffic in to it's queue.

VPN traffic details:
I'm currently attempting to match the VPN traffic based on public destination subnet of the VPN service (172.98.76.0/25). I also have the ability to specify source and destination ports and whether the tunnel is UDP or TCP. The tunnel is currently using UDP and random ports.

Current Rules:
OUTBOUND: I currently have a LAN pass rule that matches destination IP traffic to 172.98.76.0/25 and put it in a q_P2P_OUT queue. This works.

INBOUND: I currently have a WAN pass rule that matches source IP traffic from 172.98.76.0/25 that is suppose to put it in a q_P2P_IN queue. This does not work.

I've tried just using the OUTBOUND rule and putting the q_P2P_IN queue as the ACK queue but this does not work either. I originally tried to match based on source and destination UDP ports but that didn't work so I simplified things by matching on the destination subnet of the VPN service.

Current WAN Rules:

Current LAN Rules:

Current Traffic Shaping Queues:

I'm running pfsense build:
2.4.4-RELEASE-p2 (amd64)
built on Wed Dec 12 07:40:18 EST 2018
FreeBSD 11.2-RELEASE-p6

Thanks in advance!