BTW: I've had your problems with wrapping my head around certain phenomena. Like, I also had cases where everything looked alright but still: no packets would show up in certain queues. My approach solved all those issues for me.
Anyway, no matter how you do it, just remember: each queue works on exactly one interface in exactly one direction (outbound).
"My" approach just implements this behaviour by putting the queueing rules directly "on top of" the corresponding interface. One rule for each queue.