Can I give Qos priority to a particular LAN interface?

[superwormy]

I'm looking for a way to make sure my VoIP connections always get priority over other data.

Can I set things up this way to make sure that happens:

WAN -> pfSense -> VoIP switch which has all VoIP phones and PBX on it    (QoS to always give this interface priority)
                      -> another switch which has all other machines behind it  (QoS so that this interface gets throttled back if other is in use)

Or is there a better way to do this?