Slow transfering speed between VLANs

  • Hello guys,
    I'm having a problem in my setup for a few months now and I couldn't solve the issue.
    I work at a Institute, so we have around 200 hosts using the network.

    I'm using pfSense 2.4.2_1 on a regular Desktop PC.
    with the following specs:

    Intel(R) Core(TM) i5-3330 CPU @ 3.00GHz
    8 GB of RAM
    1 TB of HDD
    2x RTL8196 PCI Express Gigabit NIC

    CPU Usage stays around 4% and RAM gets around 16%.

    Network setup:
    ISP Router -> Pfsense -> Switch HP V1910 48 ports -> hosts

    I'm using 17 VLAN's and I'm using traffic shaping in some of them. The problem is:

    Transfering files between hosts on different VLAN's gives me super slow transfer speed, but when the same hosts are on the same VLAN, I get full transfer speed.
    Disabling traffic shaping doesn't solve the problem. Tried to follow this guide but it didn't help.
    This problem also affect my internet downloads and uploads. The weird thing is that some websites like Youtube or Mega show better performance but others show very poor download and upload speeds.
    To eliminate the ISP fault for the slow internet connection, I tried connecting a PC directly on the router and that didn't show any slow performance. Connecting the PC directly on Pfsense (thought it might by the Switch) also didn't show any perfomance issue. So it seems it has something to do with the VLAN's.
    Another thing that botters me is that this problem seems to get worse between 2PM and 8PM, looking at the graphics doesn't show much of a difference when compared to other times of the day.
    I have the same setup in more then a year, but this problem started to happen a few months ago.

    Any ideas?