Per IP traffic shaping limiter problem with games

[strike101]

Server ( Core2Duo 3Ghz , 2Gb RAM , Onboard and dedicated realtek fast ethernet )

Using a 100mb/100mb fiber connection

decided to Limit 5mb/5mb per IP for 20 units

limiter works when using speedtest.net and steam downloads

problem is when playing ( dota2 and LoL ) , ping/latency is fine.. around 20-30ms , but there are times that
they get sudden spikes , ping/latency reaches around 80-100ms on all clients…. i checked the traffic graph and i'm barely using any bandwidth , around 8-9mb total

if i remove the rules for limiters... they don't encounter any problems at all