Thanks for the buffer bloat explanation, figures comcast would have a huge buffer. :|
I did try the traffic shaping wizard, and that worked great to almost eliminate the upload latency spike while saturated, though download still hits about 200ms, which isn't all that bad I guess. I'll continue to research and tweak it.
The only thing left I can't figure out is why polling causes the network interfaces to stop working, I would think Intel Pro NICs would support that feature. Is there something in the kernel that isn't compiled in by default which is required for polling?