Thanks for your reply, CMB. So, at this point, we can say that my Endian firewall box is our villain. I just don't understand why it would be sending its traffic through its gateway(pfsense is the default gateway of endian) if the communication is happening on the same network. To be on the safe side, endian and pfsense are installed in two different hardware, not vms anymore.
Endian is a linux box. I looked at its route table, but there is only the default gateway route. I even cleaned up all the iptables rules, but the packets still going through pfsense. Google tells me that two more people faced the same issue when trying put an endian box behind a pfsense, but it seems that they just given up. :-(
I am running out of ideas so if anyone has any, I'd love to hear. My next test is to put a different linux box(probably an opensuse as Endian is based on RedHat) and see if I will see the same problem. I doubt it, though.