Getting "could not setup receive structures" for half my interfaces during setup

imanz

Hello guys,

This is an install on a brand new machine I just built, final piece came in today. I am using the Supermicro A1SRI-2558F-O motherboard that uses the Intel Atom C2558 Quad Core CPU with 8GB of ram. I used the pfSense-memstick-2.1.5-RELEASE-amd64 install image to install from a USB stick to my 32GB SSD. I have also added a Intel Gigabit ET Dual port PCIE-E1G42ETBLK card for a total of 6 network interfaces not including the dedicated IMPI 2.0 dedicated port.

First time around I ran the install WITHOUT the additional Intel NIC and the machine recognized all 4 ports and labeled them igb0,igb1,igb2 and igb3 - Intel PRO/1000 Netowork Connection. Plugging an active network cable into any of the ports would provide the expected "igb[X]: link state changed to UP/DOWN" for all 4 ports. I shut down the machine and added the extra network card, booted off the USB drive and did a fresh install to the hard drive. Everything came up fine and displayed the valid interfaces as igb0,igb1,igb2,igb3,igb4 and igb5, again all as the Intel PRO/1000 Network Connection which I imagine is correct. The issue came when trying to setup my WAN and identify which interface goes with which physical port. Plugging a wire into either of the ports of the external NIC registered as igb0 and igb1, then I plugged the wire into what I believed was the "first" of the four built in network interfaces and that registered as igb3 however ALL other ports on the motherboard no longer gave any messages of "igb3: link state changed". Plugging the cable into what I believe to be igb3, igb4, and igb5 showed no messages in command line but the activity lights were active on the switch and NIC itself. After plugging and unplugging the cable a handeful of times from all the different ports I received the following messages:
igb3: Could not setup receive structures
igb4: Could not setup receive structures
igb5: Could not setup receive structures

After this the command line rolled over and it seems like the network interfaces reinitialized again showing "Valid interfaces are: .. igb0 -> igb5". The issue however was still present, with the same interfaces showing activity and the same ones not showing activity. I tried to reproduce the issue and moved the cable around maybe 10 times but it has not yet came back. It seems it is a potential issue with the drivers but I do not have very much experience with freeBSD and this is my first pfsense project for my house, any suggestions would be very appreciated. Please let me know if any additional information is needed.

Thank you!

cmb

You'll want to bump nmbclusters as noted here.
https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

That probably will prevent that message from ever occurring again (and will prevent the NIC from kernel panicing the system by running it out of mbufs).

imanz

Thank you! I was looking though similar posts and appreciate the help. Do you have any recommendations on doing this without configuring the WAN/LAN interface? I cant get passed that to get into the shell; should I use single user mode? Thanks!

Side question: Knowing this configuration is recommended are there any other things to look into for tuning the interfaces?

cmb

If you only assign WAN and leave all NICs unplugged, that should be enough to get booted up.

The only thing you'll want to change is what's noted on that page I linked for igb. And you probably don't need to mess with the queues setting at all, just change nmbclusters.

imanz

I just wanted to let you know that it is working correctly after the change. Thank you for the quick response. Not that install and basic configuration is complete I will move on to correctly setting up all the subnets; will probably be back with more questions :)

Wolf666

@imanz:

I just wanted to let you know that it is working correctly after the change. Thank you for the quick response. Not that install and basic configuration is complete I will move on to correctly setting up all the subnets; will probably be back with more questions :)

An OT question. Since I just installed pfSense 2.2.Beta on a similar set up, same board, same amount of RAM but using an INTEL S3500 SSD 80GB.
I am using a M350 case, 1 Noctua Fan (40x40x10) blowing from the upper case directly to CPU and RAM.

Can you kindly report your operation temperatures, case used and fans (number, position and flow direction)?

Thanks