Solved: Dual NIC issue
-
em0 and em1 are the interfaces on the dual nic card
re1 is the single interface onboard Realtek nicI can configure the WAN to work on re1, em0 or em1. But I can only use one of the HP cards interfaces at a time. If I try to assign the WAN to em0 and the LAN to em1 the LAN will work but the WAN will not pull an IP with DHCP. I've assigned it the other way around as well with the same results (LAN=em0, WAN=em1). So the only configuration I can get to work is to assign the WAN to re1 and the LAN to either em0 or em1.
I'd like to use only em0 and em1 because the HP card is gigabit. The realtek is only 10/100. I'd love to be able to eliminate that bottleneck since everything else on the network is gigabit capable. Any way to accomplish this? And why would a dual nic not be able to run wan and lan at the same time?
Hardware Reference:
Realtek 8101E
http://www.realtek.com/products/productsView.aspx?Langid=1&PFid=7&Level=5&Conn=4&ProdID=217HP NC360T PCI Express Dual Port Gigabit Server Adapter
http://h18000.www1.hp.com/products/quickspecs/archives_North_America/12563_na_v5/12563_na.HTML -
Have you checked to see if you NIC is supported in the hardware compatibility list?
https://www.freebsd.org/releases/10.3R/hardware.html#ethernet
Also how are you doing the install and assigning the interfaces? Have you tried to only plug in the LAN during the install so only 1 MAC shows up that you know for sure is the LAN and then assigning that interface while leaving the WAN disconnected? Afterwards you can plug in the WAN when configuration is completed and go back to "assign interfaces" and assign the WAN. This way you should already know the MAC address LAN and can ensure the correct MAC Address for WAN is assigned?
I'd try that first. Disconnect everything expect for the LAN you want to use and reassign it and document its MAC address. Then plug in the WAN and do the same thing with the LAN connected.
Also sometimes for a WAN to pull a new address you may need to reboot the modem to allow it to update its routing tables.
-
Ty for the reply!
Have you checked to see if you NIC is supported in the hardware compatibility list?
Thank you for the link. I just checked. The HP NC360 I'm using isn't listed. There is a similar model number listed, HP NC370. But pfsense works without error on all of the ethernet ports. It just wasn't uing the HP NC360 Nic with wan and lan simultaneously.
Also how are you doing the install and assigning the interfaces?
I was using the shell, directly logged into the router itself by using the #1 option "assign interfaces".
Have you tried to only plug in the LAN during the install so only 1 MAC shows up that you know for sure is the LAN and then assigning that interface while leaving the WAN disconnected? Afterwards you can plug in the WAN when configuration is completed and go back to "assign interfaces" and assign the WAN. This way you should already know the MAC address LAN and can ensure the correct MAC Address for WAN is assigned?
No, I have correctly identified which port is which and know for certain that em0 and em1 are the HP NC360 Dual NIC card and that re1 is the realtek onboard NIC. I've also identified their MAC addresses But I assigned the interfaces by plugging in the lan and wan and assigning them while they are both connected.
I'd try that first. Disconnect everything expect for the LAN you want to use and reassign it and document its MAC address. Then plug in the WAN and do the same thing with the LAN connected.
Also sometimes for a WAN to pull a new address you may need to reboot the modem to allow it to update its routing tables.
So I'm not sure what I was doing wrong, or if there was some system error but here's what I tried and what worked.
First I tried going back into the system options while connected directly to the pfsense box (monitor and keyboard connected). I disconnected the lan and wan cables. Then went back into option #1, plugged in the cables, and tried to assign the interfaces that way. But when I tried that only the lan would set an IP address, WAN would hang for more than 120 seconds so I used [control+C] to exit. It would show an error, something along the lines of "grep broken pipe".
Could not get WAN to work in the shell at all. LAN was fine though. So I tried reassigning the WAN interface MAC address from the realetek onboard to the free port on the HP dual nic. I did this in the web interface, changed its mac to one of the dual nic ports (the other is the lan), and now it works. So I have both wan and lan running on the dual HP NC360 gigabit NIC. Yay!
I'm happy that I've got it all working on gigabit but I really wish I understood my or pfsenses error was.
Thanks for your reply, it got me tinkering and it's now working they way I'd like!
-
Another note, while reassigning the wan via the shell I did try restarting the modem several times to no avail. After reassigning with the web interface it now has a new ip. I had the same IP for the last 6 months and now it's changed. Not sure if that gives any clues as to why the web interface worked when the shell option #1 did not.
-
Awesome, happy to hear you got it working!
Yeah thats the greatness of troubleshooting. Gotta be smart and think of other ways to solve the problem. I had a similar issue when I first installed PFsense and I had to do it in the manner as I described to get the WAN to pull an IP properly. However, I've seen times where that didn't work either.
Process of elimination!
As for IP changing, if you are on a DHCP address on a WAN, anything can cause it to release. Its TTL in the modem could have expired, rebooting the modem etc… can all cause it to change address. As for why it worked via the web UI and not shell. That I'm not sure about, maybe the modem was in middle of the IP renewal during shell when shortly after it renewed again because it was done through the web UI as well?
As for the error during boot with WAN disconnected. That is normal. It will hang on "starting WAN" or something like that then you will receive the warning as you described simply just showing WAN is not pulling an IP.
Have you tried to restart the device since after WAN and LAN was connected? I'm sure they warning/error should be gone now and it should only pause on WAN for about half the time it did before.
-
Thanks again for your help. Although I got it working, I have changed it back to the original configuration in the last day.
I noticed there was an odd lag between lan and wan, like a second or two. I believe the card isn't great at processing both wan and lan at the same time, or the drivers in bsd aren't optimized. It's almost like the buffer in one port was filled and it took time for the other port to process it. You could even see it in the gui interface using the traffic graph. While DLing a large file you could see the packets transferring for 5 seconds then pausing for 1 second. It would happen on web pages too, load load load, pause, load load load, pause. Odd. Anyway, going back to the original configuration solved that issue.
Now I'm wondering about something else and I have no idea if it's possible. Can I use both ports on the dual nic to handle lan traffic? Is there a way to configure them to both handle wan to lan traffic? To sort of, split the work load between them? Never heard of anything like that but wondering if it's possible. Might require a new thread! :)
-
Hi Crebs,
The HP NC360T is a re-badged Intel card - just sold by HP. That's why it works with the em driver.
Sadly, I've had two of those NICs fail in the past. Both were second-hand. Generally, though, they are pretty reliable and efficient - probably more so than the Realtek.
Does Status > Interfaces show any In/Out errors?
-
Hey biggsy
I checked and no errors at the moment. But I did revert back to using the realtek as the WAN and the HP as LAN due to performance issues. So right now I'm only using one of the two dual ports and things are working well again. So I'm not using the HP nic for WAN and LAN.
When I get a chance I'll try re-configuring the interfaces to use only the HP dual nic for LAN and WAN and then checking STATUS > INTERFACES again.
Thanks