Netgate 7100-1U New NIC setup issues

Sammy2Netgo

@stephenw10 hello;-) thats what i did
Wan -ixo which i got ip from modem
Lan -ix3 setup dhcp and gateway ip

After that i cant connect to webgui or to network through switch or router -
Do i need to setup dhcp on switch ?
All i need is the sfp+ port and new nic card ethernet as wan
And when setting up dhcp and gateway ip After
When it gives me iption wan uostream ipv4 gateway i skip as its for wan ,right ? I wonder what else am i missing as i cant connect

stephenw10

You are talking about an external switch connected to ix3 I assume?

There shouldn't be anything special required there, I would expect it to 'just work'.

Are you sure the switch is connected to ix3? Do you see link LEDs?

Run at the command line: ifconfig ix3
Make sure it shows as linked.

Sammy2Netgo

@stephenw10 its very weird as my devices connected to switch detecting dhcp from router but cant connect neans the ix3 is passing data to switch and is connected

Sammy2Netgo

@Sammy2Netgo would the issue be related to switch set to l3 instead L2 im sure dhcp doesnt need to be running on the switch just the router as far as i know

Or would it be related to the port on the switch set to access port vs trunk port ?

Sammy2Netgo

@Sammy2Netgo i was thinking about reverse engineering and setting dhcp on the switch and get the router to obtain ip from switch

stephenw10

Just to confirm you are referring to some external switch connected to the on-board SFP port on the 7100 here?

Not the on-board switch in the 7100? Which is L2 only.

If your switch is set to layer 3 then it may be relaying the DHCP requests between two subnets but would not necessarily route traffic unless configured to do that.

Clients connected to the pfSense LAN expect to be in the same layer 2 segment. So, yes, having the switch setup as L3 could be causing this.

Sammy2Netgo

@stephenw10 yes its an external switch hooked up to sfp + port 10

Sammy2Netgo

@Sammy2Netgo what do i do in this section for lan ip setup ?
For a WAN, enter the new LAN ipv4 upstream gateway address. For a LAN, press enter > for none:
I leave blank ,right ? As its for a lan or does this have to do with me not connecting online through my dhcp

stephenw10

Yes just leave it empty, only WAN type interfaces should have a gateway set.

DHCP will pass the LAN address to clients to use as their gateway. Check the default route on the client.

Make sure the LAN subnet doesn't conflict with the whatever the WAN is receiving.

Sammy2Netgo

@stephenw10 omg my head feels like a pumpkin ready to explode -So the clients that are hooked to the switch are showing receiving dhcp lease from router but when i do ping to client it says no network or connection.Last night i tried to reverse the process and make my switch hand out dhcp leases and the router did receive the lease so there is some communication between router and switch but still no connectivity
I find the ports are out of whack as i factory reset router and the lan port 1 should be the wan but thats not the case -My other option idea would be to take the new card out of the router then setup the router and then put the new card in ?Not sure if that would help-What do you guys think maybe the assignments of the ports needed is gone or missing on the back end , thats why the no connection issues ?
Or maybe through console boot sector do new install of pfsense ?
One funny thing i noticed is in console it shows the sfp i wanna use as lan to external switch is ix3 so when i did that i only see rx not tx but when i put ix1 as LAN switch showing rx & tx connectivity.Originally on the front face panel it does show the sfp port labeled ix1.Any ideas ?

Sammy2Netgo

@stephenw10 both my modem router from isp and netgate router are set with 24 /cidr
255.255.255.0 subnet /how would it interfere ?
Currently my netgate router is set on lan side of the isp modem router

stephenw10

@Sammy2Netgo said in Netgate 7100-1U New NIC setup issues:

One funny thing i noticed is in console it shows the sfp i wanna use as lan to external switch is ix3 so when i did that i only see rx not tx but when i put ix1 as LAN switch showing rx & tx connectivity.Originally on the front face panel it does show the sfp port labeled ix1.Any ideas ?

When you put the expansion card in the ports on that card become ix0 and ix1.

If you are connecting to the on-board SFP+ port labelled ix1 that will no be ix3 so that's what you should use as LAN.

However, as I said, you can confirm that by running at the command line: ifconfig -vm ix3
You should see the link as up and the correct SFP module data reported.

If you disconnnect it and re-run that command you can confirm the NIC then goes down.

If you see that then the issue is probably in the switch config.

The subnet size (mask) doesn't matter as long as the two subnets do not conflict. So by default pfSense uses 192.168.1.1/24 for the LAN. The WAN must be in some other subnet.

What do you actually see reported at the console? Can you copy/paste it here?

Sammy2Netgo

@stephenw10 im currently out of office but will let you know as soon as i get back / Thanx

Sammy2Netgo

@stephenw10 how would i start command line
Is it selecting 8 shell ?
Here is my new port settings in console :
Wan >ix0
Lan >ix3
Opt1>lagg0.4090
Opt2>ix1

I finally see mac address of router showing in my switch but still no connection devices hooked to switch not connecting -Devices show dhcp lease ip
But not connecting or saying cant get an ip address is it maybe some setting in switch i need to configure ?

stephenw10

Yes use option 8 from the console menu to reach the command line.

Then run: ifconfig -vm ix3

Make sure it shows as up and linked.

Unplug the SFP and run that again. Make sure it now shows as down to confirm that's actually the LAN NIC.

Be sure the addresses shown above the menu for WAN and LAN do not conflict.

For example:

Netgate 7100 - Netgate Device ID: ba27a9eaeaeaea439bacb

*** Welcome to Netgate pfSense Plus 24.08-DEVELOPMENT (amd64) on 7100 ***

  Current Boot Environment: default_20240624180133_20240903152424 
     Next Boot Environment: default_20240624180133_20240903152424 

 LAN (lan)   -> ix2   -> v4: 192.168.88.1/24
 OPT1 (opt1) -> ix0   -> 
 WAN (wan)   -> lagg0 -> v4/DHCP4: 172.21.16.30/24

Sammy2Netgo

@stephenw10 how would i tell if wan lan has ip conflict ?
I was wondering while setting this up is it safe to have netgate router wan hooked up directly to isp modem wan that obtains external ip or behind firewall as lan under private ipv4?(netgate router wan to isp modem router lan)
Currently my wan and lan have private ip
As i put netgate wan to isp modem lan behind firewall
Wan -> 192.168.2.1
Lan >192.168.1.200
Is that ok or would it cause issues? Readon i did that is im not sure if firewall is setup properly and dont want to expose router to attacks

Sammy2Netgo

@Sammy2Netgo once i get back i will run the cli and paste the results here /Thanx

stephenw10

As long as those subnets are both /24 (or smaller) that's fine.

Sammy2Netgo

@Sammy2Netgo
I ran the cli with sfp plugged in results :

[24,03-RELERSE][root@pfSense.home,arpa]/root: ifconfig -vm ix3
1x3: flags=1008843<UP, BROADCAST, RUNNING,SIMPLEX,MULTICAST, LOWER UP> metric 0 mtu 1500
options=4138bb<RXCSUM,TXCSUM, VLAN_MTU,VLAN_ HUTACGING, JUMBO_MTU, VLAN_ HUCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HUFILTER, RXCSUM_IPV6,TXCSUM_IPV6, HUSTATS,MEXTPG>
capabilities=4/53bb<RXCSUM, TXCSUM, VLAN_MTU, VLAN_HUTAGGING, JUMBO_MTU, VLAN_ HUCSUM, TS04, TSO6,LRO, WOLLUCAST ,WOL_MCAST, WOL_ MAGIC, VLAN HUFILTER, VLAN_HUTSO, NETMAP, RCSUM_IPV6, TXCSUM IPV6, HUSTATS, MEXTPG>
ether 00:08:a2:12:87:27
inet 192.168.208.200 netmask 0xffffff00 broadcast 192.168.208.255
inet fe80::9e69:b4ff:fe63;6cca&ix3 prefixlen 64 scopeid Oxe
media: Ethernet autoselect (10Cbase-Twinax <full-duplex,rxpause, txpause>)
status: active supported media;
media autoselect media 10Gbase-Twinax
nd6 options=21<PERFORMNUD, AUTO_LINKLOCAL>
drivername: ix3
plugged: SFP/SFP+/SFP28 Unknown (Copper pigtail)
vendor: DEM PN: SFP-H10GB-CU1M SN: CSC200801600244 DATE: 2020-08-14
[24.03-RELEASE1Lroot&pfSense.home,arpa/root:
124,03-RELEASE] [root.@pfSense.home,arpal/root:

Sammy2Netgo

@Sammy2Netgo results with sfp unplugged -

(24,03-RELEASE][root&pfSense,home,arpa]/root:
(24,03-RELEASE1[root&pfSense.home.arpal/root: ifconfig -vm 1x3
ix3: flags=1008843<UP ,BROADCAST, RUNNING,SIMPLEX, MULTICAST, LOWER UP> metric 0 mtu 1500
options=4138bb<RXCSUM, TXCSUM, VLAN_ MTU, VLAM_ HUTAGGING, JUMBO_MTU, VLAN HUCSUM, WOLUCAST, WOL.MCAST,MOLMACIC, VLAN_HUFILTER, RCSUM_1PV6, TICSUM_ IPV6, HUSTATS, MEXTPG>
capabilities=4f53/bb<RXCSUM,TXCSUM, VLAN_MTU, VLAN_HUTAGING, JUMBO_MTU, VLAN_HUCSUM, TSO4, TSOG,LRO,WOLUCAST, WOL_MCAST, MOL_ MAGIC, VLAN_HUFILTER, VLAN_HWTSO,NETMAP, RICSUM_IPV6, TXCSUM_IPV,HUSTATS, MEXTPG>
ether 00:08:a2:12:87:27
inet 192,168.208.200 netmask 0xffffff00 broadcast 192.168.208.255
inet fe80::9e69;b4ff:fe63:6cca%ix3 prefixlen 64 scopeid Oxe
media: Ethernet autoselect
status: no carrier supported media:
media autoselect media 10Gbase-Twinax
nd6 options=21<PERFORMNUD, AUTO_LINKLOCAL>
drivername: ix3