PfSense on Sophos UTM 320

[bart0]

Hello everyone!

As the topic says: I'm currently trying to get pfSense up and running on a Sophos UTM 320. Everything works fine so far, but I have one strange problem I'm seeking assistance with:

The Appliance has 8 Network Interface Ports (em0-em7) however only 5 of them are working properly. It seems to me that I'm facing some sort of strange driver issue, however I could not find anything that really helped.
I hope someone of you might have faced this problem already and maybe could point me into the right direction to get all the ports up and running :)

System Log shows:

Feb 23 22:39:42 gatekeeper kernel: em0: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x8f00-0x8f1f mem 0xfd7c0000-0xfd7dffff,0xfd7fc000-0xfd7fffff irq 16 at device 0.0 on pci1
Feb 23 22:39:42 gatekeeper kernel: em0: Using MSIX interrupts with 3 vectors
Feb 23 22:39:42 gatekeeper kernel: em0: Ethernet address: 00:1a:8c:17:d4:e8
Feb 23 22:39:42 gatekeeper kernel: em0: netmap queues/slots: TX 1/1024, RX 1/1024
Feb 23 22:39:42 gatekeeper kernel: pcib2: <acpi pci-pci="" bridge="">irq 17 at device 28.1 on pci0
Feb 23 22:39:42 gatekeeper kernel: pcib2: [GIANT-LOCKED]
Feb 23 22:39:42 gatekeeper kernel: pci2: <acpi pci="" bus="">on pcib2
Feb 23 22:39:42 gatekeeper kernel: em1: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x7f00-0x7f1f mem 0xfd6c0000-0xfd6dffff,0xfd6fc000-0xfd6fffff irq 17 at device 0.0 on pci2
Feb 23 22:39:42 gatekeeper kernel: em1: Using MSIX interrupts with 3 vectors
Feb 23 22:39:42 gatekeeper kernel: em1: Ethernet address: 00:1a:8c:17:d4:e9
Feb 23 22:39:42 gatekeeper kernel: em1: netmap queues/slots: TX 1/1024, RX 1/1024
Feb 23 22:39:42 gatekeeper kernel: pcib3: <acpi pci-pci="" bridge="">irq 18 at device 28.2 on pci0
Feb 23 22:39:42 gatekeeper kernel: pcib3: [GIANT-LOCKED]
Feb 23 22:39:42 gatekeeper kernel: pci3: <acpi pci="" bus="">on pcib3
Feb 23 22:39:42 gatekeeper kernel: em2: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x6f00-0x6f1f mem 0xfd5c0000-0xfd5dffff,0xfd5fc000-0xfd5fffff irq 18 at device 0.0 on pci3
Feb 23 22:39:42 gatekeeper kernel: em2: Using MSIX interrupts with 3 vectors
Feb 23 22:39:42 gatekeeper kernel: em2: Ethernet address: 00:1a:8c:17:d4:ea
Feb 23 22:39:42 gatekeeper kernel: em2: netmap queues/slots: TX 1/1024, RX 1/1024
Feb 23 22:39:42 gatekeeper kernel: pcib4: <acpi pci-pci="" bridge="">irq 19 at device 28.3 on pci0
Feb 23 22:39:42 gatekeeper kernel: pcib4: [GIANT-LOCKED]
Feb 23 22:39:42 gatekeeper kernel: pci4: <acpi pci="" bus="">on pcib4
Feb 23 22:39:42 gatekeeper kernel: em3: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xdf00-0xdf1f mem 0xfdec0000-0xfdedffff,0xfdefc000-0xfdefffff irq 19 at device 0.0 on pci4
Feb 23 22:39:42 gatekeeper kernel: em3: Using MSIX interrupts with 3 vectors
Feb 23 22:39:42 gatekeeper kernel: em3: Ethernet address: 00:1a:8c:17:d4:eb
Feb 23 22:39:42 gatekeeper kernel: em3: netmap queues/slots: TX 1/1024, RX 1/1024
Feb 23 22:39:42 gatekeeper kernel: pcib5: <acpi pci-pci="" bridge="">irq 16 at device 28.4 on pci0
Feb 23 22:39:42 gatekeeper kernel: pcib5: [GIANT-LOCKED]
Feb 23 22:39:42 gatekeeper kernel: pci5: <acpi pci="" bus="">on pcib5
Feb 23 22:39:42 gatekeeper kernel: em4: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xcf00-0xcf1f mem 0xfddc0000-0xfdddffff,0xfddfc000-0xfddfffff irq 16 at device 0.0 on pci5
Feb 23 22:39:42 gatekeeper kernel: em4: Using MSIX interrupts with 3 vectors
Feb 23 22:39:42 gatekeeper kernel: em4: Ethernet address: 00:1a:8c:17:d4:ec
Feb 23 22:39:42 gatekeeper kernel: em4: netmap queues/slots: TX 1/1024, RX 1/1024
Feb 23 22:39:42 gatekeeper kernel: pcib6: <acpi pci-pci="" bridge="">irq 17 at device 28.5 on pci0
Feb 23 22:39:42 gatekeeper kernel: pcib6: [GIANT-LOCKED]
Feb 23 22:39:42 gatekeeper kernel: pci6: <acpi pci="" bus="">on pcib6
Feb 23 22:39:42 gatekeeper kernel: pcib7: <pci-pci bridge="">mem 0xfdce0000-0xfdcfffff irq 17 at device 0.0 on pci6
Feb 23 22:39:42 gatekeeper kernel: pci7: <pci bus="">on pcib7
Feb 23 22:39:42 gatekeeper kernel: pcib8: <pci-pci bridge="">irq 18 at device 1.0 on pci7
Feb 23 22:39:42 gatekeeper kernel: pci8: <pci bus="">on pcib8
Feb 23 22:39:42 gatekeeper kernel: em5: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xbf00-0xbf1f mem 0xfdbc0000-0xfdbdffff,0xfdbfc000-0xfdbfffff irq 18 at device 0.0 on pci8
Feb 23 22:39:42 gatekeeper kernel: em5: Setup of Shared code failed, error -2
Feb 23 22:39:42 gatekeeper kernel: device_attach: em5 attach returned 6</intel(r)></pci></pci-pci></pci></pci-pci></acpi></acpi></intel(r)></acpi></acpi></intel(r)></acpi></acpi></intel(r)></acpi></acpi></intel(r)></acpi></acpi></intel(r)>

Although the Machine has 8 Network Interfaces there is no information about em6 and em7.

Dmesg shows:

em0: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x8f00-0x8f1f mem 0xfd7c0000-0xfd7dffff,0xfd7fc000-0xfd7fffff irq 16 at device 0.0 on pci1
em0: Using MSIX interrupts with 3 vectors
em0: Ethernet address: 00:1a:8c:17:d4:e8
em0: netmap queues/slots: TX 1/1024, RX 1/1024
pcib2: <acpi pci-pci="" bridge="">irq 17 at device 28.1 on pci0
pcib2: [GIANT-LOCKED]
pci2: <acpi pci="" bus="">on pcib2
em1: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x7f00-0x7f1f mem 0xfd6c0000-0xfd6dffff,0xfd6fc000-0xfd6fffff irq 17 at device 0.0 on pci2
em1: Using MSIX interrupts with 3 vectors
em1: Ethernet address: 00:1a:8c:17:d4:e9
em1: netmap queues/slots: TX 1/1024, RX 1/1024
pcib3: <acpi pci-pci="" bridge="">irq 18 at device 28.2 on pci0
pcib3: [GIANT-LOCKED]
pci3: <acpi pci="" bus="">on pcib3
em2: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x6f00-0x6f1f mem 0xfd5c0000-0xfd5dffff,0xfd5fc000-0xfd5fffff irq 18 at device 0.0 on pci3
em2: Using MSIX interrupts with 3 vectors
em2: Ethernet address: 00:1a:8c:17:d4:ea
em2: netmap queues/slots: TX 1/1024, RX 1/1024
pcib4: <acpi pci-pci="" bridge="">irq 19 at device 28.3 on pci0
pcib4: [GIANT-LOCKED]
pci4: <acpi pci="" bus="">on pcib4
em3: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xdf00-0xdf1f mem 0xfdec0000-0xfdedffff,0xfdefc000-0xfdefffff irq 19 at device 0.0 on pci4
em3: Using MSIX interrupts with 3 vectors
em3: Ethernet address: 00:1a:8c:17:d4:eb
em3: netmap queues/slots: TX 1/1024, RX 1/1024
pcib5: <acpi pci-pci="" bridge="">irq 16 at device 28.4 on pci0
pcib5: [GIANT-LOCKED]
pci5: <acpi pci="" bus="">on pcib5
em4: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xcf00-0xcf1f mem 0xfddc0000-0xfdddffff,0xfddfc000-0xfddfffff irq 16 at device 0.0 on pci5
em4: Using MSIX interrupts with 3 vectors
em4: Ethernet address: 00:1a:8c:17:d4:ec
em4: netmap queues/slots: TX 1/1024, RX 1/1024
pcib6: <acpi pci-pci="" bridge="">irq 17 at device 28.5 on pci0
pcib6: [GIANT-LOCKED]
pci6: <acpi pci="" bus="">on pcib6
pcib7: <pci-pci bridge="">mem 0xfdce0000-0xfdcfffff irq 17 at device 0.0 on pci6
pci7: <pci bus="">on pcib7
pcib8: <pci-pci bridge="">irq 18 at device 1.0 on pci7
pci8: <pci bus="">on pcib8
em5: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xbf00-0xbf1f mem 0xfdbc0000-0xfdbdffff,0xfdbfc000-0xfdbfffff irq 18 at device 0.0 on pci8
em5: Setup of Shared code failed, error -2
device_attach: em5 attach returned 6
pcib9: <pci-pci bridge="">irq 17 at device 4.0 on pci7
pci9: <pci bus="">on pcib9
em5: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0xaf00-0xaf1f mem 0xfdac0000-0xfdadffff,0xfdafc000-0xfdafffff irq 17 at device 0.0 on pci9
em5: Setup of Shared code failed, error -2
device_attach: em5 attach returned 6
pcib10: <pci-pci bridge="">irq 18 at device 5.0 on pci7
pci10: <pci bus="">on pcib10
em5: <intel(r) 1000="" pro="" network="" connection="" 7.6.1-k="">port 0x9f00-0x9f1f mem 0xfd9c0000-0xfd9dffff,0xfd9fc000-0xfd9fffff irq 18 at device 0.0 on pci10
em5: Setup of Shared code failed, error -2
device_attach: em5 attach returned 6</intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></pci></pci-pci></acpi></acpi></intel(r)></acpi></acpi></intel(r)></acpi></acpi></intel(r)></acpi></acpi></intel(r)></acpi></acpi></intel(r)>

As you can see it basically provides the same information: The Device failed to bring up em5 (and em6 and em7 are also not mentioned). However pciconf shows that the Network Interfaces definitely exist:

hostb0@pci0:0:0:0: class=0x060000 card=0x2e308086 chip=0x2e308086 rev=0x03 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '4 Series Chipset DRAM Controller'
    class      = bridge
    subclass  = HOST-PCI
vgapci0@pci0:0:2:0: class=0x030000 card=0x2e328086 chip=0x2e328086 rev=0x03 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '4 Series Chipset Integrated Graphics Controller'
    class      = display
    subclass  = VGA
vgapci1@pci0:0:2:1: class=0x038000 card=0x2e328086 chip=0x2e338086 rev=0x03 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '4 Series Chipset Integrated Graphics Controller'
    class      = display
pcib1@pci0:0:28:0: class=0x060400 card=0x27d08086 chip=0x27d08086 rev=0x01 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family PCI Express Port 1'
    class      = bridge
    subclass  = PCI-PCI
pcib2@pci0:0:28:1: class=0x060400 card=0x27d28086 chip=0x27d28086 rev=0x01 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family PCI Express Port 2'
    class      = bridge
    subclass  = PCI-PCI
pcib3@pci0:0:28:2: class=0x060400 card=0x27d48086 chip=0x27d48086 rev=0x01 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family PCI Express Port 3'
    class      = bridge
    subclass  = PCI-PCI
pcib4@pci0:0:28:3: class=0x060400 card=0x27d68086 chip=0x27d68086 rev=0x01 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family PCI Express Port 4'
    class      = bridge
    subclass  = PCI-PCI
pcib5@pci0:0:28:4: class=0x060400 card=0x27e08086 chip=0x27e08086 rev=0x01 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = '82801GR/GH/GHM (ICH7 Family) PCI Express Port 5'
    class      = bridge
    subclass  = PCI-PCI
pcib6@pci0:0:28:5: class=0x060400 card=0x27e28086 chip=0x27e28086 rev=0x01 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = '82801GR/GH/GHM (ICH7 Family) PCI Express Port 6'
    class      = bridge
    subclass  = PCI-PCI
uhci0@pci0:0:29:0: class=0x0c0300 card=0x27c88086 chip=0x27c88086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family USB UHCI Controller'
    class      = serial bus
    subclass  = USB
uhci1@pci0:0:29:1: class=0x0c0300 card=0x27c98086 chip=0x27c98086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family USB UHCI Controller'
    class      = serial bus
    subclass  = USB
uhci2@pci0:0:29:2: class=0x0c0300 card=0x27ca8086 chip=0x27ca8086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family USB UHCI Controller'
    class      = serial bus
    subclass  = USB
uhci3@pci0:0:29:3: class=0x0c0300 card=0x27cb8086 chip=0x27cb8086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family USB UHCI Controller'
    class      = serial bus
    subclass  = USB
ehci0@pci0:0:29:7: class=0x0c0320 card=0x27cc8086 chip=0x27cc8086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family USB2 EHCI Controller'
    class      = serial bus
    subclass  = USB
pcib11@pci0:0:30:0: class=0x060401 card=0x244e8086 chip=0x244e8086 rev=0xe1 hdr=0x01
    vendor    = 'Intel Corporation'
    device    = '82801 PCI Bridge'
    class      = bridge
    subclass  = PCI-PCI
isab0@pci0:0:31:0: class=0x060100 card=0x27b88086 chip=0x27b88086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82801GB/GR (ICH7 Family) LPC Interface Bridge'
    class      = bridge
    subclass  = PCI-ISA
atapci0@pci0:0:31:1: class=0x01018a card=0x27df8086 chip=0x27df8086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82801G (ICH7 Family) IDE Controller'
    class      = mass storage
    subclass  = ATA
ahci0@pci0:0:31:2: class=0x010601 card=0x27c08086 chip=0x27c18086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family SATA Controller [AHCI mode]'
    class      = mass storage
    subclass  = SATA
none0@pci0:0:31:3: class=0x0c0500 card=0x27da8086 chip=0x27da8086 rev=0x01 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = 'NM10/ICH7 Family SMBus Controller'
    class      = serial bus
    subclass  = SMBus
em0@pci0:1:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
em1@pci0:2:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
em2@pci0:3:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
em3@pci0:4:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
em4@pci0:5:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
pcib7@pci0:6:0:0: class=0x060400 card=0x860410b5 chip=0x860410b5 rev=0xba hdr=0x01
    vendor    = 'PLX Technology, Inc.'
    device    = 'PEX 8604 4-lane, 4-Port PCI Express Gen 2 (5.0 GT/s) Switch'
    class      = bridge
    subclass  = PCI-PCI
pcib8@pci0:7:1:0: class=0x060400 card=0x860410b5 chip=0x860410b5 rev=0xba hdr=0x01
    vendor    = 'PLX Technology, Inc.'
    device    = 'PEX 8604 4-lane, 4-Port PCI Express Gen 2 (5.0 GT/s) Switch'
    class      = bridge
    subclass  = PCI-PCI
pcib9@pci0:7:4:0: class=0x060400 card=0x860410b5 chip=0x860410b5 rev=0xba hdr=0x01
    vendor    = 'PLX Technology, Inc.'
    device    = 'PEX 8604 4-lane, 4-Port PCI Express Gen 2 (5.0 GT/s) Switch'
    class      = bridge
    subclass  = PCI-PCI
pcib10@pci0:7:5:0: class=0x060400 card=0x860410b5 chip=0x860410b5 rev=0xba hdr=0x01
    vendor    = 'PLX Technology, Inc.'
    device    = 'PEX 8604 4-lane, 4-Port PCI Express Gen 2 (5.0 GT/s) Switch'
    class      = bridge
    subclass  = PCI-PCI
none1@pci0:8:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
none2@pci0:9:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet
none3@pci0:10:0:0: class=0x020000 card=0x00008086 chip=0x10d38086 rev=0x00 hdr=0x00
    vendor    = 'Intel Corporation'
    device    = '82574L Gigabit Network Connection'
    class      = network
    subclass  = ethernet

As you can see none1 - none3 are exactly the missing interfaces (em5-em7). Plus "none0" indicates that the 'NM10/ICH7 Family SMBus Controller' is also not working. Now the big question is:

I can only make em5 show up, when I've got a cable connected to the port at a reboot, however the boot process will hang at "em5: Using MSIX interrupts with 3 Vectors" in this case and all I can do is poweroff the box and reboot without a cable attached to em5.

How can I get this fixed? Has anyone of you ever seen or experienced something like this before?

Thanks in advance and best regards

[stephenw10]

They all show as em5 since em4 is last valid em interface installed. Each tries to attach as em5 and fails.

That's odd. They appear to be identical chips, same PCI product ID.

Different firmware version on those perhaps?

They could just be dead, did they work under Sophos?

Steve

[kholmqvist]

I have installed pfSense on a UTM 320 rev. 4 without any issues, are you sure those nics are working with the sophos software installed?

[Juanesptux]

@kholmqvist Hi, bro, do you have any document in any forum step by step to install pfsense in sophos utm rev5?

[mr.rosh]

@stephenw10 chances are that the UTM motherboard is some what equivalent and or designed by lanner inc [also makes WatchGuard]. in that case, some nics are configured in BIOS to act like two physical nics that pass same traffic for traffic sniffing or something along those lines

[stephenw10]

Most of those Sophos boxes are from Portwell/Caswell I believe.
I'm not aware of any that had bypass NICs but I haven't looked that hard. However when you do have bypass NICs they generally still attach OK but just no traffic.
You can see failures for NICs using odd PHYs, like attached to internal switches for example. I don't think that applies to the UTM 320.

@Juanesptux You should probably start your own thread unless you are using the same UTM 320.

Steve