Pfsense Nokıa IP380 Install



  • hello
    When ata0 is found… things start to slow down... ata1 is found... about
    20 seconds later the OHCI devices are found and then it hangs. I've let it
    si  there for an hour or more... no avail.

    Is it possible to disable the usb?

    real memory  = 268435456 (262144K bytes)
    avail memory = 244572160 (238840K bytes)
    Preloaded elf kernel "kernel" at 0xc1006000.
    Preloaded mfs_root "/mfsroot" at 0xc100609c.
    Pentium Pro MTRR support enabled
    md0: Preloaded image 11534336 bytes at 0xc0504e1c
    md1: Malloc disk
    npx0: <math processor="">on motherboard</math>
    npx0: INT 16 interface
    pcib0: <serverworks nb6635="" 3.0le="" host="" to="" pci="" bridge="">on motherboard
    pci0: <pci bus="">on pcib0
    pci_cfgintr: can't route an interrupt to 0:9 INTA
    pcic0: <ti pci-1225="" pci-cardbus="" bridge="">at device 9.0 on pci0
    pcic0: PCI Memory allocated: 0x88000000
    pci_cfgintr: can't route an interrupt to 0:9 INTA
    pcic0: No PCI interrupt routed, trying ISA.
    pcic0: Polling mode
    pcic0: TI12XX PCI Config Reg: [pwr save][FUNC pci int + CSC serial isa
    irq]
    pccard0: <pc card="" 16-bit="" bus="" (classic)="">on pcic0
    pci_cfgintr: can't route an interrupt to 0:9 INTB
    pcic1: <ti pci-1225="" pci-cardbus="" bridge="">at device 9.1 on pci0
    pcic1: PCI Memory allocated: 0x88001000
    pci_cfgintr: can't route an interrupt to 0:9 INTB
    pcic1: No PCI interrupt routed, trying ISA.
    pcic1: Polling mode
    pcic1: TI12XX PCI Config Reg: [pwr save][FUNC pci int + CSC serial isa
    irq]
    pccard1: <pc card="" 16-bit="" bus="" (classic)="">on pcic1
    pci0: <unknown card="">(vendor=0x14e4, dev=0x5802) at 10.0 irq 11
    isab0: <serverworks ib6566="" pci="" to="" isa="" bridge="">at device 15.0 on pci0
    isa0: <isa bus="">on isab0
    atapci0: <serverworks rosb4="" ata33="" controller="">port 0xf800-0xf80f at device
    15.1 on pci0
    ata0: at 0x1f0 irq 14 on atapci0
    ata1: at 0x170 irq 15 on atapci0
    ohci0: <ohci (generic)="" usb="" controller="">mem 0xc0010000-0xc0010fff irq 11 at
    device 15.2 on pci0
    usb0: OHCI version 1.0, legacy support</ohci></serverworks></isa></serverworks></unknown></pc></ti></pc></ti></pci></serverworks>



  • is there any solution for this probleme, I have the same probleme with my IP380.

    Alex



  • I could not figure out yet. ip 380 remains idle



  • @kankabir:

    I could not figure out yet. ip 380 remains idle

    Hi,

    So I have made some test under Linux (I'm more familiar with linux as BSD system) and I got the same error at the beginning using a standard debian installation.

    I tried the kernel option nousb but it was no sufficient to be able to boot correctly
    After that I have built my own kernel without ramdisk and only with the needed driver  (and without USB).
    Now my IP380 is starting without any problem under Debian.

    The next step is to build the same custom kernel for pfsense, if it is possible …
    I think the only think to do is to disable all reference to USB driver.

    Alex


  • Netgate Administrator

    Have you tried adding any of the following to /boot/loader.conf.local

    hint.usb.0.disabled=1
    hint.uhci.0.disabled=1
    hint.ohci.0.disabled=1
    

    Steve



  • Hi,

    Unfortunatly adding the these option in the loarder.conf is not working.

    I think is the same problem I had under Linux, disabling USB was not enough … I must remove all USB option in the kernel to have a booting OS.

    Alex



  • Hi,

    Good news, after 3days fight Pfsense 2.0 is up and running on my IP380

    The process was not so easy for me because I'm not a bsd user.
    Second point the csv port was blocked at work but hopefully there a some nice workaround using ssh tunneling :D

    I you want to fix you installation just download the following tgz file and use it to replace the /boot/kernel folder on your pfsense boot media.

    http://alexkachler.free.fr/perso/ip380/ip380_kernel.tgz

    If you want to rebuild a new kernel, you can reuse my conf file (I have only removed RAID, USB, WLAN support):
    http://alexkachler.free.fr/perso/ip380/pfSense_wrap.8.i386

    ALex



  • @psykok:

    Hi,

    Good news, after 3days fight Pfsense 2.0 is up and running on my IP380

    The process was not so easy for me because I'm not a bsd user.
    Second point the csv port was blocked at work but hopefully there a some nice workaround using ssh tunneling :D

    I you want to fix you installation just download the following tgz file and use it to replace the /boot/kernel folder on your pfsense boot media.

    http://alexkachler.free.fr/perso/ip380/ip380_kernel.tgz

    If you want to rebuild a new kernel, you can reuse my conf file (I have only removed RAID, USB, WLAN support):
    http://alexkachler.free.fr/perso/ip380/pfSense_wrap.8.i386

    ALex

    Congratulations Alex!!! I downloaded your kernel and I am also able to boot. I had a little problem for mounting the disk, because I installed pfSense 2.0 from the live CD on a disk that was called /dev/da0 but on the IP380 host, the disk has to be called /dev/ad0. After changing the occurences of da0 into da0 into /etc/fstab, I was able to reboot.

    But I have still issues with the NICs. Since my second reboot, I no longer have the menu for reassigning the interfaces, etc… but a root prompt (with no completion feature).

    I would like to know if you had to change the config.xml file for spoofing the mac addresses.

    Anyway, thank you very much for the custom compiled kernel.

    Bien le bonjour là-bas (en France, je suppose)!

    Pierre



  • Hi

    I must have done something wrong: my WAN NIC has a inet6 IP address but not inet (4) IP address. This is the output of ifconfig dc3:

    # ifconfig dc3
    dc3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=80008 <vlan_mtu,linkstate>ether 00:00:00:00:00:00
            inet6 fe80::7422:d7c0:c46:842%dc3 prefixlen 64 scopeid 0x8 
            nd6 options=43 <performnud,accept_rtadv>media: Ethernet autoselect
            status: no carrier</performnud,accept_rtadv></vlan_mtu,linkstate></up,broadcast,running,simplex,multicast> 
    

    How is this possible? I have no inet6 DHCP server active at home…



  • hi,

    Merci je suis bien en france :D

    I have performed some test today on the box and pfsense is running without any problem.
    I can assign all my 4 NICs. The configuration is kept over reboot or power failure by me.
    I have not changed anything in the config.xml, the MAC addresses are generated automatically when I’m configure an interface.

    I know that Nokia has released several version of the IP380. Currently I have two different version (grey and white box), both are now running pfsense. I have only some hardware issues with the grey box … it’s not always booting : (
    Maybe there is some difference between the NICs.

    Alex



  • Hello Alex

    I also have an IP380 with a grey box. I have 8 NICs. fxp[0-3] (Intel EtherExpress Pro 10/100B Ethernet) at the right and dc[0-3] (Digital DC21143 Fast Ethernet) at the left. The Intel ones are running. The Digital ones not yet. Worse: sometimes, I have an interrupt storm detected on "irq10", which is the bios irq of the dc[0-3] NICs, and my webconfigurator becomes non responsive, needing a reboot at the serial console. It is possible to unplug the Digital NICs.

    I could redesign my LAN for 4 NICs and forget about the 4 NICs at the left, but it is pity…

    I suppose you don't have these Digital DC21143 Fast Ethernet in your configuration.

    Anyway, the system is running and that's the most important.

    Vive la France.

    Pierre



  • Hi,

    I have 6 NICS in my grey box, the 4 fxp and 2 other on the first extension card.

    Unfortunately I have some trouble with this version, I think something is going wron on the hardware level because the box is not booting correctly.

    On the other box I have only the 4 standard ports.

    But for your problem if the Digital interfaces are visible using the command ifconfig the system has loaded some drivers for it.
    If the driver is good working, that’s another point.

    For the irq storm did you try to modify something in the bios?

    Alex



  • So my gray box booted again and I have check the config of the additional NIC card.
    Same error as you with the IRQ, the two port are detected but not working, the 4 default NICs are working without any problem.

    I will check if there is some way to change the irq mapping on the firewall but I remember that the bios is a special light version with a very few available options.



  • When the system was running under IPSO, the Digital NICs used irq 11, 12 and 15.

    If there is any way to assign other irq's to the Digital NIC's (through the device.hints file), it could solve the problem.

    I would try something like adding some lines in the /conf/device.hints file, like

    hint.dc.0.irq="10"
    hint.dc.1.irq="12"
    hint.dc.2.irq="13"
    hint.dc.3.irq="15"
    

    avoiding the irq that are already listed in the output of vmstat -i

    Does this make sense? I am a software man, not a hardware guru.



  • Anyway, my entries in the device.hints file are not taken into consideration. Issuing vmstat -i shows the dc card is still using irq 10.



  • Did you check the bios if some irq setting is available?

    I don't know if there is a way to force the irq for these nics.

    Alex



  • In FreeBSD the calculated IRQ for a PCI device can be overridden by an entry in /boot/loader.conf.local (preferred since it won't be changed by a pfSense install) or /boot/loader.conf of the form:```
    hw.pci1.2.3.INTA.irg=15

    
    If you have a suitable IPSO installation or the startup output from a previous IPSO boot you should be able to determine the IRQs of each of the NICs. The challenge is then to map from the IPSO device name (e.g. eth-1, eth-s1p2) to FreeBSD device name (e.g. fxp1, dc2).


  • I give a try to your solution by modifying the /boot/loader.conf but it makes no difference.

    It seems that the entry I write in the file are not affecting the kernel.
    I use the following config:
    hw.pci3.5.0.dc0.irq=12
    hw.pci3.6.0.dc1.irq=13

    I checked the pci addresse swith dmesg.

    ALex



  • @psykok:

    It seems that the entry I write in the file are not affecting the kernel.
    I use the following config:
    hw.pci3.5.0.dc0.irq=12
    hw.pci3.6.0.dc1.irq=13

    You should have used INTA rather than dc0 and dc1. See my post about this!

    It might be helpful to have a bit of background on PCI devices. Each PCI device can use up to 4 interrupt request lines. They are called INTA, INTB, INTC, INTD. Most devices use INTA. The PCI interrupt request lines are connected to IRQs according to the whim of the motherboard designer. The /boot/loader.conf.local line```
    hw.pci3.5.0.INTA.irq=12

    
    As far as I know, there is no code that would act on a line like:```
    hw.pci3.5.0.dc0.irq=12
    

    The bus, device and function numbers you have used look plausible but I have no idea if they are correct.

    Edit: I had the parameters in the loader variable name incorrect and consequently the suggested variables and values in this reply won't work.



  • Hi

    First of all, my system is running even with interrupt storm on irq10.

    The mapping of the NICs IPSO <-> pfSense can be deduced from the bootlog of IPSO, below:

    eth-s1p1 <-> dc0
    eth-s1p2 <-> dc1
    eth-s2p1 <-> dc2
    eth-s2p2 <-> dc3
    eth1 <-> fxp0
    eth2 <-> fxp1
    eth3 <-> fxp2
    eth4 <-> fxp3

    This is confirmed by the experience of connecting stuff to these ports. The four first NICs, the Digital ones, are the left ones: eth-s1p2 stands for slot 1 port 2, the other ones are the Intel NICs.

    From the same bootlog, I can extract the INTA, INTB, etc. and the corresponding irq, but not the PCI bus number. I don't know what "rev" and "onboard" stand for.

    pcidec0 <intel 21154be="" 64-bit="" pci-pci="" bridge=""> rev 0 on pci1:0:0                
    fxp0 <intel 10="" etherexpress="" pro="" 100b="" ethernet=""> rev 9 int d irq 6 onboard 1     
    netlog:eth1 .. Ethernet address 0:a0:8e:78:c1:b4                               
    fxp1 <intel 10="" etherexpress="" pro="" 100b="" ethernet=""> rev 9 int a irq 10 onboard 2    
    netlog:eth2 .. Ethernet address 0:a0:8e:78:c1:b5                               
    fxp2 <intel 10="" etherexpress="" pro="" 100b="" ethernet=""> rev 9 int b irq 11 onboard 3    
    netlog:eth3 .. Ethernet address 0:a0:8e:78:c1:b6                               
    fxp3 <intel 10="" etherexpress="" pro="" 100b="" ethernet=""> rev 9 int c irq 12 onboard 4    
    netlog:eth4 .. Ethernet address 0:a0:8e:78:c1:b7                               
    pcidec1 <intel 21154be="" 64-bit="" pci-pci="" bridge=""> rev 0 slot 1                     
    tulip0 <digital dc21143="" fast="" ethernet=""> rev 65 int b irq 11 slot 1 port 1       
    netlog:eth-s1p1 .. Generic 2114x DC21143 pass 4.1 -- 00:a0:8e:78:c1:ac         
    netlog:eth-s1p1 .. enabling 10baseT/UTP port in half duplex mode               
    tulip1 <digital dc21143="" fast="" ethernet=""> rev 65 int c irq 15 slot 1 port 2       
    netlog:eth-s1p2 .. Generic 2114x DC21143 pass 4.1 -- 00:a0:8e:78:c1:ad         
    netlog:eth-s1p2 .. enabling 10baseT/UTP port in half duplex mode               
    pcidec2 <intel 21154be="" 64-bit="" pci-pci="" bridge=""> rev 0 slot 2                     
    tulip2 <digital dc21143="" fast="" ethernet=""> rev 65 int b irq 15 slot 2 port 1       
    netlog:eth-s2p1 .. Generic 2114x DC21143 pass 4.1 -- 00:a0:8e:78:c1:b0         
    netlog:eth-s2p1 .. enabling 10baseT/UTP port in half duplex mode               
    tulip3 <digital dc21143="" fast="" ethernet=""> rev 65 int c irq 12 slot 2 port 2       
    netlog:eth-s2p2 .. Generic 2114x DC21143 pass 4.1 -- 00:a0:8e:78:c1:b1         
    netlog:eth-s2p2 .. enabling 10baseT/UTP port in half duplex mode</digital></digital></intel></digital></digital></intel></intel></intel></intel></intel></intel>
    

    For giving values for the PCI bus, I suppose I have to look into the bootlog of pfSense this time:

    pci0: <serial bus,="" usb="">at device 15.2 (no driver attached)
    pcib1: <serverworks nb6635="" 3.0le="" host="" to="" pci="" bridge="">pcibus 1 on motherboard
    pci1: <pci bus="">on pcib1
    pcib2: <pci-pci bridge="">at device 0.0 on pci1
    pci2: <pci bus="">on pcib2
    fxp0: <intel 10="" 100="" 82559er="" embedded="" ethernet="">port 0xec00-0xec3f mem 0xc0100000
    -0xc0100fff,0xc0120000-0xc013ffff irq 11 at device 3.0 on pci2
    fxp0: Disabling dynamic standby mode in EEPROM
    fxp0: New EEPROM ID: 0xfffd
    fxp0: EEPROM checksum @ 0xff: 0xffff -> 0xbbb9
    miibus0: <mii bus="">on fxp0
    inphy0: <i82555 10="" 100="" media="" interface="">PHY 1 on miibus0
    inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    fxp0: [ITHREAD]
    fxp1: <intel 10="" 100="" 82559er="" embedded="" ethernet="">port 0xe800-0xe83f mem 0xc0140000
    -0xc0140fff,0xc0160000-0xc017ffff irq 11 at device 4.0 on pci2
    fxp1: Disabling dynamic standby mode in EEPROM
    fxp1: New EEPROM ID: 0xfffd
    fxp1: EEPROM checksum @ 0xff: 0xffff -> 0xbbb9
    miibus1: <mii bus="">on fxp1
    inphy1: <i82555 10="" 100="" media="" interface="">PHY 1 on miibus1
    inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    fxp1: [ITHREAD]
    fxp2: <intel 10="" 100="" 82559er="" embedded="" ethernet="">port 0xe400-0xe43f mem 0xc0180000
    -0xc0180fff,0xc01a0000-0xc01bffff irq 11 at device 5.0 on pci2
    fxp2: Disabling dynamic standby mode in EEPROM
    fxp2: New EEPROM ID: 0xfffd
    fxp2: EEPROM checksum @ 0xff: 0xffff -> 0xbbb9
    miibus2: <mii bus="">on fxp2
    inphy2: <i82555 10="" 100="" media="" interface="">PHY 1 on miibus2
    inphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    fxp2: [ITHREAD]
    fxp3: <intel 10="" 100="" 82559er="" embedded="" ethernet="">port 0xe000-0xe03f mem 0xc01c0000
    -0xc01c0fff,0xc01e0000-0xc01fffff irq 11 at device 6.0 on pci2
    fxp3: Disabling dynamic standby mode in EEPROM
    fxp3: New EEPROM ID: 0xfffd
    fxp3: EEPROM checksum @ 0xff: 0xffff -> 0xbbb9
    miibus3: <mii bus="">on fxp3
    inphy3: <i82555 10="" 100="" media="" interface="">PHY 1 on miibus3
    inphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    fxp3: [ITHREAD]
    pcib3: <pci-pci bridge="">at device 1.0 on pci1
    pci3: <pci bus="">on pcib3
    dc0: <intel 10="" 21143="" 100basetx="">port 0xdc00-0xdc7f mem 0xc0200000-0xc02003ff irq
     10 at device 5.0 on pci3
    miibus4: <mii bus="">on dc0
    bmtphy0: <bcm5221 10="" 100basetx="" phy="">PHY 1 on miibus4
    bmtphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    dc0: [ITHREAD]
    dc1: <intel 10="" 21143="" 100basetx="">port 0xd800-0xd87f mem 0xc0200400-0xc02007ff irq
     10 at device 6.0 on pci3
    miibus5: <mii bus="">on dc1
    bmtphy1: <bcm5221 10="" 100basetx="" phy="">PHY 1 on miibus5
    bmtphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    dc1: [ITHREAD]
    pcib4: <pci-pci bridge="">at device 2.0 on pci1
    pci4: <pci bus="">on pcib4
    dc2: <intel 10="" 21143="" 100basetx="">port 0xcc00-0xcc7f mem 0xc0300000-0xc03003ff irq
     10 at device 5.0 on pci4
    miibus6: <mii bus="">on dc2
    bmtphy2: <bcm5221 10="" 100basetx="" phy="">PHY 1 on miibus6
    bmtphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    dc2: [ITHREAD]
    dc3: <intel 10="" 21143="" 100basetx="">port 0xc800-0xc87f mem 0xc0300400-0xc03007ff irq
     10 at device 6.0 on pci4
    miibus7: <mii bus="">on dc3
    bmtphy3: <bcm5221 10="" 100basetx="" phy="">PHY 1 on miibus7
    bmtphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    dc3: [ITHREAD]</bcm5221></mii></intel></bcm5221></mii></intel></pci></pci-pci></bcm5221></mii></intel></bcm5221></mii></intel></pci></pci-pci></i82555></mii></intel></i82555></mii></intel></i82555></mii></intel></i82555></mii></intel></pci></pci-pci></pci></serverworks></serial> 
    

    So, cross referencing both bootlogs, I suppose I have to put next settings in /boot/loader.conf.local (not in device.hints, by the way):

    hint.pci2.3.0.INTD.irq="6"
    hint.pci2.4.0.INTA.irq="10"
    hint.pci2.5.0.INTB.irq="11"
    hint.pci2.6.0.INTC.irq="12"
    hint.pci3.5.0.INTB.irq="11"
    hint.pci3.6.0.INTC.irq="15"
    hint.pci4.5.0.INTB.irq="15"
    hint.pci4.6.0.INTC.irq="12"
    

    I am going to try later and I'll tell the results. If this doesn't make sense, please stop me before I risk to make my system unbootable.



  • By me if I check, I got the following information about the nics:

    dmesg | grep dc
    ubsec0: Broadcom 5802
    dc0: <intel 10="" 21143="" 100basetx=""> port 0xdc00-0xdc7f mem 0xc0200000-0xc02003ff irq 10 at device 5.0 on pci3
    miibus4: <mii bus=""> on dc0
    dc0: [ITHREAD]
    dc1: <intel 10="" 21143="" 100basetx=""> port 0xd800-0xd87f mem 0xc0200400-0xc02007ff irq 10 at device 6.0 on pci3
    miibus5: <mii bus=""> on dc1
    dc1: [ITHREAD]</mii></intel></mii></intel>
    


  • Based on the posting by pierref  of the IPSO output (which gives the actual IRQs for the NICs), I suggest```

    hw.pci3.5.0.INTA.irg=11
    hw.pci3.6.0.INTA.irg=15

    if the two NICs are in slot 1 and

    hw.pci3.5.0.INTA.irg=15
    hw.pci3.6.0.INTA.irg=12

    
    The two NICs on the plugin cards are downstream of a PCI bridge which "swizzles" the interrupt lines. This may add a complication which I haven't accounted for. Please try the appropriate suggestion and report back the outcome. A further complication is that I'm not sure if the startup output reports the IRQ from the PCI configuration registers (wrong in many cases on IPSO systems) or the actual IRQ the software considers the device using. The output of pfSense shell command```
    vmstat -i
    ```is probably the best report.


  • Hi,

    So I ried with the folowing both version  without any result:

    hint.pci3.5.0.INTB.irq="12"
    hint.pci3.6.0.INTC.irq="13"
    

    and

    hw.pci3.5.0.INTB.irq=12
    hw.pci3.6.0.INTC.irq=13



  • I tried:

    hint.pci2.3.0.INTD.irq="6"
    hint.pci2.4.0.INTA.irq="10"
    hint.pci2.5.0.INTB.irq="11"
    hint.pci2.6.0.INTC.irq="12"
    hint.pci3.5.0.INTB.irq="11"
    hint.pci3.6.0.INTC.irq="15"
    hint.pci4.5.0.INTB.irq="15"
    hint.pci4.6.0.INTC.irq="12"
    

    and still have interrupt storm detected on "irq10:"; throttling interrupt source and something related to stray irq7.

    The output of vmstat -i is

    interrupt                          total       rate
    irq0: clk                          53059         99
    irq4: uart0                        13408         25
    irq7:                                 14          0
    stray irq7                            14          0
    irq10: dc0 dc1 dc2+               744325       1396
    irq11: ubsec0 fxp0*                51603         96
    irq14: ata0                         3684          6
    Total                             866107       1624
    
    

    I will try the suggestion of wallabybob and post the results here.



  • Posting the output of pciconf -lv of my configuration still having interrupt storm on irq10, in case somebody is inspired to fix it.

    hostb0@pci0:0:0:0:      class=0x060000 card=0x00000000 chip=0x00091166 rev=0x06 
    hdr=0x00
        class      = bridge
        subclass   = HOST-PCI
    hostb1@pci0:0:0:1:      class=0x060000 card=0x00000000 chip=0x00091166 rev=0x06 
    hdr=0x00
        class      = bridge
        subclass   = HOST-PCI
    cbb0@pci0:0:9:0:        class=0x060700 card=0x00000000 chip=0xac1c104c rev=0x01 
    hdr=0x02
        class      = bridge
        subclass   = PCI-CardBus
    cbb1@pci0:0:9:1:        class=0x060700 card=0x00000000 chip=0xac1c104c rev=0x01 
    hdr=0x02
        class      = bridge
        subclass   = PCI-CardBus
    ubsec0@pci0:0:10:0:     class=0x0b4000 card=0x00000000 chip=0x580214e4 rev=0x01 
    hdr=0x00
        class      = processor
    isab0@pci0:0:15:0:      class=0x060100 card=0x02001166 chip=0x02001166 rev=0x50 
    hdr=0x00
        class      = bridge
        subclass   = PCI-ISA
    atapci0@pci0:0:15:1:    class=0x01018a card=0x00000000 chip=0x02111166 rev=0x00 
    hdr=0x00
        class      = mass storage
        subclass   = ATA
    none0@pci0:0:15:2:      class=0x0c0310 card=0x02201166 chip=0x02201166 rev=0x04 
    hdr=0x00
        class      = serial bus
        subclass   = USB
    pcib2@pci0:1:0:0:       class=0x060400 card=0x00000000 chip=0xb1548086 rev=0x00 
    hdr=0x01
        class      = bridge
        subclass   = PCI-PCI
    pcib3@pci0:1:1:0:       class=0x060400 card=0x00000000 chip=0xb1548086 rev=0x00 
    hdr=0x01
        class      = bridge
        subclass   = PCI-PCI
    pcib4@pci0:1:2:0:       class=0x060400 card=0x00000000 chip=0xb1548086 rev=0x00 
    hdr=0x01
        class      = bridge
        subclass   = PCI-PCI
    fxp0@pci0:2:3:0:        class=0x020000 card=0x00000000 chip=0x12098086 rev=0x09 
    hdr=0x00
        class      = network
        subclass   = ethernet
    fxp1@pci0:2:4:0:        class=0x020000 card=0x00000000 chip=0x12098086 rev=0x09 
    hdr=0x00
        class      = network
        subclass   = ethernet
    fxp2@pci0:2:5:0:        class=0x020000 card=0x00000000 chip=0x12098086 rev=0x09 
    hdr=0x00
        class      = network
        subclass   = ethernet
    fxp3@pci0:2:6:0:        class=0x020000 card=0x00000000 chip=0x12098086 rev=0x09 
    hdr=0x00
        class      = network
        subclass   = ethernet
    dc0@pci0:3:5:0: class=0x020000 card=0x61e00040 chip=0x00191011 rev=0x41 hdr=0x00
        class      = network
        subclass   = ethernet
    dc1@pci0:3:6:0: class=0x020000 card=0x61e00040 chip=0x00191011 rev=0x41 hdr=0x00
        class      = network
        subclass   = ethernet
    dc2@pci0:4:5:0: class=0x020000 card=0x61e00040 chip=0x00191011 rev=0x41 hdr=0x00
        class      = network
        subclass   = ethernet
    dc3@pci0:4:6:0: class=0x020000 card=0x61e00040 chip=0x00191011 rev=0x41 hdr=0x00
        class      = network
        subclass   = ethernet
    

    Any help would be appreciated.

    Pierre



  • Hi,

    It's atrange that all these config are not afecting the irq maping!
    The question is: is it possible to force a config?

    Pierre do you have a running iposo? If yes do you have checked if there is some kernel option set?

    Alex



  • Hi Alex

    I don't have any IPSO running no more. I can provide you the full output of the bootlog I saved some weeks ago, before I installed pfSense. See the attached file: that is all what I kept. I also have an image of the disk, but it is several GB big.

    Meilleures salutations.

    Pierre

    mac-addresses-nokia-ip380.txt



  • Pierre and Alex, My apologies. My hasty reading of the source code led me to misinterpret the parameters in the loader variable assignments I posted earlier. I have done some testing and confirmed I can change the reported interrupts on my system through the mechanism I have suggested (after correction of the parameters).

    I haven't seen any reports that either of you have followed exactly my previous suggestions. I hope we can all pay closer attention to the details.

    Lets start with the fxps:

    | Device | IPSO irq | FreeBSD irq |
    | eth1/fxp0 | 6 | 11 |
    | eth2/fxp1 | 10 | 11 |
    | eth3/fxp2 | 11 | 11 |
    | eth4/fxp3 | 12 | 11 |

    Suppose eth2/fxp1 is in use. It interrupts on irq10 but FreeBSD thinks it interrupts on irq11 so FreeBSD will call the interrupt handler for eth2/fxp1 on an irq11 interrupt. If eth2/fxp1 interrupts then FreeBSD will call the irq10 handlers (none of which is the actual eth2/fxp1 handler) none of which will clear the eth2/fxp1 interrupt condition so irq10 interrupt will happen again and again: irq10 storm!

    The following lines in /boot/loader.conf.local should fix this:```

    hw.pci0.2.3.INTA.irq=6
    hw.pci0.2.4.INTA.irq=10
    hw.pci0.2.5.INTA.irq=11
    hw.pci0.2.6.INTA.irq=12

    where the value of the variable hw.pci0.2.3.INTA.irq specifies the interrupt line to use for the device with PCI address (PCI domain=0, PCI bus =2, PCI device number on bus = 3). Please try this, reboot after updating /boot/loader.conf.local and verify the irqs reported in the dmesg output and the output of
    vmstat -i

    
    Edit: Corrected typo in lines for /boot/loader.conf.local


  • Hi wallabybob

    I am really happy to read your post. You look like the first person understanding exactly what's happening here. Alex and I are more Linux men and noobs in the world of pfSense/FreeBSD.

    I am ready to test what you are proposing, but I will probably not be able to do it today, because of other priorities.

    I will make my file /boot/loader.conf.local look like:

    hw.pci0.2.3.INTA.irq=6
    hw.pci0.2.4.INTA.irq=10
    hw.pci0.2.5.INTA.irq=11
    hw.pci0.2.6.INTA.irq=12
    

    correcting the tipo on the last line: INTA/irq -> INTA.irq, and I will post the ouptput of vmstat -i

    Regards.

    Pierre



  • Hi,

    I tried it but it's still not working as expected.

    With the following loader.conf.local

    hw.pci0.2.3.INTA.irq=6
    hw.pci0.2.4.INTA.irq=10
    hw.pci0.2.5.INTA.irq=11
    hw.pci0.2.6.INTA.irq=12
    hw.pci0.3.5.INTA.irq=13
    hw.pci0.3.6.INTA.irq=15
    
    

    I get these results

    
    #vmstat -i
    interrupt                          total       rate
    irq0: clk                          32550         99
    irq4: uart0                          676          2
    irq7:                                 21          0
    stray irq7                            21          0
    irq8: rtc                          41663        127
    irq10: fxp1                         1002          3
    irq11: ubsec0 fxp2                 32516         99
    irq14: ata0                         2177          6
    Total                             110626        339
    
    
    vmstat -ai
    interrupt                          total       rate
    ???                                    0          0
    irq0: clk                          33828         99
    stray irq0                             0          0
    irq1:                                  0          0
    stray irq1                             0          0
    irq3: uart1                            0          0
    stray irq3                             0          0
    irq4: uart0                          728          2
    stray irq4                             0          0
    irq5:                                  0          0
    stray irq5                             0          0
    irq6: fxp0                             0          0
    stray irq6                             0          0
    irq7:                                 22          0
    stray irq7                            22          0
    irq8: rtc                          43300        127
    stray irq8                             0          0
    irq9:                                  0          0
    stray irq9                             0          0
    irq10: fxp1                         1002          2
    stray irq10                            0          0
    irq11: ubsec0 fxp2                 33794         99
    stray irq11                            0          0
    irq12: fxp3                            0          0
    stray irq12                            0          0
    irq13: dc0                             0          0
    stray irq13                            0          0
    irq14: ata0                         2195          6
    stray irq14                            0          0
    irq15: dc1 ata1                        0          0
    stray irq15                            0          0
    Total                             114891        338
    
    

    The irq mapping has changed but the NICs are not working anymore and pfsense is slower as before.
    I can configure each interface as usual but for example the local one is no more responding and the WAN gets no address.

    The good point is that the irq storm disappear …

    Is our irq mapping wrong or is something other?

    Alex



  • @psykok:

    The irq mapping has changed but the NICs are not working anymore and pfsense is slower as before.
    I can configure each interface as usual but for example the local one is no more responding and the WAN gets no address.

    The good point is that the irq storm disappear …

    Is our irq mapping wrong or is something other?

    Thanks for the data. Looks like some progress is being made. What is the front panel designation of the NICs you are using? and for for what purpose? (which is LAN? WAN? etc). From the interrupt counts it looks as if only two NICs are in use since only two have non-zero interrupt counts.

    @pierref:

    correcting the tipo on the last line: INTA/irq -> INTA.irq, and I will post the ouptput of vmstat -i

    Thanks. Not a good start in attention to details on my part. I'll correct the error in the original post.



  • Currently I'm only using fxp0(wan) and fxp1(lan), I'm testing the box :D
    But I have tried to use the other port for the lan interface with the same result.

    Alex



  • @psykok:

    Currently I'm only using fxp0(wan) and fxp1(lan), I'm testing the box

    But isn't there a front panel designation: eth1? eth2? etc.

    fxp1 and fxp2 are the only NICs with non-zero interrupt counts so I suspect you are really using fxp1 and fxp2 but that you have done a mental translation from ethx (x=1, 2, 3, 4) to fxpn (n=0, 1, 2, 3). Correct?

    One of the earlier replies gave a mapping from the ethx to the fxpn but with no mention of how that was derived. I'm suspecting that mapping is not correct. I have vague memory of some Nokia IP? boxes where the electrical ordering of NICs on PCI buses didn't correspond to the front panel ordering but its over 4 years ago that I worked on the Nokia IPxxx boxes.



  • Sorry, you are right I made a mental mapping

    On the front panel I have Eth1 to Eth4.
    I have checked and I have the following correspondence:

    Eth1 : fxp0
    Eth2 : fxp1
    Eth3 : fxp2
    Eth4 : fxp3

    In the welcome screen of pfsense I get the same information about the up links.

    And for my tests I'm using Eth1(fxp0) and  Eth2(fxp1) for the wan and lan interface.

    Alex



  • @psykok:

    I have checked and I have the following correspondence:

    Eth1 : fxp0
    Eth2 : fxp1
    Eth3 : fxp2
    Eth4 : fxp3

    How did you determine this mapping is correct rather than (say) eth1 <-> fxp3 … ?

    @psykok:

    In the welcome screen of pfsense I get the same information about the up links.

    Do you mean pfSense reports fxp0 and fxp1 up? (I don't know how pfSense would know about eth1 and eth2).

    Do you see the problem: you say you are using fxp0 and fxp1 but fxp1 and fxp2 are the NICs with non-zero interrupt counts. Maybe your translation from the front panel label to fxpn is not correct? Maybe you moved one or more cables after booting? Can you account for the anomaly?



  • Hello wallabybob

    @pierref:

    I will make my file /boot/loader.conf.local look like:

    hw.pci0.2.3.INTA.irq=6
    hw.pci0.2.4.INTA.irq=10
    hw.pci0.2.5.INTA.irq=11
    hw.pci0.2.6.INTA.irq=12
    

    With these settings I promised you to test, the irq10 storm is worse than ever and there was no connectivity on the NICs. I removed /boot/loader.conf.local and now, I can reboot and have connectivity. But the storm is still there. Up to now, I only use fxp0 (WAN) and fxp1 (LAN), just like Alex. We will have to try something different.



  • Hi wallabybob

    I promised you:

    @pierref:

    I will make my file /boot/loader.conf.local look like:

    hw.pci0.2.3.INTA.irq=6
    hw.pci0.2.4.INTA.irq=10
    hw.pci0.2.5.INTA.irq=11
    hw.pci0.2.6.INTA.irq=12
    

    I am sorry to tell you these settings are worse than before: the irq10 storm was heavier, and the NIC's were no longer working. The only ports I am using now are fxp0 (WAN) and fxp1 (LAN), just as Alex. If you want me to try something different, I am ready to do it.

    Thanks in advance.



  • I can not run, unfortunately. I get an error when installing the USB



  • Hi Alex and wallabybob

    I found a set of entries in /boot/loader.conf.local where the NICs are working, at least fxp0 and fxp1, and no irq10 storm. Here is my file:

    hw.pci0.2.3.INTD.irq="6"
    hw.pci0.2.4.INTA.irq="10"
    hw.pci0.2.5.INTB.irq="11"
    hw.pci0.2.6.INTC.irq="12"
    hw.pci0.3.5.INTB.irq="11"
    hw.pci0.3.6.INTC.irq="15"
    hw.pci0.4.5.INTB.irq="15"
    hw.pci0.4.6.INTC.irq="12"
    

    Basically, these are the values collected from IPSO but in the correct format: lines starting with hw and not hint, correct order of parameters for PCI domain/bus/device_number.

    I will connect more switches to the ports in the hope that all the NICs will go on working, and I will inform you.

    Oh yes: and this is the output of # vmstat -i
    interrupt                          total       rate
    irq0: clk                          16552         99
    irq4: uart0                          560          3
    irq7:                                 23          0
    stray irq7                            23          0
    irq10: fxp1 dc0 dc*                 2158         13
    irq11: ubsec0 fxp0*                16507         99
    irq14: ata0                         2765         16
    Total                              38588        232

    There is something strange with this output: irq6, irq12 and irq15 are not set as I expected, but… it works.

    Anyway, I hope these setting will also work for Alex: since he has a subset of my NICs in his configuration, this should do the job for him too.

    Regards.

    Pierre



  • Sorry, other commitments mean I will have very limited time to devote to this for at least the next week, maybe the next two weeks.

    There are a number of inconsistencies in the accounts so far:

    1. Alex and Pierre both say they are using fxp0 and fxp1 but fxp1 and fxp2 show significant interrupt rates for Alex while Pierre sees significant interrupt rates for fxp0 and fxp1.

    2. Pierre reports interrupt storm on irq10 with my suggested modifications to /boot/loader.conf.local but the IPSO startup reports only one device on irq10: fxp1 and fxp1 was assigned to irq10 in my suggested modifications.

    Maybe the boxes have different motherboards. Maybe the IPSO startup output we have is from a different box

    From memory, the front panels of these devices designate the onboard (not on expansion slot) NICs as eth1, eth2, eth3 and eth4 but both Pierre and Alex report devices fxp0 and fxp1 in use. To get this to work correctly we need appropriate IPSO source code (used to be available from Nokia on request if I recall correctly; don't know about Checkpoint who took over the firewalls from Nokia) or an accurate association between front panel port name, FreeBSD device name and IRQ. Both Pierre and Alex have made associations between front panel port names and FreeBSD device names. I want to know how these associations were derived. If they have been assumed then they can't be trusted and they will need to be derived from the source code or worked out by experiment..

    If anyone can track down the source code for a version of IPSO that supported IP380 I'll take a look at it to try to derive the association I requested.

    @pierref:

    There is something strange with this output: irq6, irq12 and irq15 are not set as I expected, but… it works.

    All the hw.pci….irq entries for fxp NICs and dc NICs need to specify INTA because these devices request interrupts on their INTA line. The INTx reported by IPSO refer to interrupt lines on the motherboard which is different from interrupt lines the actual devices use. More detailed explanation will have to wait.


Locked