1.2 RC2 -> RC3 upgrade fails - swap_pager: out of swap space

[steve2k7]

Hi

When performing 1.2 RC2 -> RC3 firmware upgrade the process fails returning:

swap_pager: out of swap space
swap_pager_getswapspace(16): failed
pid 660 (lighttpd), uid 0, was killed: out of swap space

Both times i've tried the lighttpd process sits on top @ ~60% cpu, all the while the available physical memory slowly decreases. Virtual memory usage slowly starts to climb and eventually the process fails returning the above error. I've got 128MB physical and 256MB swap configured. From start to death the process takes about 40 seconds.

Below is dmesg output:

Copyright 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-RELEASE-p7 #0: Fri Aug 17 17:14:44 EDT 2007
    sullrich@builder6.pfsense.com:/usr/obj.pfSense/usr/src/sys/pfSense.6
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Pentium/P55C (200.46-MHz 586-class CPU)
  Origin = "GenuineIntel"  Id = 0x543  Stepping = 3
  Features=0x8001bf <fpu,vme,de,pse,tsc,msr,mce,cx8,mmx>real memory  = 134217728 (128 MB)
avail memory = 121769984 (116 MB)
Intel Pentium detected, installing workaround for F00F bug
wlan: mac acl policy registered
kbd1 at kbdmux0
ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
cpu0 on motherboard
pcib0: <host to="" pci="" bridge="">pcibus 0 on motherboard
pir0: <pci 6="" interrupt="" routing="" table:="" entries="">on motherboard
pci0: <pci bus="">on pcib0
isab0: <pci-isa bridge="">at device 7.0 on pci0
isa0: <isa bus="">on isab0
atapci0: <intel piix4="" udma33="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 7.1 on pci0
ata0: <ata 0="" channel="">on atapci0
ata1: <ata 1="" channel="">on atapci0
uhci0: <intel 82371ab="" eb="" (piix4)="" usb="" controller="">port 0x6400-0x641f irq 11 at device 7.2 on pci0
uhci0: [GIANT-LOCKED]
usb0: <intel 82371ab="" eb="" (piix4)="" usb="" controller="">on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pci0: <bridge>at device 7.3 (no driver attached)
pci0: <display, vga="">at device 8.0 (no driver attached)
vr0: <via 10="" vt6105="" rhine="" iii="" 100basetx="">port 0x6800-0x68ff mem 0xe4000000-0xe40000ff irq 10 at device 9.0 on pci0
miibus0: <mii bus="">on vr0
ukphy0: <generic ieee="" 802.3u="" media="" interface="">on miibus0
ukphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr0: Ethernet address: 00:0f:3d:ce:7a:ca
vr1: <via 10="" vt6105="" rhine="" iii="" 100basetx="">port 0x6c00-0x6cff mem 0xe4001000-0xe40010ff irq 12 at device 10.0 on pci0
miibus1: <mii bus="">on vr1
ukphy1: <generic ieee="" 802.3u="" media="" interface="">on miibus1
ukphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr1: Ethernet address: 00:0f:3d:ce:7a:b9
vr2: <via 10="" vt6105="" rhine="" iii="" 100basetx="">port 0x7000-0x70ff mem 0xe4002000-0xe40020ff irq 11 at device 11.0 on pci0
miibus2: <mii bus="">on vr2
ukphy2: <generic ieee="" 802.3u="" media="" interface="">on miibus2
ukphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr2: Ethernet address: 00:0f:3d:ce:69:f4
pmtimer0 on isa0
orm0: <isa option="" rom="">at iomem 0xc0000-0xc7fff on isa0
atkbdc0: <keyboard controller="" (i8042)="">at port 0x60,0x64 on isa0
atkbd0: <at keyboard="">irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ed0 at port 0x240-0x25f iomem 0xd8000 irq 3 on isa0
ed0: Ethernet address: 00:60:b0:35:ac:7d
ed0: type RTL8019 (16 bit)
ed1 at port 0x220-0x23f iomem 0xd4000 irq 5 on isa0
ed1: Ethernet address: 00:60:b0:7d:da:70
ed1: type RTL8019 (16 bit)
fdc0: <enhanced floppy="" controller="">at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: parallel port not found.
sc0: <system console="">at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
unknown: <pnp0303>can't assign resources (port)
speaker0: <pc speaker="">at port 0x61 on isa0
unknown: <pnp0c01>can't assign resources (memory)
unknown: <pnp0a03>can't assign resources (port)
unknown: <pnp0501>can't assign resources (port)
unknown: <pnp0700>can't assign resources (port)
unknown: <pnp0501>can't assign resources (port)
Timecounter "TSC" frequency 200455544 Hz quality 800
Timecounters tick every 1.000 msec
Fast IPsec: Initialized Security Association Processing.
ad0: 32253MB <maxtor stm380215a="" 3.aad="">at ata0-master UDMA33
acd0: CDRW <samsung cdrw="" dvd="" sm-352b="" t803="">at ata1-master UDMA33
Trying to mount root from ufs:/dev/ad0s1a
vr0: link state changed to UP
vr1: link state changed to DOWN
vr2: link state changed to DOWN
ed0: device timeout
ed0: device timeout
ed0: device timeout
pflog0: promiscuous mode enabled
swap_pager: out of swap space
swap_pager_getswapspace(16): failed
pid 660 (lighttpd), uid 0, was killed: out of swap space</samsung></maxtor></pnp0501></pnp0700></pnp0501></pnp0a03></pnp0c01></pc></pnp0303></generic></system></enhanced></at></keyboard></isa></generic></mii></via></generic></mii></via></generic></mii></via></display,></bridge></intel></intel></ata></ata></intel></isa></pci-isa></pci></pci></host></fpu,vme,de,pse,tsc,msr,mce,cx8,mmx>

[steve2k7]

Ok, just rebuilt the system w/1G of swap and the upgrade completed ok. Only a warning about the upgrade not being signed and I proceed at my own peril (or something thereabouts).

Since this was a fresh re-install I had not configured any of my ed interfaces, nor had I set up any other services. I'll perform another re-install, configure the system for all interfaces and services and see what happens when I try and upgrade to RC3.

Steve.

[steve2k7]

Having configured all interfaces and services (nothing special, just dhcp server at the moment) the upgrade from RC2 -> RC3 completed successfully.

Having watched top a bit more closely this time I can confirm the following figures for swap usage (taken while watching the screen, so the math may not add up exactly!)

Swap: 1024M Total, 373M Used, 651M Free, 36% In Use

The 128MB of physical was exhausted pretty quickly, and 256MB of swap is definitely insuffcient! :)

Steve

[cmb]

There's a problem right now with an apparent bug in Lighttpd where it chews up a bunch of memory and dies during upgrades. If you run into this, the console upgrade will still work fine. I actually like console upgrade better because you can just paste the URL in and it downloads and upgrades automatically.