PFSense 2.0 Beta5 1/19 build system locks up

[sporocyst]

OK.  I set debug.pfftpproxy to 1 and I can no longer connect to a PPTP VPN.  If I set the flag to 0, I can connect, set it back to 1 and cannot connect.  I'm not sure why or how these are related, but they appear to be.

[jimp]

I checked with Ermal (the author of the FTP and PPTP proxies) and he said at the moment that does disable them both, though it didn't seem to be entirely intentional.

He's working on replicating the issue and fixing it now.

[cmbaker82]

I've also had the lockup occur during pptp vpn.  I believe it was the pptp connections that triggered the other lockups i reported and not the web browsing.

[jimp]

It may be then, but I have no PPTP activity on my network and yet it still locked up, and it locked up precisely when I did a LIST on an FTP site.

A couple more patches hit the repo this afternoon so the next snapshot will be worth trying, but it probably won't be uploaded until later tonight/early tomorrow.

[jimp]

FYI- On today's snap, I can no longer induce the lockup with FTP. Haven't tried PPTP yet though.

I still get failed LIST command using passive FTP (and active even more so) but the router isn't seizing, which is a good thing…

[sporocyst]

I just updated to todays snapshot.  Should know within a couple of hours if I am still getting the PPTP lockup.

[jimp]

Looks like there is a solid lead on the remaining FTP issue(s), so a fix should be coming along before too long.

[cmbaker82]

I'm using 2.0-BETA5 (i386) built on Fri Jan 21 09:14:03 EST 2011 and still experiencing the lockups immediately after starting an ftp transfer using filezilla.

[jimp]

I'm on the one from overnight:
2.0-BETA5 (i386)
built on Thu Jan 20 20:59:52 EST 2011

Can you try to load that one and see if there is any difference?

I tried both active and passive FTP and I was able to use both, but very spotty results.

There are more FTP changes to come though, so it may not really be fixed quite yet. It probably won't be on a snap from today, but maybe one from tomorrow, depending on when the patch goes in.

[mromero]

Very UNHAPPY to report my first lockup with PFSENSE since we've been using it.

Something is broken on today's snapshot. Had to power cycle to get it up again. One step forward two steps backward  ;D

Does anyone know if this bug is replicated in the 64 version? We are going back to the 64 bit version on a backup box with a week old snapshot and not updating until a fix is confirmed.

[cmbaker82]

I can try that specific build.  Is there a way to "upgrade" to it, or do i need to reflash? 
another update, I'm still on 2.0-BETA5 (i386) built on Fri Jan 21 09:14:03 EST 2011 and the ftp issue seems to only occur when transferring multiple files.  Today I was able to transfer 226 files (out of a 5000 file batch) before it locked up.

[jimp]

Just try the most recent build instead. A firmware update should work, no need to redo the whole thing.

[clahti]

I am also experiencing lockups, we do not use FTP, just normal ingress and egress traffic.  This happens on both cluster nodes will hang randomly, we are using an internal server for OpenVPN so just port forwards through pfsense.  I experienced this with 01-24.2011 snapshot, upgrading to today's 01-26-2011 and will let you know if this solves the problem.  The symptoms are things work great, then a box will just freeze, no console errors, just in hung state.  Are there logs I can look at or forward to the list?

[jimp]

Not really, when it gets stuck there really isn't anything that can be done.

FTP and PPTP are the main causes of the hang, or were. My router hasn't hung up in days (I'm on a snap from late on the 20th) The most recent snapshots should behave better all around, but there is still a panic out there that some people are hitting.

[clahti]

Ok, so now I have attempted to upgrade the master node to the latest snapshot, it seems stuck in the update process.  A ps -ef on the box shows:

PID  TT  STAT      TIME COMMAND
  26  v0  Is+    0:00.01  sh /etc/rc autoboot
  254  v0  I+    0:04.28  /usr/local/bin/php -f /etc/rc.bootup
10734  v0  SN+    0:00.50  /bin/sh /var/db/rrd/updaterrd.sh
24545  v0  SN+    0:00.00  sleep 60
26261  v0  S+    0:00.14  /usr/sbin/tcpdump -s 256 -v -l -n -e -ttt -i pflog0
26317  v0  S+    0:00.17  logger -t pf -p local0.info
46810  v0  I+    0:00.04  /usr/sbin/pkg_delete -x libwww-5.4.0_4
47319  v0  I+    0:00.00  /bin/rm -r /var/db/pkg/libwww-5.4.0_4
24777  0  R+    0:00.00  ps -ef
55403  0  Is    0:00.00  -sh (sh)
56612  0  I      0:00.00  /bin/sh /etc/rc.initial
60290  0  S      0:00.02  /bin/tcsh

It has been like this for an hour or so, it is still serving up 6 CARP vips, not sure if I should just reboot the box.  The slave node has not yet been upgraded.

[jimp]

Looks like it's still running a package upgrade. Does the console not tell you what it's doing?

(As an aside, on FreeBSD and thus pfSense, use ps uxawww)

[clahti]

yes my Linux underwear is showing :)  At any rate I hooked up a console to the master, hit enter a couple of times and the box automagically finished the upgrade process.  This box was not asleep as I could ssh into it, but only console activity made it actually complete.  Strange.  I am updating the slave node now.

[cmbaker82]

I installed 2.0-BETA5 (i386) built on Wed Jan 26 10:45:46 EST 2011 yesterday afternoon.  By the time anyone tried to connect this morning it had locked up.  I'm not sure if this is caused by the FTP /PPTP issue or not but I will test ftp transfer yet this morning and see if we can trigger another lockup

[cmbaker82]

I just tried ftp transfer again and was able to reproduce the lockup.

[clahti]

yes, lockup still persists with me as well.  The master node was up for around 10 hours last I checked and then locked up sometime during the night.  Thankfully the slave took over at that point, it has an uptime of around 18 hours right now.  Both have 2.0-BETA5 (i386) built on Wed Jan 26 09:44:03 EST 2011 installed.