LCDProc 0.5.4-dev

stephenw10 · Jan 17, 2012, 5:39 PM

There are three different hardware types that use the SDEC LCD.

The X-Core: X500, X700, X1000 and X2500.

The X-Peak: X5000, X6000 and X8000.

The X-Core-e: X550e: X550e, X750e and X1250e. And the X-Peak-e: X5500e, X6500e and X8500e.
The X-Core-e and X-Peak-e are effectively the same hardware.

Also there are some variants based on these such as the ssl-100 and ssl-core but these are far more rare.

Steve

stephenw10 · Jan 18, 2012, 8:23 PM

OK some interesting things to report.
I did a clean install of v0.8 of the package on my X-Peak and less than 24Hrs later noticed the webgui became unavailable. As before it came back again but left no trace in the logs. However this evening I happened to be connected to the box via ssh when is happened so I could poke around a bit.

I think this explains why I couldn't connect to the webgui or login.


last pid: 56372;  load averages:  4.89,  3.27,  3.44           up 2+20:13:22  20:02:54
111 processes: 10 running, 84 sleeping, 1 zombie, 16 waiting
CPU: 24.0% user,  0.0% nice, 75.7% system,  0.4% interrupt,  0.0% idle
Mem: 55M Active, 16M Inact, 56M Wired, 1036K Cache, 59M Buf, 357M Free
Swap:

  PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
50705 nobody    74  r30  3368K  1548K RUN     27.9H 100.00% LCDd

Yes LCDd using 100% CPU. ::)

At that point I didn't want to kill LCDd because I knew that would cause lcdproc_client.php to fill the logs with errors and I was hoping the logs might contain clues. So I thought I would kill lcdclient.sh first:


[2.0.1-RELEASE][root@pfsense.fire.box]/root(22): ps aux | grep lcd
root   51028  0.0  0.3  3656  1508  ??  R    Mon07PM   0:00.00 /bin/sh /tmp/lcdclient.sh
root   51368  0.0  3.4 47452 17472  ??  S    Mon07PM   0:14.73 /usr/local/bin/php -f /usr/local/pkg/lcdproc_client.php
[2.0.1-RELEASE][root@pfsense.fire.box]/root(23): kill 51028
[2.0.1-RELEASE][root@pfsense.fire.box]/root(24): ps aux | grep lcd
root   51028  0.0  0.3  3656  1508  ??  R    Mon07PM   0:00.00 /bin/sh /tmp/lcdclient.sh
root   51368  0.0  3.4 47452 17472  ??  S    Mon07PM   0:14.73 /usr/local/bin/php -f /usr/local/pkg/lcdproc_client.php

It won't die!

Eventually I did kill LCDd and everything else died with it.

The system logs show nothing unusual.

For me the package is not stable on the X-Peak but I don't know if it's a driver problem or a bug in LCDd.
Hmmm… :-\

Steve

mdima · Jan 18, 2012, 8:28 PM

Hi Steve,
I was thinking about you and your LCDproc problem this evening (no, I don't think it's awkward)…

I am afraid it could be a problem on the driver or the combination between your panel and the driver... can you send me your lcdproc.conf file?

Thanks,
Michele

Brak · Jan 18, 2012, 8:54 PM

I have multiple X-Peak units (x550e, x750e) if you all have a specific tests you want me to run.

fmertz · Jan 18, 2012, 9:58 PM

@stephenw10:

OK some interesting things to report.

So this is on X-Peak only, Version 0.5.5 only, and with the "old" 0.5.4 driver?

Can you give it a shot with the 0.5.5 driver? https://github.com/downloads/fmertz/sdeclcd/sdeclcd.so

Was it working ok on 0.5.4?

stephenw10 · Jan 19, 2012, 5:05 PM

@ Brak The X-Peak is X6000 etc see list above. ;)

@ mdima The lcd in the X-Peak is identical to the X-Core but the parallel port hardware is different.

@fmertz That's using the v0.8 package downloaded via the package manager. I thought that had the 0.55 driver version already? It is 14395 bytes in size.

So today I bootup my X-e box to test some stuff and check out this nightmare boot log:


Jan 19 16:10:44 pfSense php: : pfSense package system has detected an ip change 0.0.0.0 ->   ... Restarting packages.
Jan 19 16:10:44 pfSense check_reload_status: Starting packages
Jan 19 16:10:45 pfSense php: : Restarting/Starting all packages.
Jan 19 16:10:45 pfSense php: lcdproc: Sync: Begin package sync
Jan 19 16:10:45 pfSense php: lcdproc: Sync: End package sync
Jan 19 16:10:45 pfSense php: lcdproc: Sync: Begin package sync
Jan 19 16:10:45 pfSense php: lcdproc: Sync: End package sync
Jan 19 16:10:48 pfSense LCDd: LCDd version 0.5.5 starting
Jan 19 16:10:48 pfSense LCDd: Using Configuration File: /usr/local/etc/LCDd.conf
Jan 19 16:10:48 pfSense LCDd: Listening for queries on 127.0.0.1:13666
Jan 19 16:10:48 pfSense php: : IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
Jan 19 16:10:49 pfSense php: : Restarting/Starting all packages.
Jan 19 16:10:49 pfSense login: login on console as root
Jan 19 16:10:49 pfSense sshlockout[40285]: sshlockout/webConfigurator v3.0 starting up
Jan 19 16:10:49 pfSense php: lcdproc: Sync: Begin package sync
Jan 19 16:10:49 pfSense php: lcdproc: Sync: Restarting the service
Jan 19 16:10:49 pfSense LCDd: Connect from host 127.0.0.1:64307 on socket 11
Jan 19 16:10:50 pfSense check_reload_status: Reloading filter
Jan 19 16:10:50 pfSense LCDd: Server shutting down on SIGTERM
Jan 19 16:10:51 pfSense php: lcdproc: Sync: End package sync
Jan 19 16:10:51 pfSense php: lcdproc: Sync: Begin package sync
Jan 19 16:10:52 pfSense php: lcdproc: Sync: Restarting the service
Jan 19 16:10:52 pfSense php: lcdproc: Connection to LCDd process lost  ()
Jan 19 16:10:52 pfSense LCDd: sock_send: socket write error
Jan 19 16:10:53 pfSense LCDd: LCDd version 0.5.5 starting
Jan 19 16:10:53 pfSense LCDd: Using Configuration File: /usr/local/etc/LCDd.conf
Jan 19 16:10:53 pfSense LCDd: Listening for queries on 127.0.0.1:13666
Jan 19 16:10:54 pfSense php: lcdproc: Sync: End package sync
Jan 19 16:10:54 pfSense LCDd: Server shutting down on SIGTERM
Jan 19 16:10:55 pfSense php: lcdproc: Connection to LCDd process lost  ()
Jan 19 16:10:55 pfSense LCDd: LCDd version 0.5.5 starting
Jan 19 16:10:55 pfSense LCDd: Using Configuration File: /usr/local/etc/LCDd.conf
Jan 19 16:10:55 pfSense LCDd: Listening for queries on 127.0.0.1:13666
Jan 19 16:10:56 pfSense LCDd: Connect from host 127.0.0.1:23883 on socket 12
Jan 19 16:10:57 pfSense LCDd: Server shutting down on SIGTERM
Jan 19 16:10:58 pfSense LCDd: LCDd version 0.5.5 starting
Jan 19 16:10:58 pfSense LCDd: Using Configuration File: /usr/local/etc/LCDd.conf
Jan 19 16:10:58 pfSense LCDd: sock_create_inet_socket: cannot bind to port 13666 at address 127.0.0.1 - Address already in use
Jan 19 16:10:58 pfSense LCDd: sock_init: error creating socket - Address already in use
Jan 19 16:10:58 pfSense LCDd: Critical error while initializing, abort.
Jan 19 16:10:58 pfSense LCDd: sock_send: socket write error
Jan 19 16:10:58 pfSense php: lcdproc: Connection to LCDd process lost  ()
Jan 19 16:10:58 pfSense php: lcdproc: Connection to LCDd process lost  ()
Jan 19 16:10:58 pfSense php: lcdproc: Connection to LCDd process lost  ()
Jan 19 16:11:10 pfSense php: lcdproc: Failed to connect to LCDd process Operation timed out (60)
Jan 19 16:11:10 pfSense php: lcdproc: Failed to connect to LCDd process Operation timed out (60)
Jan 19 16:11:21 pfSense php: lcdproc: Failed to connect to LCDd process Operation timed out (60)
Jan 19 16:11:21 pfSense php: lcdproc: Failed to connect to LCDd process Operation timed out (60)
Jan 19 16:11:32 pfSense php: lcdproc: Failed to connect to LCDd process Operation timed out (60)
Jan 19 16:11:32 pfSense php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Rebooted. Almost extactly the same thing! :-\

Steve

fmertz · Jan 19, 2012, 5:30 PM

@stephenw10:

@fmertz That's using the v0.8 package downloaded via the package manager. I thought that had the 0.55 driver version already? It is 14395 bytes in size.

So today I bootup my X-e box to test some stuff and check out this nightmare boot log:
Jan 19 16:10:48 pfSense LCDd: Using Configuration File: /usr/local/etc/LCDd.conf

From memory, we never updated the actual driver file in the package from V0.5.4 to V0.5.5. Feel free to check the V0.5.5 file at the link mentioned above. But first, I would definitely cleanup the boot area. Have you checked /etc/rc.d/? Might have more than one instance of several scripts.

mdima · Jan 19, 2012, 6:21 PM

@fmertz:

From memory, we never updated the actual driver file in the package from V0.5.4 to V0.5.5. Feel free to check the V0.5.5 file at the link mentioned above. But first, I would definitely cleanup the boot area. Have you checked /etc/rc.d/? Might have more than one instance of several scripts.

fmertz, maybe it's the case to ask to the development team to update the files under files.pfsense.org? You do or I do?
Is it stable the V0.5.5 driver for SDEC?

Thanks,
Michele

power_matz · Jan 20, 2012, 10:52 AM

I have the following problem: everytime I am disconnecting a device from an ethernet port the LCDProc is stopping.

stephenw10 · Jan 20, 2012, 4:22 PM

I have a feeling that's because every time pfSense detects an IP change on an interface it restarts all the packages.
I think it may be caused by trying to start a new instance of LCDd before the old one has been killed. LCDd then cannot open local port 13366 and quits leaving lcdclient.sh running and spamming the logs. In the worst case this happens several times with the result that several instances of lcdclient are running.
Perhaps this is caused by the sleep delay in between the kill statements? Perhaps it should kill LCDd first? :-\

In my opinion lcdclient.sh needs to quit after, say 3, tries if lcdproc_client.php cannot connect to LCDd. Is that what Seth had in mind when he setup the counter variable in lcdclient?

What do you think?

Steve

mdima · Jan 20, 2012, 5:07 PM

@stephenw10:

In my opinion lcdclient.sh needs to quit after, say 3, tries if lcdproc_client.php cannot connect to LCDd. Is that what Seth had in mind when he setup the counter variable in lcdclient?

What do you think?

Steve

Hi Steve,
yes, I agree… I will do this improvement in the next update of the package...

Michele

m4f1050 · Jan 20, 2012, 9:52 PM

I tried this LCDProc 0.5.4-dev and all it does is flash a few screens real fast then it says "Thank you for using pfSense" and sits on that screen forever… :O

stephenw10 · Jan 20, 2012, 9:59 PM

Could you show us your system logs?
I suspect you are seeing the same as me. Is that on your X-Core box?

Steve

mdima · Jan 20, 2012, 10:27 PM

@m4f1050:

I tried this LCDProc 0.5.4-dev and all it does is flash a few screens real fast then it says "Thank you for using pfSense" and sits on that screen forever… :O

Hi m4f,
Did you set all the new properties in the service configuration?
Can you tell me what values did you enter?

Thanks,
Michele

m4f1050 · Jan 21, 2012, 2:33 AM

Steve: Yes, it's on the X500 (I also have an X700 but they are the same hardware)

Hrm! I had uninstalled it, when I installed it again it installed 0.5.5 v 0.8 :O

Here are my settings:

Under Server tab:
Enable LCDProc: checked
Com Port: Parallel port 1 (/dev/lpt0)
Display Size: 2 rows 20 columns
Driver: Watchguard Firebox with SDEC (x86 only)
Refresh frequency: 5 seconds
The rest are set Default and the LED option is disabled

Under Screens tab I have enabled:
Version, Time, Uptime, System, Disk, Load, States,
Load Balancer, Interfaces, CPU Freq and Internet Traffic

m4f1050 · Jan 21, 2012, 5:59 AM

I get these on boot, but after it boots my LCD works with this newer version.

 Last 50 system log entries

Jan 20 22:05:37 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:37 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:25 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:25 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:13 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:13 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:02 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:02 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:05:01 php: : Hotplug event detected for opt2 but ignoring since interface is configured with static IP ()

Jan 20 22:05:01 php: : Hotplug event detected for opt4 but ignoring since interface is configured with static IP ()

Jan 20 22:05:01 php: : Hotplug event detected for opt1 but ignoring since interface is configured with static IP ()

Jan 20 22:05:01 php: : Hotplug event detected for opt3 but ignoring since interface is configured with static IP ()

Jan 20 22:04:54 check_reload_status: Linkup starting re5

Jan 20 22:04:54 check_reload_status: Linkup starting re4

Jan 20 22:04:54 check_reload_status: Linkup starting re3

Jan 20 22:04:54 kernel: re5: link state changed to DOWN

Jan 20 22:04:54 kernel: re4: link state changed to DOWN

Jan 20 22:04:54 kernel: re3: link state changed to DOWN

Jan 20 22:04:54 kernel: re2: link state changed to DOWN

Jan 20 22:04:54 check_reload_status: Linkup starting re2

Jan 20 22:04:50 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:50 sshlockout[17089]: sshlockout/webConfigurator v3.0 starting up

Jan 20 22:04:50 php: /index.php: Successful webConfigurator login for user 'admin' from 192.168.2.1

Jan 20 22:04:50 php: /index.php: Successful webConfigurator login for user 'admin' from 192.168.2.1

Jan 20 22:04:50 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:39 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:38 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:27 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:27 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:16 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:15 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:05 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:04:04 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:53 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:53 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:42 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:41 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:30 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:30 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:19 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:18 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:07 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:03:07 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:56 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:55 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:45 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:44 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:33 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:33 php: lcdproc: Failed to connect to LCDd process Operation timed out (60)

Jan 20 22:02:32 apinger: /usr/local/bin/rrdtool respawning too fast, waiting 300s.

EDIT:

I removed everything except SYSTEM TIME and SYSTEM STATS, good enough for me :)

mdima · Jan 21, 2012, 7:06 AM

Steve,
can you pls tell me which are your options/screen?

Thanks,
Michele

stephenw10 · Jan 21, 2012, 12:48 PM

I'm running the Time and Uptime screens. Both boxes.

Steve

mdima · Jan 21, 2012, 1:25 PM

@stephenw10:

I'm running the Time and Uptime screens. Both boxes.

Steve

mmmhhh… I am trying to mess it up as much as I can, but really, after every try I only get 1 server and 1 client running (also with enable/disable of ipsec that foces a package resync)... with:
Refresh: 1 second
Driver: sureelec
Screens: ALL

I don't even find the problem on the stats and time screens, both from a machine running since 31 days or a machine just booted...

Anyway, I will change the package in order to let the client make only 3 attempts to connect to the service then quit...

stephenw10 · Jan 21, 2012, 2:01 PM

I appreciate all your efforts, thanks. :)

It's looking like a driver issue is more likely then.
There seem to be two issues here:
1. When the package is started/restarted in quick succession some delay in the process can result in LCDd trying to start before the previous version has stopped. Or possibly the previous version simply does not stop as I have found when trying to kill it manually.

2. The LCDd process can start to use a lot of CPU time causing the box to become unresponsive (but still routes traffic OK).

For testing purposes we can either:
Build the old driver against 0.55 or
Swap out LCDd with a 0.54 version and equivalent driver.

What do think? I may have 0.54 version files the fmertz mailed to me. I'll try that if I do.

Steve