Python scrip for OWL-Intuition

stephenw10 · Dec 18, 2012, 1:11 PM

Try using a virtual IP rather than a virtual interface:
http://doc.pfsense.org/index.php/What_are_Virtual_IP_Addresses%3F
I would try IP Alias.

Steve

vbhoj74 · Dec 18, 2012, 2:59 PM

That was my first try, as soon as I gave it a virtual ip alias it locked me out, saying i might me in a man in the middle attack, and then could not access the box at all. I tried it with a console cable but it looked in a crashed state, so I booted it, and it hung at every boot attempt I made. I've a vanilla pfsense flashed on another cflash, I'll try this again with this build to see how it goes.

Once I add an alias, what else do I need to do to make that subnet work. It's already passed on the firewall rules. I think i wold not need to NAT it, since this is IGMP subnet that we need to enable and not another routable subnet. Clueless what crashed my install.

stephenw10 · Dec 18, 2012, 3:21 PM

I'm pretty much guessing at this point! I've never tried adding an IP Alias to a bridge interface, could be some incompatibility you've discovered. It's an unusual config to say the least.

Steve

vbhoj74 · Dec 18, 2012, 4:34 PM

This document below does not seem to suggest usage of ip alias with ver 2 installs. This if for modem access configuration, but I guess provides a clue that ip alias may not work with ver 2?

http://doc.pfsense.org/index.php/Accessing_modem_from_inside_firewall

stephenw10 · Dec 18, 2012, 5:39 PM

Nope that's a different reason. The way PPP connections are handled changed which meant no longer had to use a virtual IP, you can just use the real interface. In fact IP Alias capability became stronger with 2.0.

Steve

vbhoj74 · Dec 19, 2012, 3:20 AM

I poped in a vanilla install which too had bridged interface (two physical interfaces excluding the wifi this time), and the system does not crash. However, the DHCP server does not start and seems to be expecting me to give it IP ranges in the alias subnet:

Dec 19 08:44:54	php: /status_services.php: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf bridge0' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.3 Copyright 2004-2011 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ bad range, address 192.168.1.200 not in subnet 224.192.0.0 netmask 255.255.0.0 If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you did get this software from ftp.isc.org and have not yet read the README, please read it before requesting help. If you intend to request help from the dhcp-server@isc.org mailing list, please read the section on the README about submitting bug reports and requests for help. Please do not under any circumstances send requests for help directly to the authors of this software - please send them to the appropriate mailing list as desc
Dec 19 08:44:54	dhcpd: exiting.

This was another msg at pfsense login screen I dont think I should much worry about but is a sore in the eye :)

You are accessing this router by an IP address not configured locally, which may be forwarded by NAT or other means. 

If you did not setup this forwarding, you may be the target of a man-in-the-middle attack.

vbhoj74 · Dec 20, 2012, 6:43 AM

I tried adding a simple alias to another embedded install, this time with address 192.168.x.0/24 range, allowed firewall rules from the alias subnet to any, also allowed any to the alias IP of the interface just in case I get lockout of admin access, saved it, was still working, rebooted the system and it was broken, never came up working. One thing to note is that, even this install had its LAN interface bridged. Really wanted the IP Alias thing to work and seems the right solution going forward, maybe I need to take time out and start with a factory image and see how it goes.

–-----------------
Program UPDATE

I did some bug fixes and upgrades to the python scrip which is now attached as ver 1.0.1 with changes as below:

1. Various bug fixes.
2. Writes two CSV files now, one as Event logger, another Day logger. Day logger just logs once at end of day.
3. You can control the frequency at which it logs to the event logger, presently it's set to log every 65th packet received. Approx writes once in 45-60 mins.
4. Added Currency Symbol to the cost.
5. Rounded figures to 2 decimal points.
6. Remember to change the NTP server to pool.ntp.org on pfsense, the default server runs couple of minutes late which kills the day logger & mail.
7. I changed the local time format to dd/mm/yy, you can probably change it back as required.

For the program to work, install python package if not already installed:

to INSTALL python ----

/etc/rc.conf_mount_rw
mkdir /home/tmp
setenv PKG_TMPDIR /home/tmp/
pkg_add -r http://files.pfsense.org/packages/8/All/python27-2.7.2_3.tbz
/etc/rc.conf_mount_ro

Installation Steps:

1. You may place both of them in /home and rename to *.py
2. Edit both files, check the comment areas to modify.
3. #chmod +x /home/owl.py
4. #chmod +x /home/send_gmail.py
5. Add under Pfsense>System>Routing>gateway
LocalNetwork Lanbridge 192.168.1.1 192.168.1.1
6. Add under Pfsense>System>Routing>routing
224.192.32. 19/32 LocalNetwork - 192.168.1.1 Lanbridge
7. Pfsense>Diagnostic>Backup>Download Backup config.xml
find /system, and add just below:
<shellcmd>python /home/owl.py</shellcmd>
save the file structure and restore.
8. Pfsense>System>general Setup>NTP time server> change to "pool.ntp.org"

Notes:

Step 5 & 6 :
edit the gateway & route as per your local LAN IP and interface names. Trying to find a better way of adding IP alias to the interface to get this working, until then the above works.

owl.txt
send_gmail.txt

vbhoj74 · Dec 21, 2012, 10:48 AM

Steve, I removed the port from LAN bridge in which I had plugged in the owl gateway and put it in a separate network. Assigned virtual ip alias to this interface, did respective firewall rules and NAT, and voila, it works. I guess virtual IP does not work with bridged interfaces properly. Thanks!

–----

So we can just add a virtual IP address in the subnet 224.192.32.0/24 to the LAN interface on which the owl gateway is plugged to and omit steps 5 & 6 in the above post.

stephenw10 · Dec 21, 2012, 1:13 PM

Nice. :)

I guess adding an Alias IP to a bridge interface is a pretty rare usage scenario.

Steve

vbhoj74 · Jan 3, 2013, 8:38 AM

Program UPDATE
–--------------

Bug fixes.
Check & create sub-directories by itself, no need to create directories manually
Support for db file log of daily kwH using sqlite
Now requires sqlite port
It has now two parts that remains resident, owl.py which write the log files and:
responder.py which responds to email query.
You can send an email (ID as defined in responder.py) with subject "OWL" and from and to dates in 1st & 2nd line of the mail body in the format yyyy-mm-dd as a query. The code will reply with an attached txt file containing statement of usage within dates, total kWH and avg kWH.

Version 1.0.2

Installation Steps:
1. Download and UNzip owl.rar https://dl.dropbox.com/u/2185098/generic/owl.rar
2. You may place all files in /home and rename to .py
2. Edit all .py files, check the comment areas to modify.
3. #chmod +x /home/.py
4. Add Firewall>Virtual IP>IP Alias 224.192.32.20/24 to your local interface
5. Add Firewall Rules>local interface:
Allow UDP * * 224.192.32.19 * * note
Allow IGMP * * * * * none
Allow * 224.192.32.19/24 * * * default none
6. Pfsense>Diagnostic>Backup>Download Backup config.xml
find /system, and add just below:
<shellcmd>python /home/owl.py &</shellcmd>
save the file structure and restore.
7. Pfsense>System>general Setup>NTP time server> change to "pool.ntp.org"

Notes:

to INSTALL python with sqlite port ----
/etc/rc.conf_mount_rw
mkdir /home/tmp
setenv PKG_TMPDIR /home/tmp/
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/old-releases/i386/8.1-RELEASE/packages/All/py26-sqlite3-2.6.5_1.tbz
/etc/rc.conf_mount_ro

I would be glad to know if you have used the code or taken any help from it.

EDIT: It seems to be working now, I made responder.py a subprocess of the main scrip instead of trying to start both the scrips using shellcmd.