IMSpector package



  • Here's what I'm working on in order:

    • Fix the MySQL logging - sql logging libraries aren't being packaged

    • ACL lists changed format between versions, fix support again

    • Update the IMSpector binary to the latest snapshot to fix MSN (the commit log says the code was untested, so who knows)

    • Fix parsing of old logs if possible (if not, just go to the shell and read the logs yourself, sheesh), new logs are FINE

    • Add new protocols and options that came with updated IMSpector (this includes message injection)

    • Output HTML escaped logs, otherwise the browser parses these (OUCH!)

    • Add Log cleanup functionality

    –Bill

    Updates: Crossed out items are believed to be completed.
    Update: March-04-2009 - Added a couple more todo's.  Should have these all cleaned up next week during the hackathon.



  • Downloading package configuration file… done.
    Saving updated package information... done.
    Downloading imspector and its dependencies... done.
    Checking for successful package installation... failed!

    Installation aborted.


    Reinstall (or install after failed reinstall) seems to b0rked. I am using 1.2.3-PRERELEASE-TESTING-VERSION
    built on Wed Feb 11 12:27:02 EST 2009

    Anyhow, thanks for the hard work billm! Its good to see active maintainer on a package as popular as imspector



  • Thanks for the heads up. I downloaded it:
    Checking for successful package installation… failed!
    Good luck Bill :) Ill test it again sometime

    jigp
    Davao



  • hi billm

    i ve been using IMSpector for a while now, but when installed, yahoo massenger audio stops working as per all my clients, so they cant communicate with their parties overseas if i install IMSpector, any reason for that?

    Thanks in advance

    hadi57



  • @n1ko:

    Downloading package configuration file… done.
    Saving updated package information... done.
    Downloading imspector and its dependencies... done.
    Checking for successful package installation... failed!

    Installation aborted.


    Reinstall (or install after failed reinstall) seems to b0rked. I am using 1.2.3-PRERELEASE-TESTING-VERSION
    built on Wed Feb 11 12:27:02 EST 2009

    Anyhow, thanks for the hard work billm! Its good to see active maintainer on a package as popular as imspector

    crap, teach me to not test the actual install after updating it.  sigh  Looks like the package installer looks in a slightly different directory than I thought - we didn't have the mysql package online.  Hoping to get it built and pushed before I head to work in a few minutes.

    –Bill



  • @billm:

    @n1ko:

    Downloading package configuration file… done.
    Saving updated package information... done.
    Downloading imspector and its dependencies... done.
    Checking for successful package installation... failed!

    Installation aborted.


    Reinstall (or install after failed reinstall) seems to b0rked. I am using 1.2.3-PRERELEASE-TESTING-VERSION
    built on Wed Feb 11 12:27:02 EST 2009

    Anyhow, thanks for the hard work billm! Its good to see active maintainer on a package as popular as imspector

    crap, teach me to not test the actual install after updating it.  sigh  Looks like the package installer looks in a slightly different directory than I thought - we didn't have the mysql package online.  Hoping to get it built and pushed before I head to work in a few minutes.

    –Bill

    Erm…in the meantime, I'm going to disable the mysql and sqlite installs again.

    --Bill



  • Installed fine now o/

    Thanks alot bill for the quick fix :)



  • @n1ko:

    Installed fine now o/

    Thanks alot bill for the quick fix :)

    0.8-5 is getting pushed now, it should fix mysql.  It takes a few minutes to sync with the package repo - I'll test it quick and roll back if it doesn't install.

    Update1: mysql installs now, made a typo in the sqlite install…syncing now.
    Update2: sqlite3 install works now, package looks to be fixed.

    --Bill



  • Great work bill, I will test the package and verify if the msn issue has been fixed.



  • Bill, just a suggestion for the future (no priority) I would suggest adding a log rotation/cleaup function.

    Delete logs older than 14 days for example. I tried finding a cronjob tutorial for pfsense ( normaly I do this with webmin on my debian machines…:P) but I couldn't find any.

    Could you tell us how to schedule some kind of weekly cronjob to delete all log files in the impsector dir?



  • @AudiAddict:

    Bill, just a suggestion for the future (no priority) I would suggest adding a log rotation/cleaup function.

    Delete logs older than 14 days for example. I tried finding a cronjob tutorial for pfsense ( normaly I do this with webmin on my debian machines…:P) but I couldn't find any.

    Could you tell us how to schedule some kind of weekly cronjob to delete all log files in the impsector dir?

    Good point, I'll see what I can do about that.  In the meantime insert this into /conf/config.xml before the and reboot (or back up, edit, and restore the config file):
    <minute>0</minute>
    <hour>0</hour>
    <mday></mday>
    <month>
    </month>
    <wday>*</wday>
    <who>root</who>
    <command></command>find /var/imspector -ctime +14 -type f -delete

    Note, I didn't test the above, it should work, I don't believe there are any special characters that the XML will barf on in there.

    –Bill



  • Awesome thanks for the new feature, I will test it sometime this weekend ( I dont' really have the option to reboot the firewall during the weekdays :P)

    Do I need to update/reinstall the imspector package to make this feature work? Or is this feature independent vs the imspector package and it doesn't display the feature in the webgui under imspector?

    Last but not least, the files are directories (example MSN\msnprofile\files) Does it do a rm -r ?



  • @AudiAddict:

    Awesome thanks for the new feature, I will test it sometime this weekend ( I dont' really have the option to reboot the firewall during the weekdays :P)

    Do I need to update/reinstall the imspector package to make this feature work? Or is this feature independent vs the imspector package and it doesn't display the feature in the webgui under imspector?

    The cron settings are built into pfSense, not part of the IMSpector package.  What I'll be doing is adding the above XML (slightly differently) to the package install.

    @AudiAddict:

    Last but not least, the files are directories (example MSN\msnprofile\files) Does it do a rm -r ?

    find will recurse.

    –Bill



  • Update the IMSpector binary to the latest snapshot to fix MSN (the commit log says the code was untested, so who knows)

    Bill just to confirm the fix you added for the MSN log bug seems to be solved!!

    The latest msn version is now being logged properly again by Imspector. Thanks allot for the help!!

    I'm hoping to see the log clean up functionality in the package aswel instead of editing files with ssh :)



  • hello bilim

    You did a great work! I have installed ver. 8.5 and it is logging (tested with irc an msn).
    I have tried the mysql function, entered die required parameters but it doesn't connect i think,
    however, its not that important as the other protocols.

    What about the other protocols such as gtalk? When are you going to tackle that?

    And for other people who own a paypal account, dont hesitate to make a donations for bilim.



  • @funnyfrish:

    hello bilim

    You did a great work! I have installed ver. 8.5 and it is logging (tested with irc an msn).
    I have tried the mysql function, entered die required parameters but it doesn't connect i think,
    however, its not that important as the other protocols.

    I've got a mysql server up local here, I'll test out the mysql here and see what's up, it should work at this point.

    @funnyfrish:

    What about the other protocols such as gtalk? When are you going to tackle that?

    And for other people who own a paypal account, dont hesitate to make a donations for bilim.

    Soon, been tied up on some contract work.  Worst case, this might wait until the hackathon.

    –Bill



  • Hello. No luck for gtalk? Been testing yahoo messenger, msn and irc but gtalk wont log in pfsense 1.2.2 impector version 8 .
    jigp
    Davao City



  • @jigpe:

    Hello. No luck for gtalk? Been testing yahoo messenger, msn and irc but gtalk wont log in pfsense 1.2.2 impector version 8 .
    jigp
    Davao City

    Do you even bother to read the threads you post in?  Two messages above you, the question was asked (by the only person financially interested in this package) and replied to in the message immediately prior to your post.  No.  It'll be in, when it's in.

    –Bill



  • @billm:

    I've got a mysql server up local here, I'll test out the mysql here and see what's up, it should work at this point.

    Me to, having a mysql 5.2.x running around for logging for other equipment on the lan.

    But imsepctor 'should' tell that the mysql 'could' be used as a logging plugin, if it's included in the build.
    I tried the -d parameter, but nothing …
    Looking at the source of the Mysql plugin code, it should some log lines, as these :


    Feb 21 11:26:59 imspector: Logging Plugin name: File IMSpector logging plugin
    Feb 21 11:26:59 imspector: Filter Plugin name: Misc IMSpector filter plugin
    Feb 21 11:26:59 imspector: Protocol Plugin name: Yahoo IMSpector protocol plugin
    Feb 21 11:26:59 imspector: Protocol Plugin name: MSN IMSpector protocol plugin
    Feb 21 11:26:59 imspector: Protocol Plugin name: IRC IMSpector protocol plugin
    Feb 21 11:26:59 imspector: Protocol Plugin name: ICQ-AIM IMSpector protocol plugin
    Feb 21 11:26:59 imspector: Protocol Plugin name: MSN IMSpector protocol plugin
    ...

    But there's nothing from the MySQL log plugin.
    I think it's not present in the build right now.

    Said that, I'll keep in mind that pfsense is a firewall - it's already marvellous it works as it does right now.



  • @Gertjan:

    @billm:

    I've got a mysql server up local here, I'll test out the mysql here and see what's up, it should work at this point.

    Me to, having a mysql 5.2.x running around for logging for other equipment on the lan.

    But imsepctor 'should' tell that the mysql 'could' be used as a logging plugin, if it's included in the build.
    I tried the -d parameter, but nothing …

    It was in the build.  I see what happened, I didn't update the packing list in the FreeBSD port when I added the mysql plugin to the build.  I'll push out a new update later today that should (really!) fix mysql logging.

    –Bill



  • Some bad news, I thought the msn issue was fixed with groupchat logs (unable to log with the correct contactname) but it seems this issue is still happening.

    It's not as bad, but it's still there, also it sometimes "fails"  to log certain messages. I know this has nothing to do with pfsense though, the makers of imspector will have to fix this.

    Keep up the good work Billm!



  • Blech, that sucks to hear AudiAddict.  Let me know if they fix the issue - I only updated the one protocol file in the package, so it'll be pretty easy to import any future fixes from them

    I haven't gotten to the MySQL fixes, been way to busy with work stuff.  Hopefully by this weekend, or during the weekend I'll update that and bump the package version.

    –Bill



  • Bill,

    Thanks for your continued work on this project.  I hope everyone in this thread has contributed to the bounty to help pay you for your time and work.



  • Can this be used if pfsense is set as a transparent bridge?  Since you bind the LAN to the WAN which interface would you monitor?

    Thanks



  • @kapara:

    Can this be used if pfsense is set as a transparent bridge?  Since you bind the LAN to the WAN which interface would you monitor?

    Presumably LAN still.  Give it a shot, report back, then we'll know and I can add the info to the wiki.  Thanks

    –Bill



  • I should have my alix system arriving next week.  Once I can replace my current pfSense I will convert it into a transparent bridge and test and report.



  • imspector not showing AIM chats….It shows Yahoo and MSN.

    Also what is your email for paypal donations?



  • @kapara:

    imspector not showing AIM chats….It shows Yahoo and MSN.

    Also what is your email for paypal donations?

    Hmmm, I'll have to see if I can remember my AIM credentials so I can get that fixed.

    bill [DOT] marquette [AT] gmail [DOT] com <– obvious substitutions apply.

    --Bill



  • @billm:

    @kapara:

    imspector not showing AIM chats….It shows Yahoo and MSN.

    Also what is your email for paypal donations?

    Hmmm, I'll have to see if I can remember my AIM credentials so I can get that fixed.

    bill [DOT] marquette [AT] gmail [DOT] com <– obvious substitutions apply.

    --Bill

    Mines logging AIM, but it's showing me as 'unknown' - as that's not in pfsense code (that I know of), I can only imagine it's something imspector related, I'll add that to my list of followups.

    –Bill



  • I was saying to myself :
    "Hey, I'm paying for a VMWare Workstation thing,", so I took a FreeBSD DVD ISO from the source (scary, BitTorrent loaded it in 35 minutes  :D) and installed a "everything but Xwindows stuff".

    I wish I had noted down what packages I had to load afterwards, but libiconv (from GNU) and gmake were needed.

    In imspector.h, this is included :
    #include <iconv.h>That one is not ok.
    I used a
    find /usr - name iconv.h (found it in /usr/local/include)
    and included THAT iconv.h file, otherwise many errors in "icqprotocolplugin.cpp" will pop up.

    I modified my Makefile slightly (only leaving the log plugging MySQL) and I had to correct icqprotocolplugin.cpp on line (appr.) 671 looks like
    const char *inbuf = string;
    because iconv() is defined as such.

    A make would actually build the thing !

    Having set the PREFIX in the Makefile correctly would even put the files on the right place.
    A ldconfig was needed to indicate the access to the .so library files, because inspector uses the high tech "late 'dynamic' binding".

    From doing this to actually building a pfSense package, well Billm, big hands down for you ^^</iconv.h>



  • Hello

    I've installed 0.8 with no problem, but why there is Gadu-Gadu (gg) plugin missing ?
    It is very popular in my country (PL), it's almost only one communicator :)
    Or how make it working with gg ?



  • Well, I'll go for the most simple answer :
    These five items need to find each other at the same time, at the same place:
    In "your Country"….,
    A Gadu Gadu user,
    With some heavy FreeBSD knowledge,
    ... and C langauge experiences,
    Capable of understanding the plug-in interface of ImSpector.
    Need to have a filter for Gadu Gadu communications.

    When these 5 joint up, your plug-in will come out very fast.

    Note that item 2 is often in conflict with item 5 ;-)



  • Thanks for the kind answer :)

    There is a reason why I'm asking about this protocol. I've seen on imspector site that gg protocol is also possible to watch/log.

    http://www.imspector.org/wordpress/?page_id=91

    icq_protocol=on
    irc_protocol=on
    msn_protocol=on
    yahoo_protocol=on
    gg_protocol=on
    jabber_protocol=on
    https_protocol=on

    GG plugin is ready to use but not included in pfSense package, I think.
    That is my question, expanded :)



  • I've managed to add GG protocol to config page and as a result to the config file. In /usr/local/pkg/imspector.xml I have added
    <option><name>GaduGadu</name>
    <value>gg</value></option>

    Now gg_protocol=on is added in config if selected on config page. First step made but no result in log. Any help ?



  • Any news on updating to new cvs? seems that march 17. build has more fixes to the 'msn2009' issues.



  • @n1ko:

    Any news on updating to new cvs? seems that march 17. build has more fixes to the 'msn2009' issues.

    I got a PM on this the other day, I haven't had time to update our sources and push a new build.  Hopefully later today.

    –Bill



  • @WielkoDuch:

    I've managed to add GG protocol to config page and as a result to the config file. In /usr/local/pkg/imspector.xml I have added
    <option><name>GaduGadu</name>
    <value>gg</value></option>

    Now gg_protocol=on is added in config if selected on config page. First step made but no result in log. Any help ?

    What you are probably missing is the redirect.  In function imspector_proto_to_port() in imspector.inc you'll find the protocol to port mappings.  Figure out what port GaduGadu is and add a case entry for it.  If you get it working, I'll update the build with the fix.  Thanks

    –Bill



  • Hi

    Seems to be working.
    gg port is 8074, after restart of imspector it is working for 1/2 hour, finally. But there is sth with config, I cannot add Gadu-Gadu as the only one protocol on config page in WebGui, only with any other protocol 'gg_protocol=on' is added to config file.
    I will watch it working and give report.
    Thanks for your advice



  • Hi ..

    I use pfsense 1.2.2 version and i installed IMSpector 0.8.9 version.BUt i can't see any log file else web gui.What should i do.Can you give me some idea .
    Thank you very much.



  • Hi. Just reinstall you imspector or install the latest package of imspector. I already encountered that before. I just upgrade it to the latest package in the webgui.
    jigp
    Davao City


Log in to reply