IGMP Proxy is broken?
-
Ok. HOLY CRAP!
This makes no sense, but, feeling dejected after my last post, I decided to throw in the towel.
I stopped the igmpproxy service. And as soon as I did, the box sounded the "chime," completed the boot sequence, and the remaining output appeared on the screen and returned to the menu.
It also wrote the last of the boot logs, which were filled with errors related to LCDproc (which I forgot I added back to this system to tinker with this morning). I've removed that package completely from the system again.
I'm going to restart one more time and see if that resolved the boot problems. Unfortunately, I don't think it has anything to do with the igmpproxy interface issue, but what the hell! I might as well just try one more time.
-
Alright, this stinks. Whenever my system starts, the boot sequence gets stuck until I stop the igmpproxy service. I removed the LCDproc package, so it definitely has nothing to do with that.
Maybe the logs will help
Aug 8 15:51:04 sshlockout[62465]: sshlockout/webConfigurator v3.0 starting up
Aug 8 15:51:04 login: login on ttyv0 as root
Aug 8 15:51:02 php-fpm[53527]: /rc.start_packages: Restarting/Starting all packages.
Aug 8 15:50:53 syslogd: kernel boot file is /boot/kernel/kernel
Aug 8 15:50:53 syslogd: exiting on signal 15
Aug 8 15:50:53 php: rc.bootup: Started IGMP proxy service.
Aug 8 15:50:53 igmpproxy[64540]: All routes removed. Routing table is empty.
Aug 8 15:50:53 igmpproxy[64540]: Got a interupt signal. Exiting.
Aug 8 15:50:53 igmpproxy[64540]: select() failure; Errno(4): Interrupted system call
Aug 8 15:50:36 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:49:42 igmpproxy[64540]: RECV Membership query from 192.168.0.1 to 224.0.0.1
Aug 8 15:49:42 igmpproxy[64540]: RECV Membership query from 192.168.0.1 to 224.0.0.1
Aug 8 15:49:07 igmpproxy[64540]: RECV Membership query from 10.10.0.3 to 224.0.0.1
Aug 8 15:49:07 igmpproxy[64540]: RECV Membership query from 10.10.0.3 to 224.0.0.1
Aug 8 15:48:45 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:48:45 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:48:45 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:48:45 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:48:27 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:48:10 php-fpm[3901]: /index.php: Successful login for user 'admin' from: 10.10.0.36
Aug 8 15:48:10 php-fpm[3901]: /index.php: Successful login for user 'admin' from: 10.10.0.36
Aug 8 15:47:37 igmpproxy[64540]: RECV Membership query from 192.168.0.1 to 224.0.0.1
Aug 8 15:47:37 igmpproxy[64540]: RECV Membership query from 192.168.0.1 to 224.0.0.1
Aug 8 15:46:57 igmpproxy[64540]: RECV Membership query from 10.10.0.3 to 224.0.0.1
Aug 8 15:46:57 igmpproxy[64540]: RECV Membership query from 10.10.0.3 to 224.0.0.1
Aug 8 15:46:36 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:46:36 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:46:36 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:46:36 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:46:23 php-fpm[55941]: /rc.dyndns.update: phpDynDNS (xxxxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Aug 8 15:46:18 igmpproxy[64540]: RECV Membership query from 10.10.0.2 to 224.0.0.1
Aug 8 15:46:14 php-fpm[55941]: /rc.dyndns.update: phpDynDNS (xxxxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Aug 8 15:46:10 php-fpm[45394]: /rc.dyndns.update: phpDynDNS (xxxxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Aug 8 15:46:01 php-fpm[45394]: /rc.dyndns.update: DynDNS (xxxxx) There was an error trying to determine the public IP for interface - wan(em0). Probably interface is not a WAN interface.
Aug 8 15:46:01 php-fpm[45394]: /rc.dyndns.update: DynDns (xxxxx): IP address could not be extracted from checkip.dyndns.org
Aug 8 15:45:55 kernel: pid 3672 (ntpd), uid 0: exited on signal 11 (core dumped)
Aug 8 15:45:46 php-fpm[7815]: /rc.dyndns.update: phpDynDNS (xxxxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Aug 8 15:45:41 check_reload_status: Reloading filter
Aug 8 15:45:41 check_reload_status: Restarting OpenVPN tunnels/interfaces
Aug 8 15:45:41 check_reload_status: Restarting ipsec tunnels
Aug 8 15:45:41 check_reload_status: updating dyndns WAN_DHCP
Aug 8 15:45:39 php-fpm[55941]: /rc.newwanip: rc.newwanip: Info: starting on em0.
Aug 8 15:45:38 check_reload_status: rc.newwanip starting em0
Aug 8 15:45:32 igmpproxy[64540]: RECV Membership query from 192.168.0.1 to 224.0.0.1
Aug 8 15:45:32 igmpproxy[64540]: RECV Membership query from 192.168.0.1 to 224.0.0.1
Aug 8 15:45:31 php-fpm[7815]: /rc.dyndns.update: DynDNS (xxxxx) There was an error trying to determine the public IP for interface - wan(em0). Probably interface is not a WAN interface.
Aug 8 15:45:31 php-fpm[7815]: /rc.dyndns.update: Dyndns debug information (xxxxx): Could not resolve checkip.dyndns.org to IP using interface IP 192.168.0.16.
Aug 8 15:45:21 igmpproxy[64540]: adding VIF, Ix 1 Fl 0x0 IP 0x0102a8c0 em1, Threshold: 1, Ratelimit: 0
Aug 8 15:45:21 igmpproxy[64540]: adding VIF, Ix 0 Fl 0x0 IP 0x1000a8c0 em0, Threshold: 1, Ratelimit: 0 -
Looking at the logs, it looks like igmpproxy is being started other than by the bootup script and is then blocking the bootup script. If it's blocking the bootup script, this suggests it's starting in the foreground rather than being started by the daemon. If imgpproxy is starting in the foreground, the bootup script won't resume until igmpproxy exits. When it does exit (e.g. because you terminate the service) the bootup script resumes and logs the "Started IGMP proxy service". This seems to be consistent with what you're seeing.
Have you got igmpstart.sh as well as having patched services.inc, as you don't need both? If you've patched services.inc (which is the better way to do it) you can delete igmpstart.sh. Alternatively, when upgrading igmpproxy have you somehow put the executable itself (or some other script) in a path that gets launched during boot (I can see from an earlier post that you're running NanoBSD).
If igmpproxy is starting too early somehow, it might also be starting before your VLANs have been initialised, which might explain your other issues?
-
I did delete the igmpstart.sh file, as I knew I did not want it started both ways. I just went and confirmed it is not there any longer, just to be sure I hadn't forgotten to remount the file system as rw before I attempted to delete it or something. It is definitely not there any longer (see attached screen capture).
As far as updating the package, I only followed your lead with the pkg commands. I didn't do anything else. I'm not smart enough to make any other changes to the way it installed. : )
Regarding starting before the VLANs have been initialized, I don't think that is the case. Watching the bootup process, I can see that it brings up the vlans, then the interfaces, then down the road is where it runs into the script being blocked, or whatever, after igmpproxy starts. Below is the code from /etc/inc/services.inc. These are the only changes that I have made to the system startup or the startup for igmpproxy.
/* NOTE: -d4 means everything LOG_WARNING and smaller */ mwexec("/usr/local/sbin/igmpproxy -v {$g['tmp_path']}/igmpproxy.conf"); log_error(gettext("Started IGMP proxy service."));
-
Which particular version /flavour of pfSense are you running please?
When you patched services. inc, did you update the entire line or just substitute d4 for v?
-
I'm running 2.2.3. When I patched services.inc I changed -d to -v -v. I removed one of the -v yesterday when I suspected the issue might be the logs getting flooded. I didn't change any other parts of the line at all.
-
OK. I think you mentioned earlier you were running the NanoBSD version? I can't think why that would make any difference, but I'm running the full version (and have only tested it on that).
On your current setup, if you terminate the igmproxy service (so you get the chimes etc), but then start the igmpproxy service again after that, does it recognise your VLAN then? (Trying to work out if the VLAN issues are related to the boot sequence not completing …)
Also, did the previous beta version of igmpproxy that's currently included in pfSense block the boot up process? If you just want to call it a day and fall back to the previous version, upgrading pfsense to 2.2.4 should do it.
-
I restarted igmpproxy just to see if it brought up the vlan interface instead, if I started/stopped the service manually. No joy. Then I thought maybe igmpproxy was confused about the interfaces due to the fact that the native interface (em1) was added after the vlan interface (elm1_vlan10), so I tried replacing the downstream interface assignment to em1 (Native in my setup), saving the changes, then reverting back to assigning the vlan interface (LAN in my setup) to see if igmpproxy would then bring up the correct interface. Again, no joy. But, I did see some weird entries in the log about the service failing, etc that I have no idea how to interpret. Starting to think going back to vanilla install (maybe via upgrade to 2.2.4) would be best. Though I REALLY want to replace my residential gateway from my ISP with my pfSense box. Maybe the updated igmpproxy will make it tinot 2.2.5, and maybe that will solve all my problems? Maybe.
Aug 11 09:59:33 php-fpm[87778]: /services_igmpproxy.php: Started IGMP proxy service. Aug 11 09:59:33 igmpproxy[7323]: All routes removed. Routing table is empty. Aug 11 09:59:33 igmpproxy[7323]: Got a interupt signal. Exiting. Aug 11 09:59:32 igmpproxy[7323]: select() failure; Errno(4): Interrupted system call Aug 11 09:59:32 php-fpm[63059]: /services_igmpproxy.php: Started IGMP proxy service. Aug 11 09:59:32 php-fpm[63059]: /services_igmpproxy.php: The command '/usr/local/sbin/igmpproxy -v -v /tmp/igmpproxy.conf' returned exit code '255', the output was '' Aug 11 09:59:32 igmpproxy[8261]: MC-Router API already in use; Errno(48): Address already in use Aug 11 09:59:32 igmpproxy[7323]: adding VIF, Ix 1 Fl 0x0 IP 0x0102a8c0 em1, Threshold: 1, Ratelimit: 0 Aug 11 09:59:32 igmpproxy[7323]: adding VIF, Ix 0 Fl 0x0 IP 0x1000a8c0 em0, Threshold: 1, Ratelimit: 0 Aug 11 09:59:32 php-fpm[63059]: /services_igmpproxy.php: Started IGMP proxy service. Aug 11 09:59:32 igmpproxy[27523]: All routes removed. Routing table is empty. Aug 11 09:59:32 igmpproxy[27523]: Got a interupt signal. Exiting. Aug 11 09:59:32 igmpproxy[27523]: select() failure; Errno(4): Interrupted system call Aug 11 09:59:17 igmpproxy[27523]: RECV Membership query from 192.168.2.1 to 224.0.0.1 -
So I've come to the conclusion that igmpproxy is completely bonkers. I took your advice, Andrew453, and upgraded to 2.2.4 so that igmpproxy would go back to default version, etc. It did. I'm running 0.1 beta2 again. And, guess what? igmpproxy now recognizes my vlan interfaces! Yay! Right?
Nope. For some reason, it configured all of my interfaces, instead of just the ones that are configured in /tmp/igmpproxy.conf!? Is it possible igmpproxy is pulling it's configuration information from somewhere else? Or just completely ignoring the .conf file?
Aug 12 13:05:23 igmpproxy: Note: adding VIF, Ix 5 Fl 0x0 IP 0x01420a0a ovpns1, Threshold: 1, Ratelimit: 0 Aug 12 13:05:23 igmpproxy: Note: adding VIF, Ix 4 Fl 0x0 IP 0x01500a0a bridge2, Threshold: 1, Ratelimit: 0 Aug 12 13:05:23 igmpproxy: Note: adding VIF, Ix 3 Fl 0x0 IP 0x010c0a0a em1_vlan12, Threshold: 1, Ratelimit: 0 Aug 12 13:05:23 igmpproxy: Note: adding VIF, Ix 2 Fl 0x0 IP 0x01000a0a em1_vlan10, Threshold: 1, Ratelimit: 0 Aug 12 13:05:23 igmpproxy: Note: adding VIF, Ix 1 Fl 0x0 IP 0x0102a8c0 em1, Threshold: 1, Ratelimit: 0 Aug 12 13:05:23 igmpproxy: Note: adding VIF, Ix 0 Fl 0x0 IP 0x1700a8c0 em0, Threshold: 1, Ratelimit: 0##------------------------------------------------------ ## Enable Quickleave mode (Sends Leave instantly) ##------------------------------------------------------ quickleave phyint em0 upstream ratelimit 0 threshold 1 altnet 224.0.0.0/4 altnet 184.60.0.0/16 altnet 184.61.0.0/16 phyint em1_vlan10 downstream ratelimit 0 threshold 1 altnet 10.10.0.0/24 phyint lagg0_vlan80 disabled phyint lagg0_vlan10 disabled phyint em1_vlan12 disabled phyint em1_vlan80 disabled phyint bridge2 disabled phyint em1 disabledWTH?!? Why, why, why?
-
So, does anyone have any idea why igmpproxy would ignore the conf file and add all interfaces, instead of just those set up in the igmp proxy service page? Any ideas where to look for an alternate conf file? I've looked everywhere and can't find it. I've restarted the service several times and it still is configuring all interfaces.
Should I just wipe clean my installation and start anew? At a loss here as to what to do next.
-
When you messed up the box to the state where you don't know what's being used, time to reformat and reinstall from scratch.
-
Hi. Re your query as to why igmpproxy is showing all interfaces, looking at the source code for igmpproxy, it iterates over all your interfaces and shows that message - it doesn't necessarily mean it's assigning them as upstream/downstream. So I think that's normal. e.g. in my post #30, the ath_wlan0 isn't upstream/downstream but is still shown.
Your install of pfSense does seem to be doing funny things though, so doing a backup of your config, followed by a reinstall and restore might be prudent anyway.
If you want to have another go at upgrading igmpproxy to 0.1, it seems that starting igmpproxy on some versions of pfSense by using mwexec means that it doesn't detach and blocks the main thread. Apparently the solution is to start it using mwexec_bg to force it into the background. That wasn't necessary for me for some reason (so I haven't tried it) but you can give it a go if you wish.
To do that, make the previously referenced change to the line in services.inc, but also change mwexec to mwexec_bg.
-
Decided to reinstall. Moved to a full install on HDD. Didn't have time to update the package to the new version yet, but will give it a try soon. Will report back results.
Thanks for all the helpful suggestions and support.
-
Well, I tried updating the package to the 0.1 version. Followed the same procedure outlined by Andrew453. Once again, the boot sequence stalled just after "Generating RRD Graphs…." and did not proceed until I stopped the igmpproxy service, at which time, the boot sequence proceeded and the menu finally appeared in the terminal. That was a bit discouraging.
So, I decided to try the suggestion to replace mwexec with mwexec_bg in services.inc. That change did allow the boot process to proceed all the way to completion without interruption or intervention. Small victory.
Also, it appears igmpproxy is only configuring 2 interfaces, instead of all interfaces. Another small victory(?). The only problem is, it is still using the em1 interface, instead of em1_vlan10. So, I deleted the interface that I created and assigned the em1 port to. Of course, this causes igmpproxy to fail.
Aug 17 22:03:59 igmpproxy[64911]: There must be at least 2 Vif's where one is upstream. Aug 17 22:03:59 igmpproxy[64911]: adding VIF, Ix 0 Fl 0x0 IP 0x0200a8c0 em0, Threshold: 1, Ratelimit: 0I personally think this is a bug. The proxy service is properly configured with an upstream and a downstream interface. The em1_vlan1 is a valid and active interface on the system. The operating system correctly reports that the underlying em1 port is up and active, even if the port is not assigned to an interface in pfSense.
$ ifconfig em1 em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500 options=5209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwfilter,vlan_hwtso>ether 00:01:69:xx:xx:xx inet6 fe80::201:69ff:fe01:24b8%em1 prefixlen 64 scopeid 0x2 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwfilter,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast>$ ifconfig em1_vlan10 em1_vlan10: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500 options=3 <rxcsum,txcsum>ether 00:01:69:xx:xx:xx inet6 fe80::201:69ff:fe01:24b8%em1_vlan10 prefixlen 64 scopeid 0xe inet 10.10.0.1 netmask 0xffffff00 broadcast 10.10.0.255 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active vlan: 10 vlanpcp: 0 parent interface: em1</full-duplex></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast>I can't see any valid reason why it won't properly function. I'd love to hear if anyone has any other input. This is driving me crazy.
-
@Andre453, I just wanted to say thank you again. I was looking at this some more and thought that there had to be more debug output than what was showing up in the system log in pfSense.
In spite of the fact that igmpproxy says to use the "-v -v" to be verbose and include debug messages, I've found it does not include the debug messages. I tried manually launching igmpproxy using "-d -v -v" and found a wealth of debug information. I'll post it here for your feedback, but I'm thinking it is probably time to start my own thread for help with this problem.
I've hijacked this one long enough. I've also thought, that perhaps you should update/edit your original post to include the information about what has worked (update to 0.1, modify services.inc, use mwexe_bg, if necessary, perhaps also to use "-d -v -v" for troubleshooting, if that might help someone else), so that anyone looking into this topic, doesn't have to read through three more pages of my rambling, whining, and foot stamping about not being able to get this service to work properly.
That said, here is the output from debug:
Searching for config file at '/tmp/igmpproxy.conf' Config: Quick leave mode enabled. Config: Got a phyint token. Config: IF: Config for interface em0. Config: IF: Got upstream token. Config: IF: Got ratelimit token '0'. Config: IF: Got threshold token '1'. Config: IF: Got altnet token 224.0.0.0/4. Config: IF: Altnet: Parsed altnet to 224/4. Config: IF: Got altnet token 184.60.0.0/16. Config: IF: Altnet: Parsed altnet to 184.60/16. Config: IF: Got altnet token 184.61.0.0/16. Config: IF: Altnet: Parsed altnet to 184.61/16. IF name : em0 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 1 Allowednet ptr : 1018040 Config: Got a phyint token. Config: IF: Config for interface em1_vlan10. Config: IF: Got downstream token. Config: IF: Got ratelimit token '0'. Config: IF: Got threshold token '1'. Config: IF: Got altnet token 10.10.0.0/24. Config: IF: Altnet: Parsed altnet to 10.10.0/24. IF name : em1_vlan10 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 2 Allowednet ptr : 1018080 Config: Got a phyint token. Config: IF: Config for interface lagg0_vlan80. Config: IF: Got disabled token. IF name : lagg0_vlan80 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 0 Allowednet ptr : 0 Config: Got a phyint token. Config: IF: Config for interface lagg0_vlan10. Config: IF: Got disabled token. IF name : lagg0_vlan10 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 0 Allowednet ptr : 0 Config: Got a phyint token. Config: IF: Config for interface em1_vlan12. Config: IF: Got disabled token. IF name : em1_vlan12 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 0 Allowednet ptr : 0 Config: Got a phyint token. Config: IF: Config for interface em1_vlan80. Config: IF: Got disabled token. IF name : em1_vlan80 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 0 Allowednet ptr : 0 Config: Got a phyint token. Config: IF: Config for interface bridge2. Config: IF: Got disabled token. IF name : bridge2 Next ptr : 0 Ratelimit : 0 Threshold : 1 State : 0 Allowednet ptr : 0 buildIfVc: Interface em0 Addr: 192.168.0.2, Flags: 0xffff8943, Network: 192.168.0/24 buildIfVc: Interface lo0 Addr: 127.0.0.1, Flags: 0xffff8049, Network: 127/8 Found config for em0 adding VIF, Ix 0 Fl 0x0 IP 0x0200a8c0 em0, Threshold: 1, Ratelimit: 0 Network for [em0] : 192.168.0/24 Network for [em0] : 224/4 Network for [em0] : 184.60/16 Network for [em0] : 184.61/16 There must be at least 2 Vif's where one is upstreamFor some reason, igmpproxy is using the loopback interface instead of the em1_vlan10, for which an "Allowednet ptr" was created.
Anyway, thanks again for your helpful suggestions.
-
No problem. Hope you can get it sorted.
It could as you say be a bug in igmpproxy, or some quirk in the combination between igmpproxy and your particular hardware. A bit odd though, as the vlans come up and work ok with igmpproxy on my setup when I tried it (post #31).
I tried to update my original post to have a consolidated set of information all in one place, but you can't modify posts on a previous page unfortunately. With a bit of luck igmpproxy 0.1 will be built into the next version of pfSense anyway.
-
This is 0.1! :(
I hope that, at least, igmpproxy will work better for the majority of folks using it, even if I continue to struggle with it. I really don't know what is wrong with it. It would be nice if we could get it worked out before 0.1 becomes included in the standard release, but I have no idea how to go about debugging it. My latest guess is that it has something to do with the order that pfSense records the interfaces. Either that, or igmpproxy prefers 192.168.x.x addresses to 10.10.x.x. ???
Any-who…thanks again.
-
So you say you have reinstalled the box. Did you reconfigure it from scratch? I cannot see any value in reinstalls if you keep importing the same (very likely broken) configuration after that.