UPnP support

rsw686

This thread has gotten relatively quiet. I'm almost done adding the ability to enable/disable logging of packets passed by miniupnpd rules. That way if your running bittorent or another app which fills the firewall log you can stop that and see packets being passed by other rules. I just need to see if Thomas will include it in the official miniupnpd source and release a new version or not?

Any other suggestions or complaints?

rsw686

I just committed version 20061029 which includes selectable logging of rules handled by miniupnpd. Embeddeds you can download the image below which is pfSense 1.0.1 with miniupnpd and the corporate theme files added.

http://wgnrs.dynalias.com:81/pfsense/pfSense-1.0.1-Embedded-Miniupnpd.img.gz
md5sum: EC25DF20C1B6B296480A480D1EE57FD2

For those wondering about the corporate theme see the following. I found it too nice a theme to not include. I personally use it on my box and its easier to not have to change the script when building the image to post here. However if theres objection I can drop it from future versions.

http://forum.pfsense.org/index.php/topic,2501.msg15220.html#new

Or if you go back a few posts you can upgrade your existing embedded install using the instructions provided.
md5sum miniupnpd: 0A42E0752ADBA5AFE94CF89C702F8609

ZPrime

@rsw686:

I just committed version 20061029 which includes selectable logging of rules handled by miniupnpd. Embeddeds you can download the image below which is pfSense 1.0.1 with miniupnpd and the corporate theme files added.

http://wgnrs.dynalias.com:81/pfsense/pfSense-1.0.1-Embedded-Miniupnpd.img.gz
md5sum: EC25DF20C1B6B296480A480D1EE57FD2

Should I be able to flash this from within a running pf? Or is this a pull-card-and-re-write image only?

I propose that you start a fresh thread in one of the other sections that is embedded-specific… I like your modifications (adding upnp, the extra theme) and frequent updates, but I wouldn't want to clutter this thread up with irrelevant information...

rsw686

@bradenmcg:

Should I be able to flash this from within a running pf? Or is this a pull-card-and-re-write image only?

I propose that you start a fresh thread in one of the other sections that is embedded-specific… I like your modifications (adding upnp, the extra theme) and frequent updates, but I wouldn't want to clutter this thread up with irrelevant information...

No you need to pull the card and rewrite the image. Or you can update within your current version if you read back a few posts.

I could start another thread however frequent updates are for full installs (can reinstall from the package manager) and embeddeds. I just post the embedded image link alongside. The last two updates adressed major issues (one major memory leak, one small, and selectable logging so your firewall log doesn't fill).

I won't be adding anything else other than miniupnpd and the coporate theme (if theres no objections) to the image. Otherwise it becomes to specific.

ZPrime

Gotcha. I already patched myself using your script (very handy!) and I'm going to go grab the corporate theme "the hard way," although it looks like I may have a fresh reflash coming since 1.0.1 seems to have broken the serial console on embedded. :(

rsw686

Been working on miniupnpd lately and now have the bytes/packets in/out implemented. I am waiting on Thomas to include this in his source and put out a new version. That way I'm not branching off from his source. Anyways for those who would like to have the latest and greatest here ya go.

http://wgnrs.dynalias.com:81/pfsense/miniupnpd/devel/miniupnpd
md5: CB9C843FD9A01CFD55AD280F497A45E2

http://wgnrs.dynalias.com:81/pfsense/miniupnpd/sh-replace-binary.sh

To install on full installs or embedded install upload the above two files. Then from ssh or the console enter the shell - option 8. Run the following commands. The resulting md5 should match above.

cd /tmp
chmod +x sh-replace-binary.sh
./sh-replace-binary.sh miniupnpd

sullrich

Nice work :)

rsw686

@sullrich:

Nice work :)

Thanks!! I worked on this some more and have added the ability to set the maximum downstream and upstream bitrates, and the option to use the pfSense system uptime instead of the daemon uptime. I am going to talk to Thomas and see if he will include these options. If not I will most likely add just include them in his future releases (reasons follow).

I like the option of seeing the pfSense system uptime as the dhcp renewal on the wan interface restarts miniupnpd every 12 hours (varying on your ISP) so its uptime serves no purpose to me. Maybe I should just have it default this way and not give the option in the package gui? I'm sure others have this same issue as well. The above post with the WinXP screenshot where is says duration is the uptime.

Also the bitrate is currently hard coded. I have a feeling some UPnP programs use this to determine the ideal amount of bandwidth to use for downloading, etc.

Heres a screenshot of the package gui. I'm debating on how the downstream and upstream bandwidths should be inputed, in bitrate as shown, Mbps, etc. I'm thinking bitrates are hard to read quickly as the values shown are 5Mbps down/2Mbps up. Then again if I do it in Mbps people on 768/128 DSL would be fractional. So maybe the best option is Kbps.

Of course these fields are all optional. The only required field is the interface. If the bitrates are omitted miniupnpd will use the nic link speed which most likely would be 100Mbps maybe 10Mbps.

Comments and suggestions are appreciated.

billm

Auto fill in the bitrate from the shaper settings (in kbps) if set :)

–Bill

sullrich

@billm:

Auto fill in the bitrate from the shaper settings (in kbps) if set :)

–Bill

Yes, please! I know for a fact a lot of folks are entering the WRONG values for the shaper. Hell, if it bit me, I know it is biting others!

bits to kilobytes to megabytes to petabytes… Who needs this burden to do math... This is one of the things I hope to fix before 2.0 comes out.

rsw686

@sullrich:

@billm:

Auto fill in the bitrate from the shaper settings (in kbps) if set :)

–Bill

Yes, please! I know for a fact a lot of folks are entering the WRONG values for the shaper. Hell, if it bit me, I know it is biting others!

bits to kilobytes to megabytes to petabytes… Who needs this burden to do math... This is one of the things I hope to fix before 2.0 comes out.

Sure I can do that. Is there a way to hide the xml fields if using the shaper values and show them otherwise? I don't see how there is, but figured I'd ask.

Otherwise I think the best option is to hide the fields and use the traffic shaper values behind the scenes. Showing them would just confuse people and they might start changing them. I did some testing with the traffic shaper setup and even if the user doesn't want the traffic shaper if they just go through the first page and define the lan and wan speeds it will save that to the config. Then they can just click next and click the pfSense logo and it will leave the shaper disabled.

Only thing left is how to inform the user of the above. I didn't see a field type I could put in the xml file to just show text on the configuration page.

sullrich

@rsw686:

@sullrich:

@billm:

Auto fill in the bitrate from the shaper settings (in kbps) if set :)

–Bill

Yes, please! I know for a fact a lot of folks are entering the WRONG values for the shaper. Hell, if it bit me, I know it is biting others!

bits to kilobytes to megabytes to petabytes… Who needs this burden to do math... This is one of the things I hope to fix before 2.0 comes out.

Sure I can do that. Is there a way to hide the xml fields if using the shaper values and show them otherwise? I don't see how there is, but figured I'd ask.

Otherwise I think the best option is to hide the fields and use the traffic shaper values behind the scenes. Showing them would just confuse people. I did some testing with the traffic shaper setup and even if the user doesn't want the traffic shaper if they just go through the first page and define the lan and wan speeds it will save that to the config. Then they can just click next and click the pfSense logo and it will leave the shaper disabled.

Only thing left is how to inform the user of the above. I didn't see a field type I could put in the xml file to just show text on the configuration page.

Sure. Simply unset the items from the xml array in the pre processing functions. I'd have to look at which one it is but IIRC its something like before_form or something or other. Grep should find it.

rsw686

@sullrich:

Sure. Simply unset the items from the xml array in the pre processing functions. I'd have to look at which one it is but IIRC its something like before_form or something or other. Grep should find it.

Sounds like a plan. I will look into that.

Alright I got it all working. When the shaper values are defined the fields are hidden and the values from the shaper are used. Otherwise the user can fill in the values. Thomas got back to me and theres a few things I still need to resolve. I should have the new version committed Wednesday night.

rsw686

I just committed version 20061107. Full installs reinstall the package via the package manager. Embedded install you can use the sh-update-miniupnpd.sh talked about a few posts back or reflash.

http://wgnrs.dynalias.com:81/pfsense/miniupnpd/sh-update-miniupnpd.sh
http://wgnrs.dynalias.com:81/pfsense/pfSense-1.0.1-Embedded-Miniupnpd.img.gz

miniupnpd 20061107: CB9C843FD9A01CFD55AD280F497A45E2

ZPrime

spectacular stuff. Looking back at last Feb. I'm amazed at how far this project has come. :) My continued thanks to all of you who have put effort into UPnP as it has made my life at home a lot easier. (No more screwing around with port forwards for random IM clients and stuff…) Plus, I can have 2 xbox 360s under one roof without any problems. ;D

Superman

It's seems with this latest package the 100% CPU usage problem is back.

I'm get the old error:```
miniupnpd[10622]: recv (state0): Operation timed out


It seems to happen every couple of days. I have to restart the miniupnpd service and then all is well. The earlier packages seemed to do it as well, but it took longer between times the problem would reappear.

I noticed that as of today (Nov 10/06) there is new version of miniupnpd on it's site, but the changelog doesn't mention any changes that would fix this problem.

rsw686

@Superman:

It's seems with this latest package the 100% CPU usage problem is back.

I'm get the old error:```
miniupnpd[10622]: recv (state0): Operation timed out


It seems to happen every couple of days. I have to restart the miniupnpd service and then all is well. The earlier packages seemed to do it as well, but it took longer between times the problem would reappear.

I noticed that as of today (Nov 10/06) there is new version of miniupnpd on it's site, but the changelog doesn't mention any changes that would fix this problem.

Hmm I've never experienced that. I really have no way to test this on as my box is using DHCP on the WAN interface and the lease is renewed every 12 hours. The packages are resynced when this happens and miniupnpd restarts.

The changes that have been made to the code were fixing a major memory leak (about 40kb leak per add/remove mapping) and additional code to read the bytes/packets in/out and system uptime. No major changes have occurred and I can't see how the changes would affect this.

I will contact Thomas the author of miniupnpd and see what he has to say. The latest version on his site has improved error handling and I was planning to update the pfsense package later today. I will hold off until I receive word from Thomas.

rsw686

I was looking at the code changes for the 20061110 version posted today and it might fix this issue. The error you receive is coming from upnphttp.c line 273: The new version adds the h->state = 100; which would close out that state instead of endlessly looping through it. I will build and update the package to the latest version and you can let me know if it addresses the issue. I sent Thomas another email asking what he thinks. I will let you know the response.

if(n<0)
{
syslog(LOG_ERR, "recv (state0): %m");
h->state = 100;
}

Superman

Thanks for the quick response. I will try out the package when it is available! I'll post my observations. Of course I'll probably have to wait a few days to see if the problem resurfaces.

Thanks!! :D

rsw686

@Superman:

Thanks for the quick response. I will try out the package when it is available! I'll post my observations. Of course I'll probably have to wait a few days to see if the problem resurfaces.

Thanks!! :D

I committed it a 15 min ago so you should be good to go. Just reinstall via package manager installed packages tab.