SpamD Package Needs Testers

[sullrich]

No problem-o.  I need to get my mail account setup shortly and forwarding to do some more tests.  I am definitely not done fixing / improving the package.

Thanks for testing!

[lambert]

I've been composing this message for several hours while fighting fires around here.  Please forgive me if you've fixed some of these things since I started.

Bugs found so far:

Once I added a few traplist entries I noticed that on the SpamD Database tab, the icons for blacklist and spamtrap are not displaying.

spamd.conf:
   the symlink needs to be in /usr/local/etc/spamd/ rather than /usr/local/etc.

The SpamD External Sources tab adds entries similar to :

test:
           :black:
           :msg="you are rejected":
           :method=http:
           :http:
           :file=www.openbsd.org/spamd/traplist.gz:

The documentation does not show the :http: entry.  My edits last night were geared toward removing the ":http:" lines.

Wierd, today I'm not getting the warning message when I start "/usr/local/libexec/spamd -d -v" manually even with those lines.

The all entry seems to be not generating correctly.  This is what is generated:

all:
       :whitelist:blacklist:nixspam:uatraps:test:

This is what should be generated, assuming you want "blacklist" to override "whitelist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist

If you want "whitelist" to override the "blacklist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist:whitelist

I would prefer to rename "whitelist" to something like "local-overrides" or "local-whitelist", and "blacklist" to "local-blacklist", just to avoid confusion when discussing having whitelist "whitelist" override entries in blacklist "blacklist".   Probably not terribly important in the grand scheme of things.

[sullrich]

@lambert:

I've been composing this message for several hours while fighting fires around here.  Please forgive me if you've fixed some of these things since I started.

Bugs found so far:

Once I added a few traplist entries I noticed that on the SpamD Database tab, the icons for blacklist and spamtrap are not displaying.

spamd.conf:
   the symlink needs to be in /usr/local/etc/spamd/ rather than /usr/local/etc.

The SpamD External Sources tab adds entries similar to :

test:
           :black:
           :msg="you are rejected":
           :method=http:
           :http:
           :file=www.openbsd.org/spamd/traplist.gz:

The documentation does not show the :http: entry.  My edits last night were geared toward removing the ":http:" lines.

Wierd, today I'm not getting the warning message when I start "/usr/local/libexec/spamd -d -v" manually even with those lines.

The all entry seems to be not generating correctly.  This is what is generated:

all:
       :whitelist:blacklist:nixspam:uatraps:test:

This is what should be generated, assuming you want "blacklist" to override "whitelist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist

If you want "whitelist" to override the "blacklist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist:whitelist

I would prefer to rename "whitelist" to something like "local-overrides" or "local-whitelist", and "blacklist" to "local-blacklist", just to avoid confusion when discussing having whitelist "whitelist" override entries in blacklist "blacklist".   Probably not terribly important in the grand scheme of things.

As far as I know every one of these are fixed.  Please reinstall and test again and report back further issues.

[lambert]

I have reinstalled the package: spamd Services 4.3.5

@lambert:

Once I added a few traplist entries I noticed that on the SpamD Database tab, the icons for blacklist and spamtrap are not displaying.

I still don't have icons for blacklist and spamtrap.  I get missing image markers in Safari and the words "blacklist" and "spamtrap" in Firefox, both on OS X 10.5.2.  Maybe you just don't have icons generated for them.

Also missing are the helo/ehlo and datestamp column headers, as was mentioned by someone else earlier.

Clicking the blacklist link does not result in the IP being added to /var/db/blacklist.txt.
Clicking the whitelist link does not result in the IP being added to /var/db/whitelist.txt.

Tried in both browsers.  The link icon dances back and forth when clicked.

@lambert:

The SpamD External Sources tab adds entries similar to :

test:
           :black:
           :msg="you are rejected":
           :method=http:
           :http:
           :file=www.openbsd.org/spamd/traplist.gz:

The documentation does not show the :http: entry.

The ":http:" line is still being created.  Seems to do no harm, but from the documentation, I can't see where it is needed/requested to be there.

@lambert:

The all entry seems to be not generating correctly.  This is what is generated:

all:
       :whitelist:blacklist:nixspam:uatraps:test:

This is what should be generated, assuming you want "blacklist" to override "whitelist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist

If you want "whitelist" to override the "blacklist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist:whitelist

I am still seeing the linear list in all as well.

[sullrich]

The missing items never existed to begin with.  This thread is concerning bugs for existing features only.

Please take your feature requests to the Bounty thread and they will be dealt with separately very soon.

Now with that said, what are the bugs with the existing features now?

[dhipo]

to extend tests …
what is the corect way to my remote users connect from thunderbird to my smtp server ( to send mails ) ?

without Spamd  i´ve been using Smtp-auth but now ... my users are treated as spammers ...

any way ??

[lambert]

@dhipo:

to extend tests …
what is the corect way to my remote users connect from thunderbird to my smtp server ( to send mails ) ?

without Spamd  i´ve been using Smtp-auth but now ... my users are treated as spammers ...

any way ??

Tell them to use port 587.  That is the mail submission port.  SpamD should only be interfering with port 25.

[pinoyboy]

On another note, the time for spamd to whitelist / blacklist seems a long time (i gfuess 25 min total as screen indicates?), is there any way to configure it for faster.  in the real world scenarion, the users will want their mail to be ASAP, but what is really "ideal" or the delicate balance for this spamd solution to be effective?

[lambert]

@sullrich:

The missing items never existed to begin with.  This thread is concerning bugs for existing features only.

Please take your feature requests to the Bounty thread and they will be dealt with separately very soon.

Now with that said, what are the bugs with the existing features now?

From the bounty starter post:
@lambert:

I've not previously managed a spamd installation so my requirements will be met by getting the package working and committed to the project with, at least the functionality it had back when sullrich initially built it plus the fixes for issues rcarr noted in his post.  I don't know enough about spamd to have further requirements at this time.  I'm trying to avoid having to learn it in too much detail too… :-)  Too many projects running right now.

http://forum.pfsense.org/index.php/topic,5660.msg33728.html#msg33728

If someone can get it working that far, I can find out what I want changed, if anything, beyond that and offer another bounty.

I believe that at least the following still applies, although he showed the issue differently with an external sources defined whitelist which still shows the same problem.

all:
       :whitelist:blacklist:uatraps:nixspam:mywhitelist:

should be according to exactly what rcarr had written:

all:
       :whitelist:blacklist:uatraps:mywhitelist:nixspam:mywhitelist:

@lambert:

The all entry seems to be not generating correctly.  This is what is generated:

all:
       :whitelist:blacklist:nixspam:uatraps:test:

This is what should be generated, assuming you want "blacklist" to override "whitelist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist

If you want "whitelist" to override the "blacklist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist:whitelist

The fix for the below is to delete line 66 of spamd.inc.  If you don't want to, don't.  I've paid my part of the bounty.

@lambert:

The SpamD External Sources tab adds entries similar to :

test:
           :black:
           :msg="you are rejected":
           :method=http:
           :http:
           :file=www.openbsd.org/spamd/traplist.gz:

The documentation does not show the :http: entry.

[dhipo]

Tell them to use port 587.  That is the mail submission port.  SpamD should only be interfering with port 25.

i will try another way .. because i don´t know how and who are my users …. i have an small wifi provider ...
i think in use another name to my mx servers and users can use normal smtp servers .. on the normal servers i only send mails from authenticated users on MX spamd do receiving ..
i this way i  think ... no user configuration was involved ..

[sullrich]

@lambert:

@sullrich:

The missing items never existed to begin with.  This thread is concerning bugs for existing features only.

Please take your feature requests to the Bounty thread and they will be dealt with separately very soon.

Now with that said, what are the bugs with the existing features now?

From the bounty starter post:
@lambert:

I've not previously managed a spamd installation so my requirements will be met by getting the package working and committed to the project with, at least the functionality it had back when sullrich initially built it plus the fixes for issues rcarr noted in his post.  I don't know enough about spamd to have further requirements at this time.  I'm trying to avoid having to learn it in too much detail too… :-)  Too many projects running right now.

http://forum.pfsense.org/index.php/topic,5660.msg33728.html#msg33728

If someone can get it working that far, I can find out what I want changed, if anything, beyond that and offer another bounty.

I believe that at least the following still applies, although he showed the issue differently with an external sources defined whitelist which still shows the same problem.

all:
       :whitelist:blacklist:uatraps:nixspam:mywhitelist:

should be according to exactly what rcarr had written:

all:
       :whitelist:blacklist:uatraps:mywhitelist:nixspam:mywhitelist:

@lambert:

The all entry seems to be not generating correctly.  This is what is generated:

all:
       :whitelist:blacklist:nixspam:uatraps:test:

This is what should be generated, assuming you want "blacklist" to override "whitelist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist

If you want "whitelist" to override the "blacklist" entries:

all:
       :nixspam:whitelist:uatraps:whitelist:test:whitelist:blacklist:whitelist

The fix for the below is to delete line 66 of spamd.inc.  If you don't want to, don't.  I've paid my part of the bounty.

@lambert:

The SpamD External Sources tab adds entries similar to :

test:
           :black:
           :msg="you are rejected":
           :method=http:
           :http:
           :file=www.openbsd.org/spamd/traplist.gz:

The documentation does not show the :http: entry.

Thanks, I'll get these taken care.  It is not that I do not wish to fix these things, I am just trying to keep this thread focused to get it knocked out quickly since I am running out of time before I have to travel for a week.  Just trying to get everyone up and running before a long delay.

[sullrich]

In regards to the ordering issue, I believe you will want to create the entries in the order you want them to apply.  This might not be optimal but it does work.  Let's discuss improvements to this in the Bounty thread.

[blueivy]

I'm unsure if this has been reported (I have a feeling it has).

The buttons on the SpamD Database page (the ones for deleting, whitelisting, blacklisting etc.) at the end of the row do not appear to work. I have a SpamTrap address I want to now remove and I'm unable to. I assume manually editing the config files will allow me to do this in the meantime?

This is hte code for the delete button that is showing up in Firefox:

===
javascript:toggle_on("d24", "/themes/nervecenter/images/icons/icon_x_p.gif");getURL("spamd_db.php?buttonid=d24&srcip=

The actual code in the source though is (Note that I changed the @ in the javascript below to AT so that it wouldn't show up as a link in the forum and I get MORE spam!)

===
javascript:toggle_on("d24", "/themes/nervecenter/images/icons/icon_x_p.gif");getURL("spamd_db.php?buttonid=d24&srcip=''''catchallATblueivy.co.uk''''&action=delete", outputrule);'

It's got something to do with the quotes around \catchallATblueivy.co.uk.

[Guest]

@blueivy:

===
javascript:toggle_on("d24", "/themes/nervecenter/images/icons/icon_x_p.gif");getURL("spamd_db.php?buttonid=d24&srcip=''''catchallATblueivy.co.uk''''&action=delete", outputrule);'

It's got something to do with the quotes around \catchallATblueivy.co.uk.

Looks like maybe just a quote mismatch, you have double quotes in the front of the address and single quotes after the address.  Hopefully a simple fix.

[sullrich]

I am currently traveling without access to fix these issues.  If you can, please hold off on further bug reports until I return so that I do not feel overwhelmed.  Thanks!

[Knobee]

I think that there may be an error in parsing "bad" e-mail addresses when displaying the SpamD database:

I'm seeing the following when "sorting by Attempts":

| Type | IP | From | To | Attempts  ↑ |
| GREY | 77.205.81.230 | | ty{n2000@HECOUNCIL.ORG> | 1209078047 |
| GREY | 83.9.63.26 | <martac-ti|< td=""></martac-ti|<> | {pyr@AOLLC.COM> | 1209076738 |

Knobee

[sullrich]

@Knobee:

I think that there may be an error in parsing "bad" e-mail addresses when displaying the SpamD database:

I'm seeing the following when "sorting by Attempts":

| Type | IP | From | To | Attempts  ↑ |
| GREY | 77.205.81.230 | | ty{n2000@HECOUNCIL.ORG> | 1209078047 |
| GREY | 83.9.63.26 | <martac-ti|< td=""></martac-ti|<> | {pyr@AOLLC.COM> | 1209076738 |

Knobee

It's a known issue when someone puts a | in the email address.

[sullrich]

@submicron:

@blueivy:

===
javascript:toggle_on("d24", "/themes/nervecenter/images/icons/icon_x_p.gif");getURL("spamd_db.php?buttonid=d24&srcip=''''catchallATblueivy.co.uk''''&action=delete", outputrule);'

It's got something to do with the quotes around \catchallATblueivy.co.uk.

Looks like maybe just a quote mismatch, you have double quotes in the front of the address and single quotes after the address.  Hopefully a simple fix.

This looks like a parsing bug of some kind, notice there is an email address for the ip address.  Can you reproduce this?

[Guest]

Here are the issues I've come across so far with our testing:

• Spamtrap email addresses don't get saved or applied.  Entering in an address doesn't add it to the database, and eventually the field is listed again as empty.  Re-Adding the entry similarly doesn't work.

• No RRD Graph reporting.  This was part of the original bounty and I'm quite sure it was in the original spamd package.

• As previously reported, entries in the "queue" cannot be whitelisted, blacklisted or deleted.  The icons just wiggle instead of doing anything.

[flachance]

I concur with those issues.  I am seeing the same on one of my client's firewall.  I just didn't have time to put it down in words.  I personally find the third one most annoying.

Add to this the parsing error that Knobee & blueivy reported earlier.  I too am seeing that in some entries.

Francois