On our production mail server we’re using a so called qmail toaster setup. It’s essentially the same setup as described by shupp.org. However, instead of using SquirrelMail for webmail, we opted for SqWebMail and maildrop. The nice thing about SqWebMail is that it offers a web interface for configuring the maildrop mailfilters. That way, every user can configure their filtering rules individually.
The problem here is that vdelivermail (of vpopmail) doesn’t support those mailfilter files. For that to work we need vdelivermail to call maildrop to process those files on an individual basis. However, this requires to setup the processing instructions for every mail account manually. Of course, this is unbearable on a large setup.
We came up with a patch for vdelivermail that automatically chain-calls maildrop if it finds a mailfilter file in the domain’s account directory. Some parts are based on another patch which source I forgot. I mainly extended it to be less strict and to publish certain internal vpopmail/vdelivermail variables as environment variables that could be used inside the mailfilter – namely username, userdir, domainname, domaindir.
If it finds the file in the domain’s directory, it will delegate the mail to the domain-wide mailfilter.
The domain-wide mailfilter will delegate the mail to the users mailfilter if it exists. If not, it will simply do a standard mail delivery to the account’s INBOX.
If vdelivermail can’t find any domain-wide mailfilter it will simply default to standard mail delivery to the account’s INBOX.
Here is the patch against vpopmail 5.4.17:
vpopmail-5.4.17-vdelivermail.c-maildrop.patch
If you’re too lazy to patch the vpopmail sourcecode yourself, here is the already patched vdelivermail.c:
vdelivermail-5.4.17-maildrop.c
(Rename it to vdelivermail.c and place it in the vpopmail source directory and compile.)
Our current mail server setup goes something like this:
qmail (with SMTP-AUTH + TLS) + vpopmail + SpamAssassin / ClamAV + maildrop + custom scripts + SqWebMail (for webmail) + Dovecot (for SSL IMAP)
In our setup by default every account has the following hierarchy:
INBOX
Spam
learn no-spam
learn spam
The Spam subfolder is automatically created by the domain-wide mailfilter once the first email hits the account.
We’re using the following domain-wide filter (/home/vpopmail/domains/yourdomainname/mailfilter):
Again, our modified vdelivermail takes care of checking the domain for that mailfilter as described above.
Here is our script used to create the Spam subfolders in the domain-wide mailfilter above:
If you want to use it, make sure to adjust the paths in the file to your setup.
So, now consider that the Spam subfolders have been created. People can use these folders to instruct what mails are spam and what mails are ham (non-spam). They can use this mechanism to report missed spam mails or to correct mistakenly classified non-spam mails.
Now, we wanted to get information out of those folders and try to educate our SpamAssassin’s bayes filter. Here is what we start in our crontab every hour:
This script will make a list of all account’s spam and ham folders in every domain under /home/vpopmail/domains and feed the mails therein into sa-learn. Again, if you want to use the script, make sure to adjust the paths to your setup.
Since our server runs Debian installing SpamAssassin is pretty easy. We just had to make sure to start SpamAssassin’s spamd with the following adjusted settings in /etc/default/spamd.conf:
# Change to one to enable spamd ENABLED=1 # Options # See man spamd for possible options. The -d option is automatically added. OPTIONS="-m 2 -H -u vpopmail --vpopmail --nouser-config" # Set nice level of spamd NICE="--nicelevel 10"
This will launch spamd globally for all vpopmail accounts. The bayes database is shared among all accounts and is stored in /home/vpopmail/.spamassassin/ with the following local.cf in /etc/spamassassin/local.cf:
Having a global shared bayes database for your SpamAssassin is controversial. I won’t go into detail on why it’s good or bad to do so. All I can say is it’s working pretty well for us.
Currently, in our setup the SpamAssassin and ClamAV scanning is integrated into qmail via Qmail-Scanner. This has the disadvantage that antivirus filtering is done pretty deep down the processing chain. I plan to migrate this to simscan, which stops and rejects any virus mail at SMTP level and thus prevents them from entering the queue, i.e. nothing will get onto your box. Qmail-Scanner on the other hand puts virus mails into a global quarantine directory.
As for IMAP access to mail accounts, we’ve previously used Courier IMAP and just recently migrated all accounts to Dovecot. Both servers are proven and very reliable. However, we found Dovecot to be more secure and way faster. This is especially noticeable when doing searches and threading on mailboxes. Also, Dovecot is easier to set up as there are fewer external dependencies than with Courier IMAP.
Dovecot offers support for vpopmail. If you want to give it a try in your setup, make sure to enable this support via the “–with-vpopmail” switch when running configure.
As already stated above, we chose to use SqWebMail for webmail and mail filter configuration. By default the latter is disabled. You’ll have to explicitly enable the mail filter interface. Check the documentation on how to do that.
Well, trying to test the patch we encountred some trouble to compile vdelivermail … and patch posted here seem to be broken.
here is our added patch solution :
— vpopmail-5.4.17/vdelivermail.c 2006-10-20 17:15:13.000000000 0200
vpopmail-5.4.17-patchmaildrop/vdelivermail.c 2006-10-20 17:12:24.000000000 0200
@@ -187,7 187,7 @@
void get_arguments(int argc, char **argv)
{
#ifdef QMAIL_EXT
– int i;
// int i;
#endif
char *tmpstr;
@@ -214,6 214,11 @@
void postprocess(void)
{
#ifdef QMAIL_EXT
int i;
#endif
lowerit(TheUser);
lowerit(TheDomain);
Jerome, thanks for the comment. Indeed there was a problem when you’re using the QMAIL_EXT option. We don’t use QMAIL_EXT over here, so that’s why I missed that one. Sorry for the trouble!
Anyway, I’ve updated the patch and the vdelivermail.c above. Thanks again!
Hi Andre,
You are right about the QMAIL-EXT option.
This kind of implementation using maildrop for filtering purpose seems to be the cleanest for us in our mind, because we do not have to change .qmail-default behavior in many ways (ie like qmailadmin stuff…). And therefore, using vdelivermail make us sure to do many more check regarding vpopmail, like for sample ‘bouncing over-quota mails over full mailboxes’.
But regarding deeper in the code and testing more your filtering solution… We encountred anothers questions : the big issue we founded is is that .qmail checks are completly bypassed if the mailfilter file is present for the domain.
Do you aggreed with our conclusion ?
Regards,
Re Andre,
As precised in our last comment, we wanted the delivery from vdelivermail to go through maildrop.
But we wanted to keep the behavior of vdelivermail on all delivering cases (like bouncing, defrals …). This is particularly important on the way that vdelivermail handle quota, and the users notifications.
We also wanted the granularity of a using a mailfilter configuration file for domains (ie /home/vpopmail/domains/onedomain/mailfilter), all domains (ie /home/vpopmail/domains/mailfilter) … and indeed a default one in /etc/mailfilter.
If no mailfilter file is present, the we would like that vdelivermail deliver the mail as it does naturally.
So we rewritted a patch from scratch, witch is working fine in ours many tests …
Regards,
Here it is :
— vpopmail-5.4.17/vdelivermail.c 2006-06-29 21:36:43.000000000 0200
vpopmail-5.4.17.patchmaildrop_actinux/vdelivermail.c 2006-10-25 16:59:17.000000000 0200
@@ -22,6 22,13 @@
*
*/
/* Patch insertion for using maildrop delivery program October 2006
* by Actinux Team
* Jerome MOLLIER-PIERRET ,
* Brian PASSANTE
*
*/
/* include files */
#include
#include
@@ -81,6 88,9 @@
/* from qmail’s wait.h for run_command() */
#define wait_exitcode(w) ((w) >> 8)
/* Maildrop binary path */
#define MAILDROP “/usr/local/bin/maildrop”
/* Forward declarations */
int process_valias(void);
void get_arguments(int argc, char **argv);
@@ -93,6 103,7 @@
void usernotfound(void);
int is_loop_match( const char *dt, const char *address);
int deliver_quota_warning(const char *dir, const char *q);
int launchmaildrop(void);
/* print an error string and then exit
@@ -610,7 621,7 @@
maildir_to_email(address));
}
– switch (deliver_to_maildir (address, DeliveredTo, 0, message_size)) {
switch (launchmaildrop()) {
case -1:
vexiterr (EXIT_OVERQUOTA, “user is over quota”);
break;
@@ -620,6 631,22 @@
case -3:
vexiterr (EXIT_BOUNCE, “mail is looping”);
break;
case -200:
/*start the old launch */
printf (“trying_normal_delivery: “);
switch (deliver_to_maildir (address, DeliveredTo, 0, message_size)) {
case -1:
vexiterr (EXIT_OVERQUOTA, “user is over quota”);
break;
case -2:
vexiterr (EXIT_DEFER, “system error”);
break;
case -3:
vexiterr (EXIT_BOUNCE, “mail is looping”);
break;
default:
return;
}
default:
return;
}
@@ -1042,6 1069,7 @@
maildir_to_email(newdir), date_header());
}
err = deliver_to_maildir (dir, DeliveredTo, read_fd, sb.st_size);
close (read_fd);
@@ -1063,3 1091,67 @@
return (strcasecmp (compare, (dt 14)) == 0);
}
int launchmaildrop(void)
{
char *prog;
int child;
char *(args[4]);
int wstat;
char mailfilter_file[256];
FILE *fs;
printf(“trying_through_maildrop: “);
sprintf(mailfilter_file, “%s/mailfilter”,TheDomainDir);
if ( (fs=fopen(mailfilter_file, “r”)) == NULL ) {
/* if no mailfilter in domain then check in vpopmail dir */
sprintf(mailfilter_file, “%s/%s/mailfilter”,VPOPMAILDIR,DOMAINS_DIR);
if ( (fs=fopen(mailfilter_file, “r”)) == NULL ) {
/* if no mailfilter in vpopmail dir check in /etc/ */
sprintf(mailfilter_file, “/etc/mailfilter”);
if ( (fs=fopen(mailfilter_file, “r”)) == NULL ) {
/* no mailfilter file present */
printf(“Ouups,_no_mailfilter_file: “);
return -200;
}
}
}
snprintf(prog, AUTH_SIZE, “| /usr/bin/env HOME=%s VDOMAINDIR=%s VUSERDIR=%s VUSER=%s VDOMAIN=%s preline \”%s\” %s”, TheDomainDir, TheDomainDir, vpw->pw_dir, TheUser, TheDomain, MAILDROP, mailfilter_file);
/*to put maildrop in debug mode uncomment theses line
snprintf(prog, AUTH_SIZE, “| /usr/bin/env HOME=%s VDOMAINDIR=%s VUSERDIR=%s VUSER=%s VDOMAIN=%s preline \”%s\” %s -V 2″, TheDomainDir, TheDomainDir, vpw->pw_dir, TheUser, TheDomain, MAILDROP, mailfilter_file);
printf(prog);
*/
while ((*prog == ‘ ‘) || (*prog == ‘|’)) prog;
switch(child = fork())
{
case -1:
printf(“Unable to fork: %d.”, errno);
vexit(EXIT_DEFER);
case 0:
args[0] = “/bin/sh”; args[1] = “-c”; args[2] = prog; args[3] = 0;
sig_catch(SIGPIPE,SIG_DFL);
execv(*args,args);
printf(“Unable to run /bin/sh: %d.”, errno);
exit(EXIT_DEFER); /* the child’s exit code will get caught below */
}
wait(&wstat);
waitpid(wstat,&child,0);
switch(wait_exitcode(wstat))
{
case 77: return -1; /* Overquota return code of maildrop */
case 0: return 0;
default: return -2; /* All other case */
}
}
Hi Jerome,
thanks for your patch. I’ll check it out one of these days.
As for the domain wide filtering problem and .qmail being bypassed, I need to look into that.