Download

CUL_ESA.tar.gz Version: Oct 25th 2010

• Supports ESA1000WZ power meter (infrared version, possibly others as well)
• Supports ESA1000Z and ESA1000GAS gas meters (infrared version and version using impulse transducer for the Elster-Kromschroeder BK-G4)
• Features proper graph files to plot current and cumulated day/month consumption

Photos / Screenshots

Installation

Copy 19_CUL_ESA.pm and all .gplot files to your FHEM directories.
Sadly adding autocreate support and extending the CUL module to understand the raw message strings requires patching. Check cul_esa-for-fhem-5.0.patch to patch the files. This was diffed against the Debian package of FHEM 5.0. The very same version of FHEM as tar.gz has some differences in the sourcecode and some files missing. I contacted the author of FHEM about this issue but he has not updated the packages yet. So if you run into issues here it is probably due to these differences.

UPDATE: Here is a completely patched version based on the Debian package of FHEM 5.0: fhem-5.0-cul_esa.tar.gz Version: Oct 25th 2010

Here are some notes in German I made while developing the module, mostly concerning the different behavior of the ESA1000GAS compared to the ESA1000WZ:

Der ESA1000GAS mit dem ESA1000Z-Messgerät verhält sich leider anders als das ESA1000WZ, sodaß ich wieder die Korrektur-Faktoren einbinden mußte.
Ich konnte nicht herausfinden, wozu der übermittelte Tick beim ESA1000Z dient. Zwar ändert sich dieser mit geänderter Zählerkonstante, allerdings ergibt der Wert im Zusammenhang mit den von der Einheit übermittelten Werten keinen Sinn, sodaß ich hier direkt mit der Zählerkonstante multipliziere, um die Kubikmeter zu erhalten, und dann zusätzlich noch mit dem Umrechnungsfaktor (bei der EWE ergibt sich dieser aus der Zustandszahl * Brennwert Hs) multipliziere, um die Kilowattstunden aus der Gasmenge zu erhalten.
Leider ändern sich damit die Parameter bei der Definition:

Stromzähler:
define CUL_ESA_286 CUL_ESA 1925 286 1 1 23.086
also:
corr1 = corr2 = 1 (und damit wirkungslos)
CostPerUnit = 23.086 Cent/kWh

Gaszähler:
define CUL_ESA_798 CUL_ESA 1429 798 0.001 9.5154 4.5339
also:
corr1 = 0.001
corr2 = 9.5154 kWh/m3
CostPerUnit = 4.5339 Cent/kWh

Ich habe ebenfalls noch einen Durchschnittswert in Kilowatt implementiert, sodaß man die Durchschnittleistung einigermaßen sinnvoll in einem Diagramm darstellen kann. Hier gab es allerdings bei meinem Drehstromzähler Probleme mit Unterabtastung, weil sich die Drehscheibe bei geringem Verbrauch zu langsam dreht, sodaß das Meßgerät zwischendurch schon Nullwerte zurückmeldet, weil kein Impuls eingegangen ist. Dafür habe ich einen entsprechenden Schwellenwert bei der Berechnung des Durchschnittswertes implementiert. Nicht schön, funktioniert aber für meine Zwecke.

First step is to install Wine either as super user or by using sudo:

sudo apt-get install wine

As normal user fire up your X server and your favourite terminal application and get the latest Inno Setup QuickStart pack:

mkdir /tmp/innosetup
cd /tmp/innosetup
wget http://files.jrsoftware.org/ispack/ispack-5.2.3.exe
wine ./ispack-5.2.3.exe

This will start the installer in Wine. Note, for the installation you will need a running X11 server since the installer obviously is graphical. If you have not set up Wine before, the default location C:\Program Files\Inno Setup 5 will install to ~/.wine/drive_c/Programme/Inno Setup 5.

Luckily the Inno Setup compiler offers a command line interface, ISCC.exe, which will run in Wine without the necessity of having a X server running. So it is ideally suited for automated runs.

Here is a simple wrapper shell script called iscc:

#!/bin/sh
unset DISPLAY
scriptname=$1
[ -f "$scriptname" ] && scriptname=$(winepath -w "$scriptname")
wine "C:\Program Files\Inno Setup 5\ISCC.exe" "$scriptname" "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9"

I installed this script in my local bin directory (~/bin) and added it to the PATH environment variable.
This will allow running the Inno Setup compiler from anywhere and it also makes it very easy to integrate into a build script. You can even feed a script via stdin, e.g. something like:

iscc - < ./myscript.iss

I’ve made some modifications to his patch to allow for sender address and sender domain based whitelisting.
You can get my patch here:

large-qmail-patch_greylisting-katastrophos.net-20061023.patch

The dbdef.sql file includes an example of how to set up a whitelist filter for a specific sender domain or sender address.

The patch is against Bill’s Large Qmail Patch 0.8.3.

The server load is also reduced because SpamAssassin and ClamAV are less busy filtering mails:

So, speaking for us, this experiment has been very successful. Exactly no real mails got lost, tons of spam was blocked at the SMTP level. Excellent. Now, let’s see how long the effects of this technique will last…

See http://projects.puremagic.com/greylisting/ to learn more about greylisting.

The problem here is that vdelivermail (of vpopmail) doesn’t support those mailfilter files. For that to work we need vdelivermail to call maildrop to process those files on an individual basis. However, this requires to setup the processing instructions for every mail account manually. Of course, this is unbearable on a large setup.

We came up with a patch for vdelivermail that automatically chain-calls maildrop if it finds a mailfilter file in the domain’s account directory. Some parts are based on another patch which source I forgot. I mainly extended it to be less strict and to publish certain internal vpopmail/vdelivermail variables as environment variables that could be used inside the mailfilter – namely username, userdir, domainname, domaindir.

If it finds the file in the domain’s directory, it will delegate the mail to the domain-wide mailfilter.
The domain-wide mailfilter will delegate the mail to the users mailfilter if it exists. If not, it will simply do a standard mail delivery to the account’s INBOX.

If vdelivermail can’t find any domain-wide mailfilter it will simply default to standard mail delivery to the account’s INBOX.

Here is the patch against vpopmail 5.4.17:

vpopmail-5.4.17-vdelivermail.c-maildrop.patch

If you’re too lazy to patch the vpopmail sourcecode yourself, here is the already patched vdelivermail.c:

vdelivermail-5.4.17-maildrop.c

(Rename it to vdelivermail.c and place it in the vpopmail source directory and compile.)

Our current mail server setup goes something like this:

qmail (with SMTP-AUTH + TLS) + vpopmail + SpamAssassin / ClamAV + maildrop + custom scripts + SqWebMail (for webmail) + Dovecot (for SSL IMAP)

In our setup by default every account has the following hierarchy:

INBOX
  Spam
    learn no-spam
    learn spam

The Spam subfolder is automatically created by the domain-wide mailfilter once the first email hits the account.

We’re using the following domain-wide filter (/home/vpopmail/domains/yourdomainname/mailfilter):

mailfilter

Again, our modified vdelivermail takes care of checking the domain for that mailfilter as described above.

Here is our script used to create the Spam subfolders in the domain-wide mailfilter above:

check-create-spamfolders.sh

If you want to use it, make sure to adjust the paths in the file to your setup.

So, now consider that the Spam subfolders have been created. People can use these folders to instruct what mails are spam and what mails are ham (non-spam). They can use this mechanism to report missed spam mails or to correct mistakenly classified non-spam mails.

Now, we wanted to get information out of those folders and try to educate our SpamAssassin’s bayes filter. Here is what we start in our crontab every hour:

sa-learn.sh

This script will make a list of all account’s spam and ham folders in every domain under /home/vpopmail/domains and feed the mails therein into sa-learn. Again, if you want to use the script, make sure to adjust the paths to your setup.

Since our server runs Debian installing SpamAssassin is pretty easy. We just had to make sure to start SpamAssassin’s spamd with the following adjusted settings in /etc/default/spamd.conf:

# Change to one to enable spamd
ENABLED=1

# Options
# See man spamd for possible options. The -d option is automatically added.
OPTIONS="-m 2 -H -u vpopmail --vpopmail --nouser-config"

# Set nice level of spamd
NICE="--nicelevel 10"

This will launch spamd globally for all vpopmail accounts. The bayes database is shared among all accounts and is stored in /home/vpopmail/.spamassassin/ with the following local.cf in /etc/spamassassin/local.cf:

local.cf

Having a global shared bayes database for your SpamAssassin is controversial. I won’t go into detail on why it’s good or bad to do so. All I can say is it’s working pretty well for us.

Currently, in our setup the SpamAssassin and ClamAV scanning is integrated into qmail via Qmail-Scanner. This has the disadvantage that antivirus filtering is done pretty deep down the processing chain. I plan to migrate this to simscan, which stops and rejects any virus mail at SMTP level and thus prevents them from entering the queue, i.e. nothing will get onto your box. Qmail-Scanner on the other hand puts virus mails into a global quarantine directory.

As for IMAP access to mail accounts, we’ve previously used Courier IMAP and just recently migrated all accounts to Dovecot. Both servers are proven and very reliable. However, we found Dovecot to be more secure and way faster. This is especially noticeable when doing searches and threading on mailboxes. Also, Dovecot is easier to set up as there are fewer external dependencies than with Courier IMAP.
Dovecot offers support for vpopmail. If you want to give it a try in your setup, make sure to enable this support via the “–with-vpopmail” switch when running configure.

As already stated above, we chose to use SqWebMail for webmail and mail filter configuration. By default the latter is disabled. You’ll have to explicitly enable the mail filter interface. Check the documentation on how to do that.