smtp-gated

transparent smtp-proxy, part of software.klolik.org

[about] [features] [supports] [security] [todo] [downloads] [bugs] [screens] [manuals] [FAQ]

About

This software block SMTP sessions used by e-mail worms and viruses on the NA(P)T router. It acts like proxy, intercepting outgoing SMTP connections and scanning session data on-the-fly. When messages is infected, the SMTP session is terminated. It's to be used (mostly) by ISPs, so they can eliminate infected hosts from their network, and (preferably) educate their users.

If you need split-host scenario, look also at proxy-helper

[back to top]

Features

Features include:
  1. Transparency - is meant to be totally transparent for users, but black-hole for worms ;)
  2. Message data is intercepted on-the-fly, and scanned just before acknowledged to SMTP server
  3. Does not break AUTH, PIPELINING or STARTTLS (TLS without scanning)
  4. Can block messages if AUTH is not used (optionally passing if AUTH is not supported by MSA)
  5. Can insert source IP (pre-NAT) and ident* into message header
  6. Can block any mail from infected hosts for defined time
  7. Logging of MAIL FROM and RCPT TO (plain or as base64-ed MD5)
  8. Logging of HELO/EHLO hostname
  9. Can impose some limits on number of SMTP sessions: total, per IP, per ident*
  10. Can reject connections when load exceeds some limit
  11. Can skip spam-scanning if load is high
  12. Executing user script on certain events
  13. Scanning limited to messages up to configured size
  14. Can be used to build scanning-farm for one or more routers*
  15. Logs all connections via syslog
  16. Has cool status screen ;)
  17. Message size limit (since 1.4.16-rc1)
  18. Outgoing XCLIENT support (since 1.4.16-rc1)
  19. Conditional content scanning depending on SMTP-AUTH status (since 1.4.16-rc1)
  20. Regular expression (regex) conditions for HELO/MAIL FROM/RCPT TO (since 1.4.16-rc1)
  21. SPF checking (since 1.4.16-rc1)
(*) ident: only when used in external mode with patched ident daemon, or with proxy-helper daemon.

[back to top]

Supports

  1. Content scanning:
    1. Clam AntiVirus daemon (clamd)
    2. mksd - daemonised version of mks_vir
    3. SpamAssassin antispam scanning
  2. Access checking:
    1. libpcre for HELO/MAIL FROM/RCPT TO regular expressions (not-)match
    2. libspf2 for SPF (tested with debian libspf2 1.2.1)
  3. Uses various NAT frameworks (for standalone mode), or ident/proxy-helper* for external mode
    1. patched ident daemon
    2. proxy-helper daemon
    3. netfilter framework of Linux
    4. ipfw on FreeBSD
    5. BSD/pf (packetfilter)
    6. BSD/ipfilter

[back to top]

Security

Please remember to protect proxy listening port or you will get into trouble by providing open-relay service to spammers.
Protection can be easily achieved by: If you want greater security about proxy code, use Grsecurity patch for linux kernels.

If you want to run proxy as root, this probably means that you should skip this page and rather read some books about Unix administration.
Do not, I repeat, do not run it as root.


[back to top]

TODO

  1. code cleanup

[back to top]

Downloads

Sources released under GNU License. If you use this software, please send me an e-mail with comment. Thanks.
Remember to check for renamed configuration variables! See ChangeLog and regression tests summary

Latest version:
[2013.02.26] smtp-gated.1.4.20.0.tar.gz


Older versions:
[2011.12.04] smtp-gated.1.4.18.8.tar.gz
[2010.08.30] smtp-gated.1.4.17.tar.gz
[2010.02.12] smtp-gated.1.4.16.3.tar.gz
[2009.01.06] smtp-gated.1.4.16.2.tar.gz
[2008.09.07] smtp-gated.1.4.16.tar.gz
[2008.01.24] smtp-gated.1.4.16-rc1.tar.gz
[2006.12.11] smtp-gated-1.4.15.1.tar.gz
[2006.12.11] smtp-gated-1.4.15.tar.gz
[2006.05.11] smtp-gated-1.4.14-rc1.tar.gz
[2005.12.02] smtp-gated-1.4.12-rc9.tar.gz
[2005.10.27] smtp-gated-1.4.12-rc8.tar.gz

Since 1.4.12-rc1 embedded .spec file allows building rpm packages using: If you receive an error during the procedure above:
  1. copy .tar.gz archive to SOURCES subdirectory of RPM tree
  2. extract .spec file from tarball SPECS subdirectory of RPM tree
  3. type rpmbuild -bb SPECS/smtp-gated.spec

[back to top]

Bugs

If something does not work as you expect, please check smtp-gated.conf.5 manual first (especially FAQ section).

About automated regression tests (make test): If you still think there is a bug, please:
  1. make sure, you are using the latest version
  2. attach description of current behaviour, and the correct one (if appropriate)
  3. attach configuration file
  4. attach smtp-gated compile-time configuration, generated by smtp-gated -V
  5. attach logs (with log_level debug)
  6. there are always some logs. if there is none, then you have broken your syslog.
  7. be more descriptive than "does not work"!

DescriptionVersionStatus*Temporary fix
ratelimiting may be buggy, any comments are welcome 1.4.18.8experimentaldisabled by default
existing sessions die with !BUG! after changing pipeline_size and reloading daemon <= 1.4.17confirmed/fixedrestart proxy after changing this variable
connection dies unexpectedly (!BUG!) *n/aupgrade libc
connection is setup with delay in tproxy mode *confirmed/unknownunload nf_conntrack_ipv4 kernel module
connection dies on MAIL FROM: without HELO/EHLO (!BUG!) <= 1.4.16.1confirmed/fixeddisable SPF
Core dump and "!BUG!" message on 64-bit architectures <= 1.4.16-rc1confirmed/fixed-
In pf mode connection fails with Lookup failed: No such file or directory <= 1.4.15.1confirmed/fixedreplace PF_IN with PF_OUT in lookup.c
Proxy dies with "accept error: Software caused connection abort" or similar <= 1.4.14-rc1confirmed/fixed-
user lock not set when spam found <= 1.4.12-rc3confirmed/fixed-
spool not deleted when pipeline is full <= 1.4.12-rc3confirmed/fixed-
compilation problem on gcc-4
(vars.h: syntax error before string constant)
<= 1.4.12-rc3confirmed/fixeduse gcc-3
FreeBSD ipfw support not detected <= 1.4.12-rc3pendingforce support with
./configure --enable-nat
(stupid) memleak in main loop <= 1.4.12-rc2confirmed/fixed-
compilation problem on gcc-2.95
(unnamed struct/union that defines no instance)
== 1.4.11confirmed/fixeduse gcc-3
SpamAssassin support may be buggy <= 1.4.9confirmed/fixed-
.pid file created with random rights == 1.4.5confirmed/fixed-
log_level does not work properly <= 1.4.4confirmed/fixedleave default
Problem with process counting <= 1.4.3confirmed/fixed-
Breaks CHUNKING (RFC3030) *unknown/pendingleave chunking support disabled (default)
user gets locked, but still can send e-mails
(when proxy running as root)
*correct behaviourdo not run proxy as root, use set_user
(*) Note: fixed means: [will be] fixed in next release (above 'Version' shown)

[back to top]

Screens

Normal session log (without syslog headers):
smtp-gated[22739]: NEW (1/1) src=194.*.*.*:17536, ident=********, dst=213.*.*.*:25, id=1110******.22739
smtp-gated[22739]: EHLO host=194.*.*.*, ident=********, helo=*****
smtp-gated[22739]: DATA:REQUEST
smtp-gated[22739]: DATA:GOING
smtp-gated[22739]: DATA:SCANNING size=23302, host=194.*.*.*, ident=*****
smtp-gated[22739]: SCAN:CLEAN size=23302, time=0, host=194.*.*.*, ident=*****
smtp-gated[22739]: SPAM:CLEAN size=23302, time=8, host=194.*.*.*, ident=*****, result=-2.800000
smtp-gated[22739]: DATA:FINISHED [250]
smtp-gated[22739]: CLOSE by=server, rcv=23438/282, trns=1, rcpts=1, time=15, host=194.*.*.*, ident=*****
... and when virus has been caught:
smtp-gated[21422]: NEW (1/1) src=194.*.*.*:48059, ident=*****, dst=194.*.*.*:25, id=1110******.21422
smtp-gated[21422]: DATA:REQUEST
smtp-gated[21422]: DATA:GOING
smtp-gated[21422]: DATA:SCANNING size=704, host=194.*.*.*, ident=*****
smtp-gated[21422]: SCAN:VIRUS size=704, time=0, host=194.*.*.*, ident=*****, virus=Eicar-Test-Signature
smtp-gated[21422]: SESSION TAKEOVER: host=194.*.*.*, ident=*****, trns=1, reason=Malware found / Znaleziono wirusa (Eicar-Test-Signature)
smtp-gated[21422]: CLOSE:TAKEN
... and when SPF kicks in:
smtp-gated[10654]: NEW (5/0) on=194.***********:9199, src=10.*******:2435, ident=, dst=64.*******:25, id=1264******.10654
smtp-gated[10654]: EHLO src=10.*******, ident=, helo=******************
smtp-gated[10654]: SPF:fail ip=194.***********, helo=******************, from=ofrmvb@***********
smtp-gated[10654]: SPF:REJECT
smtp-gated[10654]: LOCK:CREATED (/var/spool/smtp-gated/lock/10.*******): SPF
smtp-gated[10654]: SESSION TAKEOVER: src=10.*******, ident=, trns=0, reason=Connection rejected due to SPF, enable SMTP authentication
smtp-gated[10654]: CLOSE:TAKEN
... and when regular expressions vote against:
smtp-gated[21497]: NEW (2/0) on=194.***********:9199, src=10.******:4897, ident=, dst=194.*******:25, id=1264******.21497
smtp-gated[21497]: REGEX:REJECT src=10.******, ident=, ehlo=********************
smtp-gated[21497]: LOCK:CREATED (/var/spool/smtp-gated/lock/10.******): REGEX_EHLO
smtp-gated[21497]: SESSION TAKEOVER: src=10.******, ident=, trns=0, reason=Forbidden HELO/EHLO
smtp-gated[21497]: CLOSE:TAKEN
Status screen 1:
Version:      1.4.15.1
Dump time:    Tue Oct  9 13:26:56 2007
Start time:   Thu Dec 28 13:43:53 2006
Restart time: Thu Feb  1 15:27:06 2007
Uptime:   284d 22h 43m 3s
Resource: 0/0/0/0 (maxrss/ixrss/idrss/isrss)
Children: 2/40 (current/max)
Found:    173/1844/0 (viruses/spam/no-auth)
Requests: 19343876/7034/417031 (total/direct/empty)
Rejects:  297/127989/18576865/10365 (host/ident/lock/other)

slot pid   state    flags time  source          target          trns   cli_rx   srv_rx    kbps ident
  62 4453  data     A     00:04 194.*.*.*       86.*.*.*           1   322761      235   630.4 ********
  63 27409 data     A     00:08 194.*.*.*       217.*.*.*          1   340394      280   332.4 ********

Status screen 2:
Version:      1.4.16.2
Compile date: Jan  6 2009 14:22:35
Dump time:    Sun Jan 31 18:00:09 2010
Start time:   Wed Aug 19 12:07:49 2009
Restart time: Wed Aug 19 12:07:49 2009
Uptime:   165d 6h 52m 20s
Resource: 0/0/0/0 (maxrss/ixrss/idrss/isrss)
Children: 4/19/46 (current/max/buggy)
Found:    0/0/0/0/11127 (viruses/spam/no-auth/spf/regex)
Requests: 516877/14331/259155 (total/direct/empty)
Rejects:  5290/0/7499/0/4173 (host/ident/lock/dnsbl/other)
Auth:     101215/3245 (accepted/rejected)

slot pid   state    flags time  source          target          trns   cli_rx   srv_rx    kbps ident
  60 31449 pre      .     00:00 10.************ 65.************    0        0      311     0.0
  61 28779 data     A     08:21 192.168.******* 217.***********    8 43735006      972   682.0
  62 30475 spf      a     02:51 192.168.******* 193.***********    0       70      498     0.0
  63 30474 helo     a     02:51 192.168.******* 193.***********    0       70      498     0.0
[back to top]

Manuals

All documentation files are distributed within doc/ subdirectory of source package.

You can view (old) documentation online: But: configuration manual above do not necessary reflect all options available.
It can contain options that do not exist anymore, or event do not yet exist in the most recent release.
You can always get all available options by running smtp-gated -t
or even more options with possible values by smtp-gated -T

Some examples are in lib/ directory found in source tree. Here are some other examples:
[back to top]

FAQ

Antispam is not working! What is wrong?
You must change max_spam_size to something above 0 to turn on antispam scanning. It's set to 0 by default, and this means turning antispam off. Also, set antispam_type.
Spammers don't get locked! What is wrong?
If you want to block spammers, set "lock_on spam[,...]", and see answer above.
I wanted to block some user by creating lockfile manually, but now his traffic is not even scanned!
Creating lockfile with owner other than proxy process runs with, will protect that host from being locked. The contents of file does not matter.
My header is invalid after upgrading!
nat_header changed into nat_header_type since 1.4.12-rc5, so after upgrade header will not be injected unless you rename it in configuration file.
I don't have idents in my logs or headers, why?
It simple -- ident is used in REMOTE IDENT mode, which is currently not available.
I don't see X-NAT-Received header in mails, why?
Please remember, that mail sent with TLS is not scanned. Also, check nat_header_type directive in your config file, and nat_header (see MANGLING section of man smtp-gated.conf).
When X-Virus-Scan and/or X-Spam-Status headers will be supported?
The answer is: never. Reason is simple -- proxy is passing-thru email headers and body. Because headers are sent first, we can't predict spam-score or virus presence. The main assumption for this project was not to buffer mail before forwarding, so this option won't appear.
I have many locks left, even if they should be gone long time ago!
Please see note for the LOCKING section of man smtp-gated.conf.
Host gets locked, lock file appears, but he can still send e-mails!
Please *do not* run proxy as root, and set set_user properly. If you do, remove all locks manually.
action_script does not work!
Please read about ACTION SCRIPT. Script will not be called once the lock exists. If you were testing "by hand" by writing some text to some log file, make sure the script has permissions to write to it too, i.e. by temporarily doing chmod a+rw on that log file. You should also check if disabling (setting to "-1") limit_* variables will do the trick, and if the answer is yes, try to set them high enough for script needs.
spamassassin gets timeout
Please check if "spamd -L" resolves this issue.
clamd fails with: SCAN:FAILED [...] result=lstat() failed.
Antispam and antivirus scanners need to have access to spool files (spool_path, spool_perm). If you want to seal your setup, you can add clamav user to smtpgw group, and deny other to access the spool directory.
action script does not seem to work well when executed by smtp-gated, possibly failing with ENFILE or ENOMEM
Read notes about resource control and action scripts.
proxy does not work and there are no logs!
For starters, be sure to set "log_level debug" in your config file. Proxy will always log something, either to syslog or to the stdout/stderr. If it does not, then you have your syslog configuration broken. You can also check your dmesg for kernel logs like segmentation fault, but that's highly unlikely to happen. You can try make test.
I can't connect/there is timeout when I'm trying to connect through proxy and there are no logs!
Try telnet to bind_address:port (take these parameters from your config file). You should get connected and the logs you will see "NEW" entry. The rest depends on configured mode (see MODE section of man smtp-gated.conf). In case of any other than fixed modes you will see also "Avoiding loop, exiting". This is correct behaviour and means that proxy is listening and accepting connections just like it should.
I telnet to the proxy addres&port and I see "Avoiding loop, exiting"!
If you use non-fixed mode this is very correct. You telnet directly to proxy port, so it accepts this connection, looks up the destination address and figures out it should forward the connection to... itself (hence the loop). How do you expect it to work? The positive side of this is that proxy is accepting connections as it should.
I get "SESSION TAKEOVER: src=203.189.141.10, ident=, trns=0, reason=Lookup failed: Protocol unavailable"
For netfilter/tproxy+netfilter mode on Linux, you need to load nf_conntrack_netlink into kernel.
For getsockname/pf/ipfw mode on *BSD I guess you need to load corresponding modules.
I get "SESSION TAKEOVER: src=203.189.141.10, ident=, trns=0, reason=Lookup failed: Protok?? niedost?pny" and I don't speak polish!
It means the same as the above, plus you need to remove "locale pl_PL" from your config file ;)




(c) 2005-2013 Bartłomiej Korupczyński
contact