Batch mail delivery solutions

Contents:

Batched SMTP solutions

We will write messages to a file, use UUCP over TCP/IP as an authenticated transport, and unpack the batch the other end.

The Batched SMTP format is described in RFC 2422.

I use exim as the SMTP agent at both ends, and Taylor (GNU) UUCP as the middle transport. Note we do not use UUCP mail - an ugly throwback to times gone by.

Set up UUCP

For more information on UUCP over TCP, read Configuring UUCP-over-TCP: A Practical, Skeletal "How To" and some FreeBSD docs. General uucp information, including lots of history, can be found at the UUCP project homepages. Some other excellent information can be found at Linuxdoc pages.

Other Useful Links, for SSL and UUCP

  1. SSL-encrypted uucp-over-tcp using stunnel
  2. Taylor UUCP documentation
  3. Linux Documentation Project's UUCP HOWTO
  4. stunnel home page
  5. openssl home page
  6. Jim Seymour's uucp-over-tcp howto
  7. gnumonks uucp-over-ssl HOWTO project page

First, set up UUCP between the two machines. Taylor UUCP out of a RedHat RPM keeps its configuration files in /etc/uucp - if you compile your own I recommend you do the same, using configure --sysconfigdir=/etc/uucp before building. First for the local machine, armada ..

UUCP for local machine

There are 3 files of interest there, I delete the other ones. Most important is config, which might look as simple as this :-

# config for armada

nodename armada                         # The UUCP name of this system

# end of config file

Then there is the sys file, listing the systems you access. It might look like this. I include two systems here - hostpro, where I have root access, and tvsecure, where I do not. Notice the address parameter points to the fixed IP address of the machine to call.

# Taylor UUCP sys file - for armada - andyr

# default for all ports
time            any

# for rsmtp
command-path    /usr/sbin

# unbatch our mail
commands        rsmtp

# look all passwords up in the call file
call-login      *
call-password   *

system  hostpro
        address hostpro.com
        port    type    tcp

# special high port because uucico is run as a user on tvsecure
system  tvsecure
        port    type    tcp
        port    service 5540
        address tvsecure.com

# end of sys file

Though they can be put in the sys file, usernames and passwords are best kept separately, in the call file. Mine looks like this, with my login convention of U<machine name> :-

# armada call file

tvsecure        Uarmada         s3cret
hostpro         Uarmada         secr3t

# end of call file

Taylor UUCP has a very handy command called uuchk - try it on this collection of files :-

[root@armada uucp]# uuchk -s hostpro
System: hostpro
 When called using any login name
 Call out using a specially defined port
 The port is defined as:
  Port name system hostpro port
   Port type tcp
   TCP service uucp
   Characteristics: eight-bit-clean reliable end-to-end fullduplex
 Remote address hostpro.com
 Chat script "" \r\c ogin:-BREAK-ogin:-BREAK-ogin: \L word: \P
 Chat script timeout 10
 Chat script incoming bytes stripped to seven bits
 Login name Uarmada
 Password secr3t
 At any time may call if any work
 May retry the call up to 26 times
 May make local requests when calling
 May make local requests when called
 May send by local request: /
 May send by remote request: ~
 May accept by local request: ~
 May receive by remote request: ~
 May execute rsmtp
 Execution path /usr/bin /usr/sbin
 Will leave 50000 bytes available
 Public directory is /var/spool/uucppublic
 Will use any known protocol

UUCP for remote machine

The remote machine is the one listed as the lowest MX for the mail domain we wish to route. Here, it is called hostpro. A standard Taylor UUCP install, with the following files in /etc/uucp - config

# Taylor UUCP config file for hostpro

nodename hostpro                        # The UUCP name of this system

# end of config file

The sys file looks like this. It is even simpler because hostpro does not call. To call we need to know the IP address, hostpro is fixed, so armada calls hostpro, not the other way. We add another allowed command, spool.

# UUCP sys file - hostpro

# default for all ports
time            any

command-path    /usr/bin /usr/sbin

# unbatch our mail, and queue it up
commands        rsmtp spool

system armada

# end of sys file

Next we need the passwd file on hostpro, to authenticate incoming calls.

# passwd file for hostpro

Uarmada secr3t

# end of passwd file

We run the uuchk command on hostpro, and find this :-

bash-2.04# uuchk -s armada
System: armada
 When called using any login name
 May make local requests when called
 May send by local request: /
 May send by remote request: ~
 May accept by local request: ~
 May receive by remote request: ~
 May execute rsmtp spool
 Execution path /usr/bin /usr/sbin
 Will leave 50000 bytes available
 Public directory is /var/spool/uucppublic
 Will use any known protocol
 Calls will never be placed to this system

Running the daemon on the remote machine

We must now arrange for a listener on hostpro, that will accept a connection from armada. This can be done a couple of ways. Simplest is probably an entry in /etc/inetd.conf, that looks like this, and runs uucico as a recipient of connections on the uucp port 540, mentioned in the /etc/services file, subject to tcpwrapper rules :-

uucp    stream  tcp     nowait  uucp    /usr/sbin/tcpd  /usr/lib/uucp/uucico    
If you use xinetd, used on Redhat 7.0 and FreeBSD, I have this entry in my /etc/xinetd.conf file.
service uucp
{
        flags       = REUSE NAMEINARGS
        socket_type = stream
        protocol    = tcp
        wait        = no
        user        = root
        server      = /usr/libexec/uucp/uucico
        server_args = uucico --prompt
}

uucico can also be run with the --loop options, and will then service connections from the uucp port without the help of inetd. With the help of --config and --port, uucico can be run as a user process on a machine you do not have root access to - for instance my tvsecure machine above. If it is not run out of inetd, something needs to start uucico upon reboot. I run this on tvsecure. If you do this, read the Taylor UUCP documentation thoroughly, and override all the default paths for the sys file, passwd file and logs to point into your home directory.

#  tvsecure UUCP config file
# run this as user on boot
# /usr/lib/uucp/uucico --config /home/andyr/etc/uucp/config --port 5540 --loop
#
# This is the abovementioned config file

nodename tvsecure                               # The UUCP name of this system

hdb-files false
v2-files false

spool /home/andyr/var/spool/uucp                # The UUCP spool directory
lockdir /home/andyr/var/spool/uucp

pubdir /home/andyr/var/spool/uucppublic         # The UUCP public directory

logfile /home/andyr/var/log/uucp/Log            # The UUCP log file
statfile /home/andyr/var/log/uucp/Stats         # The UUCP statistics file
debugfile /home/andyr/var/log/uucp/Debug        # The UUCP debugging file


sysfile /home/andyr/etc/uucp/sys                # Default "sys"
portfile /home/andyr/etc/uucp/port              # Default "port"
passwdfile /home/andyr/etc/uucp/passwd          # Default "passwd"

Testing the UUCP link

At this point, from armada you should be able to execute
uucico -S hostpro
which should call hostpro to see if there is any mail. Progress can be followed in the log file - perhaps in /var/log or /var/spool/uucp. You might see something like this :-
uucico hostpro - (2001-03-26 21:45:36.56 6039) Calling system hostpro (port TCP)
uucico hostpro - (2001-03-26 21:45:38.77 6039) Login successful
uucico hostpro - (2001-03-26 21:45:41.38 6039) Handshake successful (protocol 't')
uucico hostpro - (2001-03-26 21:45:42.11 6039) Call complete (4 seconds 0 bytes 0 bps)

Exim setup

Exim is an excellent, easy to configure, SMTP mailer from Cambridge University, and can be downloaded from the main exim site.

Handy commands for testing your exim installation are the following :-

exim -C exim.new.conf -bV # tests new config file for errors
exim -C exim.new.conf -bt user@my-domain.com # tests how mail would be delivered
exim -C exim.new.conf -bP # prints all config values

Local Exim configuration

This is the configuration file from the machine that dials up. I keep my exim configuration in /etc/exim.conf and put this in the file. Please read the comments there for explanations.

It is only here for completeness, the default file loaded with your exim install will probably also work fine.

Things of note - you must list your domain in local_domains. Also, I suggest you pass all outbound mail to your ISP for delivery, it will save time online.

######################################################################
#                  Runtime configuration file for Exim               #
######################################################################

######################################################################
#                    MAIN CONFIGURATION SETTINGS                     #
######################################################################

# optional - I like them here. Careful with permissions.

spool_directory = /var/spool/exim
log_file_path = /var/log/exim_%slog

# must add uucp to trusted_users
trusted_users = "root:uucp:exim"

# Specify your host's canonical name here.

primary_hostname = my-domain.com

# you must list your domain here - otherwise exim will return mail to your ISP
# the @ copies primary_hostname

local_domains = localhost:@

forbid_domain_literals

never_users = root

# I run a network at home
host_accept_relay = localhost:10.1.2.0/24:10.1.1.0/24

host_lookup = 0.0.0.0/0

ignore_errmsg_errors_after = 2d

end

######################################################################
#                      TRANSPORTS CONFIGURATION                      #
######################################################################
#                       ORDER DOES NOT MATTER                        #
#     Only one appropriate transport is called for each delivery.    #
######################################################################

# A transport is used only when referenced from a director or a router that
# successfully handles an address.

# This transport is used for delivering messages over SMTP connections.

remote_smtp:
  driver = smtp

# This delivers into local mailboxes here.

local_delivery:
  driver = appendfile
  file = /var/spool/mail/${local_part}
  delivery_date_add
  envelope_to_add
  return_path_add
  group = mail
  mode = 0660

address_pipe:
  driver = pipe
  return_output

address_file:
  driver = appendfile
  delivery_date_add
  envelope_to_add
  return_path_add

address_reply:
  driver = autoreply

end

######################################################################
#                      DIRECTORS CONFIGURATION                       #
#             Specifies how local addresses are handled              #
######################################################################
#                          ORDER DOES MATTER                         #
#   A local address is passed to each in turn until it is accepted.  #
######################################################################

system_aliases:
  driver = aliasfile
  require_files = /etc/exim/exim_aliases
  file = /etc/exim/exim_aliases
  search_type = lsearch
# user = exim
  file_transport = address_file
  pipe_transport = address_pipe

# This director matches local user mailboxes.

localuser:
  driver = localuser
  transport = local_delivery

end

######################################################################
#                      ROUTERS CONFIGURATION                         #
#            Specifies how remote addresses are handled              #
######################################################################
#                          ORDER DOES MATTER                         #
#  A remote address is passed to each in turn until it is accepted.  #
######################################################################

# Remote addresses are those with a domain that does not match any item
# in the "local_domains" setting above.

# this overrides the ones below ..

# this punts all remote addresses to your local ISP. Quicker, faster, better.

smart_route:
  driver = domainlist
  transport = remote_smtp
  route_list = "* smtp.my-isp.net bydns_a"

# this does all the work yourself, and may be blocked by your ISP.
# not active here, because smart_route gets them all.

lookuphost:
  driver = lookuphost
  transport = remote_smtp

end


######################################################################
#                      RETRY CONFIGURATION                           #
######################################################################

end

######################################################################
#                      REWRITE CONFIGURATION                         #
######################################################################

end

# End of Exim configuration file

Remote Exim configuration

This is the exim configuration sitting somewhere on internet, listed as the lowest Mail Exchanger for my-domain.com. I run it as the user exim, you might prefer mail or something.

######################################################################
#                  Runtime configuration file for Exim               #
######################################################################

DM = my-domain.com:my-domain.org:my-other-domain.com

# Specify your local domains as a colon-separated list here.
# I only list domains here that are delivered to mailboxes here,
# not my UUCP transport domains.
# This works because of the "self = local" command below.

# only used if you have mailboxes on this machine.
local_domains = localhost

######################################################################
#                    MAIN CONFIGURATION SETTINGS                     #
######################################################################

spool_directory = /var/spool/exim
log_file_path = /var/log/exim_%slog

trusted_users = "root:uucp:exim"

# Specify your host's canonical name here.

primary_hostname = isp.my-domain.com

forbid_domain_literals

never_users = root

# list your domains here - see DM macro above.

relay_domains = DM

host_lookup = 0.0.0.0/0

# kill that spam.
rbl_domains = rbl.maps.vix.com
# rbl_domains = rbl.maps.vix.com:dul.maps.vix.com:relays.orbs.org

ignore_errmsg_errors_after = 2d

end


######################################################################
#                      TRANSPORTS CONFIGURATION                      #
######################################################################
#                       ORDER DOES NOT MATTER                        #
#     Only one appropriate transport is called for each delivery.    #
######################################################################

# A transport is used only when referenced from a director or a router that
# successfully handles an address.

# this transport piles up mail in one BSMTP file. This must be queued for
# UUCP transport periodically - I use the spool script.
# the substr command snips of the last 5 letters, .UUCP

bsmtp:
  driver = appendfile
  bsmtp = all
  file = /var/spool/bsmtp/${substr_-5:$host}
  user = uucp
  #group = uucp
  #mode = 0660
  prefix =
  suffix =

# This transport is easier to use, but queues each message individually.
# That translates to lots of little UUCP messages. Maybe you do not care.

uucp:
  driver = pipe
  bsmtp = all
  prefix =
  suffix =
  no_from_hack
  user = nobody
  command = "uux - ${substr_-5:$host}!rsmtp"
  return_fail_output = true


# This transport is used for delivering messages over SMTP connections.

remote_smtp:
  driver = smtp

# only used if you have mailboxes on this machine.
local_delivery:
  driver = appendfile
  file = /var/mail/${local_part}
  delivery_date_add
  envelope_to_add
  return_path_add
  group = mail
# mode = 0660

address_pipe:
  driver = pipe
  return_output

address_file:
  driver = appendfile
  delivery_date_add
  envelope_to_add
  return_path_add

address_reply:
  driver = autoreply

end


######################################################################
#                      DIRECTORS CONFIGURATION                       #
#             Specifies how local addresses are handled              #
######################################################################
#                          ORDER DOES MATTER                         #
#   A local address is passed to each in turn until it is accepted.  #
######################################################################

# not used for UUCP transport domains
system_aliases:
  driver = aliasfile
  file = /etc/exim/exim_aliases
  search_type = lsearch
# user = exim
  file_transport = address_file
  pipe_transport = address_pipe

# I keep my alias files separated by domain.
virtual:
  driver = aliasfile
  require_files = /etc/exim/virtual/$domain
  file = /etc/exim/virtual/$domain
  search_type = lsearch
  user=exim
  group=mail
  file_transport = address_file
  pipe_transport = address_pipe
# reply_transport = address_reply

# not used for UUCP transport domains
userforward:
  driver = forwardfile
  file = .forward
  no_verify
  no_expn
  check_ancestor
# filter
  file_transport = address_file
  pipe_transport = address_pipe
  reply_transport = address_reply

# This director matches local user mailboxes.
# not used for UUCP transport domains

localuser:
  driver = localuser
  transport = local_delivery

end



######################################################################
#                      ROUTERS CONFIGURATION                         #
#            Specifies how remote addresses are handled              #
######################################################################
#                          ORDER DOES MATTER                         #
#  A remote address is passed to each in turn until it is accepted.  #
######################################################################

# Remote addresses are those with a domain that does not match any item
# in the "local_domains" setting above.

# both the following route_file arguments specify a file that contains
# lines like this :-
#
# my-domain.com:	armada.UUCP
# my-other-domain.com:	armada.UUCP
#
# the UUCP part is snipped off by the transport, see above.
# put entries in the file of the transport of choice.

bsmtphost:
  driver = domainlist
  transport = bsmtp
  route_file = /etc/exim/bsmtphosts
  search_type = lsearch

uucphost:
  driver = domainlist
  transport = uucp
  route_file = /etc/exim/uucphosts
  search_type = lsearch

# This router routes to remote hosts over SMTP using a DNS lookup with
# default options.

lookuphost:
  driver = lookuphost
  self = local
  transport = remote_smtp

end



######################################################################
#                      RETRY CONFIGURATION                           #
######################################################################

end

######################################################################
#                      REWRITE CONFIGURATION                         #
######################################################################

# There are no rewriting specifications in this default configuration file.

end

# End of Exim configuration file