# Copyright (c) 2001 Sendmail, Inc. and its suppliers. # All rights reserved. # # By using this file, you agree to the terms and conditions set # forth in the LICENSE file which can be found at the top level of # the sendmail distribution. # # $Id: TUNING,v 1.16 2001/08/19 21:03:38 gshapiro Exp $ # ******************************************** ** This is a DRAFT, comments are welcome! ** ******************************************** If the default configuration of sendmail does not achieve the required performance, there are several configuration options that can be changed to accomplish higher performance. However, before those options are changed it is necessary to understand why the performance is not as good as desired. This may also involve hardware and software (OS) configurations which are not extensively explored in this document. We assume that your system is not limited by network bandwidth because optimizing for this situation is beyond the scope of this guide. In almost all other cases performance will be limited by disk I/O. This text assumes that all options which are mentioned here are familiar to the reader, they are explained in the Sendmail Installation and Operations Guide; doc/op/op.txt. There are basically three different scenarios which are treated in the following: * Mailing Lists and Large Aliases (1-n Mailing) * 1-1 Mass Mailing * High Volume Mail Depending on your requirements, these may need different options to optimize sendmail for the particular purpose. It is also possible to configure sendmail to achieve good performance in all cases, but it will not be optimal for any specific purpose. For example, it is non-trivival to combine low latency (fast delivery of incoming mail) with high overall throughput. Before we explore the different scenarios, a basic discussion about disk I/O, delivery modes, and queue control is required. * Disk I/O ----------------------------------------------- In general mail will be written to disk up before a delivery attempt is made. This is required for reliability and should only be changed in a few specific cases that are mentioned later on. To achieve better disk I/O performance the queue directories can be spread over several disks to distribute the load. This is some basic tuning that should be done in all cases where the I/O speed of a single disk is exceeded, which is true for almost every high-volume situation except if a special disk subsystem with large (NV)RAM buffer is used. Depending on your OS there might be ways to speed up I/O, e.g., using softupdates or turning on the noatime mount option. If this is done make sure the filesystem is still reliable, i.e., if fsync() returns without an error, the file has really been committed to disk. * Queueing Strategies and DeliveryMode ----------------------------------------------- There are basically three delivery modes: background: incoming mail will be immediately delivered by a new process interactive: incoming mail will be immediately delivered by the same process queue: incoming mail will be queued and delivered by a queue runner later on The first offers the lowest latency without the disadvantage of the second, which keep the connection from the sender open until the delivery to the next hop succeeded or failed. However, it does not allow for a good control over the number of delivery processes other than limiting the total number of direct children of the daemon processes (MaxChildren) or by load control options (RefuseLA, DelayLA). Moreover, it can't make as good use as 'queue' mode can for connection caching. Interactive DeliveryMode should only be used in rare cases, e.g., if the delivery time to the next hop is a known quantity or if the sender is under local control and it does not matter if it has to wait for delivery. Queueing up e-mail before delivery is done by a queue runner allows the best load control but does not achieve as low latency as the other two modes. However, this mode is probably also best for concurrent delivery since the number of queue runners can be specified on a queue group basis. Persistent queue runners (-qp) can be used to minimize the overhead for creating processes because they just sleep for the specified interval (which shold be short) instead of exiting after a queue run. * Queue Groups ----------------------------------------------- In most situations disk I/O is a bottleneck which can be mitigated by spreading the load over several disks. This can easily be achieved with different queue directories. sendmail 8.12 introduces queue groups which are collections of queue directories with similar properties, i.e., number of processes to run the queues in the group, maximum number of recipients within an e-mail (envelope), etc. Queue groups allow control over the behaviour of different queues. Depending on the setup, it is usually possible to have several queue runners delivering mails concurrently which should increase throughput. The number of queue runners can be controlled per queue group (Runner=) and overall (MaxQueueChildren). * DNS Lookups ----------------------------------------------- sendmail performs by default host name canonifications by using host name lookups. This process is meant to replace unqualified host name with qualified host names, and CNAMEs with the non-aliased name. However, these lookups can take a while for large address lists, e.g., mailing lists. If you can assure by other means that host names are canonical, you should use FEATURE(`nocanonify', `canonify_hosts') in your .mc file. For further information on this feature and additional options see cf/README. If sendmail is invoked directly to send e-mail then either the -G option should be used or define(`confDIRECT_SUBMISSION_MODIFIERS', `C') should be added to the .mc file. * Mailing Lists and Large Aliases (1-n Mailing) ----------------------------------------------- Before 8.12 sendmail delivers an e-mail sequentially to all its recipients. For mailing lists or large aliases the overall delivery time can be substantial, especially if some of the recipients are located at hosts that are slow to accept e-mail. Some mailing list software therefore "split" up e-mails into smaller pieces with fewer recipients. sendmail 8.12 can do this itself, either across queue groups or within a queue directory. For the former the option SplitAcrossQueueGroups option must be set, the latter is controlled by the 'r=' field of a queue group declaration. Let's assume a simple example: a mailing lists where most of the recipients are at three domains: the local one (local.domain) and two remotes (one.domain, two.domain) and the rest is splittered over several other domains. For this case it is useful to specify three queue groups: QUEUE_GROUP(`local', `P=/var/spool/mqueue/local, F=f, R=2, I=1m')dnl QUEUE_GROUP(`one', `P=/var/spool/mqueue/one, F=f, r=50, R=3')dnl QUEUE_GROUP(`two', `P=/var/spool/mqueue/two, F=f, r=30, R=4')dnl QUEUE_GROUP(`remote', `P=/var/spool/mqueue/remote, F=f, r=5, R=8, I=2m')dnl define(`ESMTP_MAILER_QGRP', `remote')dnl define(`confSPLIT_ACROSS_QUEUEGROUPS', `True')dnl define(`confDELIVERY_MODE', `q')dnl define(`confMAX_QUEUE_CHILDREN', `50')dnl define(`confMIN_QUEUE_AGE', `27m')dnl and specify the queuegroup ruleset as follows: LOCAL_RULESETS Squeuegroup R$* @ local.domain $# local R$* @ $* one.domain $# one R$* @ $* two.domain $# two R$* @ $* $# remote R$* $# mqueue Now it is necessary to control the number of queue runners, which is done by MaxQueueChildren. Starting the daemon with the option -q5m assures that the first delivery attempt for each e-mail is done within 5 minutes, however, there are also individual queue intervals for the queue groups as specified above. MinQueueAge is set to 27 minutes to avoid that entries are run too often. Notice: if envelope splitting happens due to alias expansion, and DeliveryMode is not 'i'nteractive, then only one envelope is sent immediately. The rest (after splitting) are queued up and queue runners must come along and take care of them. Hence it is essential that the queue interval is very short. * 1-1 Mass Mailing ----------------------------------------------- In this case some program generates e-mails which are sent to individual recipients (or at most very few per e-mail). A simple way to achieve high throughput is to set the delivery mode to 'interactive', turn off the SuperSafe option and make sure that the program that generates the mails can deal with mail losses if the server loses power. In no other case should SuperSafe be set to 'false'. If these conditions are met, sendmail does not need to commit mails to disk but can buffer them in memory which will greatly enhance performance, especially compared to normal disk subsystems, e.g., non solid-state disks. * High Volume Mail ----------------------------------------------- For high volume mail it is necessary to be able to control the load on the system. Therefore the 'queue' delivery mode should be used, and all options related to number of processes and the load should be set to reasonable values. It is important not to accept mail faster than it can be delivered otherwise the system will be overwhelmed. Hence RefuseLA should be lower than QueueLA, the number of daemon children should probably be lower than the number of queue runnners (MaxChildren vs. MaxQueueChildren). DelayLA is a new option in 8.12 which allows delaying connections instead of rejecting them. This may result in a smoother load distribution depending on how the mails are submitted to sendmail. * Miscellaneous ----------------------------------------------- Other options that are interesting to tweak performance are (in no particular order): SuperSafe: if interactive DeliveryMode is used, then this can be set to the new value "interactive" in 8.12 to save some disk synchronizations which are not really necessary in that mode.