Libmilter: worker model

Last Update 2003-04-02

Jose Marcio MARTINS DA CRUZ from Ecole Nationale Superieure des Mines de Paris kindly donated a patch to change the libmilter threading model from the current "one thread per connection" to a worker model.

Note: use this patch only if you understand what this is all about. If in doubt, do not apply these changes. The default libmilter implementation works fine in almost all cases.

Here's his explanation:

I'm including two patches: one for libmilter directory (updated 2002-12-12, new version from 2002-12-30 which requires

APPENDDEF(`conf_libmilter_ENVDEF',`-D_FFR_WORKER_MODEL')
in devtools/Site/site.config.m4) and the other for include/libmilter/mfapi.h include file.
What's the difference between this version and original libmilter
version :

* FILE engine.c
  mi_engine was modified to return at the end of each sendmail
  command instead of end of session. So, mi_engine returns three
  possible values :
  - MI_FAILURE as before
  - MI_SUCCESS as before at the end of session
  - MI_CONTINUE if the session isn't ended

* FILE handler.c
  This file was completely rewritten in order to have all sessions
  wait states handled by a single multiplexer (using accept) instead
  of a call to mi_rd_cmd inside mi_engine. Each time mi_engine will
  be called, data will be available at socket, so mi_rd_cmd inside
  mi_engine will return immediately with available data.
  Also, available work will be distributed to workers as you defined.

* FILE mfapi.h
  constant MI_CONTINUE defined.

Some precisions :

In the original libmilter version, the number of threads in the system
is equal to the number of active sessions. But most of then were idle
waiting for commands from sendmail.

There is two ways to reduce the mean number of threads in the filter.

1. As commands come from sendmail, you may distribute then to workers.
   In this case, the number of threads is not fixed, but is equal to
   the number of commands being handled. This number is lower than 
   the number of sessions.

2. You can fix the number of thread to some value. In this case, you
   shall manage a waiting queue, as you may, at some time, have more
   work available than the fixed number of threads in the system. In 
   this case, the mean latency time is greater than in the first case.
   
The version of libmilter I worked on is of the first kind. If this
isn't enough to solve problems, I can try to work on the second.

Also I modified the criteria to kill a thread : instead of exiting
when there is more than MAX_THREADS, works will exit when there are
more than MIN_THREADS idle threads. This reduce latency time resulting
from thread launching when the filter is running on a busy mail server.


[(links)] [Hints] [Avoiding UBE] [cf/README] [New]
Copyright © Claus Aßmann Please send comments to: <ca at sendmail.org>
Disclaimer: the information provided may be inaccurate or outdated or incomplete. Please contact me if you find an error.