⟦de6490a88⟧

TextFile

% -*- LaTeX -*-
\chapter {Channels}

\section {Overview}

This chapter describes the interface which provides the Message
Delivery aspects of the Message Transfer Abstract Service and MTA
Abstract Service.  The major component of this is a set of routines
for accessing the PP queue.  As these routines are usually used in the
context of a channel process, their usage is illustrated in various
types of channel.  There are also additional functions to facilitate
reformatting/conversion.

The following types of channel are supported:

\begin{itemize}
\item Protocol Outbound Channel (MTA AS Delivery)

\item Local delivery channel (MT AS Delivery)

\item Local pickup channel (MT AS Delivery)

\item Remote TWA channel (MTA AS Delivery)

\item Reformatters
\end{itemize}

\section {The Queue Interface}

\subsection {Major Structures}

The major structures in use for reading the queue are the following:-
\begin{itemize}
\item The parameter structure described in Figure~\ref{prm:struct} on
Page~\pageref{prm:struct}.

\item The queue structure described in Figure~\ref{q:struct} on
Page~\pageref{q:struct}.

\item The address structure described in Figure~\ref{adr:struct} on
Page~\pageref{adr:struct}.

\end{itemize}

There is also a separate set of routines and structures for
interacting with the queue manager. This interface makes use of the
structures defined in
Figure~\ref{struct:qmgrif}.\index{type\_Qmgr\_DeliveryStatus}
\index{type\_Qmgr\_Channel}\index{type\_Qmgr\_ProcMsg}\index{type\_Qmgr\_QID}
\index{type\_Qmgr\_RecipientId}
\tagrind[hbtp]{qmgrif}{Channel / Queue Manager Interface Structures}
{struct:qmgrif}

These structures are derived from the queue manager ROS specification
given in Volume 3.

\subsection {Basic Access}

The basic access provided to the queue is through a set of library
calls.  There are several classes of routines which can be divided up
into queue manager interaction, address file processing routines and
message body routines. Most of the operations described here are only
useful for outbound channels.

\subsubsection{Queue Manager Interaction}

The queue manager interaction is normally handled by a single
procedure, \verb|channel_control|, which manages the session with the
qmgr and calls the appropriate routines on receipt of ROS operations.

\begin{quote}\index{channel\_control}\begin{verbatim}
int channel_control (argc, argv, initfnx, 
                     procfnx, endfnx)
int    argc;
char   **argv;
IFP    initfnx;
struct type_Qmgr_DeliveryStatus   *(*procfnx)();
IFP    endfnx;
\end{verbatim}\end{quote}

The arguments are as follows:
\begin{describe}
\item[\verb|argc|/\verb|argv|:]	The argument count and vector -- normally
copied from the \verb|main| procedure. These are used to recapture the
state of the association as outbound channels are normally run under
the \man pptsapd(8) daemon.

\item[\verb|initfnx|:]	An initialisation function. This is called to
initialise the channel and inform it of it's name.

\item[\verb|procfnx|:]	A function called for each qmgr process
message operation.

\item[\verb|endfnx|:] A function called when the session with the qmgr
is terminated. The constant \verb|NULLIFP| may be used if this function
is not required.
\end{describe}
The \verb|channel_control| function invokes it's own error handlers
and does not normally return.


The two functions \verb|initfnx| and \verb|procfnx| shown above are the key to the channels
operation. The initialisation function should be defined in the
following way:
\begin{quote}\begin{verbatim}
int   initfnx (arg)
struct type_Qmgr_Channel *arg;
\end{verbatim}\end{quote}

The function should then decide whether the given channel argument is
sensible for this channel and return one of the constants \verb|OK| or
\verb|NOTOK|. This function is normally only called once by the qmgr,
but in cases where the same program can run as a number of channels it
may be called subsequently to reinitialise the channel to a different
mode of operation.

The process function is the one that does the real work of the
channel. It should be defined like this:
\begin{quote}\begin{verbatim}
struct type_Qmgr_DeliveryStatus *processmessage (arg)
struct type_Qmgr_ProcMsg *arg;
\end{verbatim}\end{quote}
The function should then attempt to process the given message for the
given recipients. This usually involves first reading the address
control file given as part of the structure passed.

When the processing of the message has finished, the function should
return a pointer to a list of recipient numbers together with a status
field. This structure is the \verb|type_Qmgr_DeliveryStatus|
structure. This specifies for each recipient whether the operation was
a success or failure. The status values are:
\begin{describe}
\item[\verb|int\_Qmgr\_status\_success|:]\index{status\_success}
The operation was successful, the next processing stage can be
considered.

\item[\verb|int\_Qmgr\_status\_negativeDR|:]\index{status\_negativeDR}
The message has failed, a delivery report {\em must} be written.

\item[\verb|int\_Qmgr\_status\_positiveDR|:]
\index{status\_positiveDR}
The message has succeeded, and a confirmation is required. This {\em
must} be written out.

\item[\verb|int\_Qmgr\_status\_successSharedDR|:]
\index{status\_successSharedDR}
The message has suceeded and a delivery report is to be sent. However,
the delivery report will be sent as the result of processing another
recipient.

\item[\verb|int\_Qmgr\_status\_failureSharedDR|:]
\index{status\_failureSharedDR}
The message has failed, and a delivery report should be sent for the
this recipient, but this will be sent as the result of another
recipients actions.

\item[\verb|int\_Qmgr\_status\_messageFailure|:]
\index{status\_messageFailure}
The message has temporarily failed, other messages on the same channel
may succeed and this message may succeed later at a second attempt.

\item[\verb|int\_Qmgr\_status\_mtaFailure|:]
\index{status\_mtaFailure}
The remote mta is not responding, the attempt should be retried at a
later time. Further messages on this channel to this mta should not be
considered at this attempt.

\item[\verb|int\_Qmgr\_status\_mtaAndMessageFailure|:]
\index{status\_mtaAndMessageFailure}
This is an indication that there was a failure whilst transferring  the
message to the mta. This channel should be attempted again in the near
future.
\end{describe}

To aid in setting these values, a set of support routines are provided
in the library. These routines all update the global variable
\verb|deliverystate| which is suitable as a return value for the
process function. Note however, that the value of
\verb|int_Qmgr_status_permanentFailure| is ``sticky'', this value may
be set, but once set can not be overwritten by these routines. This is
because normally when an address is marked with this status, a
delivery report is written out and the message is updated accordingly
(see section~\ref{set:dr} for routines to write out delivery reports).
To then set the status to something else would confuse the queue
manager. The routines are as follows:

\begin{quote}\index{delivery\_init}\begin{verbatim}
struct type_Qmgr_DeliveryStatus *delivery_init (users)
struct type_Qmgr_UserList *users;
\end{verbatim}\end{quote}

This routine initialises the structure. It does by allocating an
element for each of the users passed in the argument. The single
parameter is the list of users passed over by the qmgr.

\begin{quote}\index{delivery\_set}\begin{verbatim}
struct type_Qmgr_DeliveryStatus 
       *delivery_setstate (rno, val, reason)
int  rno, val;
char *reason;
\end{verbatim}\end{quote}

This routine sets the value of the given recipient number to the
indicated value in the global structure (with the above proviso). It
also includes a string indicating the reason for failure, if failure
occured. This string is passed back to the \pgm{qmgr} for display by
the \pgm{MTAconsole}. It should be set to the constant \verb|NULLCP|
if the recipient was handled successfully.

\begin{quote}\index{delivery\_setall}\begin{verbatim}
struct type_Qmgr_DeliveryStatus 
       *delivery_setallstate (val, reason)
int  val;
char *reason;
\end{verbatim}\end{quote}

This routine sets all users status to the given value. This is useful
when a connection can't be established for instance and all recipients
must be put in the same state (again, the above proviso about delivery
reports holds). The reason field is the same as above.

The termination function, if used, should be defined as follows.
\begin{quote}\begin{verbatim}
int endfuction ()
\end{verbatim}\end{quote}

This routine is used to close down anything required before exiting
the program. The calling routine will execute this function before sending
the final disconnect to the \verb|qmgr| and exiting.


\subsubsection{Address file routines}

The address file reading is usually the first operation a channel does
on receiving the instruction to process a message. This is usually
achieved by first building up the name of the message file to be read
in from the qid, and then invoking the procedure \verb|rd_msg|.
\begin{quote}\index{rd\_msg}\begin{verbatim}
int rd_msg (msg, prm, qp, ad_sndr, ad_recip, ad_count)
char  *msg;
struct prm *prm;
Q_struct *qp;
ADDR  **ad_sndr,
      **ad_recip;
int   *ad_count;
\end{verbatim}\end{quote}
The parameters are:
\begin{describe}
\item[\verb|msg|:] The address control file.

\item[\verb|prm|:] The message parameters.

\item[\verb|qp|:] The message queue structure.

\item[\verb|ad\_sndr|:] the originator of the message.

\item[\verb|ad\_recip|:] The list of recipients of the message.

\item[\verb|ad\_count|:] A count of the number of recipient addresses.
\end{describe}

The routine returns a status to indicate success or failure. As part
of its operation it locks the address file to prevent simultaneous
access by other channels.

When the message has been processed, the lock {\em must} be released
by calling the function \verb|rd_end|.
\begin{quote}\index{rd\_end}\begin{verbatim}
int rd_end ();
\end{verbatim}\end{quote}

If the messages status changes in some manner, the address file must
be updated. This is achieved by one of the following functions.
\begin{quote}\index{wr\_ad\_status}\index{wr\_ad\_rcntno}\begin{verbatim}
int wr_ad_status (ap, type}
ADDR *ap;
int type;

int wr_ad_rcntno (ap, rcnt)
ADDR *ap;
int  rcnt;
\end{verbatim}\end{quote}

The \verb|wr_ad_status| routine is used to update the status of the address.
This can take one of the following values.
\[\begin{tabular}{|l|l|}
\hline
	\multicolumn{1}{|c|}{\bf Value}&
		\multicolumn{1}{|c|}{\bf Meaning}\\
\hline
	\tt AD\_STAT\_PEND&	The address is pending\\
	\tt AD\_STAT\_DONE&	The address has been finished with\\
\hline
\end{tabular}\]


The \verb|wr_ad_rcntno| is used to update the reformatter count variable.

\subsubsection{Message files}

There are a number of routines to allow a channel to iterate through
the set of message files to deliver. First in this set is a function
to allow location of the relevant directory. This function maps a
given queue id as given by the qmgr into a directory basename with the
message files in relating to a given address. It is
used as follows:
\begin{quote}\index{qid2dir}\begin{verbatim}
int qid2dir (qid, ap, exists, cpp)
char   *qid;
ADDR   *ap;
int    exists;
char   **cpp;
\end{verbatim}\end{quote}
The arguments to this are
\begin{describe}
\item[\verb|qid|:]	A character string containing the queue identifier.

\item[\verb|ap|:]	The address that is being processed.

\item[\verb|exists|:]	A flag, this should be \verb|TRUE| if the directory
should exist, and \verb|FALSE| if it doesn't matter.

\item[\verb|cpp|:]	A character pointer that is assigned the
resulting directory name.
\end{describe}

The function will return \verb|OK| on successful completion and
\verb|NOTOK| on failure.

The next operation is typically to visit each of the message files in
turn. This is handled by the following routines.

\begin{quote}\index{msg\_rinit}\begin{verbatim}
int msg_rinit (dir)
char *dir;
\end{verbatim}\end{quote}

The argument here is
\begin{describe}
\item[\verb|dir|:]	The directory base of the message (usually
obtained from a call to \verb|qid2dir|).
\end{describe}

This function returns the usual \verb|OK|/\verb|NOTOK| status.

Following this, the function \verb|msg_rfile| is called repeatedly
until it returns \verb|RP_DONE| to obtain each of the files composing
the message.
\begin{quote}\index{msg\_rfile}\begin{verbatim}
int msg_rfile (buf)
char *buf;
\end{verbatim}\end{quote}
The argument is:
\begin{describe}
\item[\verb|buf|:] A pointer to a character buffer (of size
\verb|FILNSIZE|) which is filled in with the file name if the call succeeds..
\end{describe}

Finally, when all files have been processed, or if the operation is to
be aborted, the function \verb|msg_rend| is called.
\begin{quote}\index{msg\_rend}\begin{verbatim}
int msg_rend ()
\end{verbatim}\end{quote}

\subsection {Delivery Notifications}\label{set:dr}

Delivery reports should be written out by a channel when it is clear
that the message cannot be delivered or if the message has been
delivered and confirmation is requested. The routine
\verb|set_1dr| is then used to set the appropriate status in the
address structure. 

\begin{quote}\index{set\_1dr}\begin{verbatim}
int  set_1dr (qp, ad_no, reason, diag, str)
Q_struct *qp;
int      ad_no;
int      reason;
int      diag;
char     *str;
\end{verbatim}\end{quote}

The arguments to this function are as follows
\begin{describe}
\item[\verb|qp|:]	A pointer to a queue structure representing
the message. The queue structure should have the address components
filled in (this is normally the case).

\item[\verb|ad\_no|:]	The address number that this report is for.

\item[\verb|reason|:]	A reson code for the message failure. This
should be chosen from one of those in Table~\ref{tab:reason} on
page~\pageref{tab:reason}.

\item[\verb|diag|:] A diagnostic code for the message failure.
This is supplementary information to the reason code. It should be one
of those values given in Table~\ref{tab:diag} on page~\pageref{tab:diag}.

\item[\verb|str|:]	A character string which records the reason
for the failure. This should be something that describes the failure.
This field should always be filled in with something appropriate.

\end{describe}

\tagtable{reason}{Reason codes}{tab:reason}

\tagtable{diag}{Diagnostic codes}{tab:diag}

When calling this function the routine should return to the qmgr one
of the following variables:
\[\begin{tabular}{|l | p{0.6\textwidth}|}
\hline
	\multicolumn{1}{|c|}{\bf Qmgr code} &
		\multicolumn{1}{|c|}{\bf When Applicable} \\
\hline
	\tt positiveDR &	For the first recipeint with a
				positive delivery report  \\
	\tt successSharedDR&	For subsequent recipients with
				positive delivery reports\\
	\tt negativeDR &	For the first recipient with a
				negative delivery report \\
	\tt failureSharedDR&	For subsequent recipients with
				negative delivery reports \\
\hline
\end{tabular}\]

Once the recipients have been marked with \verb|set_1dr|, the last
step necessary is to write out the actual delivery report by calling
\verb|wr_q2dr|. This is defined as follows:

\begin{quote}\index{wr\_q2dr}\begin{verbatim}
int wr_q2dr (qp, msgid)
Q_struct *qp;
char *msgid;
\end{verbatim}\end{quote}

The arguments to this function are
\begin{describe}
\item[\verb|qp|:]	A pointer to the queue structure representing
the message with the address components filled in.

\item[\verb|msgid|:] The queue id for this message.
\end{describe}

This function will then write out the reports and update the queue
entries with their new status.


\subsection {Verification and Trust}

Channels are trusted processes, and as such should respect this trust.
As far as possible they should check that the data they are being
given is reasonable. This includes the following:
\begin{itemize}
\item	That the channel name and type is reasonable for this channel.

\item	That the address requested to be processed exists and is ready
to be processed (e.g. it has passed through all reformatters).

\item	That the type of address is correct for this type of channel
(e.g. RFC-822 for smtp or X.400 for X.400 channels).

\item	That the message is not locked by another channel.

\item 	That the channel is the same as the channel given in the
address structure.
\end{itemize}

For common outbound channels and filters these checks are encapsulated
in two functions, \verb|chan_acheck|\index{chan\_acheck} and
\verb|lchan_acheck|\index{lchan\_acheck}.

These two calls are similar in function, one checks the address is ok
to process for filters and outbound channels, the other for local channels.
They are called in the following way:

\begin{quote}\small\begin{verbatim}
int chan_acheck (ap, mychan, first, mta)
ADDR *ap;
CHAN *mychan;
int first,
char **mta;
\end{verbatim}\end{quote}

The arguments are as follows
\begin{describe}
\item[\verb|ap|:] The address to be checked.
\item[\verb|mychan|:] The channel that is doing this checking.
\item[\verb|first|:] If this is the first address to be checked. This
is used to determine if the mta parameter can be changed.
\item[\verb|mta|:] The mta this should connect to. If the constant
\verb|NULLVP| is used, the mta parameter will not be filled in or
checked. If the mta is the address of a pointer with \verb|NULL|
contents then it will be updated if required. If the mta is a pointer
to a string, then it is checked that the mta does not change unless
the \verb|first| flag is set. This ensures for channels that batch
messages to the same host, that all addresses in this message are
destined for the same mta.
\end{describe}
It returns either the constant \verb|OK| or \verb|NOTOK|. If
\verb|NOTOK| then the \verb|deliverystate| is updated in a suitable way.
If \verb|OK| then the address should be processed, and the
\verb|deliverystate| updated accordingly.

The function \verb|lchan_acheck| is almost equivalent, except in place
of the \verb|mta| parameter is a user.

\begin{quote}\small\begin{verbatim}
int lchan_acheck (ap, mychan, first, user)
ADDR *ap;
CHAN *mychan;
int first,
char **user;
\end{verbatim}\end{quote}


\section {Example usage}

\subsection {Outbound Channels}

Presented here is a skeleton of an outbound channel. This structure is
not required, but is a common way of achieving the results.

\tagrindfile{outbound}{Example outbound channel}{example:outbound}

\subsection {Local Pickup}

This needs to be written - both in code and here!

\subsection {TWA channel}

%Show how this can be used as ``pull only'', and as full TWA in conjunction
%with submission.   Need to have both QMGR and server initiated approach.
Not yet written

\subsection {Delivery Reports}

A typical interface to the delivery report functions might be the
following.

\tagrindfile{delv-rep}{Example Delivery Report Generation}{examp:dr}

\section {Formatters and Converters}


Formatters and converters can run in two ways. Generally the simple
filters work on single files, the complex filters work on several
files at once. The majority of filters should be of the simple kind.

\subsection{Simple converters}

The simple converters are programs that reads from the standard input,
do some formatting and write the result to the standard output.
This type of formatter is invoked by the filtercontrol program,
\verb|fcontrol|\index{fcontrol}.

These simple filters should return a suitable exit status to confirm
that the conversion completed successfully. They follow the standard
\unix/ tradition of returning \verb|0| on success and anything else on
failure. 

These filters may be anything from simple shell/awk/sed scripts to
more complex {\em C} programs.

\subsection{Complex Filters}

More complex formatters are necessary if the formatting needs to
convert more than one file. This is the case for formatters that change
the directory structure, or combine several files into one. These
formatters require more work, but are similar to the channel programs
in their operation. The \verb|qmgr| interface is in fact identical.
The only real difference is in the \verb|qid2dir|\index{qid2dir}
routine used to locate the directory. The starting directory is found
using the normal \verb|qid2dir| routine. The destination directory is
found using the same routine, but first incrementing the formatter
count in the address structure, and then calling the routine with the
flag set to \verb|FALSE|. The formatter should however place it's
output in a temporary directory which will then be renamed to the
destination directory if the transformation is successful. This allows
for failure at any point in the reformatting.

%Usually can just do filters --- describe

%Notes on model for restructuring

\subsection{Example}

to be written.
DataMuseum.dk

DKUUG/EUUG Conference tapes

⟦de6490a88⟧ TextFile

Derivation

TextFile
