Date: 20 Jul 2000 00:14:01 -0000
Message-ID: <20000720001401.19958.qmail@cr.yp.to>
From: "D. J. Bernstein" <djb@cr.yp.to>
To: drums@cs.utk.edu
Subject: Re: SMTP WG Last-Call
References: <1461297.3173012530@nifty-jr.west.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii

My web page http://cr.yp.to/smtp/klensin.html contains many objections
to this draft. A copy of the text appears below.

(Procedural notes: I have raised all of these objections before. In at
least two cases, there is already overwhelming evidence that Klensin's
position is not the consensus of the working group:

   http://cr.yp.to/smtp/drums/19990215054214-9088-qmail@cr-yp-to
   http://cr.yp.to/djbdns/namedroppers/20000204012814-28947-qmail@cr-yp-to

I'd rather discuss content than procedure. Unfortunately, certain people
habitually try to destroy productive discussions by claiming, falsely,
that there's consensus against my objections. So I feel compelled to
preemptively point out these two clearly established counterexamples.)

---Dan


smtpupd-10 was released in February 1999. This page is a list of
problems in smtpupd-10. I've excluded issues of procedure, document
organization, and 8-bit mail.

smtpupd-11 was released in March 2000. As far as I can tell, none
of these problems were fixed.

smtpupd-12 was released in July 2000. Two problems were fixed, as
noted below.

Interoperability failures in smtpupd-10
---------------------------------------

Klensin says that a client without a name registered in DNS
        SHOULD send an address literal ... optionally followed by
        information that will help to identify the client system
in HELO. The address literal is fine; a bracketed IP address is a
valid RFC 821 domain, and any server that rejected it would be
violating RFC 1123, section 5.2.5. However, ``information that will
help to identify the client system'' is a Klensin invention that
creates new interoperability problems. Clients that send such
information will find themselves completely unable to send mail to
some hosts.

Klensin says that servers ``MUST NOT recognize'' anything other
than \015\012 as a line terminator. In reality, for interoperability
with existing clients, servers recognize \012 as a line terminator
for client requests. Clients do not (and, in practice, cannot) use
\012 inside requests except as a line terminator.

Klensin says that SMTP servers ``MUST reject'' requests containing
underscores or other ``illegal character codes.'' In reality, for
interoperability with existing clients, servers accept (for example)
EHLO requests containing underscores.

Klensin has extended the Received-line syntax to allow fractional
seconds. This breaks some Received-line parsers. This problem was
fixed in smtpupd-12.

There are dozens of other interoperability issues where Klensin
does not give adequate warning to future implementors.

Safe behavior prohibited by smtpupd-10
--------------------------------------

Klensin says
        Only resolvable, fully-qualified, domain names (FQDNs) are
        permitted when domain names are used in SMTP.
This prohibits the behavior of many existing clients: for example,
a client relaying a message to a smarthost is violating the protocol
if the user typed an address incorrectly. Obeying Klensin's requirement
means checking every address in DNS. This is an unacceptable policy
violation at some corporate sites, an unacceptable expense for many
dumb clients, and an unacceptable obstacle for the vast majority
of users when their ISPs are temporarily unable to reach the rest
of the Internet. The requirement also prohibits the standard use
of domain literals as specified in RFC 1123.

Modern MTAs send ``double-bounce'' notifications to the local
postmaster when a bounce message is undeliverable. Klensin has three
requirements saying that this ``MUST NOT'' be done; only one of the
requirements has a relevant exception.

Klensin says that an SMTP client ``MUST NOT intentionally close the
transmission channel until it sends a QUIT command.'' In reality,
if an SMTP client is in the middle of sending a message, and is
unable to read the next portion of the message from disk, it
ruthlessly closes the connection. Sending QUIT in this situation
means sending a corrupted message.

Klensin says that VRFY ``MUST be listed ... in an EHLO response''
if it is supported. In reality, as of 1999, approximately 90% of
the servers that support both VRFY and EHLO do not list VRFY in
response to EHLO. If a client wants to use VRFY for some reason,
it simply tries the VRFY command, as per RFC 821; paying attention
to the EHLO response would be foolish. There's no reason to require
an announcement.

Klensin says that SMTP clients ``MUST NOT'' send message data unless
the response to DATA is 354. This prohibits the behavior of many
existing clients: if a server (in violation of RFC 821) sends (e.g.)
a 387 reply, these clients will (as encouraged by RFC 821) treat
387 the same way as 354. The correct rule is that clients must not
send message data if the reply begins with 4 or 5.

Klensin says that servers ``MUST support the EHLO command even if
they do not implement any specific extensions.'' In reality, servers
that do not support EHLO are still able to receive mail. If a server
does not support any extensions, its only benefit from supporting
EHLO is to save a small amount of time for clients that start with
EHLO. This is enough of a benefit for me to recommend that all
servers support EHLO, but there is no interoperability excuse for
requiring it.

Klensin says
        Servers MUST NOT return the extended EHLO-style response
        to a HELO command.
It is unclear what this means. Some obvious interpretations prohibit
the behavior of existing hosts.

Klensin says
        Whatever mechanisms are used, servers MUST contain provisions
        for detecting and stopping trivial loops.
It is horrendously unclear what this means. Some obvious interpretations
prohibit the behavior of existing hosts.

Safe behavior discouraged by smtpupd-10
---------------------------------------

An essential element of a proper multidrop POP mailbox delivery is
that the (single) envelope recipient is copied to a header field
in the message, typically Delivered-To. However, Klensin says
        SMTP clients and servers SHOULD NOT copy the full set of
        RCPT TO command arguments into the headers, either as part
        of trace headers or as informational or private-extension
        headers.
Klensin says that this ``requirement'' should be violated ``only
under exceptional and well-understood circumstances''; Klensin
claims that MTAs that use Delivered-To may not ``interoperate
properly'' unless ``great care is taken.'' This is garbage. Klensin's
alleged justification---namely, that such copying ``defeats the
purpose'' of mailing lists and blind carbon copies---is completely
incorrect for a message being delivered to one recipient at a time.

The normal behavior of part-time dialup hosts and dumb clients is
to connect to a relay identified by name (or sometimes IP address).
Klensin says that clients that use relays ``SHOULD'' instead perform
MX lookups on the relay name. Klensin says that this ``requirement''
should be violated ``only under exceptional and well-understood
circumstances''; Klensin claims that clients using the normal
procedure may not ``interoperate properly'' unless ``great care is
taken.'' This is garbage. There is no reason to demand that these
clients use MX lookups.

Klensin says that clients ``SHOULD preferentially utilize EHLO
rather than HELO.'' Klensin says that this ``requirement'' should
be violated ``only under exceptional and well-understood circumstances'';
Klensin claims that clients starting with HELO may not ``interoperate
properly'' unless ``great care is taken.'' This is garbage. In
reality, clients that have no need for extensions can and should
start with HELO; EHLO adds quite a bit of unnecessary complexity.
Of course, clients that want to use extensions will start with EHLO.

Klensin says that SMTP servers ``SHOULD support EXPN.'' Klensin
says that this ``requirement'' should be violated ``only under
exceptional and well-understood circumstances''; Klensin claims
that SMTP servers that always respond 502 to EXPN may not ``interoperate
properly'' unless ``great care is taken.'' This is garbage. In
reality, EXPN is obsolete. Thousands of the most heavily used mail
servers on the Internet always respond 502 to EXPN.

Klensin says that SMTP servers ``SHOULD support VRFY''; this means
``at least recognition of local mailboxes.'' Klensin says that this
``requirement'' should be violated ``only under exceptional and
well-understood circumstances''; Klensin claims that SMTP servers
that always respond 252 to VRFY may not ``interoperate properly''
unless ``great care is taken.'' This is garbage. In reality, VRFY
is obsolete. Thousands of the most heavily used mail servers on the
Internet always respond 252 to VRFY.

Klensin says that an SMTP client ``SHOULD'' wait until it receives
the reply to QUIT before closing a connection. Klensin says that
this ``requirement'' should be violated ``only under exceptional
and well-understood circumstances''; Klensin claims that clients
closing the connection immediately after sending QUIT may not
``interoperate properly'' unless ``great care is taken.'' This is
garbage. In reality, thousands of SMTP clients close the connection
immediately after sending QUIT. There is no reason to wait for the
response.

Klensin says that an SMTP server ``SHOULD attempt to send a line
containing a 421 response code'' if it is shut down by the system
administrator. Klensin says that this ``requirement'' should be
violated ``only under exceptional and well-understood circumstances'';
Klensin claims that servers that simply close the connection may
not ``interoperate properly'' unless ``great care is taken.'' This
is garbage. In reality, connections are abruptly dropped for a wide
variety of reasons, including server shutdowns.

Klensin says that every ``SMTP-capable host SHOULD support both the
alias and the list models of address expansion for multiple delivery.''
Klensin says that this ``requirement'' should be violated ``only
under exceptional and well-understood circumstances''; Klensin
claims that servers that don't support mailing lists may not
``interoperate properly'' unless ``great care is taken.'' This is
garbage. In reality, firewalls and POP toasters do not, and should
not, support mailing lists.

Klensin says that SMTP servers ``SHOULD reject'' parameters in RSET,
DATA, or QUIT. Klensin says that this ``requirement'' should be
violated ``only under exceptional and well-understood circumstances'';
Klensin claims that servers that ignore such parameters may not
``interoperate properly'' unless ``great care is taken.'' This is
garbage. In reality, thousands of SMTP servers ignore such parameters.

Klensin says that all ``fully-capable SMTP implementations'' are
``expected to support'' various client features. This is garbage.
In reality, firewalls and POP toasters often run pure SMTP servers.
They do not send mail through SMTP, and they are not expected to
support any SMTP client features. Installing unused code is a
violation of some corporate security policies.

Other misleading statements in smtpupd-10
-----------------------------------------

sendmail rewrites messages received through SMTP. Klensin claims
that this behavior was ``in response to weak SMTP clients'' that
generated incomplete messages. That's backwards. First sendmail was
designed to feed all messages, whether received locally or via SMTP,
through the same rewriting and routing mechanisms. Then people took
advantage of this and wrote dumb clients.

Klensin claims that, ``in most cases,'' SMTP clients that send all
mail through a relay host neglect to retry delivery after failures.
This might be true for some dumb clients but it's certainly not
true for serious MTAs such as sendmail.

Klensin defines ``relays'' as receiving mail by SMTP and forwarding
mail by SMTP. In reality, many relays support other protocols for
transferring Internet mail messages, such as QMTP. (These relays
generally do not change message formats, so they do not fit Klensin's
``gateway'' requirements.)

Klensin says that ``the [message] body, if structured, is defined
according to MIME.'' In reality, there are many common non-MIME
body structures.

Klensin's document begins the same way as the original SMTP
specification, RFC 821, written in 1982:
        The objective of the Simple Mail Transfer Protocol (SMTP)
        is to transfer mail reliably and efficiently.
Unfortunately, SMTP does not achieve this objective. It is painfully
slow; more importantly, its design has led to a string of
interoperability disasters.

Klensin requires that years contain at most four digits. In reality,
there will be a year 10000. This problem was fixed in smtpupd-12.