[Pkg-exim4-users] getting authentication to work with a smarthost

Discussion:

Ross Boylan

2016-01-12 09:10:12 UTC

I am try to send mail via a smarthost that requires authentication.
Some documentation (dated, I suspect) indicates I should be using port
465, but I am able to connect via 25. The smarthost advertises
STARTTLS and various authentication mechanisms, and my local exim
seems to recognize that it should try to authenticate. But as far as
I can tell it neither negotiates TLS nor attempts to authenticate. It
just tries to deliver the email, which is rejected as unauthenticated.

I would appreciate any help.
<update-exim4.conf.conf>
dc_eximconfig_configtype='smarthost'
dc_other_hostnames='ross-sas.psg.net ross-sas.epi-ucsf.org'
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost='ucsf.edu'
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost='mail.ucsf.edu:465'
CFILEMODE='644'
dc_use_split_config='true'
dc_hide_mailname='true'
dc_mailname_in_oh='true'
dc_localdelivery='mail_spool'
</update-exim4.conf.conf>
I have made no customizations beyond running dpkg-reconfigure
exim4-config and putting an entry in passwd.client.

Something, possibly the debconf questions, made me think the single
colon in the smarthost was the proper syntax for specifying an
alternate port. It seems to have been interpreted as a list separator
and ignored instead; I have since doubled it. Results at the bottom.

I think the smarthost is running MS Exchange 2012.
# exim -v -t -bm -f "" -d < test.msg
produced lots of output, finally ending up in the
remote_smtp_smarthost transport. Here's the key section, with my
comments added after the #

Connecting to mail.ucsf.edu [64.54.247.179]:25 ... connected # Port
25, not the 465 I requested
waiting for data on socket
read response data: size=95
SMTP<< 220 exht05.net.ucsf.edu Microsoft ESMTP MAIL Service ready at
Mon, 11 Jan 2016 23:00:50 -0800
64.54.247.179 in hosts_avoid_esmtp? no (option unset) # remote host
name does not match what I used to find it
SMTP>> EHLO ross-sas # my local system has no FQDN
waiting for data on socket
read response data: size=201
SMTP<< 250-STARTTLS
250-exht05.net.ucsf.edu Hello [64.54.171.2]
250-SIZE 141557760
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH GSSAPI NTLM LOGIN
250-8BITMIME
250-BINARYMIME
250 CHUNKING
64.54.247.179 in hosts_require_tls? no (option unset)
64.54.247.179 in hosts_avoid_pipelining? no (option unset)
using PIPELINING
64.54.247.179 in hosts_require_auth? no (option unset)
search_open: nwildlsearch "/etc/exim4/passwd.client"
search_find: file="/etc/exim4/passwd.client"
key="mail.ucsf.edu" partial=-1 affix=NULL starflags=0

/etc/exim4/passwd.client

End
internal_search_find: file="/etc/exim4/passwd.client"
type=nwildlsearch key="mail.ucsf.edu"
file lookup required for mail.ucsf.edu
in /etc/exim4/passwd.client
mail.ucsf.edu in "mail.ucsf.edu"? yes (matched "mail.ucsf.edu")
lookup yielded: SomeAccount:SomePasword # Recognizes as configured
for authentication
64.54.247.179 in hosts_try_auth? yes (matched "64.54.247.179")
scanning authentication mechanisms
login authenticator yielded 13 # Not sure what that means
# I would expect the next messages to the smarthost to establish TLS
# and then authenticate.
# But instead, we jump right to a mail command.
# Maybe such negotiations are note reported in the debug output?
# However, the failure of the MAIL command suggests the problem is
# that the commands were never issued.
SMTP>> MAIL FROM:<> SIZE=1716
SMTP>> RCPT TO:<***@ucsf.edu>
SMTP>> DATA
waiting for data on socket
read response data: size=40
SMTP<< 530 5.7.1 Client was not authenticated
waiting for data on socket
ok=0 send_quit=1 send_rset=1 continue_more=0 yield=0 first_address is not NULL
SMTP>> QUIT

Thanks.
Ross Boylan

P.S. Running
exim4-daemon-heavy 4.82-3ubuntu2
exim4-config 4.82-3ubuntu2

Doubling the colon in the smarthost spescification gets exim to use
port 465, but:
mail.ucsf.edu [64.54.247.179]:465 status = usable
64.54.247.179 in serialize_hosts? no (option unset)
delivering 1aIufG-00024b-Sq to mail.ucsf.edu [64.54.247.179]
(***@ucsf.edu)
set_process_info: 7977 delivering 1aIufG-00024b-Sq to mail.ucsf.edu
[64.54.247.179] (***@ucsf.edu)
Transport port=25 replaced by host-specific port=465
Connecting to mail.ucsf.edu [64.54.247.179]:465 ... connected
waiting for data on socket
ok=0 send_quit=0 send_rset=1 continue_more=0 yield=1 first_address is not NULL
LOG: MAIN
Remote host mail.ucsf.edu [64.54.247.179] closed connection in
response to initial connection
set_process_info: 7977 delivering 1aIufG-00024b-Sq: just tried
mail.ucsf.edu [64.54.247.179] for ***@ucsf.edu: result DEFER
added retry item for T:mail.ucsf.edu:64.54.247.179:465: errno=-18
more_errno=0,A flags=2
all IP addresses skipped or deferred at least one address

Ross Boylan

2016-01-12 23:04:55 UTC

Permalink

I've noticed some things and done some tweaks, but still can not get
TLS to start (at least, I see nothing in the logs suggesting it
started) or authentication to be attempted.

1. Names and IPs of the smarthost.
dig mail.ucsf.edu mx yields 3 machine names, with 3 associated IPs.
dig -x on the IP's yields the machine name. I added
^(cuda|jingo|jango)\.ucsf\.edu$:nnnn:pppp
to passwd.client without luck.
The actual IP connected to doesn't match any of those machines.
Here's a more recent fragment with DNS debugging on:
finding IP address for mail.ucsf.edu
calling host_find_byname
gethostbyname2(af=inet6) returned 4 (NO_DATA)
fully qualified name = mail.ucsf.edu
gethostbyname2 looked up these IP addresses:
name=mail.ucsf.edu address=64.54.247.179
I'm baffled that the IP is not what I get using dig. I added a line
with the IP to passwd.client; no change.

2. "login authenticator yielded 13" might indicate the host wasn't
matching in the passwd.client file according to the internet.
This and the discussion of passwd.client in the man page was the basis
of trying to expand the entries in passwd.client.
I also speculated it might mean there were no matching authenticators,
leading to ..

3. The authenticators listed, "AUTH GSSAPI NTLM LOGIN", appear not to
match those configured. Login would probably work if the connection
were encrypted, which it doesn't seem to be. I added a sap
authenticator for NTLM. No change in the results.

Ross

On Tue, Jan 12, 2016 at 1:10 AM, Ross Boylan

Post by Ross Boylan
I am try to send mail via a smarthost that requires authentication.
Some documentation (dated, I suspect) indicates I should be using port
465, but I am able to connect via 25. The smarthost advertises
STARTTLS and various authentication mechanisms, and my local exim
seems to recognize that it should try to authenticate. But as far as
I can tell it neither negotiates TLS nor attempts to authenticate. It
just tries to deliver the email, which is rejected as unauthenticated.
I would appreciate any help.
<update-exim4.conf.conf>
dc_eximconfig_configtype='smarthost'
dc_other_hostnames='ross-sas.psg.net ross-sas.epi-ucsf.org'
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost='ucsf.edu'
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost='mail.ucsf.edu:465'
CFILEMODE='644'
dc_use_split_config='true'
dc_hide_mailname='true'
dc_mailname_in_oh='true'
dc_localdelivery='mail_spool'
</update-exim4.conf.conf>
I have made no customizations beyond running dpkg-reconfigure
exim4-config and putting an entry in passwd.client.
Something, possibly the debconf questions, made me think the single
colon in the smarthost was the proper syntax for specifying an
alternate port. It seems to have been interpreted as a list separator
and ignored instead; I have since doubled it. Results at the bottom.
I think the smarthost is running MS Exchange 2012.
# exim -v -t -bm -f "" -d < test.msg
produced lots of output, finally ending up in the
remote_smtp_smarthost transport. Here's the key section, with my
comments added after the #
Connecting to mail.ucsf.edu [64.54.247.179]:25 ... connected # Port
25, not the 465 I requested
waiting for data on socket
read response data: size=95
SMTP<< 220 exht05.net.ucsf.edu Microsoft ESMTP MAIL Service ready at
Mon, 11 Jan 2016 23:00:50 -0800
64.54.247.179 in hosts_avoid_esmtp? no (option unset) # remote host
name does not match what I used to find it
SMTP>> EHLO ross-sas # my local system has no FQDN
waiting for data on socket
read response data: size=201
SMTP<< 250-STARTTLS
250-exht05.net.ucsf.edu Hello [64.54.171.2]
250-SIZE 141557760
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH GSSAPI NTLM LOGIN
250-8BITMIME
250-BINARYMIME
250 CHUNKING
64.54.247.179 in hosts_require_tls? no (option unset)
64.54.247.179 in hosts_avoid_pipelining? no (option unset)
using PIPELINING
64.54.247.179 in hosts_require_auth? no (option unset)
search_open: nwildlsearch "/etc/exim4/passwd.client"
search_find: file="/etc/exim4/passwd.client"
key="mail.ucsf.edu" partial=-1 affix=NULL starflags=0

Post by Ross Boylan
/etc/exim4/passwd.client

End
internal_search_find: file="/etc/exim4/passwd.client"
type=nwildlsearch key="mail.ucsf.edu"
file lookup required for mail.ucsf.edu
in /etc/exim4/passwd.client
mail.ucsf.edu in "mail.ucsf.edu"? yes (matched "mail.ucsf.edu")
lookup yielded: SomeAccount:SomePasword # Recognizes as configured
for authentication
64.54.247.179 in hosts_try_auth? yes (matched "64.54.247.179")
scanning authentication mechanisms
login authenticator yielded 13 # Not sure what that means
# I would expect the next messages to the smarthost to establish TLS
# and then authenticate.
# But instead, we jump right to a mail command.
# Maybe such negotiations are note reported in the debug output?
# However, the failure of the MAIL command suggests the problem is
# that the commands were never issued.
SMTP>> MAIL FROM:<> SIZE=1716
SMTP>> DATA
waiting for data on socket
read response data: size=40
SMTP<< 530 5.7.1 Client was not authenticated
waiting for data on socket
ok=0 send_quit=1 send_rset=1 continue_more=0 yield=0 first_address is not NULL
SMTP>> QUIT
Thanks.
Ross Boylan
P.S. Running
exim4-daemon-heavy 4.82-3ubuntu2
exim4-config 4.82-3ubuntu2
Doubling the colon in the smarthost spescification gets exim to use
mail.ucsf.edu [64.54.247.179]:465 status = usable
64.54.247.179 in serialize_hosts? no (option unset)
delivering 1aIufG-00024b-Sq to mail.ucsf.edu [64.54.247.179]
set_process_info: 7977 delivering 1aIufG-00024b-Sq to mail.ucsf.edu
Transport port=25 replaced by host-specific port=465
Connecting to mail.ucsf.edu [64.54.247.179]:465 ... connected
waiting for data on socket
ok=0 send_quit=0 send_rset=1 continue_more=0 yield=1 first_address is not NULL
LOG: MAIN
Remote host mail.ucsf.edu [64.54.247.179] closed connection in
response to initial connection
set_process_info: 7977 delivering 1aIufG-00024b-Sq: just tried
added retry item for T:mail.ucsf.edu:64.54.247.179:465: errno=-18
more_errno=0,A flags=2
all IP addresses skipped or deferred at least one address

Nick Guerette

2016-01-13 00:39:29 UTC

Permalink

To remove extra possible failure modes (wrong regex or new server name),
try using a wildcard instead of a regex in passwd.client and also try
adding the @domain to the username to be used for authentication if you
weren't already:

*.ucsf.edu:***@dddd.edu:pppp

However, in a quick search I found this document indicating you should
use port 465 for outgoing mail, which suggests that the server expects
you to use the nominally obsolete but still very popular SMTPS protocol:

https://it.ucsf.edu/services/email/tutorial/ucsf-email-pop-and-imap-settings?page=show

If this is a requirement, then the server may be lying about supporting
STARTTLS. For using SMTPS with a smarthost, see a thread from one of
the last few times this came up:

https://lists.alioth.debian.org/pipermail/pkg-exim4-users/2014-December/002186.html

Post by Ross Boylan
I've noticed some things and done some tweaks, but still can not get
TLS to start (at least, I see nothing in the logs suggesting it
started) or authentication to be attempted.
1. Names and IPs of the smarthost.
dig mail.ucsf.edu mx yields 3 machine names, with 3 associated IPs.
dig -x on the IP's yields the machine name. I added
^(cuda|jingo|jango)\.ucsf\.edu$:nnnn:pppp
to passwd.client without luck.
The actual IP connected to doesn't match any of those machines.
finding IP address for mail.ucsf.edu
calling host_find_byname
gethostbyname2(af=inet6) returned 4 (NO_DATA)
fully qualified name = mail.ucsf.edu
name=mail.ucsf.edu address=64.54.247.179
I'm baffled that the IP is not what I get using dig. I added a line
with the IP to passwd.client; no change.
2. "login authenticator yielded 13" might indicate the host wasn't
matching in the passwd.client file according to the internet.
This and the discussion of passwd.client in the man page was the basis
of trying to expand the entries in passwd.client.
I also speculated it might mean there were no matching authenticators,
leading to ..
3. The authenticators listed, "AUTH GSSAPI NTLM LOGIN", appear not to
match those configured. Login would probably work if the connection
were encrypted, which it doesn't seem to be. I added a sap
authenticator for NTLM. No change in the results.
Ross
On Tue, Jan 12, 2016 at 1:10 AM, Ross Boylan

Post by Ross Boylan
/etc/exim4/passwd.client

End
internal_search_find: file="/etc/exim4/passwd.client"
type=nwildlsearch key="mail.ucsf.edu"
file lookup required for mail.ucsf.edu
in /etc/exim4/passwd.client
mail.ucsf.edu in "mail.ucsf.edu"? yes (matched "mail.ucsf.edu")
lookup yielded: SomeAccount:SomePasword # Recognizes as configured
for authentication
64.54.247.179 in hosts_try_auth? yes (matched "64.54.247.179")
scanning authentication mechanisms
login authenticator yielded 13 # Not sure what that means
# I would expect the next messages to the smarthost to establish TLS
# and then authenticate.
# But instead, we jump right to a mail command.
# Maybe such negotiations are note reported in the debug output?
# However, the failure of the MAIL command suggests the problem is
# that the commands were never issued.
SMTP>> MAIL FROM:<> SIZE=1716
SMTP>> DATA
waiting for data on socket
read response data: size=40
SMTP<< 530 5.7.1 Client was not authenticated
waiting for data on socket
ok=0 send_quit=1 send_rset=1 continue_more=0 yield=0 first_address is not NULL
SMTP>> QUIT
Thanks.
Ross Boylan
P.S. Running
exim4-daemon-heavy 4.82-3ubuntu2
exim4-config 4.82-3ubuntu2
Doubling the colon in the smarthost spescification gets exim to use
mail.ucsf.edu [64.54.247.179]:465 status = usable
64.54.247.179 in serialize_hosts? no (option unset)
delivering 1aIufG-00024b-Sq to mail.ucsf.edu [64.54.247.179]
set_process_info: 7977 delivering 1aIufG-00024b-Sq to mail.ucsf.edu
Transport port=25 replaced by host-specific port=465
Connecting to mail.ucsf.edu [64.54.247.179]:465 ... connected
waiting for data on socket
ok=0 send_quit=0 send_rset=1 continue_more=0 yield=1 first_address is not NULL
LOG: MAIN
Remote host mail.ucsf.edu [64.54.247.179] closed connection in
response to initial connection
set_process_info: 7977 delivering 1aIufG-00024b-Sq: just tried
added retry item for T:mail.ucsf.edu:64.54.247.179:465: errno=-18
more_errno=0,A flags=2
all IP addresses skipped or deferred at least one address

_______________________________________________
Pkg-exim4-users mailing list
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exim4-users

--
Nick Guerette
Embedded Systems Engineer
Mosaic Industries, Inc.
510-790-8222
http://mosaic-industries.com/embedded-systems

Ross Boylan

2016-01-13 20:39:26 UTC

Permalink

Thanks to Nick's suggestions, I got things working.

1. As mentioned in the thread he referenced, I needed to add
"protocol = smtps" to the transport. With this change, connection to
port 465 worked and negotiated TLS (as opposed to the the previous
behavior, an immediate disconnection). I specified port 465
explicitly on the smarthost, but I don't think that was necessary
since the log shows "Transport port=465 replaced by host-specific
port=465".

2. The account name in passwd.client needed to be in the form
domain\name, since the remote host is Windows.

3. After authenticating I got lots of "Client does not have
permissions to send as this sender" until I set both the envelope
sender and the From: header to match my email edress on the remote
host. I'm not sure if the From: header matters.