[Pkg-exim4-users] [Solved] Use a System Filter to AutoBCC/AutoSave Sent Msgs

groups, freeman

2014-05-17 23:59:14 UTC

********************************************
** AutosaveToMailBox_AllRemotelySentItems **
********************************************

The Exim docs said to use a system filter so I did, though it was a bit
of a challenge.

This email is to outline the config I used so that anyone else can have
a quicker path to bliss.

I'm using Debian v6.0, with Exim v4.72-6+squeeze3 in un-split configuration.

My goal is to have all (or actually, just some) emails that get SENT
from an account get auto-bcc'd (actually: automatically saved) into the
inbox of the email address that sent out the message in the first place.
(e.g. This is helpful to keep a copy of all outbound messages sent from
your cellphone, for your records).

First, about my setup.

All my email is retrieved via POP3. All my sent email goes out via Exim.
Same server. The mailbox is one file per user, located as /var/mail/username

I use Thunderbird on my desktop as the permanent, complete warehouse for
all emails.

However I have migrated two of my email addresses to a laptop PC also
running TB, and those addresses are primarily used from there nowadays.
(They get copied over to the desktop warehouse from time to time to keep
the warehouse up-to-date). However when I send an email from the
desktop, using one of the now-laptop-resident addresses, I'd still like
a copy to show up in the sent folder on the laptop.

Of note: In addition to those two ('upper') email addresses on my
Blackberry I also have two 'lower' addresses on it as well.

The Blackberry was the instigating factor for me getting into this
'auto-BCC' stuff: I migrated from the Blackberry_BIS-based email, to the
new Q10. The old BIS service had a built-in AutoBCC function so that any
messages sent from the Blackberry could be delivered to another email
address for eventual filing into the sent folder of the sender's TB
account. The new Q10 had no such automated function built in, so because
all my Blackberry-originated emails are routed through my personal Exim
server I could intercept them on their way out and place a copy directly
into the mail file of the sender, on my server, for later retrieval and
placement into the proper 'sent' folder.

Finally, I wanted to NOT have duplicate copies on my laptop or desktop.
When I send from that device I wanted Exim to be able to realize that a
copy would be already going into my sent folder (for certain addresses)
so in that case Exim DIDN'T need to place a copy into the sender's
mailbox file.

- Regarding the sample config below:
- Four email addresses are of interest to me, because that encompasses
all that are used outside of the Desktop PC
- I have two email addresses (upper1@ and upper2@) that are located on
the laptop AND on my Q10
- I have two other email addresses (lower1@ and lower2@) that are
located on my desktop AND my Q10
- my email addresses are in format of ***@applepie.mysite.com
(possibly ***@apple.mysite.com)
- my laptop's hostname is 'laptop'
- my desktop PC(s) are hostnamed MainPC5, MainPC6, PC5 and PC6
- I use auth on my SMTP - if you don't then comment out the
'sender_host_authenticated' check

So below is the config I used to get the results I wanted.

Though I solved my initial need some questions still remain, if there
are any takers...
- where in the docs can I read about 'mode_fail_narrower = false' and
'no_mode_fail_narrower' ... or what do they do?!
- I'd prefer to have the BCC'd addresses remain in the copy of the email
that gets placed into the sender's mailbox but Exim strips the BCC's.
However it appears that $recipients contains such addresses - if I had
more time I'd master the coding to parse them out and then use
'add_header bcc: ***@yyy.zzz' or similar since that would then perfectly
duplicate what happens when you send an email from your actual client,
and any BCC's addresses get included in the saved copy. Maybe one day.

===============================

Overview:
FWICT I needed to do two distinct steps:
A) activate a system filter
1) create a new file that is the filter
2) adjust /etc/exim4/exim4.conf.template to point to the new file as the
filter (and the proper transport, too)

B) create a new transport for the filter's use since no baked-in
transport does what we need

===============================

A1) Create New Filter File:
- filename is /etc/exim4/systemfilter.txt
- permissions for me are 0640
- contents are:
= = = = = = = = = = = = = = = = = = = = = = = =

# Exim filter <<== do not edit or remove this line!
##############################################################
if error_message then
finish
endif
logfile /var/log/exim4/system_filter.log 0664
logwrite "\n"
logwrite "[$tod_log] SystemFilter - ENTRY -
sender_address_local_part=$sender_address_local_part"
if not first_delivery then
logwrite " 1-first_delivery=false, so just finish"
finish
endif
if $sender_address_domain is not applepie.mysite.com then
logwrite " 2-domain is NOT applepie.mysite.com, so just finish"
finish
endif
if $sender_host_authenticated is not plain_saslauthd_server then
logwrite " 3-auth ($sender_host_authenticated) is not good
('plain_saslauthd_server'), so just finish"
finish
endif
if not foranyaddress $header_from: ($thisaddress matches
logwrite " 4-not a sought email sendername, so just finish"
finish
endif
#logwrite " host=$host"
#logwrite " host_address=$host_address"
#logwrite " received_ip_address=$received_ip_address"
#logwrite " received_port=$received_port"
#logwrite " self_hostname=$self_hostname"
#logwrite " sender_fullhost=$sender_fullhost"
#logwrite " sender_host_address=$sender_host_address"
#logwrite " sender_host_name=$sender_host_name"
# Certain email addresses have their actual sent
# folder stored on particular PC's, so...
# we don't need to Autosave a copy of what we sent since
# it will be saved there automatically by the email client
# (Thunderbird in my case). In fact it is preferable since
# it avoids duplication (or auto-deletion of duplicates).
#
# Sender_field host_name
# ===============================================================
# (upper1|upper2) laptop.apple(pie)?.mysite.com
# (lower1|lower2) (Main)?PC(5|6).apple(pie)?.mysite.com
#
#############################################################################
# Stored on laptop?
########################################
if ${lc:$sender_host_name} matches
"^laptop\\.apple(pie)?\\.mysite\\.com\\$" then
if foranyaddress $header_from: ($thisaddress matches
logwrite " 5-Sent from laptop client (TB) for upper1/upper2, so just
finish"
finish
endif
endif
# Stored on Main desktop?
########################################
if ${lc:$sender_host_name} matches
"^(Main)?PC(5|6)\\.apple(pie)?\\.mysite\\.com\\$" then
if foranyaddress $header_from: ($thisaddress matches
logwrite " 6-Sent from MainPC client (TB) for lower1/lower2, so just
finish"
finish
endif
endif
unseen save /var/mail/${lc:$sender_address_local_part} 660
logwrite " DONE - we copied the email to the sender's inbox"
finish

= = = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = = = = = =

A2) Activate New Filter:
- edit file /etc/exim4/exim4.conf.template
- at effectively the very beginning of the file add these 3 lines:
= = = = = = = = = = = = = = = = = = = = = = = =

system_filter=/etc/exim4/systemfilter.txt
system_filter_file_transport=AutosaveToMailBox_AllRemotelySentItems
#NotNeeded system_filter_user=Debian-exim

= = = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = = = = = =

B) Create New Transport:
- edit file /etc/exim4/exim4.conf.template
- right after the comment lines that are beneath the line saying:
begin transports
(so about line 1400 in the file, for me)
add in the following
= = = = = = = = = = = = = = = = = = = = = = = =

# For system filters,
# $local_part="system-filter"
# so it's of no use; instead, derive it from sender_address
########################################
debug_print = "T: {CUSTOM - 'AutosaveToMailBox_AllRemotelySentItems')
appendfile for ${sender_address}"
driver = appendfile
#NFG!# file = /var/mail/$local_part
file = /var/mail/${lc:$sender_address_local_part}
mode = 0660
user = ${lc:$sender_address_local_part}
group = mail
delivery_date_add
envelope_to_add
return_path_add
# We'll need to add a (custom!) filter in TB to move to the
# sent folder emails which have this header set to 'true'
##################################################################################
headers_add = X-AutosaveToMailBox_AllRemotelySentItems: true
# Envelope-to: system-filter
# since that's a hallmark that the email is
# an Autosaved copy of a remotely-sent msg
##########################################################################
#? mode_fail_narrower = false
#? no_mode_fail_narrower
# Note!...
##########################
# $recipients
# This variable contains a list of envelope recipients for a message.
A comma and a space separate
# the addresses in the replacement text. However, the variable is not
generally available, to prevent
# exposure of Bcc recipients in unprivileged users’ filter files. You
can use $recipients only in these
# (1) In a system filter file.
# ...

= = = = = = = = = = = = = = = = = = = = = = = =

And that has it. Have fun!

groups, freeman

2014-05-25 01:09:03 UTC

Permalink

Drat, I overlooked one niggling detail in my recipe.

The copies of the messages that are sent back to the original sender's
email address - that of course means that on my Blackberry I would
receive a msg every time I sent one out - definitely sub-optimal.

So I created new accounts: upper1_sent@, upper2_sent@, lower1_sent@,
upper2_sent@ and have the copies sent there.

I don't monitor those addresses on my Blackberry so the problem of
receiving copies that device just sent out is resolved.

On my laptop & main PC's I can add the _sent accounts to my monitored
email addresses, and then with filters just move items received there to
the appropriate sent folder of the originating email address.

Below is what I modified and it's now all dandy - I've been using it for
a week and a half now and am pleased.

Thanks all for open-source and consequently great products!

Changes in exim4.conf.template:
-- old line was:
file =
/var/mail/${lc:$sender_address_local_part}
-- new line is:
file =
/var/mail/${lc:$sender_address_local_part}_sent
AND
-- old line was:
user = ${lc:$sender_address_local_part}
-- new line is:
user =
${lc:$sender_address_local_part}_sent

Changes in systemfilter.txt:
-- old line was:
unseen save /var/mail/${lc:$sender_address_local_part} 660
-- new line is:
unseen save /var/mail/${lc:$sender_address_local_part}_sent 660

Have more fun!

===================================================================
===================================================================
===================================================================

Post by groups, freeman
********************************************
** AutosaveToMailBox_AllRemotelySentItems **
********************************************
The Exim docs said to use a system filter so I did, though it was a
bit of a challenge.
This email is to outline the config I used so that anyone else can
have a quicker path to bliss.
I'm using Debian v6.0, with Exim v4.72-6+squeeze3 in un-split
configuration.
My goal is to have all (or actually, just some) emails that get SENT
from an account get auto-bcc'd (actually: automatically saved) into
the inbox of the email address that sent out the message in the first
place. (e.g. This is helpful to keep a copy of all outbound messages
sent from your cellphone, for your records).
First, about my setup.
All my email is retrieved via POP3. All my sent email goes out via
Exim. Same server. The mailbox is one file per user, located as
/var/mail/username
I use Thunderbird on my desktop as the permanent, complete warehouse
for all emails.
However I have migrated two of my email addresses to a laptop PC also
running TB, and those addresses are primarily used from there
nowadays. (They get copied over to the desktop warehouse from time to
time to keep the warehouse up-to-date). However when I send an email
from the desktop, using one of the now-laptop-resident addresses, I'd
still like a copy to show up in the sent folder on the laptop.
Of note: In addition to those two ('upper') email addresses on my
Blackberry I also have two 'lower' addresses on it as well.
The Blackberry was the instigating factor for me getting into this
'auto-BCC' stuff: I migrated from the Blackberry_BIS-based email, to
the new Q10. The old BIS service had a built-in AutoBCC function so
that any messages sent from the Blackberry could be delivered to
another email address for eventual filing into the sent folder of the
sender's TB account. The new Q10 had no such automated function built
in, so because all my Blackberry-originated emails are routed through
my personal Exim server I could intercept them on their way out and
place a copy directly into the mail file of the sender, on my server,
for later retrieval and placement into the proper 'sent' folder.
Finally, I wanted to NOT have duplicate copies on my laptop or
desktop. When I send from that device I wanted Exim to be able to
realize that a copy would be already going into my sent folder (for
certain addresses) so in that case Exim DIDN'T need to place a copy
into the sender's mailbox file.
- Four email addresses are of interest to me, because that encompasses
all that are used outside of the Desktop PC
the laptop AND on my Q10
located on my desktop AND my Q10
- my laptop's hostname is 'laptop'
- my desktop PC(s) are hostnamed MainPC5, MainPC6, PC5 and PC6
- I use auth on my SMTP - if you don't then comment out the
'sender_host_authenticated' check
So below is the config I used to get the results I wanted.
Though I solved my initial need some questions still remain, if there
are any takers...
- where in the docs can I read about 'mode_fail_narrower = false' and
'no_mode_fail_narrower' ... or what do they do?!
- I'd prefer to have the BCC'd addresses remain in the copy of the
email that gets placed into the sender's mailbox but Exim strips the
BCC's. However it appears that $recipients contains such addresses -
if I had more time I'd master the coding to parse them out and then
perfectly duplicate what happens when you send an email from your
actual client, and any BCC's addresses get included in the saved copy.
Maybe one day.
===============================
A) activate a system filter
1) create a new file that is the filter
2) adjust /etc/exim4/exim4.conf.template to point to the new file as
the filter (and the proper transport, too)
B) create a new transport for the filter's use since no baked-in
transport does what we need
===============================
- filename is /etc/exim4/systemfilter.txt
- permissions for me are 0640
= = = = = = = = = = = = = = = = = = = = = = = =

= = = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = = = = = =
- edit file /etc/exim4/exim4.conf.template
= = = = = = = = = = = = = = = = = = = = = = = =

system_filter=/etc/exim4/systemfilter.txt
system_filter_file_transport=AutosaveToMailBox_AllRemotelySentItems
#NotNeeded system_filter_user=Debian-exim

= = = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = = = = = =
- edit file /etc/exim4/exim4.conf.template
begin transports
(so about line 1400 in the file, for me)
add in the following
= = = = = = = = = = = = = = = = = = = = = = = =

= = = = = = = = = = = = = = = = = = = = = = = =
And that has it. Have fun!
_______________________________________________
Pkg-exim4-users mailing list
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exim4-users