[Coco] yahoo groups
John E. Malmberg
wb8tyw at qsl.net
Sat Apr 21 14:20:12 EDT 2007
Kevin Diggs wrote:
> John E. Malmberg wrote:
>>
>> Your mail server should be writing a line in the headers that shows
>> the host name that the sending mail server claimed to be (can be
>> forged), and in parentheses, the rDNS name and the I.P. address of the
>> sender which can not be forged.
>>
>> Every other line in the message header is suspect and can be forged.
>>
> This is the last received from entry (farthest from your server), right?
> Sometimes I see the phrase "may be forged" in there somewhere. Where
> does that come from?
That line comes from your mail server. It is detecting something wrong
with either the rDNS or the "hello" message from the sending mail
server. If you are seeing it on legitmate e-mail, you should notify the
sender that there is a defect in their mail server configuration.
It will probably be rare that you see this, if ever, on legitimate
e-mail. Of course you are more likely to be looking at the headers of
spam than or real e-mail.
This type of defect on the sender side can cause post processing spam
filters to silently discard messages from them.
A spam filter that is integrated with the mail server will cause
detected spam to be rejected with a 550 code and a diagnostic. So if
this catches a legitimate e-mail, the sender should get a non-delivery
message. This is the only "reliable" method of doing spam filtering so
sender gets a notification of the problem. In the bulk of the cases I
have seen investigated, the rejection a legitimate e-mail has been a
result of a security breach on the sender side. The next most common
case is a mis-configuration of the receiving mail server. Very rarely
is it from an incorrect listing in a blocking list.
As some mail servers and mail programs are incorrectly set up to
silently delete non-delivery messages, these users may not be aware that
some of their e-mail is not getting through. So there is no totally
reliable method to indicate non-delivery because of broken mail servers
and e-mail programs.
And the bulk of commercial spam filters and all "HOME" spam filters
effectively silently delete what they detect as spam. Client side spam
filters have only to options, tag/sort, or silently delete.
> Could you possibly show an example of what can and cannot be forged?
At-signs are replaced by (at) to prevent gmane munger from messing
things up more. Letter x replaces some information that I do not want
displayed. Header lines prefixed by "> " to make them stand out. Lines
that start with "> " are actually continuations.
Unless stated otherwise, assume that a header line is forgeable.
> Return-path: <xxxxxx(at)yahoo.com>
Will usually be either <> for a NDR message or a message from a <> robot
mailer. Otherwise will have an alleged e-mail address of the sender to
receive an NDR.
Do not filter out <>, especially on postmaster or abuse addresses,
because that will cause you to miss legitimate non-delivery messages.
Since mail servers should use reject codes instead of bounces, this
header is mostly worthless for spam filtering.
> Received: from CONVERSION-DAEMON.Encompasserve.org by Encompasserve.org
> (PMDF V6.2-X27 #31448) id <01MFOIU3X2Q800D0RR<at>Encompasserve.org> for
> xxxxxxxx<at>Encompasserve.org (ORCPT xxxxxxx<at>encompasserve.org); Sat,
> 21 Apr 2007 10:50:51 -0500 (CDT)
This was generated by my mail server, so I know that I can trust it.
> Received: from n6.bullet.re3.yahoo.com
This is a line that can mostly be trusted. The first part of this line,
the name after the "from" can not be trusted. It is the name that the
mail server claims to be. Most legitimate mail servers give their
public DNS names. A few do not. Apparently there is no RFC that
requires this. I am aware of a few postmasters that refuse mail from
"lying" mail servers.
A known spammer trick is to put your mail server I.P. address here in
order to fool poorly written spam filters. This actually makes such
spam easier to reject, except that tests of some commercial spam
filtering products show that they do not know how to do this.
This test can be done before the body of the message is accepted into
the mail server, so doing it saves bandwidth and capacity costs for the
mail operator.
The rest of the line continues below and can be trusted.
> ("port 35629"(at)n6.bullet.re3.yahoo.com [68.142.237.91])
> by Encompasserve.org (PMDF V6.2-X27 #31448)
> with SMTP id <01MFOITZYZIQ00DFTT<at>Encompasserve.org> for
> xxxxxxxx(at)Encompasserve.org (ORCPT xxxxxxxx(at)encompasserve.org);
> Sat, 21 Apr 2007 10:50:50 -0500 (CDT)
Now if you lookup the I.P. address for the hostname for the yahoo mail
server, you will find that it matches the I.P. address. That is the
case for the majority of legitimate e-mail.
If it does not match then it is about a 90% percent indicator that you
have a spam message. If the hostname is missing then it is over a
99.99% chance that it is spam.
The I.P. address can then be looked up in real time blackhole lists.
Most commercial mail servers know how to do this check.
On this header you can also white list or black list based on the rDNS name.
If this I.P. address (with the exception of web mailers) and rDNS match,
and the I.P. address is not in any conservative blocking list, then it
is probably not spam. Content filtering after passing this step may
find real viruses, but is more likely to produce a false positive than
to detect additional spam.
> Received: from [68.142.230.29] by n6.bullet.re3.yahoo.com with NNFMP; Sat,
> 21 Apr 2007 15:50:15 +0000
Not trusted, but the by matches the trusted line, and so you can lookup
the rDNS of the receiver which matches the next line.
Spammers routinely put fake lines here.
The main reason for verifying at this point is if you have a mail
forwarder or mailing list, and you want to sort the spam from it.
However rejecting mail from a mailing list or a forwarder can cause it
to unsubscribe you. So you have to either live with the spam, or
collect information to help that mail service improve their spam filtering.
> Received: from [66.196.101.131] by t2.bullet.re2.yahoo.com with NNFMP; Sat,
> 21 Apr 2007 15:50:15 +0000
The same as before.
> Received: from [127.0.0.1] by rrr2.mail.re1.yahoo.com with NNFMP; Sat,
> 21 Apr 2007 15:50:15 +0000
This is an internal hand off through localhost.
> Received: (qmail 14785 invoked by uid 60001); Sat, 21 Apr 2007 15:50:15 +0000
Useless for spam filtering.
> Received: from [72.254.20.168] by web63806.mail.re1.yahoo.com via HTTP; Sat,
> 21 Apr 2007 08:50:15 -0700 (PDT)
The via HTTP is significant. Web mailers tend to get infested with
various spammers, particularly 419 scams. Via DAV is also a way of
connecting to a web mailer.
Usually these messages are sent through infected computers, but
sometimes from internet cafes.
Generally the true source I.P. has usually sent enough spam that it is
listed somewhere.
This I.P. address can be looked up against the real time lists. It is
normal for it to show up in the DHCP lists.
> Date: Sat, 21 Apr 2007 08:50:15 -0700 (PDT)
If missing on the sender, it may be added by your mail server. Can not
be trusted.
> From: John Malmberg <xxxxxxx(at)yahoo.com>
> Subject: test
> To: xxxxxx(at)Encompasserve.org
> Message-id: <392417.14337.qm<at>web63806.mail.re1.yahoo.com>
> Message-id: <571367.55337.bm<at>rrr2.mail.re1.yahoo.com>
The message-id: are used by news-readers and mail clients to group
messages. They are preserved on "replies". Which is why "reply" should
not be used to start a new thread.
> MIME-version: 1.0
> Content-type: multipart/alternative;
> boundary="Boundary_(ID_0tMa0DK9gUdjdomtC0brBQ)"
Yahoo sent the message in HTML. I have since found out how to fix that.
> DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com;
> h=X-YMail-OSG:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID;
> b=ajhrwwov8AEbzxORxMn7Sohgs3ge7HJytAnICj1WwO0cAf8mmfpRqOkAfWhFyLNcsbLbNkjBeqcG5c1tbdUOb517oRY297WwPQ1ahxnBEQDv7FT/bqnA1k1P0q+p0ukMkYtdVWaQXbMvDwjZG43n0rxkOovsCkiew0/8K2OVVRo=;
This is some type of key that can be looked up to verify that the e-mail
actually came from a yahoo.com server.
Note that the simple rDNS check performed by your mail server already
did an equivalent check with a lot less resources used.
> X-YMail-OSG:
> rs652kYVM1mnjZXj_MkGTVUUn.OJuc4L7ydwtCJoB_DRsszE_915.VvA_Q8Tue7EQ2X.fXZ.CT77U_r7XTE65Y42qHogTG0MyVjb7XpcnVeO.wgzF985o3sruFNkIm9h
> X-Yahoo-Newman-Id: 571367.55337.bm at rrr2.mail.re1.yahoo.com
> X-Yahoo-Newman-Property: ymail-3
I have no idea what these are. For all I know, they can be decrypted to
reveal the information I have given to Yahoo.
> Original-recipient: rfc822;xxxxxxx(at)encompasserve.org
This appears to have been added by my mail server.
No X-originating headers were found in this message, but I could find
the I.P. address that submitted to yahoo.
Now here is an extract from some spam that got through to me.
> Received: from mail.pitel.net (mail.pitel.net [76.164.128.7])
> by mail.qsl.net (Postfix) with ESMTP id 47D473F46C2 for <wb8tyw at qsl.net>; Fri,
> 20 Apr 2007 13:57:14 -0400 (EDT)
This is a real mail server that is currently listed in three aggressive
anti-spam lists. I do not know if this got through to me before the
listings.
It looks like it is an open relay that has not yet been detected, or
this is a multi-hop exploit.
http://www.moensted.dk/spam can be used to help diagnose things by
searching the various anti-spam databases.
Be aware that some of them are slowly listing the entire internet, so do
not panic if you find your own I.P. address in some.
> Received: from localhost (mail.indianheadtel.net [76.164.128.9])
> by mail.pitel.net (Merak 8.9.1) with SMTP id ZVZ23414; Fri,
> 20 Apr 2007 12:57:14 -0500
No listings found.
> Date: Fri, 20 Apr 2007 12:57:14 -0500
> From: Wilson Ron <w_r2<at>indianheadtel.net>
> Subject: From Mr.Wilson Ron
> X-Originating-IP: 196.1.190.6
Untrusted, but listed in both spamhaus and spamcop.
http://www.spamhaus.org/SBL/sbl.lasso?query=SBL45528
No rDNS present for this I.P. address. It looks like the above mail
server is now effectively an open relay, but has not yet made it into
the major open relay lists.
So treating X-Originating-IP the same as a VIA HTTP received line can
help detect additional spam, as long as you make sure the check is done
in the header portion and not the message body. Otherwise this message
would be flagged as spam.
Now back to Gene's original issue: Here is a sample from a message sent
to gmane from the Yahoo groups list.
> X-Mailer: Yahoo Groups Message Poster
> X-Originating-IP: 66.94.237.25
> X-eGroups-Msg-Info: 1:6:0:0
> X-Yahoo-Post-IP: 24.222.98.155
It appears that if the message comes from Yahoo Groups, the message
header X-Yahoo-Post-IP shows it's origin.
In this case it is a DHCP address belonging to eastlink.ca and is not on
any major anti-spam list.
The yahoo server is on the UCEPROTECT list, which is apparently gaining
usage, especially in Europe, even though it appears to be an aggressive
list.
Based on previous discussions in the spamcop.net forums, Yahoo appears
to let the spam outflow from their servers build up to some point where
people start showing up complaining on the public forums about Yahoo
and/or groups being blocked. About a week after the complaints begin,
apparently Yahoo cleans out the spammers, and the blocks expire.
Gene, it would be interesting if the X-Yahoo-Post-IP is present in the
spam that you are getting, what happens if you put it in the box at
http://www.moensted.dk/spam.
I suspect that in most cases you will find it in the SBL,SBL-XBL,XBL,
DSBLLIST, NJABL, SPAMCOP zone. You will probably never see that with a
legitimate message from Yahoo Groups.
The other check that has shown to be reliable is look up the URLs in a
message that you suspect could be spam against the above zones.
SpamAssasin 4.0 or later can do this. Spamhaus.org recommends a
different setting than the default.
http://www.spamhaus.org/effective_filtering.html
Unlike spamhaus, I do not recommend doing content filtering unless there
is something suspicious in the header. Otherwise you will filter out
legitimate e-mail discussing spam filtering.
And of course content filtering except to de-fang viruses should never
be done on the ABUSE or POSTMASTER addresses, because that content is
expected to be in abuse reports.
But as spamhaus.org statistics show, over 75 percent of spam can be
reliably detected with out the body of the spam ever entering your mail
server.
For further discussions about spam filtering, I would recommend using
the news.spamcop.net private news servers and the spamcop newsgroup.
-John
wb8tyw(at)qsl.net
Personal Opinion Only
More information about the Coco
mailing list