Spam filter gone crazy?

Hi

The last few days most of my emails have ended up in the junk folder. It took some time for me to discover. I thought it was a glitch with either my ISP or my email provider that made my emails not get through.

In most emails I find two spam related headers, both clearing the emails. Yet Pandors gives them a junk score and puts them in the junk folder. Yesterday all got the score 70. Today they get lower numbers and different for different emails.

I use mainly two email addresses, from the same provider. Some of the emails to those addresses aren't marked as spam. Test mails I've sent to myself from other email addresses aren't marked as spam. But most are.

I can't find anything about how Pandora does this. And above all, how do I get it to stop?

I should mention that I never got the spam filtering to work when I mark an email as junk. With the single exception for e Japanese spammer. So all I usually have in the junk folder is one or two of these emails in Japanese. Now most of the about 50 emails I get a day end up there and only the occasional one really is spam.

Many of the emails are sent in bulk, mailing lists and so on, if that can have to do with it. Today an email from the w3c was marked as junk. It contained a large number of spam headers, all clearing the email.

I recently updated Pandora. I don't remember how many days ago, but I don't think this started immediately after the update. I *think* everything worked normally for a few days, but I'm not sure and could very well be wrong.

Current Pandora version: 4.8.0.1

Hi croos
Take a look at Settings | General | Junk Mail. Your Junk Threshold might be set too low. Try raising it.
All the best. Davo

It's set to 80. I've never changed it. And the emails that end up in junk only was marked as 70 originally, now mostly below 50.

How does Pandora set its junk score? Obviously it doesn't go by the spam headers. Not any longer anyway.

Well, I'm still living with this. Marking all the presumed spam as not spam has had no effect, except in one single case. I'm getting used to using the Junk folder as my inbox. Most of my emails are from mailing lists or news letters and they end up in junk.

croos wrote:Hi

The last few days most of my emails have ended up in the junk folder. It took some time for me to discover. I thought it was a glitch with either my ISP or my email provider that made my emails not get through.

In most emails I find two spam related headers, both clearing the emails. Yet Pandors gives them a junk score and puts them in the junk folder. Yesterday all got the score 70. Today they get lower numbers and different for different emails.

I use mainly two email addresses, from the same provider. Some of the emails to those addresses aren't marked as spam. Test mails I've sent to myself from other email addresses aren't marked as spam. But most are.

I can't find anything about how Pandora does this. And above all, how do I get it to stop?

I should mention that I never got the spam filtering to work when I mark an email as junk. With the single exception for e Japanese spammer. So all I usually have in the junk folder is one or two of these emails in Japanese. Now most of the about 50 emails I get a day end up there and only the occasional one really is spam.

Many of the emails are sent in bulk, mailing lists and so on, if that can have to do with it. Today an email from the w3c was marked as junk. It contained a large number of spam headers, all clearing the email.

I recently updated Pandora. I don't remember how many days ago, but I don't think this started immediately after the update. I *think* everything worked normally for a few days, but I'm not sure and could very well be wrong.

Current Pandora version: 4.8.0.1

The Pandora version you are using is not the current release version.  

The current Pandora version is 5.2.0 but will require a Series 5 license upgrade if you have not already done so.  You can order the Series 5 License while using the Pandora you have installed.  When the email with new License is received, install the License, test that the new License works,  and then upgrade to the current Pandora version.

In Settings | General+Shift+Click is an option JunkAutoTransfer which has a default value of 1.  If you set that option to zero, I expect automatic transfer of junk mail will not occur.  I do not use Pandora Junk mail filtering, have the JunkAutoTransfer set to zero, and I never get mail transferred to the Junk Folder.

I hope this helps.

Butch

Wouldn't that rather make it move all mail to Junk?

But the problem isn't so much that they get moved, but it's rather that 99% of my incoming mail is marked as spam. As I said, almost all emails that are some kind of bulk mail get marked as spam. They are not. Most are posts from mailing lists, among them several w3c lists. They should have their headers in order, one would think. But also emails from some companies that I want to get. Private emails are left alone. That's what I'm trying to get help with, why does Pandora see these emails as spam all of a sudden and what can I do to change that?

Again, I hadn't touched any settings when this started to happen and me removing the spam mark doesn't make Pandora learn - more than in one case when it actually did. I had received these emails for a long time without any of them being marked as spam before this mess started and they are from many different servers to several of my email addresses. The only common denominator is that they are some kind of bulk mail.

No, I don't have the latest version. I want to get this sorted before I buy I new version. But I updated the old version, which didn't make a difference. I'm now on 4.9.0.1.

Thanks for trying to help. It's appreciated.

croos wrote:Wouldn't that rather make it move all mail to Junk?

But the problem isn't so much that they get moved, but it's rather that 99% of my incoming mail is marked as spam. As I said, almost all emails that are some kind of bulk mail get marked as spam. They are not. Most are posts from mailing lists, among them several w3c lists. They should have their headers in order, one would think. But also emails from some companies that I want to get. Private emails are left alone. That's what I'm trying to get help with, why does Pandora see these emails as spam all of a sudden and what can I do to change that?

Again, I hadn't touched any settings when this started to happen and me removing the spam mark doesn't make Pandora learn - more than in one case when it actually did. I had received these emails for a long time without any of them being marked as spam before this mess started and they are from many different servers to several of my email addresses. The only common denominator is that they are some kind of bulk mail.

No, I don't have the latest version. I want to get this sorted before I buy I new version. But I updated the old version, which didn't make a difference. I'm now on 4.9.0.1.

Thanks for trying to help. It's appreciated.

I suspect the Service that hosts your email has changed something to do with the junk score evaluation and Pandora is just responding to that change.  You need to know what the junk evaluation level  is before Pandora processes the message.  So If a web email interface is available, check the junk scores before Pandora downloads the messages.

Change the value of JunkAutoTransfer to zero and see what happens. You can always set it back; if you do not like the results.

The value set by JunkTreshold controls the evaluation level where Pandora takes action.  You could try raising it up to see what happens.

There are settings that control acceptance of Junk Scores set by AcceptJunkScoreBarracuda, AcceptJunkScoreSpamAssassin, AcceptJunkScoreSpamPal, and AcceptJunkScoreYahoo.  Look at the message headers to see if one of them is present.

I also suggest browsing through the Settings and the Extra Settings to understand all the options Pandora has.

Butch

[quote="butchkemper"]

I suspect the Service that hosts your email has changed something to do with the junk score evaluation and Pandora is just responding to that change.

[/quote]

I've seen examples in the past where spam scores (inserted into various headers inside an email by a mail provider's system before the mail is processed by your email client's filters) could either be a fraction between 0 and 1, so eg "0.80" or 100 times that (so between 0 and 100, eg 80) ... and some providers' system insert both numbers.

If something is reading a number from a header that used to show eg "80" and now it reads "0.80" it might interpret the latter as almost "0" meaning eg "very likely to be spam" rather than "80" meaning "almost certainly perfectly ok".

To find out if a change like that has occurred you'd need to look at the spam-scoring headers in emails you received a while ago (when perhaps the provider's scoring mechanism worked one way) and headers inside recent emails, to see if the format of those headers has changed.

I don't think that's likely, since it concerns several email addresses from different providers and it started all of a sudden for all of them. And if I check with webmail that aren't marked as spam. I don't see anything "spammy" in the headers either, but I'm no expert an email.

For instance the notifications from this forum get marked as spam. As you can see the provider (GMX) has cleared it for spam.

Note, alas I had already removed the spam mark before I thought of copying the headers, but that should only change what Pandora possibly had entered. I'll post headers from an untouched (by me) email when I get a new one. But I don't think it will look any different. If someone could kindly answer in this thread, type anything, I'll get a new one quickly.

I've obfuscated my email address, nothing else is touched. Except I bolded the bits I saw that mark it as NOT spam.


Authentication-Results: gmx.net; dkim=pass header.i=@forumotion.com
Content-Transfer-Encoding: quoted-printable
Content-Type: multipart/alternative; charset="utf-8"; boundary="----=_NEXT_PART_6474058b10930"
Date: Mon, 29 May 2023 03:53:15 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=forumotion.com;
s=default; t=1685325196;
bh=YGS87RzJLlr1oFpbV9uB3k2uRaT/DRD+ubxpI3A+nC8=;
h=To:Subject:From:Date:From;
b=ymlz2xeNInI5GavgyjO/is6xKqzByu1R4sFNTFnLO1oRlCw7IeeKChI1orXwZAP1L
bSz/sllsiu3cOEkT0XjvCT5quZyAIMNAnxg5aJj8uZWz2UANUX19ePdMxKSHTkpysD
zbZyOigquQXm4C5s3msKGfUnttGj5i3iB2mH0ITQ=
Envelope-To: <xxx@gmx.com>
From: "Pandora Mail Forum" <noreply@forumotion.com>
Message-ID: <20230529015315.123acb1ec6@pandoramail.forumotion.net>
MIME-Version: 1.0
Received: by f48.mail.maxns.net (Postfix, from userid 33)
id 14FEE21F53; Mon, 29 May 2023 03:53:15 +0200 (CEST)
Return-Path: <noreply@forumotion.com>
Subject: Topic Reply Notification - Spam filter gone crazy?
To: "croos" <xxx@gmx.com>
UI-InboundReport: notjunk:1;M01:P0:C6/9Cp+EOH4=;t86AahzZlm8fI9s0oB+mDB9WONY4
Z+5ifrhbP8s9oIj3pQcoauzz6mVPpv74LiYZje1RMGGzUoqy9NwSjoakjHp12MJQ+YgiTySeq
K3fWGRGt0oKhuhNOwyFpkWC7R6ruzsrAn5hqk9MtvAX2TnqfbJlgJAa2XFnjX5U+VdPj0PKpJ
qrBuQ069qRctGhv/x+eom7qpWjRb5Gx5W92yrAZNA7dd18IQdzN9UB9VhhH7gXi5bDdK7VTmM
6fdZXdW2e+o9VKQ8nUyT+scSj1Sn4pnV0RQwkqd4S80+OSHM2rzUQaVYfKo25Mw9Rpgq83kvp
Ce9Et88S+sQxpoqrNM/pNTr7g3Vtli/Naij+oqnlj22v6jiFPFTgXcb1Xk9a350ixIZJaVeha
S0F3CK06mEBmvW0ZnRsVFBO3NYlvbWZKrg+QqOMcf0Z8XG9gO2Xwsq/kwtzxNtTcjG2O3tFoE
OHWaY+SvGcNA+8kWwfSPX5hp9008fBijbNiU25PE9ruyveSrVx8sabz17Hfln4sK+9MCNPE/0
kVLUyP5o8fZf2SkViMSppFpsLzB4qzieOhZ9fQE/GI+pASsErfAZ8LUdDVmUJthS+aTklPs8m
2Dy4GlHXt8vVNc5kSuCHN7tO2A/CCTtkyAYo/w9aJGbssf1R5c/duUX4z8pSp+DnYELVgygCG
xLGGyHI+MFGZCgtJPyQcwfUZEqQF9HaCayzZnWyDN/zjsU3LmToQKrTE10QGgnbKubhFG90jK
bTQf7wNlhDK6P02nZo3WE0yMYV2s4/XBd7+d1usYfYKItPrp0NG7bmBk8L/va9G6LHuufj6nx
oeB2Q5CvwiHjLJkRGygZACQBCu/YX6EqkDyqdZYnBNpYU+7CtHyByZ7EAGqmPHx6TeZAr4ISC
ru5LpZhpf+5g2wEpJ41A123Qv3Ko5NsR7o5sHpEU9S4Ih1qOO9BMYxBdM6WNZMl/rwvzauNeV
m0gtxWRoDzmzzliJMu4ejaA/9IyUbASmp5cbHnH4GGrCfH02cxjw8kDr+QWwZ0ygOfmSdVmcO
Xt1BrT83qQTYNGCggUn7DJaeUQ7zazmDOipxjidTnCACuzZCpnSZZAF54xdOM5onAZPoFqAcz
8nA9GSDTQjEaOn9puL87cqR66HdMoxf5mdD5QUe+h98aKtzMrf/j96B55BP36u+Fl8bMAPXC9
iGYi/8fGuFEoioEDhh0MmvjV31sSZN/7OUjVJIo4yU+wqP4iItUZJIGCaJraUE/OnP0MXogBI
2hPzRiuB9v1tQHcUAZBvdzoqfEZgaPHihCuHiYtPLGNt7VUIQQ/u38d40NYFl3APpFM2C4r8z
woocgwVqDHZRMIpxSdwCLYLR+pZ3TxkiJQdVb1Wx2QgOp8F69Acboozr8LomLgtFyhZPJFZsn
058MSDZgDyHQvQdltjiOLaSK0vpaKpUqnn1jZFsf28LRCz630mZwek/GGCR1ey0pH0MYzXnX1
6VbZpuyReGLqry+hzrVd6IsIU0Ywc0KvR9xfY+gZbSE6CN2/0o0bcNferqmlVeZXgpweBwWVX
PYpn8OwED2VJjcE3vPk9MNnztcWOlDT8YP5RPHBvXASeGHP7V/FqwPFuxvo1QotMFqZnZYbq0
OrB8j8gdmgrlVhpFKG9qtwXbs9qmlBfII8d5DURW5I+T4+nAaqqKrTT/BEBFQH8+zdizwqsub
RXahlgilXpVCFsFMfpG20uU9M06lJfn6885mF20fujFh5+x4wc9E05Mu9RM3OZ9EJh8yho/eN
lKms+E5j1s8wgHiS9e7YFaw5B4snTI5F6g7a6z1fMvT5FvuDfb6JhYtO6A/GAwThjuM6oNtne
NQqpMooFunYKZLfyxD+6z3bv3K/v+B6E6IYHkSTG6CqDCLD67V2qK9XJFcEZm5sLa2lteR3rE
cyeOWvOlq8b4EmhcEkmSOcVV5Cvb42EJjiEAmlXhFx284VmV2aJSpH7dqbCleM+d8V/3z3jru
bemTwt7eAFODGnbJVJj4Xqa590HJ6fuUjH6VCPaxq2vPIFxl9vgIKsZsJTJQ3HoL7GXwdDVKI
xk2QlzUqB2lFZzFaAdbjkh3rFLHBn0dSHBqkd39AuT4b8bR5+trwG9VTXSs8LC0iASUW9oqKZ
B1LrSPIthi/+StcNPNmoh2XEgRf6gHiY7ZRMRSPMEXWInl3UmHKMlLlAjuDlvd+nuFE+fDndV
Pwf29C5wqpMN0zVaIJbDafzbKIoAUQSFr9uhgWXofRby217trixt1As0PfwKiCZYhPPAfIJv/
T1ifEKDVhss2765ghxsMWIWYJCulCHdGo9Z+4tc0SsPPr+JgYZvx54tyjuQbGApJ9mWlL8x5c
JMB7xC26E2+vwCRsNh4dRy6sTaxLnLCrBYy1ULBKujlf2J/I6xzV9aj4p8lhfil/Hhi4x3+ej
qkjQmviHFvX1boeSOXQnxg5akEs2mmKmzM5wta7KEbiYXL9pYmUnEFTVTQeZqwx3LQp8TER+a
+F0k76uqzEv17kGjDJ4BW76rDZpi7a1EilOyg7qA8ZOU5VIn6tmjyvL20L/DufYTX68PHhg4O
nSvNCeFvhdEOngjYMlIyd5n7d49nX/n7yFja9wGUxepLjvMPVdycaDs0Jufby53Bb6h/ll3PD
D9nfPk/IMxxyPGhAWpzaBNicJJXOilJ24UhGMB0X0nuQmgQ4rh9i6L8ihQVUAKIGoN1jscRDQ
7wWxlX9PYWmp9UFGYqd2zWxxs6jPiVq5yFX6AC7k6Kk2OW43NoHQEr756wErnHmZt3etSGfRn
5UsFlK3ve7HMneEJ5VsFxfrIFjYlJSgStMOBOeR6lss49XMWGjXvHlJFqF8g+cNB6BEtMyGRY
t0I1dPWyP9Yozf73y13LisGQCiE/qqRlh7WvZ0ASUYc75mDzpJLdIpLquh0C1El8FQ0bNeA0Q
tJJO1DeZy7knsEDlcbCYZsCr8f1u3zT/LW6uLew1Wp16Z15of6owmjnaMGYWa659fNyB0ZFpj
eKgelas9iSEAQyhmlMGWxc1p/NiQWr41QLruxUTObCZi1RJKcI+kXCA25zG07mkuFjITeutSq
qrbI2pc1iQ/KP6qel5UwWGS3l4lfq6lzdVte2gENqa/vCvjTbblVLIzQ7POYZpfYfxPIZe60p
V90ydemvBBUOwA295P2I7VJ7jC+YVH+IfbX+119Wg25XZzuI9mPHE7LG0atDh/AeYzfVUu59y
WWlgUpUwyWvq8FOsy3N7lg2i/b/Ogmo05Wf+U4UUYnT5gWOgxSVXbto7XQ8ke3JjypyoBIGQq
XfPn2kSrBJB6a0G9Qn0hKihL898OyhlzjYQI1lxEYTURDuj8RyOAFudPs5HTNF2wBztGphsDf
qJtvREtaBugoEA+Bpq2Accr86+YeQAWgWfyFwzTGoehStZW+X+30+WLFFmRG7j+QY0yyLpz81
0c2u1AqcSmEJKVqu9uBb8g69GnYZK8a4X56EBpfJnyWfeK1RQM49SPx13AKTJtptfNiX96yzb
2MWodVhTGVLw1fszQnb5ahc9uah6kkhdnDUlsboIouKJDT8NNiseQjP3vgX+7mH8YfaARUQLV
ldrrkY9O1NebDEuvP3ay68Kj8H8YeaM9ShYinNvRh+vD3cXMtJx2dxW28TM5WXDDNeyKVOxk/
Sd1pH+7aJSwbBnZrsYWNCO36sjwupmFVQe1H7BaHTdAkunG7+Yr5JQiUYWCkPcnk7bO2zSPZp
Q1q/qUl0xZvR1PkqZhokKeQpnRiyKbNRuhgXkHgjjXJJemGzOgQRINi595UXGbFoTJDvMnwCM
D2P+ws7Cq3JAPbjq6JcuHGpZW9b0YH9el+BmFH0ofGyqFlu1lNXItaU/YilQrHTy2Y0LMac7d
SAheAeJADdzGN/WUTVprxaeyQR6EB+/TrRxRc+oRbAfPy75jQoysQREDuWt2kjw71egWk9GZ2
eRKi/vY8EWXNghrByH7QVwBcLQca8JrMixWT9onW2m7Cu/P+z/RALO9mR2xeHvxpt/UVOOlfz
P9u5bAY5pvcliWI2FJrPYaS4XTFdSTAgW73ErWk5gKdUh+YGIa1+S+VJXp1923yprFRP1DvGW
jnYRmuO9LxpUDX0TT8ViWFoSYwZWkm9rcUObnJhxpwPwy0w0hTNLDfbehXXe21iD8D6sjAPFa
MEe8iAWMiNRpAO7gCYM4oKsvJWQdKog8NBqzuIfvH9PfGW9dbt5p4FuJrNg28jzuuuMhkn4aR
lxKuG2BgUA/Z3LCKezWXESn/90o5jMTJR5eHPKtFrWiyBy+p9t220y7CMcHxLb4G1K1R79kWW
e5IAHIrgkp6cBSq5ZCt0nngz3ZS1DHGBACX+m5SXZoaeQfLB+Q2wLBY3kAp6dobqz3MD3OkMD
6yPtWXcuNoGXWFhBQhhMyTCf5oVqKiC0JN1dTN1XRQGqe9rloTtH+HGhz3F5ZrmDYFxkOG8kV
HhsYwbVk82WGEli2EMDNSJRPqZ5Tur9gg0Y0YBN0EjeM2sXAHuHFWE613XePex5+VOxiCaqG/
eZ4Sb285wUg3Pl42St/K93fi6uVi/ftc8jhcizW4++r9P4NQ/AFl7SV4DzqNdl4EgNndKuE/F
SyRl5H99r5cCVVG3EwXOyLo7U7iqZU7DmMASBYYAHz2MzlQcWp4KMw390jabmJSkTD7xkeldp
TK5ddeveHzcST1vjSlr4pt/m5ijNSV21GKT1

X-GMX-Antispam: 0 (Mail was not recognized as spam); Detail=V3;

X-Pandora-Index-Message-Type: 0
X-Persona: <Dominant>
X-Spam-Flag: NO
X-UIDL: 1MTAib-1pf5T11ouV-00UhsP

That's interesting - a weird collection of headers with many missing ones, then a huge wodge of
cryptic data just after that: "UI-InboundReport: notjunk" header. It's hardly user-friendly, is it?

The sort of thing I was describing is demonstrated by this set of headers from an email I
received a couple of months ago. As usual headers prefaced by "X-" are non-standard, in that
one email provider's use of a set of X-xxx headers is not required to have the same sort of
contents as any other provider's use of same-named headers.

X-Spam-Score: 2.9
X-Spam-Score-Int: 29
X-Spam-Bar: ++
X-Spam-Report: Spam detection software, running on the system "mail.xxxxxxxxxx.net.uk", has
processed this message and it scored (2.9 points).
pts rule name description
---- ---------------------- --------------------------------------------------
-0.4 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)
[195.140.184.244 listed in wl.mailspike.net]
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5004]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 FROM_NOT_REPLYTO From does not match Reply-To
0.0 HTML_IMAGE_RATIO_04 BODY: HTML has a low ratio of text to image
area
0.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
0.1 HTML_MESSAGE BODY: HTML included in message
1.5 MKE_HIDDEN1 RAW: Contains CSS-hidden text
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
valid
-0.1 DKIM_VERIFIED No description available.
0.0 RCVD_NOT_IN_IPREPDNS Sender not listed at
http://www.chaosreigns.com/iprep/
0.0 RCVD_IN_MSPIKE_WL Mailspike good senders
X-AA-Info: Message ran through Aliases
X-AA-Info: Message ran through Aliases
X-Spam-Mark-Threshold: 10
X-Spam-Reject-Threshold: 50
X-Spam-User: my.xxxxxxxxxxxxxxxxxxxx.org.uk
X-Spam-Flag: NO

and in particular see the three lines

X-Spam-Score: 2.9
X-Spam-Score-Int: 29
X-Spam-Bar: ++

where the score is presented as both a rational number and an integer. And there's even the spam bar so that
an email client that cannot work with a numeric score can see it got "++" rather than say "+" or "+++++" so could
for example filter likely-spam emails based on X-Spam-Bar containing "++++++" (ie having a spam score of 60
or more).

The benefit of a set of X-Spam-xxxxx headers like this being in an email is that one can see which aspects of the
spam-check contributed most to the score, and also which aspects reduced the score.

So, in that example, the things that looked spammiest in that email were the result of a Bayes test on the
contents, and - worst - the fact that CSS in the HTML part of the message body was set up to hide some
parts of the text.

The two headers

X-Spam-Mark-Threshold: 10
X-Spam-Reject-Threshold: 50

tell me (if I'd forgotten) that I've configured that provider's system so that a mail scoring more than 10 in
their system will be marked (ie have "[SPAM} " placed in its Subject, and a mail would need to score more
than 50 to be rejected. Those 10 and 50 scores are compared with the 2.9 that the sample mail scored
so it was neither marked nr rejected.