Solved Pop3 download (gmail not involved) broken with latest snapshots

fuxs

Sorry for posting a snapshot issue here, but I got no response in the corresponding snapshot threads and would like to check whether this is known and how to possibly troubleshoot the issue.

Since 2817.3 both of my two pop3 accounts will not fetch any new msgs. Those accounts have been working with the same settings before and are still with other clients. They are located on gmx,net and 1und1.de. Msgs are kept on the server until local deletion. Verifying accounts in the settings works fine. Download connects, posts a msg to status bar 'NNNN mails exist on POP account <account name>' and after that 'finished indexing - account name'. But no new msg fetched. The number of mails reflects new or deleted mail on that server, so M3 seems to receive correct counts from the remote box. I see no errors or anything helpful in the mail status.
But, there is a change: in older builds M3 went through the whole number of msgs (shown in status bar, took quite some seconds) and then downloaded the new ones.

IMAP and SMTP work as expected. This happens on Win11 Prof

So, anyone else seeing this?
Is there a way to influence or debug that less brute forced as with reindex? I seem to recall a setting in M2 containing the last downloaded mgs-identifier, one could reset this to a lower number I think - didn't find anything like that in Preferences or MailDB. I've read that ..\Storage\ext\mpognobbkildjkofajifpdfhcoklimli\def\IndexedDB - contents only contain IMAP stuff.

gmg

@fuxs

Fix is out in latest snapshot
https://vivaldi.com/blog/desktop/rocker-gestures-and-pop-issues-vivaldi-browser-snapshot-2867-19/

mib2berlin

@fuxs
Hi, I can remember one user report this for a Gmail POP3 account and there is a bug report I verified.
I can test this with my Yahoo account on the latest snapshot but I don´t get many mail on this account, only spam.
Will send some test mails from other accounts.

Cheers, mib
EDIT: Do you have the same error message on your 1und1 account?

fuxs

@mib2berlin
Hi mib,
Thanks for checking. No error msg here, neither from gmx nor 1und1. Both verify happily.

Cheers,
Carsten

mib2berlin

@fuxs
Hrm, setup my vivaldi.net account gives me the same error.
I guess this should work.
I will test on my other system on Windows 11.

Cheers, mib

fuxs

@mib2berlin
Hi mib, did you have the chance to play with that yet?

Still broken for me with 2824.8, just checked, this starts hurting...

mib2berlin

@fuxs
Yes but no time to reply until now.
I have setup an POP3 vivaldi.net account and send mails from my other accounts.
Send with an other mail client or from a web interface, all mails fetched correctly.
To be honest, I have no idea why this doesn't work anymore after an update.
First idea would be reindex, it only repairs the database.
If your mails are save on the server you can even try to delete the account but not the mails.
You can choose this during the deleting process.
I can ping a mail developer but as usual it´s weekend now .

Cheers, mib

fuxs

@mib2berlin

Thanks for checking, mib.

To be honest, I have no idea why this doesn't work anymore after an update.

This seems quite strange, indeed. But it happens for two different accounts with the same symptoms

account verifies correctly, all settings are the same as before and fit
when checking for new msgs
-- account connects
-- shows number of msgs on server in status bar, also after new msgs have arrived
-- shows 'indexing finished' on status bar
-- does not fetch new msgs

So, obviously M3 does not mind the gap between number of already fetched msgs and new msgs on server to be fetched. This seems to be broken somehow and I guess resetting those download counters / IDs might fix the issue.
Can't find out, where they are stored, though. Not in MailDB it seems, but reindex must also reset this mechanism. According to the status bar notification former M3 builds seemed to compare downloaded msgs and server msgs somehow (by some ID perhaps) because they counted on to the number of msgs on server, which took quite a while on my main account. This is not the case anymore.

So, would be great if you could ping a mail dev, thanks. Would be bad if that happened to more users in 5.6 stable.

mib2berlin

@fuxs
You can make a bug report with all information and add a link to this thread.
I guess developers can resist to look at the tracker even at weekends.
The problem is, if I cant reproduce it a dev cant either and the first recommendation is always, try in a clean profile.

May I can ask if you use a third party anti virus software?
I had a situation with a PC and Avast were I cant add a new mail account getting strange error messages from Vivaldi but nothing from Avast.
It takes me a while to check this and after disabling Avast it work.
This happen always after an update of Vivaldi.

For information on how to report a bug, see this URL: https://vvld.in/how-to-report-bugs
Once that is done, please share the bug number (beginning with VB-). Thanks.
On the form, you can add your email address. Once submitted, you'll get a confirmation. You can reply to this with any logs or further info.

Cheers, mib

fuxs

@mib2berlin
VB-92562, first bug I've filed for vivaldi. 15 years (nearly straight) after starting as Postal-Elektran in the early days of M2, funny coincidence.

I use Nod32, had checked disabling that before, did not help.

fuxs

@mib2berlin
I wasn't really aware of the console log. It gives an reproduceable error when checking both of my pop3 accounts:

background-bundle.js:1 TypeError: Cannot destructure property 'searchListId' of 'e' as it is undefined.
    at $e (background-bundle.js:1:30791)
    at async Object.getFirstBatch (background-bundle.js:1:33608)
    at async t (background-bundle.js:1:34469)
    at async je (background-bundle.js:1:34371)
    at async Object.qe [as checkForMail] (background-bundle.js:1:35153)
    at async background-bundle.js:1:35666
(anonymous) @ background-bundle.js:1
t @ background-bundle.js:1

Does that help?

mib2berlin

@fuxs
The developer should get this.
Make a .txt file and attach it in reply of the bug confirmation mail and it will land in the bug report.
I can double check, mib

fuxs

@mib2berlin
Thanks mib, done.

mib2berlin

@fuxs
Mail and .txt has arrived.
I've seen this error before, cant remember where but a user saw messages arrives in the status bar but not show up in the mail window, iirc.

Cheers, mib

mib2berlin

@fuxs
Hi, there is a new snapshot with a bug fix for POP3 accounts.
It is not your report but maybe it´s relevant:

[Mail] Disable OAuth for Pop3 (Pop3 with OAuth won’t download email) (VB-91669)

Cheers, mib

fuxs

@mib2berlin
Hi mib,
Thanks for your hint. I've tried the snapshot - to no avail, pity.
Same error as before for each check of the pop3 accounts.

<edit>
I've added my vivaldi.net as pop3 account - vivaldi is happily fetching mail from there. So something specific on both 'old' accounts must be triggering this behavior.
</edit>

fuxs

@mib2berlin
Played around with that some more (on 2839.3). I tried a reindex with renaming “Default/Storage/ext/mpognobbkildjkofajifpdfhcoklimli/def/IndexedDB” (as advised in help) to check what would happen.

Results

all accounts visible in Mail settings
all offline accounts gone from mail panel
active pop3 accounts gone from mail panel
active IMAP accounts shown in mail panel, headers had been fetched again - they seem OK and even preserved custom labelin
no mail displayed from not shown accounts
RSS feeds gone from mail panel

So, this not only has killed Pop3 mail not being present on Server any more but the presentation of all Pop3 mails, plus offline IMAP, plus RSS. The msgs were not deleted from MailDB or Mail-folder, though.
I've tried several things before to identify the Pop3 switches to further play with but to no avail. Running out of ideas now, some additional hints would be great.

mib2berlin

@fuxs
Hi, I don´t have any ideas.
I delete my Gmail IMAP account, add POP3 Gmail account, I had to create an app password for this, and it fetch new mails immediately.
POP3 does not support AOuth in M3.
If you still have the second clean profile try it there or create a new one.
I guess you default profile is messed up anyway, mail wise.

Cheers, mib

fuxs

@mib2berlin
Lazy Sunday afternoon, took a deeper dive and debugged this issue (first time for me to do this for js in the browser).
I could identify the issue, there is an inconsistency in the local databases, most likely leveldb, but I can't see, which data store is involved. Mail tries to build up a msg array to check it against the server store it seems. On both broken accounts (at least) one msg is present in internal index somewhere but not in MailDB and another index. Fancy arrays being connected to each other, perhaps to level out locally deleted msgs to not download them again.
Both missing msgs are older. Adding a msg with the missing searchListID to MailDB does not change a thing, pity, I was hoping this could be the solution.
Fot one account M3 quits its building up of the 'compare'-array on position 678 of 776. There is only one undefined entry for this account. Most likely this would be no problem, if this error would simply be ignored and the bogus entry would have just been taken out of the server comparison array. Maybe there was a change in this area in 2817.3, this would make sense for those symptoms.

Adding a fake msg with the missing searchListID 43848 to MailDB does not change anything. The reference to this ID must be taken from somewhere else.
I'll also update the bug report with this information, hopefully the devs will further investigate, at least to check, whether there have been changes, which also might affect others.

Edit: "[Mail] Prefetching already imported bodies: further fix (VB-81283)" might be a suspect, mentioned in the 2817.3 changelog.

Popanz

I have the same problem here with one account on the latest snapshot. If i delete that account and set it up anew. I'm able to receive mails again. But then all old mails are gone. Reindexing leads to the same problem as described above. Accounts are there in settings but the Mail panel is empty and asks for setting up an email account.

fuxs

@Popanz
Thanks for adding here. Do you also see the same error msgs in console (Cannot destructure property 'searchlistid' of 'e'...)?

Also, thanks for confirming the broken reindex. I have not checked whether this is connected to the same issue, but might be worth an own bugreport.