Pop3 download (gmail not involved) broken with latest snapshots

mib2berlin

@fuxs
Yes but no time to reply until now.
I have setup an POP3 vivaldi.net account and send mails from my other accounts.
Send with an other mail client or from a web interface, all mails fetched correctly.
To be honest, I have no idea why this doesn't work anymore after an update.
First idea would be reindex, it only repairs the database.
If your mails are save on the server you can even try to delete the account but not the mails.
You can choose this during the deleting process.
I can ping a mail developer but as usual it´s weekend now .

Cheers, mib

fuxs

@mib2berlin

Thanks for checking, mib.

To be honest, I have no idea why this doesn't work anymore after an update.

This seems quite strange, indeed. But it happens for two different accounts with the same symptoms

account verifies correctly, all settings are the same as before and fit
when checking for new msgs
-- account connects
-- shows number of msgs on server in status bar, also after new msgs have arrived
-- shows 'indexing finished' on status bar
-- does not fetch new msgs

So, obviously M3 does not mind the gap between number of already fetched msgs and new msgs on server to be fetched. This seems to be broken somehow and I guess resetting those download counters / IDs might fix the issue.
Can't find out, where they are stored, though. Not in MailDB it seems, but reindex must also reset this mechanism. According to the status bar notification former M3 builds seemed to compare downloaded msgs and server msgs somehow (by some ID perhaps) because they counted on to the number of msgs on server, which took quite a while on my main account. This is not the case anymore.

So, would be great if you could ping a mail dev, thanks. Would be bad if that happened to more users in 5.6 stable.

mib2berlin

@fuxs
You can make a bug report with all information and add a link to this thread.
I guess developers can resist to look at the tracker even at weekends.
The problem is, if I cant reproduce it a dev cant either and the first recommendation is always, try in a clean profile.

May I can ask if you use a third party anti virus software?
I had a situation with a PC and Avast were I cant add a new mail account getting strange error messages from Vivaldi but nothing from Avast.
It takes me a while to check this and after disabling Avast it work.
This happen always after an update of Vivaldi.

For information on how to report a bug, see this URL: https://vvld.in/how-to-report-bugs
Once that is done, please share the bug number (beginning with VB-). Thanks.
On the form, you can add your email address. Once submitted, you'll get a confirmation. You can reply to this with any logs or further info.

Cheers, mib

fuxs

@mib2berlin
VB-92562, first bug I've filed for vivaldi. 15 years (nearly straight) after starting as Postal-Elektran in the early days of M2, funny coincidence.

I use Nod32, had checked disabling that before, did not help.

fuxs

@mib2berlin
I wasn't really aware of the console log. It gives an reproduceable error when checking both of my pop3 accounts:

background-bundle.js:1 TypeError: Cannot destructure property 'searchListId' of 'e' as it is undefined.
    at $e (background-bundle.js:1:30791)
    at async Object.getFirstBatch (background-bundle.js:1:33608)
    at async t (background-bundle.js:1:34469)
    at async je (background-bundle.js:1:34371)
    at async Object.qe [as checkForMail] (background-bundle.js:1:35153)
    at async background-bundle.js:1:35666
(anonymous) @ background-bundle.js:1
t @ background-bundle.js:1

Does that help?

mib2berlin

@fuxs
The developer should get this.
Make a .txt file and attach it in reply of the bug confirmation mail and it will land in the bug report.
I can double check, mib

fuxs

@mib2berlin
Thanks mib, done.

mib2berlin

@fuxs
Mail and .txt has arrived.
I've seen this error before, cant remember where but a user saw messages arrives in the status bar but not show up in the mail window, iirc.

Cheers, mib

mib2berlin

@fuxs
Hi, there is a new snapshot with a bug fix for POP3 accounts.
It is not your report but maybe it´s relevant:

[Mail] Disable OAuth for Pop3 (Pop3 with OAuth won’t download email) (VB-91669)

Cheers, mib

fuxs

@mib2berlin
Hi mib,
Thanks for your hint. I've tried the snapshot - to no avail, pity.
Same error as before for each check of the pop3 accounts.

<edit>
I've added my vivaldi.net as pop3 account - vivaldi is happily fetching mail from there. So something specific on both 'old' accounts must be triggering this behavior.
</edit>

fuxs

@mib2berlin
Played around with that some more (on 2839.3). I tried a reindex with renaming “Default/Storage/ext/mpognobbkildjkofajifpdfhcoklimli/def/IndexedDB” (as advised in help) to check what would happen.

Results

all accounts visible in Mail settings
all offline accounts gone from mail panel
active pop3 accounts gone from mail panel
active IMAP accounts shown in mail panel, headers had been fetched again - they seem OK and even preserved custom labelin
no mail displayed from not shown accounts
RSS feeds gone from mail panel

So, this not only has killed Pop3 mail not being present on Server any more but the presentation of all Pop3 mails, plus offline IMAP, plus RSS. The msgs were not deleted from MailDB or Mail-folder, though.
I've tried several things before to identify the Pop3 switches to further play with but to no avail. Running out of ideas now, some additional hints would be great.

mib2berlin

@fuxs
Hi, I don´t have any ideas.
I delete my Gmail IMAP account, add POP3 Gmail account, I had to create an app password for this, and it fetch new mails immediately.
POP3 does not support AOuth in M3.
If you still have the second clean profile try it there or create a new one.
I guess you default profile is messed up anyway, mail wise.

Cheers, mib

fuxs

@mib2berlin
Lazy Sunday afternoon, took a deeper dive and debugged this issue (first time for me to do this for js in the browser).
I could identify the issue, there is an inconsistency in the local databases, most likely leveldb, but I can't see, which data store is involved. Mail tries to build up a msg array to check it against the server store it seems. On both broken accounts (at least) one msg is present in internal index somewhere but not in MailDB and another index. Fancy arrays being connected to each other, perhaps to level out locally deleted msgs to not download them again.
Both missing msgs are older. Adding a msg with the missing searchListID to MailDB does not change a thing, pity, I was hoping this could be the solution.
Fot one account M3 quits its building up of the 'compare'-array on position 678 of 776. There is only one undefined entry for this account. Most likely this would be no problem, if this error would simply be ignored and the bogus entry would have just been taken out of the server comparison array. Maybe there was a change in this area in 2817.3, this would make sense for those symptoms.

Adding a fake msg with the missing searchListID 43848 to MailDB does not change anything. The reference to this ID must be taken from somewhere else.
I'll also update the bug report with this information, hopefully the devs will further investigate, at least to check, whether there have been changes, which also might affect others.

Edit: "[Mail] Prefetching already imported bodies: further fix (VB-81283)" might be a suspect, mentioned in the 2817.3 changelog.

Popanz

I have the same problem here with one account on the latest snapshot. If i delete that account and set it up anew. I'm able to receive mails again. But then all old mails are gone. Reindexing leads to the same problem as described above. Accounts are there in settings but the Mail panel is empty and asks for setting up an email account.

fuxs

@Popanz
Thanks for adding here. Do you also see the same error msgs in console (Cannot destructure property 'searchlistid' of 'e'...)?

Also, thanks for confirming the broken reindex. I have not checked whether this is connected to the same issue, but might be worth an own bugreport.

Capushon

Описание

bug only GMAIL

Popanz

@fuxs
Yes I got the same error:

Meanwhile, I could fix my problem by overwriting the folder .../vivaldi-snapshot/Default/Storage/ext/mpognobbkildjkofajifpdfhcoklimli with an two-week-old folder from my backups.

BUT I still get the error "Cannot destructure..." so this shouldn't be the culprit.
By the way, the affected account is an ionos account - so probably the same technology as your 1und1.

Even so my mails are getting downloaded again, this problem makes me somewhat nervous, because there is no working database repair tool. Just deleting the folder which I replaced leads to an unusable Vivaldi Mail. I also tried to start with a clean profile and imported from a backuped profile path. That wasn't successful at all.
Even so the import tool told me success with importing bookmarks, passwords and history, there was nothing imported. As well, there was no option to import mails from this old profile. This option is shown when you try to import from old opera profile, but not when you want to import from a vivaldi instance.

fuxs

@Popanz
Not sure, wether the error would not be the culprit or at least be connected with the culprit.
I've also tried resetting the mpognobbkildjkofajifpdfhcoklimli - folder with prior versions older than the last received mails in both accounts.
Vivaldi indicated an update of the mail version in status bar after restart.

This process most likely does something unexpected with the index.

The result for both older versions of the folder contents was the same. One account started downloading msgs again, the other did not. I also still get the error, but only for the non-working account.
I do share your worries regarding the robustness of the mail backend. Some bOrked or orphaned index entry seems to render an account useless without the chance to keep | bring back | import old mail not present on the Pop3 Server as of now it seems. The same for sent mails. Even backups do not resolve the situation. Those would only work with an older version but you can't downgrade.

@mib2berlin FYI, I think this is worth being looked at.

mib2berlin

@fuxs
Hi, a developer asked 2 mail developers to take a look a few days ago.

Cheers, mib

gmg

@fuxs

Issue here is reproduction.
I set up a POP3 mail account with a vivaldi.net email address on a version prior to the v186 upgrade.
Then tried to run the upgrade and I didn't run into any issues. Even with pending mail still to come in while not having finished the sync first.

We'll try some more, otherwise we'll need to fine some way of making a blind fix for this...

Edit: Think we found a possible culprit. We'll look into it.