Solved Pop3 download (gmail not involved) broken with latest snapshots

mib2berlin

@fuxs
Hi, there is a new snapshot with a bug fix for POP3 accounts.
It is not your report but maybe it´s relevant:

[Mail] Disable OAuth for Pop3 (Pop3 with OAuth won’t download email) (VB-91669)

Cheers, mib

fuxs

@mib2berlin
Hi mib,
Thanks for your hint. I've tried the snapshot - to no avail, pity.
Same error as before for each check of the pop3 accounts.

<edit>
I've added my vivaldi.net as pop3 account - vivaldi is happily fetching mail from there. So something specific on both 'old' accounts must be triggering this behavior.
</edit>

fuxs

@mib2berlin
Played around with that some more (on 2839.3). I tried a reindex with renaming “Default/Storage/ext/mpognobbkildjkofajifpdfhcoklimli/def/IndexedDB” (as advised in help) to check what would happen.

Results

all accounts visible in Mail settings
all offline accounts gone from mail panel
active pop3 accounts gone from mail panel
active IMAP accounts shown in mail panel, headers had been fetched again - they seem OK and even preserved custom labelin
no mail displayed from not shown accounts
RSS feeds gone from mail panel

So, this not only has killed Pop3 mail not being present on Server any more but the presentation of all Pop3 mails, plus offline IMAP, plus RSS. The msgs were not deleted from MailDB or Mail-folder, though.
I've tried several things before to identify the Pop3 switches to further play with but to no avail. Running out of ideas now, some additional hints would be great.

mib2berlin

@fuxs
Hi, I don´t have any ideas.
I delete my Gmail IMAP account, add POP3 Gmail account, I had to create an app password for this, and it fetch new mails immediately.
POP3 does not support AOuth in M3.
If you still have the second clean profile try it there or create a new one.
I guess you default profile is messed up anyway, mail wise.

Cheers, mib

fuxs

@mib2berlin
Lazy Sunday afternoon, took a deeper dive and debugged this issue (first time for me to do this for js in the browser).
I could identify the issue, there is an inconsistency in the local databases, most likely leveldb, but I can't see, which data store is involved. Mail tries to build up a msg array to check it against the server store it seems. On both broken accounts (at least) one msg is present in internal index somewhere but not in MailDB and another index. Fancy arrays being connected to each other, perhaps to level out locally deleted msgs to not download them again.
Both missing msgs are older. Adding a msg with the missing searchListID to MailDB does not change a thing, pity, I was hoping this could be the solution.
Fot one account M3 quits its building up of the 'compare'-array on position 678 of 776. There is only one undefined entry for this account. Most likely this would be no problem, if this error would simply be ignored and the bogus entry would have just been taken out of the server comparison array. Maybe there was a change in this area in 2817.3, this would make sense for those symptoms.

Adding a fake msg with the missing searchListID 43848 to MailDB does not change anything. The reference to this ID must be taken from somewhere else.
I'll also update the bug report with this information, hopefully the devs will further investigate, at least to check, whether there have been changes, which also might affect others.

Edit: "[Mail] Prefetching already imported bodies: further fix (VB-81283)" might be a suspect, mentioned in the 2817.3 changelog.

Popanz

I have the same problem here with one account on the latest snapshot. If i delete that account and set it up anew. I'm able to receive mails again. But then all old mails are gone. Reindexing leads to the same problem as described above. Accounts are there in settings but the Mail panel is empty and asks for setting up an email account.

fuxs

@Popanz
Thanks for adding here. Do you also see the same error msgs in console (Cannot destructure property 'searchlistid' of 'e'...)?

Also, thanks for confirming the broken reindex. I have not checked whether this is connected to the same issue, but might be worth an own bugreport.

Capushon

Описание

bug only GMAIL

Popanz

@fuxs
Yes I got the same error:

Meanwhile, I could fix my problem by overwriting the folder .../vivaldi-snapshot/Default/Storage/ext/mpognobbkildjkofajifpdfhcoklimli with an two-week-old folder from my backups.

BUT I still get the error "Cannot destructure..." so this shouldn't be the culprit.
By the way, the affected account is an ionos account - so probably the same technology as your 1und1.

Even so my mails are getting downloaded again, this problem makes me somewhat nervous, because there is no working database repair tool. Just deleting the folder which I replaced leads to an unusable Vivaldi Mail. I also tried to start with a clean profile and imported from a backuped profile path. That wasn't successful at all.
Even so the import tool told me success with importing bookmarks, passwords and history, there was nothing imported. As well, there was no option to import mails from this old profile. This option is shown when you try to import from old opera profile, but not when you want to import from a vivaldi instance.

fuxs

@Popanz
Not sure, wether the error would not be the culprit or at least be connected with the culprit.
I've also tried resetting the mpognobbkildjkofajifpdfhcoklimli - folder with prior versions older than the last received mails in both accounts.
Vivaldi indicated an update of the mail version in status bar after restart.

This process most likely does something unexpected with the index.

The result for both older versions of the folder contents was the same. One account started downloading msgs again, the other did not. I also still get the error, but only for the non-working account.
I do share your worries regarding the robustness of the mail backend. Some bOrked or orphaned index entry seems to render an account useless without the chance to keep | bring back | import old mail not present on the Pop3 Server as of now it seems. The same for sent mails. Even backups do not resolve the situation. Those would only work with an older version but you can't downgrade.

@mib2berlin FYI, I think this is worth being looked at.

mib2berlin

@fuxs
Hi, a developer asked 2 mail developers to take a look a few days ago.

Cheers, mib

gmg

@fuxs

Issue here is reproduction.
I set up a POP3 mail account with a vivaldi.net email address on a version prior to the v186 upgrade.
Then tried to run the upgrade and I didn't run into any issues. Even with pending mail still to come in while not having finished the sync first.

We'll try some more, otherwise we'll need to fine some way of making a blind fix for this...

Edit: Think we found a possible culprit. We'll look into it.

gudmundurg74

@fuxs Did you see any errors in the log during the upgrade?

fuxs

@gmg @gudmundurg74

Thanks guys for investigating. Glad you found a possible culprit, nevertheless I'll answer your questions as well.
During the upgrade no errors came to view and to the first-hand log (Mail icon in status bar). In console I came across the 'Cannot destructure property' error for the account but nothing else. So, no, no error I think but I could try again.

I'd guess that this cannot be easily reproduced. Both orphaned index entries which cannot be 'destructured' are older ones as of their SearchListIds. Might have been an unsuccessful delete or housekeeping, as they are present somewhere in the index but not in MailDB.
From the outside it looks like Mail quits building up the 'final array' of msg-entries which is checked against the Pop3 server but happily steps in again afterwards, showing the 'finished indexing for' msg in status bar. So to handle the symptom it might be a way to catch the error and just kill this item from the array.

From a deeper and and more cause based perspective it would be great if such glitches in the index could be found und healed by some housekeeping job or a dedicated function to call for the user. I have no idea, how this inconsistency was produced - perhaps also a crash and this might happen inbetween? If so, I'd much more like to heal the inconsistency than to reset the account and loose all msgs not present on the server (incl. sent). And, if a msg is not present in MailDB anymore, it should also not be in the index - why not clean that up.

gudmundurg74

@fuxs thanks for reporting this. A fix is on the way. This was not a problem with the data itself but with a piece of code which didn't correctly handle a (rather rare but) intact structure of the data.

fuxs

Great news, @gudmundurg74
Looking forward to verifying the fix.

gmg

@fuxs

Fix is out in latest snapshot
https://vivaldi.com/blog/desktop/rocker-gestures-and-pop-issues-vivaldi-browser-snapshot-2867-19/

fuxs

@gmg

Verified, both accounts happily download msgs again. I am pretty much relieved, phew.
Thanks a lot guys!