Search Mail to include All Mail (Gmail) / Archive question

[Ayespy]

@DavyLJones My stored email is six accounts - four GMail accounts and two others. It's those six that make up the 19GB of storage. M3 uses less storage space than GMail for some reason - probably partly because it doesn't have folders, just a unified database that fakes folders by using labels in the headers.

It definitely searches the "all mail" folders from GMail accounts.

It only takes a few minutes to rebuild the mail search database. I've done that more than once.

[WildEnte]

@DavyLJones I just created a new profile (takes 10 seconds) and added my mostly unused GMail account to it without downloading all message data. Search works just for all terms I have entered.

Please just for testing create a new user profile and set up your GMail account there. Does the search fail just as in the profile you are currently working on? In that case - holy moly. No idea what's wrong. In case that works fine, something is borked in your profile.

[DavyLJones]

So I think I have worked out what the essence of the problem is.

Gmail has a folder all called All Mail. It's not exactly a separate folder - it contains everything - stuff in the inbox, the folders, and stuff with no folder or label. Every email in my Gmail is in here.

The way I handle emails is that once I am done with them, if I do not want to file them in a folder I just remove the inbox label or folder and they get dumped into Gmail's All Mail. By default, when I hit backspace on Gmail the email simply has the inbox or other labels removed, and it ends up in All Mail, where I ignore it. The only time I ever access Gmail's All Mail is when I search.

In this way, All Mail acts like an archive.

Right now in one of my accounts All mail there are about 84,000 emails

Vivaldi Mail also has an All Mail folder for that account. But in this folder there are just over 1000 emails only.

So I don't exactly understand what Vivaldi Mail's All Mail refers to on the server.

I suspect Vivaldi is not seeing Gmail's All Mail at all - that is also why my searches don't find emails that have no labels, folders or stars.

So - is it possible for my to manually point Vivaldi Mail's All Mail to Gmail's All Mail? And if not, is there another way for me to help Vivaldi Mail to find Gmail's All Mail.

@WildEnte I will follow your steps next. (new profile)

@Ayespy Apologies if I didnt make it clear but I tried rebuilding the database but it has had no impact on the issue.

[DavyLJones]

@WildEnte I made a new profile and added my gmail account - just one for now. I ran a search which on gmail pulls several hundred emails, all uncategorised (not in folders, not labelled, no stars).

I ran the same search on Vivaldi mail. It found zero messages.

[Ayespy]

@DavyLJones So I have tens of thousands in a given GMail account in "all mail" as well. Difference is, almost none of them are without any other folder label. The all are "inbox" or "sent" or some custom folder, except those that are in trash waiting to be permanently deleted. I have never systematically gone through deleting folder labels.

Do you have a folder in GMail named "Archive?" In order to download an email, M3 has to be subscribed to its folder. If it has no folder, it "doesn't exist." I suspect if you created a folder on the GMail server named "Archive" and shoved all those unlabeled emails into it, M3 would download, index, and be able to find them all.

[Ayespy]

@DavyLJones While I am setting up email accounts in M3, I instruct them to create an "Archive" folder on the server. It shows up on the server as [IMAP]Archive.

When I want to archive a GMail from the webmail interface, I have GMail "move" the email to Archive. That removes the existing folder label(s), applies an "Archive" label and moves it to that folder. All such emails are downloaded and indexed by M3.

[DavyLJones]

I thought that might be the issue.

I would have to work out how to tell my other clients to work that way. I use two clients that specifically are designed for Gmail so they also by default archive in the same way gmail does - as in, to All Mail. One is Airmail for mac - I think I know how to do it there - and the other is Gmail's own email app for android.

[DavyLJones]

@Ayespy

You said:
When I want to archive a GMail from the webmail interface, I have GMail "move" the email to Archive. That removes the existing folder label(s), applies an "Archive" label and moves it to that folder.

Is that something gmail does in a single click/button press? Did you have to set it up?

[Ayespy]

@DavyLJones It's in the email settings, but it is also presented when you are setting up an account.

If you check it now, I suspect it will set up an archive folder in all of your IMAP accounts.

[Ayespy]

@DavyLJones In GMail on line, you right-click an email or multi-selected batch of mails and select "Move" in the context menu.

You could do the same in M3, and it should have the same effect. But first, of course, it has to have been downloaded.

[WildEnte]

@Ayespy @DavyLJones I now also played with this a bit.

TL;DR: removing all "labels" in GMail adds an archive tag in Vivaldi, so in the Vivaldi Mail client I find those emails in the GMail account Archive subfolder, as well as in the All Mail and the GMail account root if I toggle the view filter for archived messages. So far all more or less as expected. But I did have to right-click folders in Vivaldi and choose "Rerun Filter for Folder" a couple of times, which is odd. @DavyLJones I recommend you do that once on all your GMail folders in Vivaldi, including the root folder.

Long version:

Steps:

I start off with an email in the GMail Web interface which is in Inbox, marked Important, and (as all emails) is also in all Mail.

GMail:

We see here that the email is marked important (yellow >> next to the Subject) and it's in the inbox folder (again to the right of the subject)

Now Vivaldi - I (my mail address redacted):

The email header shows where the email appears in Vivaldi, and additionally it can also be found in the "root" account view. Marked by blue dots, I find the email here:

@DavyLJones said in Search Mail to include All Mail (Gmail) / Archive question:

The way I handle emails is that once I am done with them, if I do not want to file them in a folder I just remove the inbox label or folder and they get dumped into Gmail's All Mail. By default, when I hit backspace on Gmail the email simply has the inbox or other labels removed, and it ends up in All Mail, where I ignore it

When I hit backspace in GMail nothing happens. So instead, in GMail's web Interface I remove the "Inbox" mark (to the right of the email subject) - now Vivaldi stops showing the email in the Inbox view (as expected).

As a third step, in GMail's web interface I remove the Important label (by clicking on the yellow >> next to the Subject). So in GMail, I don't see it any longer in Inbox or Important. I see it only in All Mail.

In Vivaldi, removing the last of what GMail calls labels has now done two things: first, the email no longer shows in the "Important" folder (as expected). It still shows in the All Mail folder (as expected) BUT it has also received the Archive marker. So I only see it in Vivaldi in the All Mail folder when I have the "show archived" toggle selected, and it also shows in the Archive folder.

So I always have the email, but GMail's quirky nonstandard use of folders for labels also does a hidden archive tagging. I just notice that in the GMail Interface I have a custom [Imap]/Archive label that - I guess - Vivaldi created with the setting @Ayespy mentioned above "Add Archive Folder if Missing".

This all makes sense but I did have to "rerun filter for folder" a couple of times along the way. Seems like Vivaldi has some issue here.

[Ayespy]

@WildEnte Yes. It's known that removing labels moves an email in M3 to Archive. If course in order to do that, one must first have the email in M3 and it can only fetch it if it belongs to a folder on the server in the first place.

[WildEnte]

@Ayespy I removed the labels on the Gmail interface, not in M3.

I suggest hitting "rerun filter for folder" on the Archive folder in Vivaldi. That made a whole lot of emails appear here that weren't there previously.

[Ayespy]

@WildEnte Amazing. Were these emails that M3 already had, or ones it had never downloaded before? In other words, did you remove all folder labels and THEN fetch the emails, or vice-versa?

[WildEnte]

@Ayespy said in Search Mail to include All Mail (Gmail) / Archive question:

Were these emails that M3 already had, or ones it had never downloaded before? In other words, did you remove all folder labels and THEN fetch the emails, or vice-versa?

Hard to say because I hardly use the GMail account at all. I just noticed that the one email I used to test, after removing the labels in GMail, showed in the Gmail Root & All Mail folders with the Archive toggle enabled, but it didn't appear in the GMail Archive folder in M3. When I chose to rerun folder, 30 emails appeared in a previously empty folder. Most of that are unimportant notification emails from GMail, so ... have they not been fetched before? I didn't have IMAP logging on, can't say.

[Ayespy]

@WildEnte OK.

So I suspect those mails were already locally stored.

The way M3 gets an IMAP email is to access the folder on the server where it lives, to fetch it.

I can't say for sure whether M3 subscribes to "All Mail" on the GMail servers, or whether it even can, since the All Mail category has no folder label of its own. It includes all folders, but it also includes "folderless" emails. I do know that M3 does have an "All Mail" view for GMail accounts, but I'm not aware of that view including any emails that don't have, or have never had, an M3 folder label (inherited from GMail)

I'm going to have to see if I can run an experiment that tries to download folderless emails from a GMail account. Let me think on this...

[DavyLJones]

@Ayespy and @WildEnte

thanks so much for your effort.

Firstly a clarification - for simplicity I said when I hit backspace in Gmail it removed the label and so it is left in All Mail. That wasn't strictly true - I rarely use the Gmail Web platform. Mostly (until Vivaldi Mail) I was using the desktop client Airmail, which is designed for good interaction with Gmail. And on mobile I was using the Gmail app.

On Airmail if I hit backspace it has the described functionality - removing folder-labels and effectively archiving in All Mail. On the Gmail app I swipe right for that function.

I tried your suggestions.

• Firstly I realised that I had not set the setting to add Archive folder if missing so I switched that on - no change though and I think there already is an archive folder in Gmails IMAP folders.
• As you suggested I did "rerun filter for folder" on many folders - no change initially, but later I did notice that it made a significant change to some folders, but no effect on Archive or All Mail.
• I removed both my accounts and then re added one. but there was no noticeable change.

I took a closer look around and it all got a bit confusing!

Spam

In Spam there are loads of emails that are definitely not spam - in fact Vivaldi is pulling up emails that it acknowledges are filed in other folders - loads from old clients filed accordingly for example. Vivaldi sees the folders they are in on Gmail but still presents them as spam. Now I am worried Vivaldi will try to delete them!

Sent

Sent only shows 1018 even though on Gmail I can see over 35k sent mails. This 1018 consist of 1000 sent emails that have no labels, going back 6 and a half months, and another 18 that have been filed in other folders. I think. If that is true is it the case that Vivaldi Mail will only search through any folder to 1000 mails? Or will only search through 1000 unlabelled mails? It might explain some but not all my issues.

Trash

Trash pulls up emails in the Gmail Bin (as expected, but also emails in a folder I have never heard of before and cannot find on Gmail, called "deleted messages" going back many. years. Turns out that is in fact a gmail label though I dont understand it or how anything ended up there. But let's put that aside - it could be Mail, it coulfd be Airmail or who knows what. Vivaldi Mail's Trash also And it also pulls up things field in weird ways like "~~Outbox->~~Local Copy". - I guess that means I wrote the email on MAil, didnt send it and then Mail has saved it?

Archive

Archive is only displaying around 400 messages. They are all from All Mail. And they go back exactly 4 months. So again I wonder if In Archive Vivaldi is only looking back a set timescale?

So I did "rerun filter for folder" a few more times but to no avail.

All in all it's running really screwy for me, and I wonder if I have just set it up wrong - I just let it set itself up. Or is there are a bunch of settings I am not using correctly?

I have other issues, like I cannot get the alias on my account to work correctly even though it is set up just as it is on all my other clients. But there's no point in fixing that until I fix this issue.

In the end I might have to admit defeat. I love the browser and the idea of itegrating mail with it is powerful. The further thought that I might be able to deep search multiple Gmail accounts simultaneously was the real draw for me, but thats not working at all at present.

Do I keep going? I hate giving up, especially since I use the browser every day....

[Ayespy]

@DavyLJones Your GMail folders are polluted by earlier email clients. "Deleted Messages" is not a GMail label. It has been added by another client. The folders prefixed [IMAP] are also not natively GMail folders. They are GMail adding folder labels from clients. It also does not have an Archive folder. This is added from an IMAP client (like M3).

There is no numeric or date limitation for M3 to download or search for mails. My numbers of downloads and results of searches go into the hundreds of thousands and back to 2001 (the oldest email currently on my system)

M3 will not delete or try to delete messages labeled as Spam. You have to pro-actively do that yourself. Also, unlike GMail, it does not permanently delete trash of a certain age. It retains it even after GMail deletes it on server. You have to clean out your local trash bin yourself. You can select any email(s) labeled Spam in M3 and re-label it "not spam." That should return it to its original folder.

Don't give up, by any means. You have created a unique problem by using rare email clients and removing labels wholesale on the server, and we should work out what the solution for that problem is. Once we're over that hump, M3 should be a really good client for you.

[Ayespy]

@DavyLJones For starters, take a look at the emails that don't belong in your Spam folder. Do any of them have ONLY the spam email header label? Or do all of them also have another label?

[mib2berlin]

@DavyLJones
Hi again, I can search back to 2019 and to 2021 in Archived mails but this is not Gmail.
I guess M3 does not limit anything in time or count of mails.