Search Mail to include All Mail (Gmail) / Archive question

[DavyLJones]

I have just moved to Vivaldi mail so forgive a bunch of possibly noob idiot questions!

I have a few gmail accounts. The way I archive my mail in gmail is just to remove the inbox label - then they go to All Mail.

When I run a search in Vivaldi mail, it does not seem to pick up all my messages at all. I think the issue is that it is not picking up anything in "All Mail" and only finding those that are in folders or have stars (flags). I am not sure that is the only issue I am experiencing because sometimes even after I move a mail from all mail to a folder (doing so inside gmail), and I refresh vivaldi mail and run the search again, it does not find the message that has now been moved to a folder.

So for example - I ran a search for messages from John. Vivalid found 2 messages from John. Gmail found 50. The two messages vivaldi found had been filed in folders, while the other 48 that only showed up in Gmail were in All Mail.

To test the theory I moved one of those to a folder and ran the test again in Vivaldi - it still only displayed the original 2.

Therefore I am not 100% sure this is the issue but I think it is.

Am I doing something wrong with the search - I do have it set to include the archive. Do I need to manually point the archive to All Mail?

All the emails in the archive tab are in All Mail but they are just a small number of the total emails in All Mail.

Thank you for your help

[WildEnte]

@DavyLJones make sure you have all the messages downloaded for full text search (which is a setting in the account setup options). When doing a search, you have filtering options above the message list. Make sure that archived messages and custom folders are included in the search using those filter options. Gmail has a quirky behavior when it comes to folders and labels, because they abuse folders as labels.

See also this bug report (not the same but somewhat related)
https://forum.vivaldi.net/topic/74974/archived-still-showing-even-though-archive-button-not-selected/15?_=1713139891333

[DavyLJones]

Thank you @WildEnte

do you mean the option: "download messages for offline viewing and searching"?

I do not have it currently checked because I am worried about the size of space it might take up - and then any CPU resources that my use. If my 2 gmail accounts are 15gb each and nearly full, and I select that option, will it require 30gb storage? I do have the space but I will need to manage it wisely (and work out how to keep it out of my time machine backups).

Do you know where it is stored?

I was hoping the search function could just search the whole account online the way gmail can from the broswer - is that not an option?

[WildEnte]

@DavyLJones yes, that is the option that's required for Vivaldi to know anything more about your email than the message header. It downloads all messages from that account (yes, all those gigabytes) and indexes them to build its own search database. Otherwise the search database only knows email header information and the content of emails you clicked on (which downloads them).
Emails are stored as .eml files in your browser profile folder (in settings, right click the account and choose "reveal data folder)

Vivaldi mail does not do a server side search, if it did, it would be painfully slow - Gmail is fast on the web interface because it uses its own proprietary search database which Vivaldi cannot access.

[DavyLJones]

@WildEnte

thank you for this information.

I think the basic header info is enough for my uses for now. But the issue is that vivaldi's search is not locating many emails.

I would expect that if it has access to the headers of all my emails, then it would find all the emails from John, whether they are in a folder or in "all Mail". But vivaldi is not finding the ones in All Mail - is downloading all the mail the only way to solve that? isnt it possible to tell vivaldi to search in "all mail" the same way as it appears to search in the folders - even when it has not downloaded the mail?

[WildEnte]

@DavyLJones of john appears as the sender in the header, Vivaldi should find them, in particular if you see them in the archive folder in the All Accounts section. What happens if you right click any of the emails from John and then choose to show all messages from that sender? Does Vivaldi subscribe to the Archive folder?

Vivaldi's email search is extremely good so I wonder if I'm missing something obvious. Please also see https://help.vivaldi.com/mail/mail-get-started/search-messages-in-vivaldi-mail/

[mib2berlin]

@DavyLJones
Hi, it depends on how many mails you store.
I delete all I don't really need and my mail folder is only 90 MB, it's about 1000 mails and I always empty the trash can like in my office.

Cheers, mib

[Ayespy]

@mib2berlin It doesn't really matter how many. I matters where/how you search, and how you store.

I store the contents of over a quarter million emails going back to 2001. This requires 19GB of storage or so.

I can search any of these emails and get a result in a couple seconds. I can search, say "Melissa," and find 2,595 messages (because one of my contacts is named Melissa), and more than one client has been named Melissa. The newest of these emails is a week old and the oldest is from 22 Feb 2002. The search takes less than a second.

It's important to make sure that your search parameters include "All Messages" or that you start your search in the "All Messages" view and that your default search is "current folder." Make sure your view filters (in the folder you are searching) are set to show Read, Mailing Lists, Custom Folders and Archives. Leave your Start Date and End Date for your search blank, to capture everything.

Before you begin, make sure you are subscribed to all folders for that account. You should already be, unless you told it not to subscribe to some or one.

if this does not reveal the same 50 emails GMail did, then rebuild your search database. (mail help files tell you how to do this).

[mib2berlin]

@Ayespy
Hi, I have no issues with search, @DavyLJones meant download of 5 mail accounts let his hardisk explode.
With 19 GB for each account this could happen.
@DavyLJones
How much space of your 15 GB Google space is used, this include all files, images and so forth.

Cheers, mib

[DavyLJones]

Hi @WildEnte

You said

@DavyLJones of john appears as the sender in the header, Vivaldi should find them, in particular if you see them in the archive folder in the All Accounts section. What happens if you right click any of the emails from John and then choose to show all messages from that sender? Does Vivaldi subscribe to the Archive folder?

I tried this both through right clicking and also clicking on the sender in the header of the email but in both cases it did not find any new emails.

I checked the link you sent thank you - it's interesting but I did not find anything to help me with this specific issue.

[DavyLJones]

@Ayespy

Thanks for your message.

I can confirm that I am searching from All Mail, and that the filters are set to include everything, with no date parameters or other limiting factors in applied.

I have no unsubscribed to any folders yet (though I plan to unsubscribe to some folders relating to 3rd party apps).

The only thing I believe I could do differently is set vivaldi to download the full emails - I didnt want to do this for storage reasons, but if it is neccesary to have this function run then I will.

Basically this function is the main reason I want to use vivaldi for email - right now when I want to find an old email, my current desktop application is too slow to search such a big amount of data, so I end up having to go through multiple gmail accounts to search, because I cant recall which email account has the message.

So it sounds like I would need to rebuild the database? Is that the conclusion?

It just seems odd that it will not search gmail's All Mail, very specifically. It feels like it must be a setting I have failed to set or something. I cant help wonder if it is something to do with "Archive" - I use All Mail as an archive.

I didnt tell Vivaldi where to Archive and I have not set up an Archive folder. The Archive tab shows emails in All Mail, but just a few hundred of them rather than the many thousands I would expect if it were accessing All Mail on both accounts fully.

[DavyLJones]

@mib2berlin Would have to check but I think in both accounts they are probably 90% emails, and both accounts are at about 80% capacity. So I guess roughly 10-11GB per account of emails.

My desktop could handle it though my laptop probably couldn't.

Is it neccessary in order to achieve the search results I want through - is this the obstacle to searching "All Mail"? If so I will do it.

Before I set it up, if it is neccesary, I would like to understand first where on my computer (mac) the emails are stored, so I can ensure it is excluded from cloud synching and from my Time Machine backups.

Thanks to you all for your support.

[mib2berlin]

@DavyLJones said in Search Mail to include All Mail (Gmail) / Archive question:

Is it neccessary

I think no, I just test this but had not any issues searching without download.
You can try to rebuild the search database from the gear in the mail icon at the status bar.

[DavyLJones]

@mib2berlin cool - I am running that. will let you know result later many thanks as ever

[DavyLJones]

i rebuilt it but no change I am afraid.

Still a fraction of the results showing up and from what I can tell none of the emails in gmail's "all Mail" - ie those without any other label or folder - are coming through.

I can't say for sure if there are others that are also not coming through - I couldnt be sure until I see the All Mails coming through - but I suspect that is the issue.

Is there something else I can try - I would really love to set this up - it is the reason I am switching to Vivaldi mail.

[DavyLJones]

I wonder if it is related to the fact that I cannot archive to all mail.

In my other email application, it has been set up so that if I hit ARCHIVE, it basically removed all labels and sits in ALL MAIL - so all mail is being used as the archive.

If I then find that email in Vivaldi mail it has the archive symbol next to it.

but If I try to archive something in vivaldi mail it asks me to set up an archive folder.

So this suggests to me that may be I have not told vivaldi how to work with gmail's "all mail"

could these be related?

[mib2berlin]

@DavyLJones
I am sorry, no further ideas.
Gmail doesn't respect any mail standards, they create there own, like Microsoft.
Gmail labels are not M3 labels, for example.
If you are used to use Gmail for all your mail activities I am not sure M3 will fit your needs.
Give it a week or something to get familiar with it.

[Ayespy]

@DavyLJones My stored email is six accounts - four GMail accounts and two others. It's those six that make up the 19GB of storage. M3 uses less storage space than GMail for some reason - probably partly because it doesn't have folders, just a unified database that fakes folders by using labels in the headers.

It definitely searches the "all mail" folders from GMail accounts.

It only takes a few minutes to rebuild the mail search database. I've done that more than once.

[WildEnte]

@DavyLJones I just created a new profile (takes 10 seconds) and added my mostly unused GMail account to it without downloading all message data. Search works just for all terms I have entered.

Please just for testing create a new user profile and set up your GMail account there. Does the search fail just as in the profile you are currently working on? In that case - holy moly. No idea what's wrong. In case that works fine, something is borked in your profile.

[DavyLJones]

So I think I have worked out what the essence of the problem is.

Gmail has a folder all called All Mail. It's not exactly a separate folder - it contains everything - stuff in the inbox, the folders, and stuff with no folder or label. Every email in my Gmail is in here.

The way I handle emails is that once I am done with them, if I do not want to file them in a folder I just remove the inbox label or folder and they get dumped into Gmail's All Mail. By default, when I hit backspace on Gmail the email simply has the inbox or other labels removed, and it ends up in All Mail, where I ignore it. The only time I ever access Gmail's All Mail is when I search.

In this way, All Mail acts like an archive.

Right now in one of my accounts All mail there are about 84,000 emails

Vivaldi Mail also has an All Mail folder for that account. But in this folder there are just over 1000 emails only.

So I don't exactly understand what Vivaldi Mail's All Mail refers to on the server.

I suspect Vivaldi is not seeing Gmail's All Mail at all - that is also why my searches don't find emails that have no labels, folders or stars.

So - is it possible for my to manually point Vivaldi Mail's All Mail to Gmail's All Mail? And if not, is there another way for me to help Vivaldi Mail to find Gmail's All Mail.

@WildEnte I will follow your steps next. (new profile)

@Ayespy Apologies if I didnt make it clear but I tried rebuilding the database but it has had no impact on the issue.