Super Speedy Smart Cache

[code3]

Vivaldi needs to make sure it is not categorized as a slow browser due to its slow (but getting better) UI. So, I suggest crawling the web. Go to Alexa.com and you can browse people’s browsing histories (yes, people really do install the Amazon browser extension) and get the most popular sites, then crawl them.

How will crawling sites help speed, you might ask? Well, many sites use the EXACT same resources. If you crawl the sites, you can find out all the locations where, say, Vuejs or fontawesome are hosted, and serve all of those from a local copy.

This will also make local cdn and decentraleyes unnecessary, building yet another extension function directly into the browser. Also, if Decentraleyes adopts the dataset it could be good publicity for V.

And one more idea: when connected to power, on WiFi on a good connection, Vivaldi could preload cached resources that it uses a lot and are about to expire.

Also, the dataset should be released under a copyleft license.

[Catweazle]

@code3 , and super speed Amazon user tracking (see Privacy policy; Amazon is worse than Google and Facebook combined, thanks.

[code3]

@Catweazle Yes, and just as hard to escape.

But what do you think of a smart cache?

[Catweazle]

@code3 , you know a FOSS alternative? I don't. Vivaldi can't use proprietary soft without to pay for the licence and developing our own system is beyond the capacity of our long-suffering developers, who already have more than 4000 requests on the list.

[code3]

@Catweazle There are plenty of FOSS web crawlers, Alexa is just one place you could find sites to crawl.

Of course, Vivaldi may not have the capacity to operate a web crawler right now. But maybe they do - they already operate mail servers.

[Pathduck]

@code3 Are you thinking of something like the caching proxy service as used by Opera Mini?
https://en.wikipedia.org/wiki/Opera_Mini#Functionality

I used it last more than a decade ago on mobile phones and it worked fine. Of course, you have to accept that the operator of the proxy (in this case Opera) would be able to see traffic originating from the device so you lose a certain degree of privacy. It worked by caching and compressing the data before being sent to clients, so it could be re-used by many users from a single point.

The purpose of the feature was not necessarily to save speed but to save bandwidth as mobile data was very expensive back then - and it still is in some parts of the world, which is why Opera Mini still exists I guess.

Back when Opera introduced Mini with caching/turbo mode they were already a pretty big company and Opera Mini had a large market share on mobile devices, so it made good business sense at the time.

Vivaldi is a small company and to operate such a service with dedicated people to manage/support it and buying the necessary hardware/bandwidth to operate would be very expensive so I don't think this is going to happen for the foreseeable future.

There are several FOSS caching proxies like Varnish and Squid that could be used if this ever happens.
https://en.wikipedia.org/wiki/Varnish_(software)
https://en.wikipedia.org/wiki/Squid_(software)

[Catweazle]

@Pathduck
"2021-03-16 - Denial of Service in varnish-modules
Some versions of the separate varnish-modules bundle allow for a potential denial of service attack when the header.append() or header.copy() functions are used."

Squid since 2013 without updates

[Pathduck]

@Catweazle You expect any complex piece of software to not have bugs or vulnerabilities?

Squid: Stable release 4.14[2] / 8 February 2021; 2 months ago

[code3]

@Pathduck No!

I am talking about a web crawler that identifies resources that are the same across many sites, such as jQuery and Vue and Fontawesome, and serves them all from a local copy, similar to decentraleyes or IPFS.

I am also talking about an opportunistic cache system, where instead of old cache files being refreshed upon access, they are refreshed before they are needed, at a time when the device is plugged in, on WiFi, and not using much CPU RAM or Network.

These changes would save RAM, speed, network usage, and storage space.

[Pathduck]

@code3 Ok then.

Well, best of luck with that

[code3]

@Pathduck Well I thought it was a good idea...

It would improve out of the box speed.

It would also improve bandwidth while using less resources than a proxy server. (Crawling 10K webpages is probably easier than providing a proxy to 2M people)

But maybe it’s just me.