Vivaldi not syncing

Pathduck

@LazyLama Sounds to me (just theorizing wildly) a DNS issue.

Chromium browsers use DNS-over-HTTPS by default, that might be what's causing the issue.
It will try to "guess" your preferred DoH provider based on for instance your OS/resolv.conf. I have no idea what happens if it can't guess - possibly it falls back to using Google's DNS or falls back to your OS DNS resolver.

For instance for me on Windows, I have 1.1.1.1 set as my DNS primary. This results in all Chromium browsers defaulting to Cloudflare unless DoH is explicitly disabled. Which I do, as I want browsers to use my OS DNS provider not DoH which I believe is just another potential failure point.

Possibly it resolves the bifrost IP to the wrong address. So do what I said and test what IP the browser thinks Bifrost is.

DoctorG

Name: bifrost.vivaldi.com
Address: 31.209.137.10

LazyLama

Possibly it resolves the bifrost IP to the wrong address. So do what I said and test what IP the browser thinks Bifrost is.

@Pathduck, so I put bifrost.vivaldi.com in the field on vivaldi://net-internals/#dns, this is what happens:

Resolved IP addresses of "bifrost.vivaldi.com": ["31.209.137.10"].
No alternative endpoints.

Still no syncing.

irz

Update:

No need to open https://bifrost.vivaldi.com/vivid-sync in Firefox

Just run curl https://bifrost.vivaldi.com/vivid-sync in terminal

Then sync is working again

I am still trying to find the cause of this issue

Pathduck

@LazyLama OK so this means Vivaldi is able to resolve the correct IP of Bifrost.
Did you do this after resolving the IP somewhere else, like from the shell/Firefox?

No need to open https://bifrost.vivaldi.com/vivid-sync in Firefox
Just run curl https://bifrost.vivaldi.com/vivid-sync in terminal

Here's why this happens (I think...)

Running curl in the shell will resolve the correct IP for Bifrost
The IP will be cached on the OS level for some time (no idea how long DNS records are cached)
The browser will use the OS DNS cache over its own cache/resolver (DoH)
After the DNS TTL expires, the OS will clear the cache entry and the browser will need to do another resolve, using DoH, and getting the wrong IP (possibly)

EDIT: Looks like Cloudflare sets a TTL for Bifrost at 120 seconds, not a lot but then again it's no problem just resolving it every time Sync is triggered maybe once every 10 mins or so.

λ dig bifrost.vivaldi.com @ns.cloudflare.com

; <<>> DiG 9.11.9 <<>> bifrost.vivaldi.com @ns.cloudflare.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36494
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;bifrost.vivaldi.com.           IN      A

;; ANSWER SECTION:
bifrost.vivaldi.com.    120     IN      A       31.209.137.10

;; Query time: 3 msec
;; SERVER: 173.245.58.100#53(173.245.58.100)
;; WHEN: Thu Sep 26 19:03:26 WEST 2024
;; MSG SIZE  rcvd: 64

Pathduck

Another thing to do is do a network capture. Here I'm using tshark (WireShark) on Win10 but the standard tcpdump has the same option. You might need to figure out what interface to capture on first by doing -D and then using -i to specify interface. Maybe on Linux it will always use eth0 by default, I dunno.

$ tshark -i4 host bifrost.vivaldi.com
Capturing on 'Ethernet'
    1   0.000000 192.168.0.10 62749 31.209.137.10 443 TCP 66 62749 → 443 [SYN] Seq=0 Win=65228 Len=0 MSS=1460 WS=256 SACK_PERM
    2   0.056274 31.209.137.10 443 192.168.0.10 62749 TCP 66 443 → 62749 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1418 SACK_PERM WS=512
    3   0.056378 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=1 Ack=1 Win=262144 Len=0
    4   0.057181 192.168.0.10 62749 31.209.137.10 443 TCP 1472 [TCP segment of a reassembled PDU]
    5   0.057181 192.168.0.10 62749 31.209.137.10 443 TLSv1 365 Client Hello (SNI=bifrost.vivaldi.com)
    6   0.113822 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=1 Ack=1730 Win=64000 Len=0
    7   0.115625 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 1472 Server Hello
    8   0.115735 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 1472 Certificate
    9   0.115735 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 219 Server Key Exchange, Server Hello Done
   10   0.115771 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=1730 Ack=3002 Win=262144 Len=0
   11   0.118955 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 180 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
   12   0.175262 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 312 New Session Ticket, Change Cipher Spec, Encrypted Handshake Message
   13   0.176734 192.168.0.10 62749 31.209.137.10 443 TCP 1472 [TCP segment of a reassembled PDU]
   14   0.176734 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 634 Application Data
   15   0.177850 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 422 Application Data
   16   0.233292 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=3260 Ack=3854 Win=64000 Len=0
   17   0.234169 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=3260 Ack=4222 Win=64512 Len=0
   18   0.262367 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 263 Application Data
   19   0.262367 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 329 Application Data
   20   0.262460 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=4222 Ack=3744 Win=261376 Len=0
   21   0.279622 192.168.0.10 62749 31.209.137.10 443 TCP 1472 [TCP segment of a reassembled PDU]
   22   0.279622 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 1472 Application Data
   23   0.279622 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 584 Application Data
   24   0.335666 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=3744 Ack=7588 Win=64000 Len=0
   25   0.372864 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 394 Application Data
   26   0.416869 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=7588 Ack=4084 Win=261120 Len=0

LazyLama

@Pathduck said in Vivaldi only syncing when on VPN:

Here's why this happens (I think...)

Running curl in the shell will resolve the correct IP for Bifrost
The IP will be cached on the OS level for some time (no idea how long DNS records are cached)
The browser will use the OS DNS cache over its own cache/resolver (DoH)
After the DNS TTL expires, the OS will clear the cache entry and the browser will need to do another resolve, using DoH, and getting the wrong IP (possibly)

That sounds like a reasonable hypothesis. It would explain why I and @irz are seeing. Below, you suggest a network capture. Would this show how Vivaldi resolves the IP address? Above, you mentioned using vivaldi://net-internals/#dns. Wouldn't that work as well?

Pathduck

@LazyLama said in Vivaldi only syncing when on VPN:

Would this show how Vivaldi resolves the IP address?

It wouldn't show how it resolves the IP. To do that you could capture DNS requests:

λ tshark -i4 port 53
Capturing on 'Ethernet'
    1   0.000000 192.168.0.10 49891 router.asus.com 53 DNS 79 Standard query 0x9b75 A bifrost.vivaldi.com
    2   0.000212 192.168.0.10 64669 router.asus.com 53 DNS 79 Standard query 0x74ec HTTPS bifrost.vivaldi.com
    3   0.002631 router.asus.com 53 192.168.0.10 49891 DNS 95 Standard query response 0x9b75 A bifrost.vivaldi.com A 31.209.137.10
    4   0.003298 router.asus.com 53 192.168.0.10 64669 DNS 138 Standard query response 0x74ec HTTPS bifrost.vivaldi.com SOA carl.ns.cloudflare.com

In this capture my PC asks my router for the IP of Bifrost, the router replies with the correct IP.

But it would show at least the request timing out, and to what IP it goes. But come to think of it, since the OS obviously resolves the IP correctly, it wouldn't actually capture any traffic going to the wrong IP.

Unless of course you capture all HTTPS traffic. But you'd get a lot of data. So close all browser tabs, close all other networked programs and restart the browser then run a
tcpdump dst port 443
and trigger the sync issue from chrome://sync-internals

Above, you mentioned using vivaldi://net-internals/#dns. Wouldn't that work as well?

Yes, and you did that and it resolved correctly. I don't know why it did.

Another thing to do:

Close all tabs
Open chrome://net-export
Click Start Logging...
Trigger the Sync issue and wait for the timeout
Stop the logging
Share the network capture (zip it first) or analyze it yourself

LazyLama

@Pathduck

I have done the following to test your hypothesis:

Restart the computer
Open Vivaldi and check whether I can sync. I could not.
Open vivaldi://net-internals/#dns. This resulted in Resolved IP addresses of "bifrost.vivaldi.com": ["31.209.137.10"]. No alternative endpoints
Run curl https://bifrost.vivaldi.com/vivid-sync.
Sign out of sync and sign back in to trigger syncing.
Syncing succeeds.

So, you seem correct.

Pathduck

@LazyLama said in Vivaldi only syncing when on VPN:

So, you seem correct.

Nope - because the resolved IP is the correct one. That works against my theory. It should be wrong. So maybe it's not DNS but something else on your system blocking the browser. Only you can know.

Or, at least I'm partly correct on the caching bit, as the act of running curl will resolve the correct IP, at least for a time. Or possibly just "open" something that was blocked.

It's also quite possible that the net-internals page does things differently and I don't know why.

Do a network capture from net-export and share it.

LazyLama

@Pathduck I did the tcpdump before and after running the curl command. So, without syncing working (before the curl command) and with syncing working (after curl).

The only difference I can see is that when things don't work, the host used differs from when things work. So, somehow, running the curl command changes how my ISP (in fact, my institution) handles my internet traffic.

So, what is see is before curl
ucfsu.xxx.xxx.xxx.56938 > 31.209.137.10.https
and
windows-j5s3qlt.xxx.xxx.xxx.34758 > 31.209.137.10.https
after curl.

(I have replaced some identifying parts of the strings with x's).

Maybe I should just write an sh script that runs the curl command before starting Vivaldi.

Pathduck

@LazyLama said in Vivaldi only syncing when on VPN:

in fact, my institution

So there's a clue, and a good one. You're on an enterprise network, probably proxied, firewalled and NAT'ed to hell and back.

Why it would change your client hostname I have no idea, you'd have to ask your IT people.
But the target IP and port is the same so should work no matter.
But if the client host is changed then the return packets have no way of making it back to the client over the NAT'ed firewall/proxy, and you'd get a timeout.

Talk to your IT people. If Vivaldi is at all supported by your company/institution. Knowing most IT depts. they will just say they don't support it and that's that. "We only support major browsers."

Still a capture from net-export would be interesting to see, but then again you X'd out your hostname so such a log would probably contain more than you're willing/allowed to share. So you'll need to analyze it yourself I guess

LazyLama

@Pathduck said in Vivaldi only syncing when on VPN:

Still a capture from net-export would be interesting to see, but then again you X'd out your hostname so such a log would probably contain more than you're willing/allowed to share. So you'll need to analyze it yourself I guess

I appreciate your offer. However, I am not sure whether my institute would appreciate me sharing these logs

irz

I have tried launching Vivaldi with the command:

vivaldi --host-resolver-rules="MAP bifrost.vivaldi.com 31.209.137.10"

but I still can't sync

In the same network environment, Vivaldi sync works fine on Windows OS

DoctorG

Could be a hardware firewall appliance which checks behavior of requesting connections by rules. Who knows.

irz

I compared the logs captured via chrome://net-export on Debian 12 and Windows under the same network environment.

After opening the log files on https://netlog-viewer.appspot.com, I noticed that in the DNS section, Debian 12 doesn't have any entries for bifrost.vivaldi.com, regardless of whether sync works properly.

On Windows, however, the DNS logs do include bifrost.vivaldi.com.

The records that are present in both logs include login.vivaldi.net, mimir2.vivaldi.com, and downloads.vivaldi.com.

DoctorG

Perhaps some settings for DNS are managed in your LAN?
Check internal pages

vivaldi://policy
vivaldi://management

irz

I searched through previous forum posts and found that this issue indeed started from version 6.7. The last version of 6.6, which is 6.6.3271.61 (Stable channel), was able to sync without any problems. However, starting from the first version of 6.7, which is 6.7.3329.17 (Stable channel), syncing no longer works.

From these two forum posts https://forum.vivaldi.net/topic/97615/vivaldi-on-linux-sync-broken https://forum.vivaldi.net/topic/97482/initializing-sync-stalled-new-version, it appears that the sync error started around May of this year, corresponding to the update from Chromium v122 to Chromium v124. I am certain that this sync issue is caused by the Chromium update.

Therefore, I downloaded several Chromium-based browsers for testing. The test focused on whether I could directly open https://bifrost.vivaldi.com/vivid-sync. Here are the test results:

The last version of google-chrome-unstable that could open the webpage directly is 124.0.6342.3-1. Information from chrome://version

Google Chrome: 124.0.6342.3 (Official Build) dev (64-bit)
Revision: 144c1c9bfe50c83a7304a43491a071f2befefdeb-refs/branch-heads/6342@{#7}
OS: Linux
JavaScript: V8 12.4.148
User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
Command Line: /usr/bin/google-chrome-unstable --flag-switches-begin --flag-switches-end
Executable Path: /opt/google/chrome-unstable/google-chrome-unstable
Profile Path: /home/username/.config/google-chrome-unstable/Default
Active Variations: f38ef081-ca7d8d80

The last version of Brave Browser Release Channel that could open the webpage directly is v1.64.122 based on Chromium 123.0.6312.122. Information from version

Brave: 1.64.122 Chromium: 123.0.6312.122 (Official Build) (64-bit)
Revision: a5b964f45dca7b87fc0f2adf2660d6fd6a6608b8
OS: Linux
JavaScript: V8 12.3.219.16
User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36
Command Line: /opt/brave.com/brave/brave --disable-domain-reliability --enable-dom-distiller --enable-distillability-service --origin-trial-public-key=bYUKPJoPnCxeNvu72j4EmPuK7tr1PAC7SHh8ld9Mw3E=,fMS4mpO6buLQ/QMd+zJmxzty/VQ6B1EUZqoCU04zoRU= --sync-url=https://sync-v2.brave.com/v2 --lso-url=https://no-thanks.invalid --variations-server-url=https://variations.brave.com/seed --variations-insecure-server-url=https://variations.brave.com/seed --flag-switches-begin --flag-switches-end --component-updater=url-source=https://go-updater.brave.com/extensions
Executable Path: /opt/brave.com/brave/brave

The last version of Microsoft Edge Beta that was able to open the webpage directly is v123.0.2420.65-1 based on Chromium 123.0.6312.87. Information from version

Microsoft Edge: 123.0.2420.65 (Official build) beta (64-bit)
Revision: 49b6a5859239c32604bcefee0b98e103c0a9bdae
Chromium version: 123.0.6312.87
Operating system: Linux
JavaScript: V8 12.3.23.7
User agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 Edg/123.0.0.0
Command-line: /usr/bin/microsoft-edge-beta --flag-switches-begin --flag-switches-end
Executable path: /opt/microsoft/msedge-beta/microsoft-edge-beta

It can be concluded that Chromium versions prior to v123 are able to sync normally. However, starting from v124, sync no longer works. It is unclear what new feature Chromium added that prevents the resolution of bifrost.vivaldi.com, although other subdomains of vivaldi.com can still be resolved correctly.

This is definitely unrelated to the network being used, as version 6.6 and earlier versions can sync normally.

Therefore, if you wish to sync directly without using a proxy or VPN, it is recommended to install version 6.6.3271.61 (Stable channel) and avoid upgrading until the issue is resolved.

I hope this issue can be fixed by the Vivaldi team soon.

DoctorG

@irz said in Vivaldi only syncing when on VPN:

I hope this issue can be fixed by the Vivaldi team soon.

Issues can only be fixed when reproducible on Operating Systems by the internal tester and developer team.

Currently i have no idea whats is wrong on your side.

May i ask in which country you live and how you are connected to Internet?

@irz Do you have special commandline for starting Vivaldi?

I can not reproduce your Sync issue on Mint 21.3 Cinnamon "Virginia".

We do not know how your Mint is caching and resolving DNS.

irz

@DoctorG It is indeed difficult to reproduce, although the network environment is not very special. It might be possible to reproduce the issue in a virtual machine using VirtualBox or VMware by assigning an internal address, like 10.0.0.x.

I found that this issue doesn't seem related to DNS resolution; it might be that Chromium has updated its restrictions on Private Network Access, leading to the ERR_TIMED_OUT error.

However, in the same network environment, I can sync normally on the Windows OS.

On Debian 12, 6.6.3271.61-1 and earlier versions can sync normally, while the problem only started appearing from 6.7.3329.17-1 onwards.

I’ve uploaded five netlog files (available for download here) and will explain the process for obtaining each of them below:

chrome-net-export-log-6.6.3271.61-1.json

Completely fresh installation of 6.6.3271.61-1, without any previous configuration files, and can open Bifrost directly.

chrome-net-export-log-6.7.3329.17-1.json

Completely fresh installation of 6.7.3329.17-1, without any previous configuration files, and cannot open Bifrost directly.

chrome-net-export-log-6.7.3329.17-1-with-proxy.json

Completely fresh installation of 6.7.3329.17-1, without any previous configuration files, and opening Bifrost through a proxy.

chrome-net-export-log-6.7.3329.17-1-with-curl.json

Completely fresh installation of 6.7.3329.17-1, without any previous configuration files. Initially, opening Bifrost returns ERR_TIMED_OUT. Running curl https://bifrost.vivaldi.com/vivid-sync in the terminal returns a response, and then Bifrost can be opened.

chrome-net-export-log-6.7.3329.17-1-after-curl.json

Completely fresh installation of 6.7.3329.17-1, without any previous configuration files. The difference from the with-curl file is that network requests before running curl are not logged.

Please let me emphasize again: I can sync normally on Windows. And on Debian 12, 6.6.3271.61-1 and earlier versions can also sync normally, so it’s impossible that my network is blocking the sync.

I believe others have also encountered this issue, mainly on devices using internal addresses. So I hope these netlog files can help in resolving this issue.