Vivaldi not syncing

DoctorG

@LazyLama said in Vivaldi only syncing when on VPN:

The weird thing is that the same computer connected to the same network booted to Windows doesn't have a problem to sync.

Then your Debian/Mint could have a network configuration issue.

Pathduck

@irz Hi, just use some basic logic.

If it times out in the browser but connects from the shell, what could cause the browser to time out?

Do you have any software that would block the browser process from connecting?
Does it work in other browsers?
If it works in other browsers, are these browsers configured differently?
Does the bifrost host resolve to the correct IP in the browser?
Try vivaldi://net-internals/#dns to test the resolve
The resolved IP will also show in the dev.tools for the request.
Are you using any proxies or VPN?

irz

@Pathduck Thanks for your reply.

It seems like this might be a Chromium issue, rather than something specific to Vivaldi.

I was able to open https://bifrost.vivaldi.com/vivid-sync in Firefox, and it returned Not found. But in Chromium, the result was the same as in Vivaldi.

Interestingly, after I opened it in Firefox, I was able to sync in Vivaldi. However, the sync only lasted for about 5 to 10 minutes.

LazyLama

@DoctorG said in Vivaldi only syncing when on VPN:

Then your Debian/Mint could have a network configuration issue.

Weirdly (and similar to @Pathduck), I can visit https://bifrost.vivaldi.com/vivid-sync in Firefox. However, in Vivaldi, visiting the same address times out. However, after visiting the URL using Firefox, I can reach the same site using Vivaldi and syncing happens.

Pathduck

@LazyLama Sounds to me (just theorizing wildly) a DNS issue.

Chromium browsers use DNS-over-HTTPS by default, that might be what's causing the issue.
It will try to "guess" your preferred DoH provider based on for instance your OS/resolv.conf. I have no idea what happens if it can't guess - possibly it falls back to using Google's DNS or falls back to your OS DNS resolver.

For instance for me on Windows, I have 1.1.1.1 set as my DNS primary. This results in all Chromium browsers defaulting to Cloudflare unless DoH is explicitly disabled. Which I do, as I want browsers to use my OS DNS provider not DoH which I believe is just another potential failure point.

Possibly it resolves the bifrost IP to the wrong address. So do what I said and test what IP the browser thinks Bifrost is.

DoctorG

Name: bifrost.vivaldi.com
Address: 31.209.137.10

LazyLama

Possibly it resolves the bifrost IP to the wrong address. So do what I said and test what IP the browser thinks Bifrost is.

@Pathduck, so I put bifrost.vivaldi.com in the field on vivaldi://net-internals/#dns, this is what happens:

Resolved IP addresses of "bifrost.vivaldi.com": ["31.209.137.10"].
No alternative endpoints.

Still no syncing.

irz

Update:

No need to open https://bifrost.vivaldi.com/vivid-sync in Firefox

Just run curl https://bifrost.vivaldi.com/vivid-sync in terminal

Then sync is working again

I am still trying to find the cause of this issue

Pathduck

@LazyLama OK so this means Vivaldi is able to resolve the correct IP of Bifrost.
Did you do this after resolving the IP somewhere else, like from the shell/Firefox?

No need to open https://bifrost.vivaldi.com/vivid-sync in Firefox
Just run curl https://bifrost.vivaldi.com/vivid-sync in terminal

Here's why this happens (I think...)

Running curl in the shell will resolve the correct IP for Bifrost
The IP will be cached on the OS level for some time (no idea how long DNS records are cached)
The browser will use the OS DNS cache over its own cache/resolver (DoH)
After the DNS TTL expires, the OS will clear the cache entry and the browser will need to do another resolve, using DoH, and getting the wrong IP (possibly)

EDIT: Looks like Cloudflare sets a TTL for Bifrost at 120 seconds, not a lot but then again it's no problem just resolving it every time Sync is triggered maybe once every 10 mins or so.

λ dig bifrost.vivaldi.com @ns.cloudflare.com

; <<>> DiG 9.11.9 <<>> bifrost.vivaldi.com @ns.cloudflare.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36494
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;bifrost.vivaldi.com.           IN      A

;; ANSWER SECTION:
bifrost.vivaldi.com.    120     IN      A       31.209.137.10

;; Query time: 3 msec
;; SERVER: 173.245.58.100#53(173.245.58.100)
;; WHEN: Thu Sep 26 19:03:26 WEST 2024
;; MSG SIZE  rcvd: 64

Pathduck

Another thing to do is do a network capture. Here I'm using tshark (WireShark) on Win10 but the standard tcpdump has the same option. You might need to figure out what interface to capture on first by doing -D and then using -i to specify interface. Maybe on Linux it will always use eth0 by default, I dunno.

$ tshark -i4 host bifrost.vivaldi.com
Capturing on 'Ethernet'
    1   0.000000 192.168.0.10 62749 31.209.137.10 443 TCP 66 62749 → 443 [SYN] Seq=0 Win=65228 Len=0 MSS=1460 WS=256 SACK_PERM
    2   0.056274 31.209.137.10 443 192.168.0.10 62749 TCP 66 443 → 62749 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1418 SACK_PERM WS=512
    3   0.056378 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=1 Ack=1 Win=262144 Len=0
    4   0.057181 192.168.0.10 62749 31.209.137.10 443 TCP 1472 [TCP segment of a reassembled PDU]
    5   0.057181 192.168.0.10 62749 31.209.137.10 443 TLSv1 365 Client Hello (SNI=bifrost.vivaldi.com)
    6   0.113822 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=1 Ack=1730 Win=64000 Len=0
    7   0.115625 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 1472 Server Hello
    8   0.115735 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 1472 Certificate
    9   0.115735 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 219 Server Key Exchange, Server Hello Done
   10   0.115771 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=1730 Ack=3002 Win=262144 Len=0
   11   0.118955 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 180 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
   12   0.175262 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 312 New Session Ticket, Change Cipher Spec, Encrypted Handshake Message
   13   0.176734 192.168.0.10 62749 31.209.137.10 443 TCP 1472 [TCP segment of a reassembled PDU]
   14   0.176734 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 634 Application Data
   15   0.177850 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 422 Application Data
   16   0.233292 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=3260 Ack=3854 Win=64000 Len=0
   17   0.234169 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=3260 Ack=4222 Win=64512 Len=0
   18   0.262367 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 263 Application Data
   19   0.262367 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 329 Application Data
   20   0.262460 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=4222 Ack=3744 Win=261376 Len=0
   21   0.279622 192.168.0.10 62749 31.209.137.10 443 TCP 1472 [TCP segment of a reassembled PDU]
   22   0.279622 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 1472 Application Data
   23   0.279622 192.168.0.10 62749 31.209.137.10 443 TLSv1.2 584 Application Data
   24   0.335666 31.209.137.10 443 192.168.0.10 62749 TCP 60 443 → 62749 [ACK] Seq=3744 Ack=7588 Win=64000 Len=0
   25   0.372864 31.209.137.10 443 192.168.0.10 62749 TLSv1.2 394 Application Data
   26   0.416869 192.168.0.10 62749 31.209.137.10 443 TCP 54 62749 → 443 [ACK] Seq=7588 Ack=4084 Win=261120 Len=0

LazyLama

@Pathduck said in Vivaldi only syncing when on VPN:

Here's why this happens (I think...)

Running curl in the shell will resolve the correct IP for Bifrost
The IP will be cached on the OS level for some time (no idea how long DNS records are cached)
The browser will use the OS DNS cache over its own cache/resolver (DoH)
After the DNS TTL expires, the OS will clear the cache entry and the browser will need to do another resolve, using DoH, and getting the wrong IP (possibly)

That sounds like a reasonable hypothesis. It would explain why I and @irz are seeing. Below, you suggest a network capture. Would this show how Vivaldi resolves the IP address? Above, you mentioned using vivaldi://net-internals/#dns. Wouldn't that work as well?

Pathduck

@LazyLama said in Vivaldi only syncing when on VPN:

Would this show how Vivaldi resolves the IP address?

It wouldn't show how it resolves the IP. To do that you could capture DNS requests:

λ tshark -i4 port 53
Capturing on 'Ethernet'
    1   0.000000 192.168.0.10 49891 router.asus.com 53 DNS 79 Standard query 0x9b75 A bifrost.vivaldi.com
    2   0.000212 192.168.0.10 64669 router.asus.com 53 DNS 79 Standard query 0x74ec HTTPS bifrost.vivaldi.com
    3   0.002631 router.asus.com 53 192.168.0.10 49891 DNS 95 Standard query response 0x9b75 A bifrost.vivaldi.com A 31.209.137.10
    4   0.003298 router.asus.com 53 192.168.0.10 64669 DNS 138 Standard query response 0x74ec HTTPS bifrost.vivaldi.com SOA carl.ns.cloudflare.com

In this capture my PC asks my router for the IP of Bifrost, the router replies with the correct IP.

But it would show at least the request timing out, and to what IP it goes. But come to think of it, since the OS obviously resolves the IP correctly, it wouldn't actually capture any traffic going to the wrong IP.

Unless of course you capture all HTTPS traffic. But you'd get a lot of data. So close all browser tabs, close all other networked programs and restart the browser then run a
tcpdump dst port 443
and trigger the sync issue from chrome://sync-internals

Above, you mentioned using vivaldi://net-internals/#dns. Wouldn't that work as well?

Yes, and you did that and it resolved correctly. I don't know why it did.

Another thing to do:

Close all tabs
Open chrome://net-export
Click Start Logging...
Trigger the Sync issue and wait for the timeout
Stop the logging
Share the network capture (zip it first) or analyze it yourself

LazyLama

@Pathduck

I have done the following to test your hypothesis:

Restart the computer
Open Vivaldi and check whether I can sync. I could not.
Open vivaldi://net-internals/#dns. This resulted in Resolved IP addresses of "bifrost.vivaldi.com": ["31.209.137.10"]. No alternative endpoints
Run curl https://bifrost.vivaldi.com/vivid-sync.
Sign out of sync and sign back in to trigger syncing.
Syncing succeeds.

So, you seem correct.

Pathduck

@LazyLama said in Vivaldi only syncing when on VPN:

So, you seem correct.

Nope - because the resolved IP is the correct one. That works against my theory. It should be wrong. So maybe it's not DNS but something else on your system blocking the browser. Only you can know.

Or, at least I'm partly correct on the caching bit, as the act of running curl will resolve the correct IP, at least for a time. Or possibly just "open" something that was blocked.

It's also quite possible that the net-internals page does things differently and I don't know why.

Do a network capture from net-export and share it.

LazyLama

@Pathduck I did the tcpdump before and after running the curl command. So, without syncing working (before the curl command) and with syncing working (after curl).

The only difference I can see is that when things don't work, the host used differs from when things work. So, somehow, running the curl command changes how my ISP (in fact, my institution) handles my internet traffic.

So, what is see is before curl
ucfsu.xxx.xxx.xxx.56938 > 31.209.137.10.https
and
windows-j5s3qlt.xxx.xxx.xxx.34758 > 31.209.137.10.https
after curl.

(I have replaced some identifying parts of the strings with x's).

Maybe I should just write an sh script that runs the curl command before starting Vivaldi.

Pathduck

@LazyLama said in Vivaldi only syncing when on VPN:

in fact, my institution

So there's a clue, and a good one. You're on an enterprise network, probably proxied, firewalled and NAT'ed to hell and back.

Why it would change your client hostname I have no idea, you'd have to ask your IT people.
But the target IP and port is the same so should work no matter.
But if the client host is changed then the return packets have no way of making it back to the client over the NAT'ed firewall/proxy, and you'd get a timeout.

Talk to your IT people. If Vivaldi is at all supported by your company/institution. Knowing most IT depts. they will just say they don't support it and that's that. "We only support major browsers."

Still a capture from net-export would be interesting to see, but then again you X'd out your hostname so such a log would probably contain more than you're willing/allowed to share. So you'll need to analyze it yourself I guess

LazyLama

@Pathduck said in Vivaldi only syncing when on VPN:

Still a capture from net-export would be interesting to see, but then again you X'd out your hostname so such a log would probably contain more than you're willing/allowed to share. So you'll need to analyze it yourself I guess

I appreciate your offer. However, I am not sure whether my institute would appreciate me sharing these logs

irz

I have tried launching Vivaldi with the command:

vivaldi --host-resolver-rules="MAP bifrost.vivaldi.com 31.209.137.10"

but I still can't sync

In the same network environment, Vivaldi sync works fine on Windows OS

DoctorG

Could be a hardware firewall appliance which checks behavior of requesting connections by rules. Who knows.

irz

I compared the logs captured via chrome://net-export on Debian 12 and Windows under the same network environment.

After opening the log files on https://netlog-viewer.appspot.com, I noticed that in the DNS section, Debian 12 doesn't have any entries for bifrost.vivaldi.com, regardless of whether sync works properly.

On Windows, however, the DNS logs do include bifrost.vivaldi.com.

The records that are present in both logs include login.vivaldi.net, mimir2.vivaldi.com, and downloads.vivaldi.com.