Importing huge HTML bookmarks does nothing, and claims success quickly.

ThePythonicCow

I am trying to migrate my 26,514 bookmarks from Firefox to Vivaldi, in HTML format.

I can successfully import smaller HTML bookmark files into Vivaldi. The biggest one that I've successfully imported had 6517 bookmarks (the *.html file size was 10253676 bytes, or 9.8 MBytes).

The larger sizes that I tried all failed, quickly and quietly, announcing the import is "Done" (in green font) within a fraction of second after I click the "Open" button, on the selected HTML bookmark file in the file selection screen, as popped up from the "Import Bookmarks and Settings" screen.

When I run the Linux system call tracer strace(2) on these failing imports, I notice that Vivaldi tries to accomplish the read(2) system call in a single, massive, read, and then I see some timeouts occurring, a few tenths of a second (if I remember exactly) later, before the read(2) completes. If this strace(2) is critical, let me know, and I'll report with greater attention to accuracy and detail. There seemed to be multiple co-ordinating tasks involved, so I'll have to take an extra cup of coffee before reliably reporting on just what happens.

This massive, apparently timed out, read(2) system call occured on a test using a 24571016 byte (23 MBytes) HTML file.

My original HTML file (with more icon data, but the same number of bookmarks) fails at the user interface level in the same way, but I've not run strace(2) on that case. That largest HTML file has 106921443 bytes (102 MBytes).

I'm running on an 8 core Ryzen with 48 GBytes of RAM, and loading these files off a PCIe 3 attached NVME.

A Former User

Can you import the bookmarks if you strip all icons &c? The size limitation still has to be resolved, but you’d have your bookmarks…

ThePythonicCow

@potmeklecbohdan

<< Can you import the bookmarks if you strip all icons &c? >>

Good idea. I might try that.

There are some other trimmings I can do, in order to "search" the space between sizes of 9.8 MBytes that loaded OK, and 23 MBytes that failed. I might explore these.

If I learn more, I'll report back here, both to further refine this "bug" investigation, and to potentially provide me with a sufficiently "good enough" migration of my bookmarks.

Hmm ... I could do a RAM disk, these sizes more or less fit in my RAM. NVME is fast, but it can slow down over time, as Steve Gibson has been reporting the last couple of weeks on his "Security Now" program. RAM is faster, and stays faster.

My current speculation is that there are two problems showing up here:

A strict "time to load from disk" (depends on file size and disk speed), versus wall clock time due to some programmed timeout values.
A failure to report back to the user any sign of failure, not even a cryptic "Failed. Error code D9YKWJ."

That D9YKWJ error code is a six char random alpha-numeric string, over the 24 char set of easily distinguished letters and numbers. This allows correlating customer problem reports with source code, years later, when no person, and no code, has a clue what broke or why. I've long found this to be a useful tactic, when writing error messages that I can't express in any more useful manner at the time.

(earlier versions of this post failed to post, with Error: Post content was flagged as spam by Akismet.com, perhaps because I named the source of that 24 char set, a well known product of a very well known large and long standing PC computer software company in Seattle.)

A Former User

@ThePythonicCow said in Importing huge HTML bookmarks does nothing, and claims success quickly.:

(earlier versions of this post failed to post, with Error: Post content was flagged as spam by Akismet.com)

That’s okay, IIRC it’s very strict when you have reputation lower than 2 (but it could be a different number). Nothing you’ve done wrong.

ThePythonicCow

@potmeklecbohdan said in Importing huge HTML bookmarks does nothing, and claims success quickly.:

Nothing you’ve done wrong.

I'd do much the same strict checking myself, were I running this site. All is well.

ThePythonicCow

Aha - I loaded my 24571016 byte (23 MBytes) HTML file, with almost all my bookmarks, but more or less no icons, Easily!

No need to futz around with a RAM disk.

Just use the Linux kernel buffer cache:

Unload my other memory hogs I had running (Firefox, et al).
Pull the HTML bookmark into my buffer cache (48 GBytes of RAM allows for a very large buffer cache) by reading the file with some other random app (I ran the command 'b2sum' on the HTML file a couple of times).
Open the HTML file off the "Import Bookmarks and Settings" screen.

This time, when I clicked the "Open" button on that 23 MByte HTML file, instead of seeing the (false) "Done" immediately (faster than my eye could discern) rather I saw some message that said something equivalent to "import in progress" for a few tenths of a second, then got the (legitimate) "Done".

I then checked, and all (or at least most -- need to inventory closer) my just imported bookmarks are loaded. Yeah!

The above is consistent with my prior speculations above -- the initial read(2) system call is failing on these big HTML files, unless that read can be completed quickly. Since reading a file already in the buffer cache (in RAM) is much quicker than reading from (any "normal") external storage, one can read a bigger HTML bookmark file this way.

P.S. However -- another problem appears. It seems that my bookmark hierarchy is somewhat smashed. Some folders that should be a level or two below the top, are appearing at the top level. I have no idea yet whether this is something (stupid or covertly genius) that I did, or something imperfect in the Firefox export, or something imperfect in the Vivaldi import, or some confusion in my initial hasty inventory of the imported results. I'll look further.

ThePythonicCow

@ThePythonicCow << It seems that my bookmark hierarchy is somewhat smashed. >>

This turns out to be my problem, not Vivaldi's.

My bookmark collection on Firefox had massive duplication and confusion, hidden beneath the portions of the bookmark folder hierarchy that I probably had not visited in years.

Whether or not the Firefox export + Vivaldi import managed to exactly duplicate the smoldering dumpster fire that was my bookmark collection is probably not worth investigating or worrying about, unless some masochistic Vivaldi bookmark importing developer is getting way too bored. Let me know if you're that person <grin>.

I will next perform some major trimming of my Firefox collection, and try moving that over to Vivaldi. So far as I know now, that effort should proceed OK, barring more surprises, of course.

A Former User

@ThePythonicCow Now that you have so much information, would you please report it to the bug tracker?

ThePythonicCow

@potmeklecbohdan << report it to the bug tracker? >>

Ah - thanks for the suggestion and link! I'll put that on my queue for today.

... right behind getting a cup of coffee and recovering my Firefox bookmarks .

Last night I made sure to use the Firefox menu option to backup my bookmarks (and also verified that Firefox automaticallly backs up daily anyway.)

Then I did some major pruning of the html formatted bookmark file and asked Firefox to re-import the trimmed results, totally replacing whatever bookmarks it thought it had. That html import succeeded, but I didn't like the results. I had trimmed too much.

So I asked Firefox to restore its latest json formatted bookmark backup file.

Then I went to bed, while the Firefox bookmark import went to work.

After eight hours of my sleeping, and four hours of accumulated CPU time ... Firefox was still churning away on the import ... of its own pure, unmodified, backup json backup file.

... So it's not just Vivaldi that can't import my Firefox bookmarks ... it's also Firefox <grin>.

Fortunately, I have multiple flavors of backups (that's kind of my hobby). So all is well here ... I just relate the above for a bit of amusement.

(P.S. -- I just "discovered" the "backup key goes to the previous web page" feature of Vivaldi, when the cursor is not in an editable form field. I cast my virtual vote to the winds to change Vivaldi to not do that as the default.)

A Former User

@ThePythonicCow said in Importing huge HTML bookmarks does nothing, and claims success quickly.:

(P.S. -- I just "discovered" the "backup key goes to the previous web page" feature of Vivaldi, when the cursor is not in an editable form field. I cast my virtual vote to the winds to change Vivaldi to not do that as the default.)

Upvote the first post here pls

ThePythonicCow

@potmeklecbohdan said in Importing huge HTML bookmarks does nothing, and claims success quickly.:

Upvote the first post here pls

Upvote done, and my own two cents added in a couple more posts on that thread.

ThePythonicCow

ThePythonicCow

OK - my (trimmed down to a mere 14,000, w/o icons) bookmarks are alive and well - in both Firefox and Vivaldi.

Now to consider what useful bug I might file.

ThePythonicCow

Now I'm having trouble reproducing the bookmark loading problem. Even my biggest, baddest HTML bookmark file, exported from Firefox, loads into Vivaldi just fine.

Next, I'll have to see if one of the few extensions that I had loaded is needed to reproduce this.

ThePythonicCow

Well dang - I cannot get HTML bookmark import to fail.

Unless I can find a way, consider this "bug" closed before it was even opened - Unable to reproduce.