Duplicated bookmark detection / cleanup



  • @bonetone I used to run Python script that I wrote especially for that but because notes sync is incomplete (without attachments) and clones have usually random attachments it is not so easy to find duplicates sometimes...
    I just got tired with hunting clones every week and stopped using notes



  • @ROTFL Yeah, I find OneNote to be superior to the built-in Notes Panel anyways. Plus, I've long been able to sync that with Android. So I prety much keep all my notes in that. Still running OneNote 2013 on Windows, but considering installing OneNote 2016. I definitely prefer those versions over the one included in Windows due to the maturity of the feature set. I think that the UWP version will approach parity eventually, just not yet.

    After spending several years using and evaluating a plethora of note-taking software, I settled on OneNote as the best solution in terms of richness of formatting, elegance of organization, ease of sharing & synchronization, integration with other programs, and multiplatform support. Since then OneNote has only gotten better, and distanced itself further from the competition. To top it all off, it is free and available to pretty much everyone whether they use Windows, Linux, Mac, iPhone or Android.

    I had a small number of notes in Vivaldi, ~25 or so, but as soon as I saw the duplication issue happen and having read the posts of people experiencing such large numbers of duplicates, I recently migrated them all into OneNote.



  • @ROTFL
    Now that I think about it, it would probably be easier to write something that works on the data in the Notes file directly. Obviously work on a backup copy of the file... but the structure of the data should make it fairly easy to do.

    I just did a quick analysis of my Notes file, which currently does have at least 1 duplicate, and they have different IDs. Simply keep the one with the smallest ID number and delete all the other dupes.



  • @bonetone

    • ID is just ID and means nothing but identifier. Sometimes creation date is more important but not always
    • As I said problem is more complicated when you have partially synced attachments between computers and notes with random attachments and older versions of notes and other chaotic artifacts of a buggy sync...


  • Finally got the time to finish the tool I was working on, an initial version is published at https://github.com/kamenminkov/vivaldi-bookmarks-dedupe
    Feedback is very welcome. The project contains a detailed readme, let me know if you think something is missing.

    Still figuring out the best way to do it as Vivaldi sometimes likes to still duplicate the bookmarks after syncing. Found out that renaming a folder that duplicates appear in, then letting that folder appear twice, then deleting the one with the duplicates kind of helps. Also found out that it's probably a good idea to reset sync, close the browser, copy the cleaned up bookmarks, then sign in again.



  • @killchain I'm definitely going to look at this in detail when I get a little time. I am also developing a bookmark management tool, so we could possibly help each other out with this.



  • @bonetone Yeah, it would be cool 🙂



  • @killchain BTW, I have not found any issues with duplicates being created from the sync process. And I've got over 80,000 unique bookmarks. I recently accidentally deleted a major folder, and to recover it I simply imported a recent exported html bookmarks file and then deduped. I had forgotten to turn off sync beforehand, and so I had like 160k bookmarks sync'ing between a few desktop and 2 Android Vivaldi instances.


Log in to reply
 

Looks like your connection to Vivaldi Forum was lost, please wait while we try to reconnect.