Backup Search Engines

AltCode

@tam710562 So if we can use this to get the collection in 5.2 snapshots, what do we use in place of

chrome.storage.local.set({'SEARCH_ENGINE_COLLECTION':"....."})

to set it (and thus overwrite it)?

luetage

@altcode I’m just writing the restore function. It seems like there is no way to set the whole thing. But there is a way to remove and add engines. In the worst case scenario we can remove existing engines and add all engines from the backup array one by one and then set the default engine settings anew. Shouldn’t be too hard.

luetage

Following is a working version. It can backup your collection and it will restore the collection by first deleting all but the default engines (we can’t delete defaults) and then adding the backup. Manually setting new defaults and deleting the old defaults works out fine. If you want to try it, backup your profile first! Although this doesn’t seem to do any harm, I take no responsibility for lost data or a broken profile. This will not work with your old backups, create a new backup first.

// Backup Search Engines
// version 2022.3.2
// https://forum.vivaldi.net/post/277594
// Adds functionality to backup and restore search engines in
// vivaldi://settings/search.

(function () {
  function msg(print) {
    clearTimeout(msgTimeout);
    if (print === "backup") {
      info.innerText = "Backup copied to clipboard";
    } else if (print === "restore") {
      info.innerText = "Search engines restored";
    } else {
      info.innerText = "Code error, aborted";
    }
    msgTimeout = setTimeout(() => (info.innerText = ""), 5000);
  }

  function exec(collection) {
    vivaldi.searchEngines.getTemplateUrls((engines) => {
      const defaults = [
        engines.defaultSearch,
        engines.defaultPrivate,
        engines.defaultImage,
        engines.defaultSearchField,
        engines.defaultSearchFieldPrivate,
        engines.defaultSpeeddials,
        engines.defaultSpeeddialsPrivate,
      ];
      console.log(defaults);
      engines.templateUrls.forEach((engine) => {
        const id = engine.id;
        if (defaults.indexOf(id) === -1) {
          console.log(id);
          vivaldi.searchEngines.removeTemplateUrl(id);
        }
      });
      collection.templateUrls.forEach((entry) => {
        vivaldi.searchEngines.addTemplateUrl(entry);
      });
      msg("restore");
    });
  }

  function restore(e) {
    e.preventDefault();
    e.stopPropagation();
    let backupCode;
    let collection;
    if (e.type === "paste") {
      const clipboardData = e.clipboardData;
      backupCode = clipboardData.getData("text");
    } else {
      backupCode = e.dataTransfer.getData("text");
    }
    try {
      collection = JSON.parse(backupCode);
    } catch (err) {
      msg("error");
      return;
    }
    if (
      "defaultImage" in collection &&
      "defaultPrivate" in collection &&
      "defaultSearch" in collection
    ) {
      exec(collection);
    } else {
      msg("error");
    }
  }

  function backup() {
    vivaldi.searchEngines.getTemplateUrls((engines) => {
      const backupCode = JSON.stringify(engines);
      navigator.clipboard.writeText(backupCode);
      msg("backup");
    });
  }

  function ui() {
    const check = document.getElementById("vm-backup");
    if (!check) {
      const place = document.querySelector(
        ".setting-section > div > .setting-group.unlimited > .setting-single"
      );
      const backupBtn = document.createElement("input");
      backupBtn.setAttribute("type", "button");
      backupBtn.setAttribute("value", "Backup");
      backupBtn.id = "vm-backup";
      place.insertBefore(backupBtn, place.lastChild);
      backupBtn.addEventListener("click", backup);
      const restoreInput = document.createElement("input");
      restoreInput.setAttribute("type", "text");
      restoreInput.setAttribute("placeholder", "Restore Backup");
      restoreInput.id = "vm-restore";
      place.insertBefore(restoreInput, place.lastChild);
      restoreInput.addEventListener("paste", restore);
      restoreInput.addEventListener("drop", restore);
      info = document.createElement("span");
      info.id = "vm-msg";
      place.insertBefore(info, place.lastChild);
    }
  }

  const css = `
    #vm-restore {
      width: 130px;
      margin-left: 6px;
      margin-top: 6px;
    }
    #vm-restore::-webkit-input-placeholder {
      opacity: 1;
      color: var(--colorHighlightBg);
      text-align: center;
    }
    #vm-msg {
      margin-left: 12px;
    }
  `;

  let msgTimeout;
  const settingsUrl =
    "chrome-extension://mpognobbkildjkofajifpdfhcoklimli/components/settings/settings.html?path=";
  chrome.tabs.onUpdated.addListener(function (tabId, changeInfo, tab) {
    if (changeInfo.url === `${settingsUrl}search`) {
      setTimeout(ui, 100);
      const check = document.getElementById("vm-engines");
      if (!check) {
        const style = document.createElement("style");
        style.id = "vm-engines";
        style.innerHTML = css;
        document.getElementsByTagName("head")[0].appendChild(style);
      }
    }
  });
})();

My goal is to set new defaults after restoring the engines and deleting the old defaults, for complete automation. The api seemingly provides a way to set default engines, but I somehow keep messing up the syntax. Here one simple example of what I tried: vivaldi.searchEngines.setDefault({defaultImage: "24"}, () => console.log("success"));. The first argument is the DefaultType, the second argument is the ID of the new default engine and it’s supposed to work like this: searchEngines.setDefault(searchEngines.DefaultType defaultType, string id, function callback). I’ll keep trying, but if someone should figure this out, please share. I’ll check again tonight.

edit: figured it out, the syntax is vivaldi.searchEngines.DefaultType.DEFAULT_IMAGE, now I can hopefully automate this fully…

iAN CooG

@luetage Thanks for your work, I'll try it now. They just announced new snapshot will revert the changes and reset all the engines to defaults, so better start backupping them now with this new script.
EDIT: thinking about it, I made a backup of the "Web Data" db3, I'll eventually restore the "keywords" table from it, way faster.

mib2berlin

@luetage
Hi, I use this script for the first time and after change and start Vivaldi I cant choose Search in settings. It blinks for a second and revert to the last item.
After restart Vivaldi it open the search settings but the entry for Export/import in not there.
Vivaldi 5.2.2623.4 on Linux.

Cheers, mib
EDIT: Hehe, this is not a .css script, changing to .js working fine now, thank you.

luetage

@mib2berlin Hmm, doesn’t sound good. But since the script works on fresh load of the search settings, switching to general settings and back to search settings should trigger the UI. In your case it’s likely the page loads before the script has a chance to add the listener.

mib2berlin

@luetage
I edit my post, thanks.

luetage

@ian-coog You can make the backup in any case, this gives you a clean json with the engines. The restore script will be done before the future snapshot comes out and you can use it then to get your engines back.

luetage

Ok, managed to automate setting of default engines, new version is up in first post and should work very well with the new backend. Haven’t had a single crash all this time, so we should be good. Still, profile backups are a good idea in any case.

Pathduck

@luetage Seems to work for backup at least

Backed up profile (and Web Data just in case)
Exported as JSON
Reset engines
Restarted Vivaldi just to make sure
Imported JSON

Restored favicons (including the ones I manually added to the DB), sorting, even added the Suggest URL for Google (default missing).

One strange thing though - got a duplicate "bing" keyword. I use b for Bandcamp.
I'm assuming that prepopulateId != 0 means it's the default one.

{
	"extensionId": "",
	"faviconUrl": "https://www.bing.com/sa/simg/bing_p_rr_teal_min.ico",
	"id": "3",
	"imagePostParams": "imageBin={google:imageThumbnailBase64}",
	"imageUrl": "https://www.bing.com/images/detail/search?iss=sbiupload&FORM=INCOH2{bing:Referral}#enterInsights",
	"keyword": "bing",
	"name": "Bing",
	"postParams": "",
	"prepopulateId": 3,
	"readOnly": false,
	"suggestPostParams": "",
	"suggestUrl": "https://www.bing.com/osjson.aspx?query=%s&language={language}",
	"url": "https://www.bing.com/search?q=%s&FORM=INCOH2{bing:Referral}"
},
{
	"extensionId": "",
	"faviconUrl": "https://www.bing.com/sa/simg/bing_p_rr_teal_min.ico",
	"id": "78",
	"imagePostParams": "imageBin={google:imageThumbnailBase64}",
	"imageUrl": "https://www.bing.com/images/detail/search?iss=sbiupload&FORM=INCOH2{bing:Referral}#enterInsights",
	"keyword": "bing",
	"name": "Bing",
	"postParams": "",
	"prepopulateId": 0,
	"readOnly": false,
	"suggestPostParams": "",
	"suggestUrl": "https://www.bing.com/osjson.aspx?query=%s&language={language}",
	"url": "https://www.bing.com/search?q=%s&FORM=INCOH2{bing:Referral}"
},

I also notice there are several values present in the Web Data table not exported in JSON. Here's the full column names list from the keywords table:

"id", "short_name", "keyword", "favicon_url", "url", "safe_for_autoreplace", "originating_url", "date_created", "usage_count", "input_encodings", "suggest_url", "prepopulate_id", "created_by_policy", "last_modified", "sync_guid", "alternate_urls", "image_url", "search_url_post_params", "suggest_url_post_params", "image_url_post_params", "new_tab_url", "last_visited", "created_from_play_api", "is_active", "position"

Especially the position value is one I still wonder what does. It's shown as a BLOB in the tool, data type is VARCHAR - it's just a string of random hex values, not ASCII.

safe_for_autoreplace = 1 also seems to indicate a default engine.

BTW - I'm assuming this won't work for the old version? You might want to link to the old version on Github or somewhere in case there's a need to make a backup using the data types.

luetage

@pathduck Thanks for checking it out. This will only store data provided by the api, which should be enough. Starting from a fresh and clean backup, with neither errors nor duplicates, the restore functionality should be able to get back to the same clean state you had when making the backup. The only thing that will change are the IDs in the background, because Vivaldi keeps count of them.

Pathduck

@luetage Thanks

By the way, would you have a "quick" one-liner or basic method to just paste into the console to generate the backup, without the UI?

Sometimes I find it more convenient than having the mod loaded at all times

I tried just running the backup() function into the console but apparently it's not enough.

luetage

@pathduck You have to leave out the msg part and the function itself, or you at least got to call the function. But to be honest I didn’t get it to work when I tried it either just now, it complains the document is not focused… I know the old code worked when we put it in the console. Not sure. On the other hand I’m not too concerned about it, it works just fine when issued from the UI itself. Might be they changed something in the backend so it isn’t possible any longer.

edit: this works, but you got to copy it out of the console manually…

vivaldi.searchEngines.getTemplateUrls((engines) => {
  console.info(JSON.stringify(engines));
});

luetage

I made a few updates over the last couple of days. The new version restores all search engines now, including search field, speed dials and the private versions of both. This should be good to go once everything settles in and the new version is up again for snapshot and in future for stable.

ukanuk

FYI search engine sync still isn't in the latest 5.2.2623.24 (Stable channel) (64-bit), see blog post https://vivaldi.com/blog/desktop/syncable-search-put-on-hold-for-now-vivaldi-browser-snapshot-2623-8

mib2berlin

@ukanuk
HI, it was rejected for 5.2, we hope they add it back in one of the next snapshots for 5.3.

Cheers, mib

luetage

@mib2berlin Yeah, I generally develop towards snapshot, as this is the version most users who mod are running. That the feature got taken out again was a surprise, but I’ll keep the mod up until new search will be implemented again, no point in going back now. We’ll have to wait and see, maybe they do something that will need fixing down the line.

aneesamjad

@luetage Hello and good day to you,

can you please confirm if this mod is still working in Vivaldi 5.2 I have been trying this but isn't working for me anymore.

luetage

@aneesamjad Search engine changes were rolled back for 5.2.

aneesamjad

@luetage so, I tried this version "2022.4.0" on multiple times, on multiple machines but it didn't worked. Fortunately I had version version "2021.10.0" available somewhere which worked flawlessly as expected.