Solved Guide | Search Suggestion URL

ukanuk

@luetage, @QuHno, and @Pathduck shared some really helpful info about POST queries. Unfortunately I still wasn't able to create a query for Deezer.

luetage

@ukanuk Post a link to the search? Deezer seems to be some kind of music portal you need an account to access… so you might be on your own until someone with access comes along.

ukanuk

@luetage you can register on Deezer for free, but you don't actually need any account just to search and see suggestions automatically appearing as you type in their search bar. The search URL is https://www.deezer.com/search/%s, an example search is https://www.deezer.com/search/stuff, but we are having trouble finding/creating the suggest URL.

See these previous posts, https://forum.vivaldi.net/post/530664 where you @stardepp originally asks about it and https://forum.vivaldi.net/post/560897 where I make an initial investigation and find the POST request but don't know how to convert it into a GET request.

Here's Deezer without being logged in, and suggestions appearing:
Screenshot_20220806-112922_Vivaldi Browser.jpg

luetage

@ukanuk Oh, it’s about suggest urls… that’s something different. I always delete those and never enter them, so I have zero experience. Can’t help you there.

Pathduck

@ukanuk Yeah, suggestions not going to work for that one I'm afraid.

First of all, suggestions from the site search field uses POST and an API token, usually this is to avoid "abuse" from outside requests:

Request URL: https://www.deezer.com/ajax/gw-light.php?method=search_getSuggestedQueries&input=3&api_version=1.0&api_token=p4bCVtDQq8wuYexil0T2QeH1SGQCD12L&cid=92442938
Request Method: POST

Search results are not in a format Vivaldi can use (custom JSON, not OpenSearch standard):

{"error":[],"results":{"SUGGESTION":[{"QUERY":"The Afghan Whigs","HIGHLIGHT":"The Afghan Whigs"},{"QUERY":"the afghan whigs","HIGHLIGHT":"the afghan whigs"},{"QUERY":"the afghan whigs \u2013 congregation","HIGHLIGHT":"the afghan whigs \u2013 congregation"},{"QUERY":"the afghan whigs \u2013 a line of shots","HIGHLIGHT":"the afghan whigs \u2013 a line of shots"},{"QUERY":"the afghan whigs \u2013 algiers","HIGHLIGHT":"the afghan whigs \u2013 algiers"},{"QUERY":"the afghan whigs \u2013 gentlemen","HIGHLIGHT":"the afghan whigs \u2013 gentlemen"},{"QUERY":"the afghan whigs \u2013 do to the beast","HIGHLIGHT":"the afghan whigs \u2013 do to the beast"},{"QUERY":"the afghan whigs \u2013 my curse","HIGHLIGHT":"the afghan whigs \u2013 my curse"},{"QUERY":"the afghan whigs \u2013 if i were going","HIGHLIGHT":"the afghan whigs \u2013 if i were going"},{"QUERY":"the afghan whigs \u2013 in spades","HIGHLIGHT":"the afghan whigs \u2013 in spades"}],"HISTORY":[]}}

For those engines I just copy the DDG search suggestion URL, works fine as a workaround.

wudini

Suggestions for startpage don't work anymote, here is the new URL.

https://www.startpage.com/suggestions?q=%s&segment=startpage.vivaldi&format=json

ukanuk

@wudini Updated the table, thank you!

stardepp

Neeva search engine:

Search URL:

https://neeva.com/search?q=%s

Search Suggestion URL:

https://neeva.com/suggest?q=%s&src=opensearch&pk_source=vivaldi&pk_medium=external_product&pk_campaign=neeva_vivaldi_o

https://neeva.com

JonesBones

Couple others that I do not see mentioned. Neeva looks very interesting by the way.

1 - Brave - https://search.brave.com

Search

https://search.brave.com/search?q=%s

Suggestion

https://search.brave.com/api/suggest?q=%s

2 - mojeek - https://mojeek.com

Search

https://www.mojeek.com/search?q=%s

Engine has no suggestions

3 - https://you.com

Search

https://you.com/search?q=%s

Suggestion

https://you.com/api/ac?q=%s

4 - searchcode - https://searchcode.com/

Search

https://searchcode.com/?q=%s

Engine has no suggestions

5 - Openverse (formerly CC Search) - https://wordpress.org/openverse/

Search

https://search-production.openverse.engineering/search/?q=%s

Engine has no suggestions

ukanuk

Can anyone find suggest URLs for mojeek, searchcode, or Openverse (formerly CC Search)?

@JonesBones Thank you! I've added Brave and You to the table above with your suggest URLs, albeit I used https://you.com/search?q=%s&utm_source=vivaldi_browser instead of https://you.com/search?q=%s to help support Vivildi's revenue stream.

There's already a great existing pinned guide post by @stardepp at Guide | Vivaldi search engines collection for sharing search engines in general. My table above is only special for including every suggest URL I can find, and search engines I personally use (see "Inclusion Critera" in my post).

Pathduck

@ukanuk said in Guide | Search Suggestion URL:

Can anyone find suggest URLs for mojeek, searchcode, or Openverse (formerly CC Search)?

There are none
Just use DDG or whatever functioning suggestions string you want.

stardepp

pranjalKumar

@ukanuk
Thank you
It really helped me solve most of my advanced queries🦚

mib2berlin

@geek4hire62511
Hi, I test this with the Google page itself and even Google cant find files on my Google drive set to public.
Or do I not understand your question?

Cheers, mib

Pathduck

@geek4hire62511 Should be as simple as:
https://drive.google.com/drive/search?q=%s

Since it seems to use GET it shows up in the url so easy to make into a search engine.

You're not going to get search suggestions working.

Pathduck

@geek4hire62511 Ah yeah, missed that

If the site does not support the OpenSearch standard for suggestions, it's not possible.

Usually you can check this yourself by looking at the page source.
There should be a link element with a type of application/opensearchdescription+xml pointing to an XML file.

Interestingly, Google Drive does have an OS XML - but no suggest URL.
https://drive.google.com/opensearch.xml

<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
	<ShortName>Google Drive</ShortName><LongName>Google Drive</LongName><Description>Search the contents of your Google Drive.</Description>
	<Url type="text/html" method="get" template="https://drive.google.com/drive/search?q={searchTerms}"/>
	<InputEncoding>UTF-8</InputEncoding>
	<Image height="16" width="16" type="image/png">https://ssl.gstatic.com/docs/doclist/images/infinite_arrow_favicon_4.ico</Image>
</OpenSearchDescription>

Compare to for instance Ecosia:
https://cdn-static.ecosia.org/static/opensearch.xml

<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
	<ShortName>Ecosia</ShortName>
	<Description>Search Ecosia</Description>
	<InputEncoding>UTF-8</InputEncoding>
	<Contact>[email protected]</Contact>
	<LongName>Ecosia Search</LongName>
	<Image width="16" height="16"> data:image/x-icon;base64,AAABAAMAEBAAAAE...AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA </Image>
	<Url type="text/html" method="get" template="https://www.ecosia.org/search?q={searchTerms}&addon=opensearch"/>
	<Url type="application/x-suggestions+json" template="https://ac.ecosia.org/autocomplete?q={searchTerms}&type=list"/>
</OpenSearchDescription>

This can be tested with curl for instance:

$ curl "https://ac.ecosia.org/autocomplete?q=test&type=list"
["test",["test","testosterone","testing","test microphone","testament","testout","testbook","testicular cancer"]]

For more on OpenSearch:
https://en.wikipedia.org/wiki/OpenSearch
https://developer.mozilla.org/en-US/docs/Web/OpenSearch

That's the beauty of open standards; you can use it to figure out how to add search engines (and suggestions if supported) yourself

Alternatively, one can use Devtools to examine outgoing requests, for instance if the site uses POST for suggestions and create your URL from that. But won't work everywhere.

Pathduck

@geek4hire62511 What page? This forum? Some sites use fancy JS instead of proper HTML forms I guess.

OSD file is here:
https://forum.vivaldi.net/osd.xml

Of course anything that submits data can be used as a "search engine" in Vivaldi. URL shorteners, Translators, basically anything that has an input argument and produces some output.

Pathduck

@geek4hire62511 That site works just like the Gdrive one, you perform a search, look at the resulting URL and replace your search term with %s.
https://explainshell.com/explain?cmd=%s

It's got a pretty standard form that calls /explain with the cmd as argument.

<form action='/explain' method="get">
  <input id='explain' class="span10" type="text" name="cmd" autocapitalize="off" autocorrect="off"></input>
  <button class="btn" type="submit">EXPLAIN</button>
</form>

Again, you don't need to understand how the site does the search to add as a search engine. Just look at the resulting URL.

Barruel

@ukanuk said in Guide | Search Suggestion URL:

Several suggest URLs exist, but none currently work with Vivaldi

y

Yahoo!

• ~~http://sugg.search.yahoo.net/sg/?output=jsonp&nresults=10&command=%s~~
• ~~http://sugg.search.yahoo.net/sg/?output=json&nresults=10&command=%~~
• ~~http://sugg.search.yahoo.net/sg/?output=xml&nresults=10&command=%s~~

https://us.search.yahoo.com/yhs/search?hspart=iry&hsimp=yhs-fullyhosted_009&type=dpp_vvldnu_00_00&param1=1&param2=pa%3Ddowncoll%26b%3DVivaldi&p=%s
• [Obsolete] This URL was the previous official version (I noticed the change in 4.3.2439.63, but it probably happened earlier):
•~~https://us.search.yahoo.com/yhs/search?hspart=iry&hsimp=yhs-fullyhosted_009&type=dpp_vvldnu_00_00&param1=1&param2=pa%3Ddowncoll&p=%s~~

Hello.

Got Yahoo! suggestions working nicely.

URL:

https://search.yahoo.com/search?p=%s&ei=UTF-8&guccounter=1

Suggest URL:

https://ff.search.yahoo.com/gossip?output=fxjson&command=%s

I've got it looking at PaleMoon headers. PM is actually a quite old Firefox, so that "ff" at the beginning of the url is targeting it and Vivaldi happily understands the string sent to it:

curl "https://ff.search.yahoo.com/gossip?output=fxjson&command=testword"
["testword",["typing test words per minute","spelling test words","pacer test words","typing speed test words per minute","test words per minute","practice typing test words per minute","free online typing test words per minute","free typing test words per minute","online typing test words per minute","test word"],[],[]]

It's noteworthy that

https://search.yahoo.com/gossip?output=fxjson&command=%s"

throws a 404 error, but

https://ff.search.yahoo.com/gossip?command=%s"

sends the correct reply when using curl, but Vivaldi doesn't show the suggestions if the "output=fxjson" part of the string is missing.

Pathduck

@Barruel Good finding!

It's a big problem with these search engines - no OpenSearch definitions and we have to guess from the basics to try and find a working suggest url. Especially with st00pid sites like Yahoo thinking OpenSearch = Firefox ... - no, it's a standard they should follow...

With a bit of tweaking and a web search I found another working url:
https://search.yahoo.com/sugg/ff?output=fxjson&command=%s

$ curl "https://search.yahoo.com/sugg/ff?output=fxjson&command=test"
["test",["test internet speed","test keyboard","testosterone","test book","test mic","test camera","testicular cancer","test microphone","testosterone supplements","test key"],[],[]]

This has the advantage it can be used for language-specific results:

$ curl "https://no.search.yahoo.com/sugg/ff?output=fxjson&command=test"
["test",["test","testosteron","test internet speed","test mobiltelefoner","test-in-prod","testament","testikkelkreft","test vinterdekk","test pinnekjøtt","test varmepumper"],[],[]]

$ curl "https://de.search.yahoo.com/sugg/ff?output=fxjson&command=test"
["test",["test","teste dich","testberichte","testament","testosteron","testzentrum","test digitalkameras","test navigationsgeräte","test kaffeevollautomat","test smartphones"],[],[]]

$ curl "https://se.search.yahoo.com/sugg/ff?output=fxjson&command=test"
["test",["test","testamente","testosteron","test mobiltelefoner","test dammsugare","test robotgräsklippare","test vinterdäck","test kaffebryggare","test tvättmaskiner","testikelcancer"],[],[]]

We don't know how long ff.search.yahoo.com will exist. Then again we don't know how long they will continue supporting "ffjson" as a format.

Then again, I don't use Yahoo as it sucks as a search engine