Spoofing the referrer does not work anymore?

[Beholder]

I made my own extension and also there are a few more advanced extensions to spoof the referrer on an arbitrary website. This obviously used to work but NOW it does not seem to work. I have tested latest Vivaldi and latest Chrome. Something has changed which prevents the extensions to spoof the referrer. Here is code of my own background script, this method used to work, now it does not:

chrome.webRequest.onBeforeSendHeaders.addListener(
    function(details) {
		var headers = details.requestHeaders;
        for (var i = 0; i < headers.length; ++i) {
			console.log(headers[i].name);
			if (headers[i].name === "Referer") {
				var url = new URL(details.url);
				console.log(headers[i].value);
				headers[i].value = "http://www.mysite.com";
				return { "requestHeaders": headers};
			}
		}
		headers.push({'name': "Referer", 'value': "http://www.mysite.com"});
		return { "requestHeaders": headers };
	},
	{ urls: ["http://*/*", "https://*/*"] },
	[ "blocking", "requestHeaders", "extraHeaders" ]
);

Can anyone please confirm that referrer modifying extensions installed on latest Vivaldi are not able to spoof the referrer? Here you can easily get to them: https://www.google.com/search?q=spoof+referrer+chrome+extension

[Pathduck]

I tried this one:
https://chrome.google.com/webstore/detail/referer-control/hnkcfpcejkafcihlgbojoidoihckciin

Then checked with:
http://browserspy.dk/headers.php

Referer header was set to what I selected in the extension, or blocked if set.

So likely something in your code that needs to be updated, but I have no idea what. Maybe some of the JS experts here can help out

[luetage]

@Pathduck No idea, “Referer” should really be spelt with 2 r in the middle?

[Komposten]

@luetage said in Spoofing the referrer does not work anymore?:

@Pathduck No idea, “Referer” should really be spelt with 2 r in the middle?

The header is called "Referer" with 1 r.

[luetage]

@Komposten Then I stick with no idea ^^

[Beholder]

@Pathduck Hey, thanks for testing. I guess something is either wrong with MY vivaldi or perhaps we're using different versions, since NONE of the referrer spoofing apps works here. Believe me, yesterday I spent 5 hours on this problem with no results. Can you please post here your version of Vivaldi and also the screenshot of the Referer Control extension options, so I can see how you set it up? I will try to reset the extension to your setting and see if it works somehow.

[Pathduck]

@Beholder I use the latest version of Vivaldi Stable to test:

Vivaldi	3.1.1929.45 (Stable channel) (64-bit)
Revision	1eb3263017ed42270818939fbff241845938a81f
OS	Windows 10 OS Version 1909 (Build 18363.900)
JavaScript	V8 8.3.110.13
Flash	32.0.0.363 C:\WINDOWS\system32\Macromed\Flash\pepflashplayer64_32_0_0_363.dll
User Agent	Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.119 Safari/537.36
Command Line	"D:\bin\Vivaldi-standalone\Application\vivaldi.exe" --vivaldi-standalone --flag-switches-begin --flag-switches-end --enable-audio-service-sandbox --ppapi-flash-path="C:\WINDOWS\system32\Macromed\Flash\pepflashplayer64_32_0_0_363.dll" --save-page-as-mhtml

Referer Control settings, only thing I changed for test was to Custom and set URL:

Result on https://www.whatismybrowser.com/detect/what-http-headers-is-my-browser-sending

HOST	www.whatismybrowser.com
REFERER	https://www.nsa.gov/

What sites are you using to test? I struggled with the browserspy site now because it only answers over http. Note that you need to actually come from a site that allows to set a referer header, like a search engine.

What other extensions are you running?
Try in a clean profile to exclude extensions and settings.

[Beholder]

Well, this should work on ANY sites actually. I managed it to work a bit just like you did, by using Referer Control and its "default referer for all sites" setting. That was the only way I made it work and it works in a limited way on a limited subset of pages, for example the one you posted: https://www.whatismybrowser.com/detect/what-http-headers-is-my-browser-sending

I test it mostly on keep2share sites like k2s.cc, for example this page: https://k2s.cc/page/terms
Here it simply didn't work and if I come from some other site, even google, one simply does not spoof referrer easily. I am pretty sure it SHOULD NOT work this way, especially if we're using a spoofer which should be completely competent in this. There must be something that makes this not work.

I will test with a clean test version of this browser and will report later with my findings. But I don't think it's in browser settings because even the newest Chrome, updated just today, seems to not work properly with referrer spoofing extensions.

[Beholder]

So I tried with a clean and a bit older version of Vivaldi and it's the same thing. So it's not any extensions or anything else. 'Nuff said.

[Pathduck]

@Beholder Then I really have no idea. I tried with the site you posted above and examined the headers in Devtools.

I went to:
https://duckduckgo.com/?q=keep2share&t=vivaldi&ia=web

This sets referrer-policy: origin = Only send the origin of the document as the referrer.
See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy

Clicked the first link to Keep2Share, this sets referer: https://www.nsa.gov/

Maybe someone with more knowledge of internet specs will come along and do a more thorough test.

@luetage @Komposten For an interesting bit of internet trivia on this subject, see:
https://en.wikipedia.org/wiki/HTTP_referer
https://groups.google.com/forum/#!original/alt.folklore.computers/7X75In21_54/JgV9Rw04f-EJ

"Its like when I did the referer field. I got nothing but grief for my choice
of spelling. I am now attempting to get the spelling corrected in the OED
since my spelling is used several billion times a minute more than theirs."

[Komposten]

@Pathduck said in Spoofing the referrer does not work anymore?:

"Its like when I did the referer field. I got nothing but grief for my choice
of spelling. I am now attempting to get the spelling corrected in the OED
since my spelling is used several billion times a minute more than theirs."

[Beholder]

Thank you for contributing to this thread. I actually made real progress on the spoofing extension today but I did the spoofing in a very rudimentary way:

1. Upon loading the spoofedSite.com my extension catches it at the "document_start" time, so no content has been running yet for the site.
2. The page is redirected to the desired referrer location; the location also contains a query string with a special code word and a "return link" to the spoofed site, for example: "http://myreferer.com/?CodeWordReturnLink=http://spoofedSite.com/34324de/e42342d.html"
3. The same extension as before catches it at the actual referer location and redirects the location back to the spoofed site, based on the link from the query string.
4. All redirects above are made by creating actual DOM links and using the click() method on them.
5. The spoofed site will now have a proper referer because the extension just redirected the user from there.

It's very quick but of course not as quick as proper referer spoofing. I am fine with it, !FK! the proper referer spoofing, it simply DOES NOT WORK as it should, I have tried in several browsers. If anyone has any problem with it as well, I can help, share code, just let me know here.