Cracking the code: How to tell what News Corp really thinks about the price of links

It's in the code

News Corp is telling us what Google should really pay for linking to its sites. It’s telling us in code. And the answer is … $0.00.

What is an Internet link worth? For most of the Internet’s life, this question has been ignored. On the Internet, links have been free, unless they were paid ads displayed by agreement between the linker and the target site.

Now that has changed. Australia has just passed a “news media bargaining code”, which throws this links-are-free philosophy out the window.

The nation’s competition regulator, the ACCC, dreamed up the code. It requires effective agreement about the price of a link that goes from a digital firm like Google or Facebook to a news site like those of News Corp or Nine/Fairfax. And if the parties don’t agree, an arbitrator has to decide between their two offers.

(Update: In June 2021, Reuters reported ACCC chief Rod Simms as saying that the code’s negotiating system was actually proposed by News Corp. They don’t quote him, though, so it’s not entirely clear what he actually said.)

News Corp and other media companies are saying or implying that if only the digital giants were not so powerful, those giants would pay the media companies for links. The ACCC is pretty much saying this outright; it’s the justification underlying the entire code.

But in fact News Corp seems to have already declared that it really believes Google is right, and the correct price is $0.00.

News Corp is saying it in code.

Literally. “Code” is not a metaphor here. News Corp is disclosing its intent right there in its HTML code. If you know where to look, you can see it immediately.

So here’s where to look, and why.

The strangeness of pay-for-links

I want first to spell out what’s actually going on in Google’s online news pages, because many people seem to misunderstand it. Google isn’t “stealing” from News Corp, any more than it steals from any other sites it links to. It doesn’t even run ads on the pages where it links to news stories. Despite what journalists keep writing, Google isn’t “paying for content” or “running news” on its site. It’s just linking to news stories, like this: Betting agency Sportsbet has opened markets for Dan Andrews’ much anticipated press conference to reveal whether the snap lockdown will …

As plenty of people have already found, it’s actually very tough to make money producing list of links like this to other people’ news. Google makes the vast majority of its profits from areas that News Corp never played in to start with. Heck, Google doesn’t even run classified-ad sites of any note: Realestate, Carsales, eBay and a bunch of smaller players took that set of markets. And in an economic sense, this destruction of the classified oligopolies is what has destroyed newspapers’ revenue streams, moreso than even the destruction of their news oligopolies.

All Google’s really guilty of in this space is offering links to a wider variety of news stories than News Corp would like. As the analyst Benedict Evans has put it: “Google didn’t take their money, any more than Boeing took money from the ocean liners. The internet destroyed the model.” If Google had never existed, the classified-ad market – newspapers’ “rivers of gold” – would still be worth far less today than 20 years ago.

But Google does exist, and it makes a lot of money from advertising. And media companies such as News Corp reckon they have a right to some of that money. Or at least, they reckon they might be able to convince the Australian government that they do.

Robots.txt and HTML suggests what News Corp really believes about links

Maybe you think that by posting links to News Corp news stories, Google is actually intruding on News Corp sites. And maybe you think that News Corp is powerless to stop that.

In fact, the exact opposite is true. We know this because companies like News Corp can bar Google from their sites in an instant if they so choose, using nothing more complex that a two-line text file called “robots.txt”. Here’s an example of such a coded instruction to Google not to look at a specific folder on a web server:

User-agent: Googlebot Disallow: /example-subfolder/

Not too complicated, right?

Yet News Corp and its colleagues never do use robots.txt in this way. They never do bar Google. That’s because they want the money that comes with Google traffic.

Now, maybe you know this, but you still think the text in the digital giants’ links to news stories gives away too much for free.

Again, we can check whether this is true by looking at what people do. One obvious place to look is the meta tag attribute values at, which are publicly available right there in the site’s easily-read HTML code.

Meta tags are essentially labels which News Corp posts for the likes of Google and Facebook to read. Here News Corp gets to dictate just how Facebook, in particular, describes each news story. (I’ll use Facebook as the example here only because Google uses a slightly more complicated mix of sources, but the principle is basically the same for both companies.)

You can click “view source” on any web page and see these tags for yourself.

So what does News Corp do when it has the chance to completely control how links are displayed, via meta tags?

Sure enough, News is positively verbose about how it populates these values. Google usually describes a link in about 20 words, often stopping mid-sentence. In the first story I pulled up, News itself asks Facebook to use a full sentence of 39 words. On top of that, it specifies an image to be used with the link. (For those who can read HTML, I’m referring to the og:description and og:image values.)

It’s hard to escape the conclusion that News Corp is on the one hand saying to the government “digital giants should pay for descriptive links”, while out of Josh Frydenberg’s earshot it whispers to Google and Facebook themselves (indeed, in literal code): “We really, really want you to describe these links with as much detail as possible”.

What a fair market solution would look like: $0 or less

The logic of the ACCC’s utterances to date seems to be that the government must act to repair the “power imbalance” in digital media. For instance, it suggests that Google doesn’t pay News Corp for ads because of the market power imbalance between the two.

OK, stop sniggering about that last sentence, and ask yourself: how can we tell the true nature of that power imbalance?

It turns out that the existing media landscape presents a way to solve that puzzle too. The question to ask here is: are there any places where News Corp and its fellow media firms would not be at such a market disadvantage?

Well, how about the situation between News Corp and, say, Yahoo parent company Altaba? Altaba has a $US10 billion market capitalisation, a little lower than News Corp’s $US13.7 billion.

So how much does Yahoo pay to link to News Corp stories?

Honestly, I don’t know for absolute sure. But I think the amount is $US0. And I suspect it’s the same for the news pages at Microsoft Bing, and DuckDuckGo, and so on. (Yahoo and Duck may use Google News technology, but I don’t think that changes the underlying case.)

And I do know what the current rate is when Club Troppo links to News Ltd stories with 20-word descriptions. here’s one: Betting agency Sportsbet has opened markets for Dan Andrews’ much anticipated press conference to reveal whether the snap lockdown will …

We pay exactly nothing for this.

Prediction: no lawyer’s letter for Club Troppo

Maybe News Corp will write Club Troppo a lawyer’s letter tomorrow asking us to take down that link above. But I don’t think so.

And I don’t think it’s just because we’re too small to worry about. I think it’s because News Corp – and other traditional media companies – know exactly which way value runs. They just pretend something different when they talk to the Australian government. The ACCC and the government just haven’t noticed.

In its 2020 bargaining code concepts paper last year, the ACCC even went so far last year as to invite suggestions on market benchmarks that would indicate an appropriate price for links “in circumstances where there is no bargaining imbalance”. The ACCC said it “recognises that such benchmarks may be difficult to identify”. Please let me know in the comments if I’m ignoring something here, but the prices paid by Yahoo, DuckDuckGo and Club Troppo – again, $0.00 – seem to me like they’re saying something.

It seems to me we can tell that the media companies’ stated beliefs about link prices are not quite true. Their audible words say one thing, but their quiet, computer readable speech says something else. They’re speaking it in code.

Update: Australian economist Joshua Gans, a competition expert now at Rotman, expects the news media bargaining code to simply entrench the big players:

“It will end up with the large digital platforms doing deals with the largest news outlets. Those deals will be multi-year lump-sum payments which enable everyone to go about their business.

“There will be no new digital platforms, no new content providers, no more competition. The shareholders of the large digital platforms will be a few million dollars poorer and the shareholders of large Australian news outlets a few million dollars richer.

“There will be no improvement in any competitive outcome whatsoever. As often happens in Australia, oligarchies will consolidate, and consumers will get nothing.”


About David Walker

David Walker runs editorial consultancy Shorewalker DMS (, editing and advising business and government on reports and other editorial content. David has previously edited Acuity magazine and the award-winning INTHEBLACK business magazine, been chief operating officer of online publisher WorkDay Media, held policy and communications roles at the Committee for Economic Development of Australia and the Business Council of Australia and run the website for online finance start-up eChoice. He has qualifications in law and corporate finance. He has written on economics, business and public policy from Melbourne, Adelaide and the Canberra Press Gallery.
This entry was posted in Bullshit, Business, Economics and public policy, Information, Intellectual Property, IT and Internet, Media, Politics - national, regulation, Web and Government 2.0. Bookmark the permalink.

19 Responses to Cracking the code: How to tell what News Corp really thinks about the price of links

  1. Antonios says:

    Companies pay to get their links listed in prominent positions on Google search.

    They’re called ads, but it’s paying to be linked to for specific keywords, the price determined by an auction. The fact that the link is accompanied by text saying Ad actually lowers the value of the link, but still companies pay for the privilege.

    The idea that this should be inverted and Google pay newspapers for links when they can freely avoid being linked to should the newspapers wish is preposterous.

  2. Graham Young says:

    Good argument. I think you could further it by looking at what SEO companies will pay to link to good quality sites. I get frequent requests for links, or for guest posts with links. They’ve never got past the price stage because none of these guys wants the word “sponsored” on their material, however, the cost of soliciting sites, writing content and then paying the site for hosting it, gives you a basis for making some calculations about what links are worth to the site being linked to.

    Of course the links on our site would be permanent, so you’d need to take that into account.

    They normally offer something like $50, and I generally tell them something like $150. As I say, I’ve had a number accept the price, but not the condition that any article has to be marked as sponsored, or that the links have to actually relate to the article in a meaningful way (or could be in the author’s biog).

  3. Pingback: Why “final offer arbitration” is Russian Roulette for Google | Club Troppo

  4. David a possible data point,
    there are plenty of cases where photographers and others have taken action re copyright breaches against newspapers and the like for embedding links etc to their photos on other websites or in this case on instagram .
    ( this kind of thing has been a long running bone of contention in the US and Europe)

    • David Walker says:

      That’s definitely a data point. Figuring out how its logic would apply here is interesting.

      Moral point first: If News Corp is populating OG:image tags, which only really exist as part of Facebook’s Open Graph protocol, it’s hard to see that as anything but an invitation from News for Facebook to display images. Same for Twitter if they’re populating twitter:image tags. Does that invitation extend to Google?

      But my guess is that Google would stop displaying the images if asked to do so. After all, Google has always observed the robots.txt protocol, which allows sites to stop Google displaying anything at all.

      Legal point second: To the extent that there’s a copyright issue, News Corp or an independent photo supplier like AAP might argue that Google had not been granted a sub-licence to use those same images. Again, my guess is that Google would take them down if asked.

      Underlying point last: this seems to me to remain the same – News Corp is not seeking to assert anything like copyright. Indeed, it consistently avoids asserting any such right, because it does not want the result that would flow from asserting such a right.

      The last thing News Corp has ever sought is for Google to stop linking to its sites or displaying the contents of the public meta tags it provide. Its overall course of conduct now and in the past demonstrates a clear desire to have other sites, including Google, display what it provides. It just wants Google’s cash.

      That is why the new law, and the ACCC’s Sims, both make the economically ridiculous assertion that net value flows from News to Google. Every piece of economic evidence says net value flows the other way.

      [Note: Edited to change “value” to “net value”.]

      • “Figuring out how its logic would apply here is interesting. ”
        More than I can figure out- the cases I’m more familiar with involve people who really didnt want whoever from linking to or using their stuff, regardless of the money etc.

        Curious thing is as far as I can workout the Google News page doesn’t display Ads ( am i Correct?) So an argument that Google is stealing Newspaper Audiences and thus advertising revenues, from the newspaper sites seems a bit of a stretch to boot.

        Suspect that the big media types would actually be fairly happy if the likes of Google and Facebook simply disappeared, it was so much ‘nicer’ back in the old days .

        BTW The robot text is only a protocol , while it seems that Google doesn’t display pages that have ‘robot keep-out’, there is no way of knowing that they dont still scan those pages- the data could still be useful-rewarding to Google.

        • David Walker says:

          “The cases I’m more familiar with involve people who really didn’t want whoever from linking to or using their stuff, regardless of the money etc.”

          Yep. Those are the cases everyone is used to solving. No-one has ever previously posed a case of a demand that a site pay to link to news. That’s because it seemed quite pointless – until someone got a national government to back up their demand.

          This is one reason it takes so much work to convince most people that something strange is going on here.

          You are also correct that Google News doesn’t run ads.

          And you are certainly correct that News Corp and others would be delighted to have Google and Facebook – indeed, all of web search – disappear. It was indeed much nicer back in the old days. I remember them fondly, along with my developing realisation through the mid-1990s that newspapers were about to become a declining industry.

          The funny thing is, though, that the disappearance of web search still would not help newspapers. Their economic fundamentals changed forever when their oligopoly control of news disappeared. To reverse that they would have to get a lot more help from governments – including somehow killing a lot of the world’s websites. (First and foremost, they would need to kill all major weather forecast websites here and overseas.)

          BTW on protocols, web pages disallowed by robots.txt remain public. You have no way of knowing that anyone doesn’t scan those pages for useful data. Anyone can. It’s just not actually useful to Google to scan pages if they can’t present results to users and have them click on something. It’s the clicks that send Google useful signals.

          • Think it was their control of classifieds, least those that everybody turned to on Saturday for everything from second hand cars, share houses and kitchen hand jobs that really mattered. Do you disagree?

            Re what’s, useful to google.
            A few years ago new scientist published an interview with a senior scientist( can’t remember his name) for google ,collecting absolutely vast amounts of examples of usages in various languages is critical to developing better automatic translation systems.

            Data is core to everything re google.

          • BTW there is something of a whiff of government backed cartel about the proposed legislation, no ?

            And if the real value of the links in question is close to zero then there is also a hint of mandated redistribution about this biz, outside the tax act.

            Section 55 anybody?
            “Laws imposing taxation shall deal only with the imposition of taxation, and any provision therein dealing with any other matter shall be of no effect.”

      • Chris Lloyd says:

        “That is why the new law, and the ACCC’s Sims, both make the economically ridiculous assertion that value flows from News to Google. Every piece of economic evidence says value flows the other way.” Surely it flows both ways like any deal. Are you saying that it is net towards Newcorp? If so, why do Google not change Newscorp?

        • Chris Lloyd says:

          Sorry. CHARGE newscorp.

          • David Walker says:

            Thanks Chris. You are correct – NET value. I’ll change the text.

            Why does Google not charge News Corp for links? It’s partly technological, but mostly cultural – Internet culture, more than Google culture. To my knowledge, when Google started in the late ’90s, no other major search engine had done this. Google certainly determined right at the start not to do so; it was a central element in the informal “don’t be evil” mantra. Google carried that through when they started up Google News around 2003.

            Indeed, there’s a scenario in which we look back on this 20 years from now and say that the worst aspect of the Australian government’s policy was that it began to erode the “no-one pays for links” mentality.

  5. I am and will always be Not Trampis says:

    Thanks David. Really interesting article.
    good to see Troppo getting backs to its roots

  6. derrida derider says:

    Oh come on. The ACCC is only “responding to political reality” because “it’s no use making recommendations in your report if it is just immediately buried” and “you can’t influence policy if you get government offside”. Criticising the inevitable intellectual shoddiness of that sort of report is a bit beside the point.

    Note these are words I have personally heard a number of times in a long career from the mouths of heads of supposedly “independent” inquiries.

    The government really really would like to have the near monopoly Murdoch and Nine medias continuing to cheer for them. So what the greedy buggers want, they get. Whether what they want is actually in their own long term interest is also a bit beside the point.

  7. ianl says:

    It is true I have no respect for either the MSM or Facebook. None whatsoever; the pox on both their houses.

    In this particular stoush though, the weight of self-serving hypocrisy falls squarely on the MSM, with the Fed Govt coat-tailing it to avoid damaging, although dishonest, headlines.

    To show this, one simple question is avoided. If FB (or Google-bot) refuse to pay, why can people not go directly to the various MSM sites ? The answer to this suppressed question is obvious: if they do, they then have to subscribe, actually pay real money. Not nearly enough people are willing to do this for the tendentious rubbish they are then offered, so the MSM smartypants decided to try and make someone else pay for it.


    • Not that simple, back in the 80s most of my friends bought the Saturday SMH for the classifieds did not read the actual paper. And the costs of placing an ad for ,housemate wanted were in hindsight fairly significant.
      In short decent journalism has nearly always involved a degree of cross subsidy.

      I don’t care for the governments solution however the problem is real enough.

      Curious aspect is that Facebook is truly unattractive to those under 40 is for “ageing babyboomers “. I am too old to understand the various platforms the young turn to for the latest info, but they aren’t Facebook.

Leave a Reply

Your email address will not be published. Required fields are marked *

Notify me of followup comments via e-mail. You can also subscribe without commenting.