Silicon Valley is Doing Just Fine, Thanks

Tech companies are doomed. Just look at this chart from JP Morgan Asset Management!

Source: Matt Levine
Source: Matt Levine

The chart is supposed to show that unicorns do poorly in the public markets, and I assume that whomever made it was high. Not just because there’s a unicorn vomiting a rainbow, but who the hell plots stock price as a function of days-since-IPO?

Screen Shot 2016-02-21 at 12.20.40 PM

Here are the unicorns again, plotted on a common timeline. You know who else has trouble surviving in the wilderness? The S&P 500.

The problem does not lie with Silicon Valley investors so much as the journalists reporting on the tech industry. Over the weekend, we had this WSJ story declaring Silicon Valley’s Hangover:

Today, venture capital is drying up for less successful startups. Investors, eyeing collapsing tech stocks and economic sloth, are culling their portfolios and forcing cash-starved companies to retrench or shut down.

Guess what: 75% of venture-backed companies fail. That’s a number from 2012, and I bet it’s even higher today. Venture Capitalists have always culled the herd. What did you expect them to do, double down on the losers?

When there are tens of thousands of venture-backed companies, three-quarters of whom are doomed to die, it is very easy to find stories of failing startups, no matter what the conditions are.

Last year, we mostly ignored those stories, because the popular narrative was that any college dropout with a pitch deck could get a billion-dollar term sheet on Sand Hill Road.

Then the Fed raised interest rates, and China/oil/whatever happened, and somehow those events unleashed a plague upon the fledgling tech industry.

The media fashions its headlines from the public markets, because that’s the only reference they have to draw from. If markets were rallying, we’d be hearing about how Uber raised yet another $200 million, Campbell Soup launched a $125 million fund, and even JetBlue announced a venture capital fund in Silicon Valley. All in the past week!

Instead, we hear about layoffs and down-rounds, stories that have always happened and will always continue to happen.

Venture capitalists have an investment horizon of over 10 years. If investors are collectively soiling themselves over two months of market turmoil, they should seriously consider a different line of work (my company is hiring!).

Reports of unicorn deaths have been greatly exaggerated. When rental prices regain sanity, and 101 traffic starts to move faster than snail shit, then we can start worrying about Silicon Valley. Until then, everyone is doing just fine.

Of Paywalls and Piracy

home_taping_is_killing_music_logo

I have a problem. I want something (access to WSJ’s content), but I don’t want to pay money for it. 😞

Isoroku’s tutorial yesterday resulted in some debate, much of which concerned business models, ethics, and 17 U.S. Code § 1201.

We’ve seen this story play out before. Remember Napster? Kazaa? What happened to illegal music downloads? People didn’t collectively decide to stop pirating music in 2004 and become law-abiding citizens. Instead, music became much easier to obtain through legitimate means.

I refer to Stratechery, which (believe it or not) I pay for. Ben Thompson discusses the evolution of music distribution:

The fact of the matter is that pirating music is a huge pain in the rear end relative to the alternative. Consider how music distribution has evolved from the time of the original iPod…

Legitimate Pirate Advantage
Buy CD, Rip, Sync Download, Sync Piracy
iTunes, Sync Download, Sync Legitimate (Slightly)
iTunes on iPhone Download, Sync Legitimate (Huge)
Stream Download, Sync Legitimate (Massive)

Convenience killed piracy, and the iTunes on your phone is already pretty damn convenient.

Today, it’s so easy to pull up iTunes that no one bothers looking for music on The Pirate Bay.

Until Kanye.

Kanye West released his latest album exclusively on Tidal, a premium streaming service. He suddenly made the legitimate channel much, much more onerous:

Legitimate Pirate Advantage
Sign up for Tidal Download, Sync Piracy (Massive)

Anyone who wants streaming music already has an account with a streaming music service, and that service sure as hell isn’t Tidal. So those who want Kanye’s music now need to pay another $156/year for Tidal. It’s easier just to pirate the damn thing.

And that’s what everybody did.

Paywall publications have put up the same barriers: A Wall Street Journal subscription costs $348 per year. WSJ readers probably also read Barron’s ($132/year) and FT ($335/year). Throw in the occasional Washington Post ($99/year), Economist ($152/year), or NY Times ($195/year) article, and you’re running quite a tab.

More importantly, casual readers don’t actually visit any of these sites. They click on links through Twitter, newsletters, or aggregators.

For one-off access, readers will choose the Google-search-paywall trick every time, because the subscription barrier is just too high.

To stop people from sneaking around paywalls, publishers need to make it much, much easier to become a paying customer.

It doesn’t help to make it harder to swipe the content. If piracy becomes impossible, people will simply stop clicking on paywall links. Information is a commodity, and most of the content published in the Wall Street Journal is later regurgitated on Business Insider anyway.

Prediction:

Digital publishers will plug the Referer/User-Agent hack in Isoroku’s post. As commenters pointed out, websites could perform a reverse-DNS lookup to verify the identity of the visitor.

Casual readers who still want the content, but not enough to pay the subscription fee, will share subscriptions. Content-providers should be okay with this. Netflix and HBO GO encourage account-sharing — HBO president Richard Plepler called it a “terrific marketing vehicle,” because sharers tend to get their own accounts later.

Maybe readers will create a pooled subscription vehicle to share subscriptions to many different publishers. Until then, modifying headers to look like Googlebot is just too convenient.

See Also:
Kanye West and Tidal, The Problem with Exclusivity –stratechery (paywall)

How Google’s Web Crawler Bypasses Paywalls

by Isoroku Yamamoto

Update: A newer version of the chrome extension is available here.

Wall Street Journal fixed their “paste a headline into Google News” paywall trick. However, Google can still index the content.

Digital publications allow discriminatory access for search engines by inspecting HTTP request headers. The two relevant headers are Referer and User-Agent.

Referer identifies the address of the web page that linked to the resource. Previously, when you clicked a link through Google search, the Referer would say https://www.google.com/. This is no longer enough.

More recently, websites started checking for User-Agent, a string that identifies the browser or app that made the request. Wall Street Journal wants to know that you not only came from Google, but also that you are an agent of Google.

By providing this information in request headers, anyone can appear to be a Google web crawler. In fact, I will show you how to make a Chrome extension that does just that.

1. Create a file called manifest.json. Paste the following in the file. Add any sites you would like to read to the permissions list.

{
  "name": "Innocuous Chrome Extension",
  "version": "0.1",
  "description": "This is an innocuous chrome extension.",
  "permissions": ["webRequest", "webRequestBlocking",
                  "http://www.ft.com/*",
                  "http://www.wsj.com/*",
                  "https://www.wsj.com/*",
                  "http://www.economist.com/*",
                  "http://www.nytimes.com/*",
                  "https://hbr.org/*",
                  "http://www.newyorker.com/*",
                  "http://www.forbes.com/*",
                  "http://online.barrons.com/*",
                  "http://www.barrons.com/*",
                  "http://www.investingdaily.com/*",
                  "http://realmoney.thestreet.com/*",
                  "http://www.washingtonpost.com/*"
                  ],
  "background": {
    "scripts": ["background.js"]
  },
  "manifest_version": 2
}

2. Create a file called background.js. Paste the following into the file:

var ALLOW_COOKIES = ["nytimes", "ft.com"]

function changeRefer(details) {
  foundReferer = false;
  foundUA = false

  var reqHeaders = details.requestHeaders.filter(function(header) {
    // block cookies by default
    if (header.name !== "Cookie") {
      return header;
    } 

    allowHeader = ALLOW_COOKIES.map(function(url) {
      if (details.url.includes(url)) {
        return true;
      }
      return false;
    });
    if (allowHeader.reduce(function(a, b) { return a || b}, false)) { return header; }

  }).map(function(header) {
    
    if (header.name === "Referer") {
      header.value = "https://www.google.com/";
      foundReferer = true;
    }
    if (header.name === "User-Agent") {
      header.value = "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)";
      foundUA = true;
    }
    return header;
  })
  
  // append referer
  if (!foundReferer) {
    reqHeaders.push({
      "name": "Referer",
      "value": "https://www.google.com/"
    })
  }
  if (!foundUA) {
    reqHeaders.push({
      "name": "User-Agent",
      "value": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    })
  }
  console.log(reqHeaders);
  return {requestHeaders: reqHeaders};
}

function blockCookies(details) {
  for (var i = 0; i < details.responseHeaders.length; ++i) {
    if (details.responseHeaders[i].name === "Set-Cookie") {
      details.responseHeaders.splice(i, 1);
    }
  }
  return {responseHeaders: details.responseHeaders};
}

chrome.webRequest.onBeforeSendHeaders.addListener(changeRefer, {
  urls: ["<all_urls>"],
  types: ["main_frame"],
}, ["requestHeaders", "blocking"]);

chrome.webRequest.onHeadersReceived.addListener(blockCookies, {
  urls: ["<all_urls>"],
  types: ["main_frame"],
}, ["responseHeaders", "blocking"]);

Save both files in one directory. These should be the only files in the directory. If you were too lazy to copy and paste, you can download the source code here.

Now type chrome://extensions/ in the browser address bar.

Click Load unpacked extension... (Make sure Developer Mode is checked in the upper right if you do not see the buttons.)

Screen Shot 2016-02-18 at 10.49.25 PM

Select the directory where you saved the two files. Enable the chrome extension and visit wsj.com.

Remember: Any time you introduce an access point for a trusted third party, you inevitably end up allowing access to anybody.

Information Wants to be Free, But Academic Publications Do Not

The first scientific journal: Philosophical Transactions, volume 1.
The first scientific journal: Philosophical Transactions, volume 1. Optick Glasses and Pendulum-watches [1].

It costs $76,140 for a one-year subscription to the collection of IEEE technology journals. $58,645 for online-only access.

That’s a bit steep for most people, which is why individual articles can be downloaded for $33 apiece.

What a racket. But what are you gonna do, drive to the public library and check out the latest issue of IEEE Transactions on Information Theory? Good luck with that. Academic publishing is a for-profit enterprise, and there is very much profit.

A few years ago, Harvard University notably whined about the high price of journal subscriptions. Come on, no one feels sorry for you Harvard. Slightly more sympathetic are the international universities that don’t have Harvard’s bankroll and can’t afford $3.75 million in subscription fees. Publishers sometimes negotiate a discounted price for the lesser-endowed.

Wait a minute. I’ve published articles in scientific journals. I’ve reviewed articles for scientific journals. Not only was I never paid a dime, I had to pay a processing fee for my papers to go into print!

Do the publishers even do anything?

They provide prestige. The purpose of academic publication is first and foremost to benefit the author. We can wax poetic about the communication of research and advancement of knowledge, but the real assumption is that nobody reads academic papers. If academics thought otherwise, they would actually try to make the papers readable. Skip the journal processing fees and post the work to a blog.

Publications are status symbols. The first rule of academic publishing is that you always submit your paper to the most prestigious journal that might accept it. It’s like applying to college. The right publications lead to research grants and faculty positions and tenure.

And just like universities, scientific journals have rankings. One widely-used metric is the impact factor, a measure of how frequently an average paper in a given journal is cited each year. Nature, the highest-impact scientific publication in the world, rejects over 90% of paper submissions.

High-impact publications can charge a lot for subscriptions, because universities want their scholars to stay on top of impactful research.

Open-access publications do exist. Most journals allow authors to archive their papers in public repositories (“green open-access“) after an embargo period [2]. These repositories are funded by donations. One example is arXiv for things related to mathematics and physics.

There is also “gold open-access,” where authors submit papers directly to an open-access journal, complete with open-access peer review. These journals make money by charging authors a processing fee upon acceptance. The Public Library of Science is a gold open-access publisher.

The open-access business model is inherently flawed. Revenue comes from the authors, so in order to be sustainable the publications must accept a lot of papers. When publications accept a lot of papers, the exclusivity is gone.

Academic publishing (all of academia, really) is rooted in exclusivity. The first scientific journal, Philosophical Transactions, was created in 1665 by the Royal Society. The Society was an elite group of physicians and philosophers who established the concepts of scientific priority and peer review. It still exists today as a coterie of fellows, and Philosophical Transactions is still in circulation.

An 1862 engraving of the Royal Society fellowship of eminent scientists
An 1862 engraving of the Royal Society fellowship of eminent scientists

It’s unlikely that open-access journals could attain the same level of prestige as private publications. Here, I just uploaded this terrible blog post to SSRN, an open-access repository for the social sciences [3]. I am now published alongside Nobel laureate Eugene Fama [4]. That felt way too easy.

Even the peer-reviewed open-access journals will have a tough time competing with private publishers, the largest of whom had over a century to build prestige and impact.

Nature, Issue 1. Nov 4, 1869. (click for full page)
Nature, Issue 1. Nov 4, 1869. (click for full page)

Most of the criticism about academic publishing comes from outside of academia. First of all, academics themselves have no idea how much a journal subscription costs because they receive access through their university. More importantly, they want to preserve the scarcity of their status symbols.

I worked hard to create peer-approved papers. I have an Erdős number of 4. Building a publication record is not supposed to be easy!

Academia is getting more competitive, not less. There are far more unemployed PhD students than exist faculty positions, and far more hungry PIs than available research funding. The protectionism will only grow stronger.

As long as academics care about gaining respect in the field, they will submit their work to the highest-ranked journals. And as long as the most impactful papers appear in prestigious journals, for-profit publications will remain a staple in university libraries. Academic publishing isn’t broken; academia is.

Appendix A
Useful note: Sci-Hub.io is a site for pirated academic papers. It downloads journal papers by attempting credentials from various universities. I am glad this service exists.

References
1. Philosophical Transactions (1665) 1, no. 1.
2. Suber, Peter. Open Access. Cambridge, MA: MIT, 2012.
3. Ou, Elaine, Information Wants to be Free, But Academic Publications Do Not (February 17, 2016). Available at SSRN: http://ssrn.com/abstract=2732821
4. Fama, Eugene F. and French, Kenneth R., A Five-Factor Asset Pricing Model (September 2014). Fama-Miller Working Paper. Available at SSRN: http://ssrn.com/abstract=2287202

Costly Mating Rituals

Screen Shot 2016-02-15 at 1.07.56 AM

Call it a form of signaling. That is, a method of conveying information between not-trustworthy parties by performing a costly action, where the cost is honestly linked to signaler quality.

In evolutionary biology, this is called the handicap principle. The ability to afford to squander a resource signals biological fitness. For peacocks, a big showy tail is costly and useless, but honestly communicates the peacock’s ability to sustain such a burden.

Peacock_With_Fanned_Tail_600

According to human contract theory, signaling is necessary when there is no other way to believably convey important information. It is a socially-learned strategy, and by definition, horribly inefficient.

In today’s mating ritual, the information being conveyed is “I am genetically fit.” We assume the ability to obtain overpriced food is somehow linked to genetic fitness. It is a costly signal for someone unable to afford food; less costly for someone who can already afford food.

But let’s try to maximize signaling efficiency here. If you want to convey a costly signal to a potential mate, why not just hand over a wad of cash before mating? The informational value of the cash comes from the belief that the ability to accumulate resources is positively correlated with genetic fitness.

There, I just saved everyone a ton of trouble. Happy Valentine’s Day.