Dating site OKCupid has admitted it experimented by deliberately “matching” incompatible partners to see how much weight its recommendations carry.
The revelation has caused some controversy, following shortly after it emerged that Facebook deliberately emphasized negative posts in some users’ news feeds to see if it changed the tone of their own posts.
It’s certainly no surprise that OKCupid analyses its data to look for informative trends: previously it’s “revealed” that iPhone users have more sexual partners, pics taken just after sunrise or before sunset get more responses, and atheists write posts with the widest vocabulary. However, this seems to be the first time the site has admitted to experiments involving manipulation, rather than merely observing “naturally occurring” data.
In the bad match experiment, the site took pairs of users whose mutual compatibility it rated as 30 percent or 60 percent and told them they were a “90 percent match.” It founds that the chances of at least one of a pair that really had a 30 percent rating sending a message to the other was 14.2 percent, while for those with a 60 percent rating, 16.5 percent sent a message. That compared with only a slightly higher rate, 16.9 percent, among those who the site really did rate as 90 percent compatible.
Further analysis showed that although telling people they were a good match got things going, the “true” level of compatibility was genuine. Those who the site legitimately rated as 90 percent compatible were almost twice as likely to carry on the conversation for at least four messages as those rated 30 percent compatible who’d been given the duff info.
The site then took things a step further, deliberately giving some “compatible” pairs a low score. This turned out to reduce the chances of an initial approach leading to a four message conversation — in other words, thinking you are incompatible may be a self-fulfilling prophecy:
While this part of the experiment was certainly valid (and arguably even necessary) from an analytical perspective, for me it’s crossing the line. Given OKCupid believes (with some statistical justification) that its compatibility rating has validity, intentionally telling a well-matched pair that they won’t get on runs counter to the stated purpose of the site in introducing people to compatible would-be partners. The users were given the real rating once the experiment concluded, but that may have been too little, too late.
OKCupid also detailed a couple of other experiments with some depressing conclusions. For seven hours it removed all photos from the site and found people became more likely to reply to messages, the chains of message exchanges extended, and people were more likely to exchange contact details — a pattern that abruptly ended when the photos returned.
Another experiment hiding profile text for some users discovered that the rating visitors gave to the profile (which is meant to combine perceived physical attractiveness and personality) was driven almost entirely by the photograph, with the text making little difference.