When Candice Morgan joined Pinterest in 2016 as the company’s first-ever head of diversity and inclusion, her goals initially centered on improving diversity within the company. The San Francisco company had set lofty goals for bringing more women and minorities into its engineering team, and Morgan had spearheaded new apprenticeship programs to bring in young people from diverse backgrounds.
But on Pinterest’s platform, Morgan saw other diversity problems. On her “home feed,” Pinterest’s recommendation engine showed her images similar to the ones she’d saved on her boards—hair tutorials for black hair, editorial images showing black women. But when she used Pinterest’s search bar to look for “hair ideas,” she found images of beachy waves and elegant up-dos, mostly featuring white women. “My hair type is what’s called ‘4C hair,’ given the level of coiliness,” says Morgan. “I learned that I needed to add that to my searches in order to find things. It shouldn’t be that way.”
Across the internet, search engines suffer the same problems with who gets represented. One study, from researchers at Brazil’s Universidade Federal de Minas Gerais, found that 80 percent of top search results for “beautiful woman” on Google and Bing showed white women. Safiya Umoja Noble, an informatics researcher at UCLA and the author of Algorithms of Oppression: How Search Engines Reinforce Racism, began writing her book after observing that Google searches for “black girls” mostly surfaced pornography.
On Pinterest, at least, the search results were less inflammatory. But it still took specific keywords to find hair, makeup, or fashion ideas showing women who weren’t white. In some cases, the level of specification meant a person couldn’t find what they were searching for at all. “We were thinking, how we can look at our database of pins and make sure we were in some way matching and labeling content across different groups of people?” says Morgan.
Today, Pinterest is introducing its answer: a way to narrow down beauty searches by skin tone. When you type in a beauty-related term, like “orange lipstick,” a set of skin tone options appears below it. Click on one and the search results show only those skin tones. It’s a small, subtle feature—one that many of Pinterest’s users will hardly notice. But Morgan, and others at the company, consider it the first step toward making Pinterest feel like a more diverse place. A team at Pinterest has spent months creating the feature, building their own skin tone taxonomy to categorize pins, and training an algorithm to recognize them, so that more users can find pins across a spectrum of skin tones. The result, they hope, will send a strong message: You shouldn’t have to work harder to search Pinterest just because you’re a person of color; you shouldn’t have to qualify “orange lipstick” just because you’re not white.
Skin tone taxonomies often lump complexion into broad categories, focusing on the spectrum of light and dark. One method, the Von Luschan chromatic scale, invented 36 different categories by comparing skin color to opaque colored tiles. Another method, the Fitzpatrick phototyping scale, reduced Von Luschan’s 36 shades into six broad categories based on how skin responds to ultraviolet light. (The Fitzpatrick scale would later become the basis of the five skin shades for emoji). But the original Von Luschan’s scale was also used as a defense of eugenics, a way to definitively separate “white” from “non-white” in the forced sterilizations committed by the German Society for Racial Hygiene. That history horrified Morgan. “We were like, ‘We’re not using that,’” she says.
Pinterest’s new filter shows up just below the search bar after beauty-related queries. You type in ‘makeup ideas’ and it appears, inviting you to ‘pick a skin tone range to narrow your search.’
But they team still needed some way of categorizing Pinterest’s 8 billion beauty-related pins. Early in their exploration, they found Brazilian artist Angélica Dass, who had photographed hundreds of people and identified their skin tones by Pantone colors. Unlike other taxonomies, Dass’s project divorced skin tone from ethnicity—Pantone 51-6 C could be a blonde-haired little boy, or a biracial woman with an afro. “It makes you look at skin tone like a paint chip,” says Larkin Brown, Pinterest’s qualitative user experience researcher. “We looked at this and thought, well, we don’t want to use Pantone colors, but there could be a way to turn skin tone into a digital value.”
Rather than thinking about skin tone as a range from light to dark, Morgan’s team thought about the biological elements that make up its color. Melanin determines how light or dark skin appears; hemoglobin influences rosiness, and carotene influences yellowness. Those elements matched up pretty well with the Lab color space, a mathematical system that perceives color on three axes: light to dark, green to red, and blue to yellow.
The team gathered an array of beauty images on Pinterest and began to analyze the faces—not as black or white or Asian or Latina, but as digital values of color. From this system, they devised their own spectrum of 16 shades. They brought that taxonomy to beauty software company Modiface, which helped Pinterest to build a machine learning system to decode skin tone in images.
Like all machine learning projects, this one isn’t perfect. Pinterest’s engineers have been training it to recognize shadows and bright lights, and making sure skin tone categories don’t get conflated with racial stereotypes. Pinterest also wants to make sure filtering for skin tone never feels creepy, so the site never saves the skin tones you’ve selected. That’s partly practical—if Pinterest makes assumptions that are wrong, then the site becomes unusable—but also for privacy. Information about someone’s race or ethnicity can easily be used against them online, even if it comes from an innocuous search for eyeshadow. So Pinterest promises not to store information from the feature, nor to use it to target ads, nor to use it to produce a user’s personal information.
The new filter shows up just below Pinterest’s search bar, only after queries related to beauty. You type in “makeup ideas” and it appears, inviting you to “pick a skin tone range to narrow your search.” The 16 tones have been pared down to four groups, represented by circular icons that each showcase four different colors, designed to look like a bronzer palette. Click on one of them and the search results focus on faces in that range of skin tones.
At least, in theory. Broader searches, like “bold makeup” or “contour tips,” tend to work well. But more specific searches often fall short. A general search for “winged eyeliner” delivers thousands of pins, but when the search term is paired with the darkest skin tone range, only eight pins remain (and only two of those actually show women with dark skin).
Right now, there appear to be more pins categorized in the first three groups than in the fourth. That means it’s sometimes easier to tack a few more words onto a text search—like “orange lipstick on dark skin”—than use the visual filters to find some specific types of pins. Over time, Pinterest expects to improve the way beauty content is categorized on the site, creating more results in all of the skin tone groups. The whole point of adding visual filters was to remove the barriers to content discovery. But right now, the filtering mishaps can have an unintended effect. When the search tool can’t come up with anything at all, it makes the lack of representation even more sobering.
So no, a search tool cannot fix the internet’s diversity problems. But Pinterest is at least trying. “Today our search results aren’t as inclusive as they should be, which is why we’re taking this first step,” says Omar Seyal, Pinterest’s head of discovery. “The beauty content is there, and the skin tone ranges feature allows it to be easily surfaced when a pinner wants it. The next step is to learn what pins people are engaging with in those ranges so that we can eventually incorporate those pins into more prominent spaces.”
Patience is part of Pinterest’s ethos, setting it apart from much of the Silicon Valley tech scene. It’s never been about moving fast and breaking things. Instead, the company tends to release new tools quietly, in a way that makes users second-guess whether the feature had been there all along. The skin tone filter is just such an enhancement. Many users might not even notice it. Those who do might find that it helps, if only a little.
For Pinterest, that’s fine. The company doesn’t have a habit of overpromising (unless you count the pins promising that you, too, can craft like Martha Stewart). As Morgan puts it, it’s the very beginning of a longer journey toward bringing greater diversity to Pinterest’s platform, through showing different complexions, body shapes, disabilities, and ages. Those are complex problems. But today’s update is one very real step toward a more representative version of Pinterest, one where anyone can find the shade of orange lipstick that works for them.
More Pinteresting Stuff
Want more news and reviews you can use? Sign up for the Gadget Lab newsletter.