Watching the aggressive use of Facebook, Twitter, Reddit to disrupt American political and social systems makes me think that organizations above a certain size should have a Chief Devil’s Advocate on their team. This exec should be focused on every way that your product could be used for misdeeds.
The term Devil’s Advocate is commonly used when someone wants to sound smart in a meeting by being contrary about the topic under discussion. (See: https://xkcd.com/1432/) But it’s original use was as a formal position in the Catholic church to argue against the canonization of individuals to sainthood.
It was this person’s job to take a skeptical view of the candidate’s character, to look for holes in the evidence, to argue that any miracles attributed to the candidate were fraudulent, and so on. — Wikipedia
They should be thinking every day how your product could be used, as designed for evil. Things like:
- Using your advertising service to racially filter your targeted housing adds.
- Making fake profiles to amplify lies and drown out fact.
- Creating automated trolls to harass others to suppress their speech.
- Undermining democratic elections and institutions.
- Use your platform to convert and radicalize people.
- Anything not against the law, you wouldn’t want on the front page of the Washington Post.
- Government use to oppress their population.
The devil’s advocate should be supported by a cross functional team to understand of your products effects on politics, society, technology, gender, and ethnic equality, markets, and more. Staff their office with individuals experienced in counterintelligence, psychological operations, inform and influence operations, history.
Engage them early in the product development cycle, and continue to look over the entire portfolio of the company. Each new feature should be looked at in how it can be used in combination with everything else you produce and every other tool in the ecosystem around you. They shouldn’t just be looking at your product in isolation, but also in combination with twitter, facebook, bitcoin, etc.
Not only should they think about how you could be evil, they should search for actual evil in progress. It isn’t impossible to find when you start looking. As demonstrated by the success of ProPublica and their work on Machine Bias. It should be even easier for a fully empowered executive inside of the company with the full run of internal data, metrics, and user content.
It is important to discuss what this isn’t, as much as what it is.
This is different from your security red team who is trying to break your product. They aren’t looking for a bad line of code that results in dumping credit card and social security numbers onto the dark web. Instead they are looking for features used as designed to damage the society and systems of trust around them.
This is different from your legal department. The legal department looks to reduce risk for the organization by crafting terms of service that are legally defensible and help you avoid breaking the law. The Devil’s Advocate is looking to understand how your TOS can be exploited or ignored in ways that you won’t notice for the maximum negative impact.
This is different from your Chief Product Officer. Your CPO is trying to build products that delight your customers. You want a CPO that is full of positive vision of what your product could be doing to make your customers lives better. The Devil’s Advocate is working to find the things that will delight evil doers in your CPOs work.
Ultimately, as the men and women who build the technology shaping the world today, we have a responsibility to think about how it could be used against us or our society. As the engineers, and product leaders who built it we are in the best position to understand how the technology works and monitor it’s effects.
As product leaders and engineers we are understandably excited about the great things we create. We see the value for our customers and our shareholders. Many even see their products as drivers of enormous social good. It will take strong leaders with a mandate to see the worst in our wonders to help us reach our highest potential.