“All fine architectural values are human values, else not valuable.” — Frank Lloyd Wright
The Promise of Open Information I am a strong believer in the idea that information wants to be open. Society benefits from access to information. However, with that openness we have a great responsibility to help each other, not harm each other.
I was part of one of the biggest transformations to society in recent history: moving news consumption online. I was eager and excited to get more information to more people, as access to information is the basis of democracy. I worked closely with newsrooms during the transition, building over 350 news websites. In this period I watched something I did not expect, and it taught me so much about capitalism and human fear.
I remember the first time I saw it. Two headlines and two images running in an experiment. An algorithm checking each one against different personas, determining which would get the most clicks. The reasoning was not “this information is important” or “people need to read this,” but simply: more clicks equals more advertising money. Which headlines and images got the most clicks? The most inflammatory. The scariest. If it bleeds, it leads, as the saying goes. Let that sink in. Your attention, your feelings, are an ongoing experiment. Your view of the world is being shaped by your stress response in a ploy to make more money.
“We shape our tools and thereafter they shape us.” — Marshall McLuhan
What came next should not surprise us. If all our information becomes driven by our stress response, we as a society go into fight-or-flight mode. As a result our country turned on itself. Neighbors stopped talking to each other, families split apart, and a few people became wildly rich through a new form of propaganda, not driven by any focused societal agenda, but driven by algorithms designed to keep people engaged and coming back.
The Facebook Factor
This phenomenon was not driven solely by news companies. In fact, news companies at some point just started trying to survive any way they could. Social media companies took over the role of information sharing we used to rely on news companies for. Interestingly, in their quest for a profitable business model, social media companies looked around and saw everyone else on the internet following the same approach, attention equals money, and then they built the most powerful attention algorithms ever created. The main difference is that they were much bigger machines than the news could ever be. With two billion people’s attention up for grabs, engineers taught at the largest best institutions in the world and billions of dollars in backing, they had a lucrative future and far stronger tools to manipulate emotions and keep people coming back.
As we saw in recent litigation, Facebook was not well-intentioned in its actions toward society, and was more than happy to profit by manipulating the most vulnerable among us, children:
- A bipartisan coalition of 42 attorneys general alleged Meta knowingly designed features that purposefully addict children and teens. New York State Attorney General
- Meta’s own research showed 13.5% of teen girls said Instagram makes thoughts of suicide worse, and 17% said it makes eating disorders worse; and they shipped anyway. PBS
- Plaintiffs argued the platforms were designed not to show kids what they want to see, but what they can’t look away from. NPR
- Plaintiffs alleged Meta’s attorneys instructed researchers to alter teen mental health studies to lower the company’s potential liability. TruLaw
- There are currently over 2,000 federal lawsuits pending, each representing a minor whose mental health was harmed. King Law
Once again, technology companies knowingly experimented on humans, saw that the results on society were negative, and continued to manipulate people while completely disregarding their responsibilities to society.
Why Would the Engineers Do This?
I work at GitHub. I purposely decided never to apply to some of these companies after I saw what was happening. I lost trust in the machine. But I never really lost trust in the engineers.
Engineers have been motivated in the wrong ways. There are plenty of people in this industry who will point out the successes of social media and place them in a vacuum to motivate their employees. It is easy to point to the amazing changes that happened around organizing political movements like the Arab Spring. That makes you feel good, and you ride that high.
We bake the wrong incentives into our workflows. Ask any engineer about the nightmare of being on call, when the system goes down you are pressured to hit SLAs and get things back up as quickly as possible. Or the pressure cooker of shipping features to keep up with the competition, and the resulting excitement when you build a complex feature and get it to production.
We built dashboards and metrics to measure system health and scaled these practices across the industry. We talk about craft as a way of shaping pixels or perfecting data lookups in complex data structures. These are feats we are proud of. We complete this work, put it into the world, and joyfully move on to the next feature to build.
“The good building is not one that hurts the landscape, but one that makes the landscape more beautiful than it was before.” — Frank Lloyd Wright
We think we are making society better because we are focused on the wrong metrics. What would a dashboard that measures the joy of your users look like? Or the suicide rate among the youth using your feature? How would a company feel at an all-hands showing a graph with a 17% increase in eating disorders linked to what they built? Apparently we know the number. We just don’t seem to care or we hide it from those who do.
Our Next Chance
We have a completely new technology to deploy now, and it is happening within the same incentive system, with the same engineers, and faster than ever. The technology is much more powerful. The blast radius is much larger. We are not just shaping what people read and how that affects their thinking. We are affecting the entire conversation that leads to their thinking. We can completely remove the critical thinking step through manipulation during conversation. We must do better this time. And maybe we can, by empowering more people.
Empowering All to Change the World
The world needs us all. My dream of open information was born out of this idea, democracy thrives when we have honest, truthful discourse available to everyone. I feel the same about the ability to create. This is embedded in the capitalist idea that the market will choose the best option. But the best option cannot exist if the means of building it are gatekept by high costs of living, education barriers, and knowledge barriers to creation.
We need creators with the right incentives watching the right metrics. A nurse who deeply cares about her patients should be able to build the tools to help other nurses. Someone who cares deeply about helping children learn to make friends should build kids’ social networks. As long as the tools of creation are limited to a few profiteers with the wrong incentives, we will not solve problems, we will seek capital. The Meta lawsuit is a strong signal that we are not building things that make society better. Their disinterest in helping society shows neglect and disrespect toward the people they claim to serve. Are these really the people we want to entrust our society to? We cannot trust the legal framework to solve this, it is just the basement enforcement pattern, not an incentive to do right.
Organic Software
Software is the bicycle of the mind. AI is the steam engine. These tools are meant to amplify human achievement. The way we design them can aim to bring out the best in society, or the worst, or they can simply focus on capital and let society become whatever it will. Much of what we see today is manipulation that serves the ego of creators and the economics of developers.
Software should seek the maximum in human happiness. It operates in the realm of the mind. It should be measured by the way it affects human lives. Let’s imagine a world with software we actually love.
What would a social media network that genuinely helps you create human connection, make friends, and build community look like? The closest Facebook ever got to this was laughably weak. One wonders if they ever seriously considered it. Maybe it is time we shift who builds these tools, toward people whose concerns align not just with engineering, but with deep domain expertise in the lives they are building for. What if instead of AI taking our jobs, AI made society productive enough that the four-day work week was common? Or the three-day work week, and no one was unemployed? We could choose to go there. It would require deciding that human time or happiness has more value than shareholder return. Rules and values of society are determined by us, we can change them.
We can move from the information age to the social age, I think we deeply need to reconnect as a society and world. Technology can help us by giving us more time to focus on each other, or building tools that drive human connection.
Working With Education
At one point in time, the smartest person in society was the one who knew how to make fire. I imagine this person was central to their community and deeply important. But then technology evolved, and knowing how to make fire became common, then simple, because of improvements in the tools used to make it.
At that point society shifted. It was no longer making fire that mattered most, but harnessing fire to improve society; through better cooking, heating, and eventually firefighting. A fundamentally different skill, but essential to the evolution of society.
We are at a similar place now. As a programmer, educator, or anyone else in society, it is disorienting to have your sense of expertise dissolve in front of you. We need to understand that to evolve, we must reposition what we value. And I would argue that what we have valued up to this point; while beneficial for a group of engineers, educators, and entrepreneurs, has not been as beneficial to society as it could be.
At some point our values need to shift from prizing technical knowledge toward wisdom in how to apply technology toward human flourishing.
It is time we take a hard look at the kinds of people we are producing for the workforce, because we have gotten it wrong. The complexity of the systems we needed to build made that almost inevitable. But we are now building the abstraction layers that allow us to train a new kind of engineer, not one who has to master complex logical expression, but one who grapples with the complexities of the world we actually live in. We have a chance to realign technology toward human flourishing. But it starts with the people who build these systems. And I think it needs to be everyone.
I am not in these rooms because I have all the answers. I am in these rooms because I have seen what happens when the people building the tools never ask the right questions. I helped build the anxiety machine once; not because I was careless, but because I was looking at the wrong things. We all were.
The most important technology decision we will make in the next decade is not which model to deploy or which platform to build on. It is what kind of person we train to make those decisions. An engineer who only knows how to measure system uptime will build systems that stay up. An engineer who understands human flourishing will build something worth keeping up.
This post is a continuation of The Split. For an understanding of how to enable everyone to write code, start there.