Multilevel Selection (MLS) is a theoretical framework that originated for the study of genetic evolution but is just as useful for the study of cultural evolution. MLS theory begins with the fact that evolution takes place in a nested hierarchy of units, such as from genes to ecosystems in biological systems and from small groups to the global scale in human systems. Natural selection at the smallest scale--between individuals within socially interacting groups—tends to favor disruptive self-serving behaviors. It is only at the scale of between-group selection that cooperation within groups is favored, and cooperative groups become selfish actors in between-group competition unless selection acts at still higher scales.
An intuitive way to appreciate the logic of MLS theory is with the game of Monopoly, where the objective is to own all the real estate and drive everyone else bankrupt. This is competition among individuals within a single group, pure and simple, and it provides almost no scope for cooperation among the players. Now imagine taking part in a Monopoly tournament, where the trophy goes to the team that collectively develops their real estate the fastest. Almost every decision that you make as a member of a team in a Monopoly tournament would be different than as a player in a single game of Monopoly.
This logic repeats itself at every rung of a multi-tier hierarchy of groups. In human terms, individual preservation is a good thing—until it becomes self-dealing. Helping family and friends is a good thing—until it becomes nepotism and cronyism. Nations growing a strong economy is a good thing—until it overheats the earth.
Despite its simplicity, MLS is a new perspective in economics and public policy. Against the background of orthodox economics, which seeks the foundations of macroeconomic theory in the behavior of individuals (Homo economicus) it is paradigmatically different. Even behavioral economics, which has gone beyond orthodox economic theory in many respects, remains rooted in a form of individualism that MLS theory goes beyond.
Yet, when we liberate ourselves from the assumptions of formal economic theory, cultural MLS is taking place all around us. New cultural innovations originate in a certain time and place, spread to a degree on the basis of their success, and diversify along the way. Eventually, they come up against boundaries, like the geographical distribution of a biological species, beyond which they remain unknown. And time is required for all of this to happen, which can range from years, to decades, to centuries, depending upon the speed of the cultural evolutionary process.
The combination of a theoretical framework that is simultaneously new and deeply familiar also describes Darwin’s theory of evolution. Pointing out the importance of variation, selection, and heredity was new, but it made sense of volumes of natural history information that already existed, in addition to organizing the search for new information.
With this in mind, there is added value in examining a cultural practice called Agile, which originated in the context of software development, has diversified to a number of other contexts (mostly within the business world) and is largely unknown beyond those borders. Our guide is Laurent Alt, who became part of the Agile movement early in its history and also has become literate in cultural MLS theory and its practical applications in real-world settings. Thus, not only is Laurent qualified to serve as our guide about Agile but also to comment upon the added value of thinking about Agile from an explicitly MLS perspective.
David Sloan Wilson (DSW): Greeting Laurent! Please introduce yourself, realizing that you will be known to some of our readers but not to others.
Laurent Alt (LA): Hi David, first thanks for having me in. I am from France, I have a scientific background, as an alumnus of the Ecole Normale Supérieure in Paris. After some experience in software development and leadership, I became very interested in why people behave the way they do. With my scientific background, I found that behavioral science was the model I needed. Not only to understand but also to make change happen in a sustainable way. I first started with Organizational Behavior Management (OBM), and then was also very enthusiastic about how this plays with Acceptance and Commitment Training. And as I went up the ladders of organizations, I got more and more interested in how this all works together in behavioral systems. I am now an Associate Director at the Boston Consulting Group (BCG), advising large organizations on how to implement Agile ways of working at a large scale, which has impacts on all aspects of business and organizations, and I am also coaching executives on their way to Agile.
DSW: Thank you! Part of an explicitly evolutionary approach is to ask four questions about any product of evolution, concerning its function, history, mechanism, and development. Let’s begin with the historical question. What is Agile, when did it originate in time and space, how did it spread and diversify, and what is its current “distribution and abundance”, to use a phrase from ecology? Please add links and footnotes so that our readers can easily learn more.
LA: Agile is rooted in the software industry of the 90s when software engineers found out that classical project management methods used in other disciplines like construction or manufacturing didn’t apply well to software. Software project failures and delays were commonplace, due to misunderstandings between customers and engineers. To solve this problem, several practitioners started thinking about iterative development and frequent feedback from their customers. They found that they were doing things quite similar, and they gathered to work together and define what they agreed on. This gave birth to the Agile Manifesto in 2001, with the core values and principles that are the foundations of Agile.
Since then, Agile has spread across various organizations. It started in small software houses, primarily by software engineers who wanted to bring a new mindset in companies together with new technology.
It became obvious that Agile made it possible to create better products, faster, and all this while bringing a lot of fun and motivation in the teams. Some of the startups became big companies, which proved that Agile could scale. One very well-known example of this is Spotify, but companies like Google, Amazon, and others are using Agile principles too.
These results drew the attention of other well-established corporations and big consulting companies like BCG, which wanted to take Agile to a larger scale, and also outside of software or IT organizations. Now, we see Agile working in IT, marketing, hardware engineering organizations. The most visible illustration of this mindset of iterative product development is how SpaceX designed, built, and flew a vessel fully equipped with sensors, learning from the data, and made the next version of it.
DSW: Fantastic! It is obvious from your description that iterative product development is an explicit process of evolution, with a rapid “generation time” of variation, selection, and replication. Now let’s proceed to Tinbergen’s Function question. How would you describe the functional principles that make Agile work? Are these principles explicitly recognized? Are there any that are more implicit—in other words, that work without anyone knowing how or why they work?
LA: In his book The Modern Firm, D.J. Roberts describes how successful organizations need to align their strategy, organization, and environment. Agile contains two evolutionary elements that make this possible and sustainable. One is the ability of agile organizations to adapt the products they create to an evolving context through iterative product delivery and customer feedback, and the other is the ability of Agile organizations to improve their own way of working through self-reflection on internal routines to better deliver products. The reason why Agile sticks is by its ability to constantly evolve on these two dimensions. The Agile ways of working are actually indirectly selected by the customer buying or not the products.
DSW: Let’s dwell on this. First, our readers should know that most business corporations are not very good at adapting to changing environments, no matter how well they are performing at the current moment. Instead, most innovation takes place by what Joseph Schumpeter called creative destruction: businesses being replaced by other businesses. In evolutionary terms, being well-adapted to one’s current environment is not the same as being adaptable to environmental change, including intentional change efforts such as new product development. A good book on this subject is The Ambidextrous Organization: Exploring the New While Exploiting the Now, by Jens Maler.
You are saying that Agile helps an organization become more adaptable in two respects: The services it provides and its internal organization. In both cases, adaptive change is accomplished by an iterative process. There is some target of selection, different ways of achieving the target are explored, better solutions are adopted, and the process is iterated again and again. Please describe how this takes place in more detail, perhaps with an illustrative example or two.
LA: Well, let’s start with how a company delivers products. Going back to software companies when delivering software products to their customers, they pay attention to all the customer feedback they can get, and drive the evolution of their products by adding the features that their customers most need. But they also organize internally to do this as frequently as possible, by delivering product improvements in small batches rather than doing one new version every year, for example. This creates faster feedback loops and improves the evolution speed. The ability to collect customer feedback has radically improved with the cloud and mobile apps since companies can now collect massive feedback in real-time. Today, Amazon is known to deliver improvements to their platform every ten seconds or so, usually very small ones. They have reached a level called “continuous delivery.” And if a modification in the platform does not bring positive change, it can be rolled back as easily as it was introduced. Their platform is becoming kind of a living thing.
Now looking at how Tesla can update their cars remotely with new features, we see that this is exactly the same Agile process applied to very different products. One may observe that cars are merely big software products and that updating a car remotely does not change the story.
But let’s now look at SpaceX. A few weeks ago, the media commented on the fact that one of the rockets exploded right after landing, and some commented on this as if it was a failure. Actually, this is part of their product development process. SpaceX rockets are full of sensors, and all the flight data are recorded to later be reused and improve the next version of the rocket. It may seem expensive to build and lose whole rockets to develop the right product, but this is exactly what an Agile product development process looks like. This is a sign that we are more and more competing on time, not money.
It is also interesting to notice that it is a software guy, Elon Musk, who started as the founder of Paypal, who brought this Agile mindset to Tesla and SpaceX.
DSW: In evolutionary terms, this is a carefully managed process of cultural evolution—but do the people who develop and use Agile actually think of it that way? How do they describe it to themselves? And even if they use words such as “evolve” and “adapt”, do they think of consulting evolutionary science as a source of insight?
LA: In the 2000s, Agile professionals mostly described Agile as a group learning process. One of the key meetings – Agile folks call them ceremonies, not meetings – is the retrospective. Every other week, the team members list what’s good in their way of working, what needs to be improved, pick one or two things to change in the forthcoming weeks, and test them. They keep what works and drop what doesn’t, and so they improve. Some authors describe Agile organizations as Complex Adaptive Systems (like Sunil Mundra in “Enterprise Agility”), illustrate evolution as cross-team pollination (like in Spotify’s videos), or refer to Ostrom’s work (like Mary Poppendieck in her keynote at Scrumday Paris 2015). But this is not the biggest part, unfortunately. Since Agile has now become a big business, it has also become a battlefield for models and certifications, and there is much more media coverage now about how to do Agile right than about how flexible Agile can be.
DSW: So far, we have described Agile as a process that makes an organization adaptable. Now I’d like to focus on the role of small groups in Agile and how those groups are organized. Later, we will compare Agile to Elinor Ostrom’s Core Design Principles, but first I want to know how the Agile Community first recognized the need for interactions to take place in small groups and then derived its own set of principles.
LA: The good news is that this is where all Agile schools actually meet! Everyone agrees that at its core, Agile is based on small autonomous groups of six to nine people (sometimes called “two-pizza teams”) having end-to-end ownership on a part of the product. These Agile teams include all the skills to do the job, so they are usually “cross-functional”, meaning they are across typical organizations based on hierarchies. For example, a typical Agile team can include 1 marketer, one UX designer, three software developers, one data scientist, and one tester. In order to scale Agile, companies gather several of these small teams together in a Tribe, and when the Tribe reaches the Dunbar number (100 to 150 individuals), they think of creating two Tribes instead of one. Typically, people inside Agile teams interact every day, but people from two Agile teams inside the same Tribe interact less frequently. So these are nested social groups.
DSW: Wonderful! I am reminded of the last print conversation I did—on the seemingly completely different topic of Ancient Greece—but illustrating the same principles! The Athenians invented a multi-tier social organization in which the smallest units, called demes, were nested within larger units called tribes. The tribes were intentionally constructed to be diverse, including demes from coastal, rural and urban areas. This was part of the social engineering that enabled Athens to function as a democratically governed city-state. Closer to the present, there is also convergence with Lean methodology, which evolved within the manufacturing industry. My favorite book on that topic is The Toyota Kata by Mike Rother. Please compare Agile with Lean, historically in addition to functionally.
LA: Agile and Lean do not share much in their origins - they come from two very distinct business areas, and were born at different times - but interestingly, they share a lot in how they work.
Historically, Lean started at Toyota and is based on demand-driven production flow. The idea was, rather than create massive stocks of cars in advance and try to sell them, to have the production be pulled by customer demand. It was introduced after WWII as the context in Japan was very different from the times when Ford had started the Ford T, where mass production was okay. Back then, Toyota had a small addressable market, and Toyoda Kiichiro said he wanted to catch up with America in three years (Taiichi Ohno 1978 - “Toyota Production System – Beyond Large Scale Production”). But things had changed since the beginning of the century and the Ford T, there were then different models and colors. The ability to be profitable by delivering such a diversity of cars on a small market like Japan required a new way of thinking.
Working for just-in-time delivery draws upon the ability for the system to self-adapt and learn to optimize flow in a moving context, and this is how Lean actually works. Of course, when talking about Lean, the first thing that comes to mind is about cutting costs; this is how it is been unfortunately depicted decades ago by consulting firms. Even the word Lean itself is a creation of westerners that does not serve its cause. For clarity, let’s keep the word Lean but let’s also consider its meaning in the light of the Toyota original mindset. Lean is about removing all kinds of things that get in the way of natural production flow. They come in three main categories: muri (unnecessary constraints and burden), mura (variability), muda (useless work). And since these are contextual to the work actually being done at the team level, there is no such thing as a big top-down decision about what to do or not, but rather a decentralized learning system, based on frequent retrospectives, as described earlier about Agile.
Later, when Agile methodologies started in software development, Mary and Tom Poppendieck drew a parallel between Agile and Lean, showing how Agile was very similar to Lean when applied to software development practices, and more generally to intellectual work. In this context, muri compares to stress or cognitive load, mura to changing priorities and task switching (which we know humans are not gifted at), and muda to repeating information or doing long useless meetings (where humans excel).
Both systems also rely on a global and synchronized feedback system that operates at all levels of the organization for all teams, based on real data. From the beginning of TPS, Toyota measured all sorts of production KPIs (Key Performance Indicators) as the backbone of their improvement decisions. It is key to not overlook how important this measurement system is.
Historically, even though the Lean body of knowledge was long known when the Agile manifesto was written in 2001, Agile was clearly a new thing that emerged as the convergence of several new software development techniques. It is surprising to see how many similarities these two approaches have knowing that they come from two different business environments.
DSW: That’s what convergent cultural evolution is all about. And the importance of a measurement system maps onto Ostrom’s fifth core design principle—monitoring.
I could list more examples of convergent cultural evolution, such as a business change method called Rapid Results and even how military groups must be organized to adapt to rapidly changing conflict situations. One thing that distinguishes ProSocial World from these many examples of convergent cultural evolution is its general conceptual framing. Once we understand the salient principles as necessary for cooperation and adaptation in all their forms, we can apply them across all contexts without being trapped within a single context, such as manufacturing, software development, or the military. With this in mind, let’s compare Agile with the generalized version of Elinor Ostrom’s core design principles, shown in Figure 1. Has there been complete convergence? If not, then what explains the non-overlap?
LA: There is a strong correspondence between Ostrom’s view and Agile, in particular when looking at how to make organizations sustainable. But at the same time, I do not find a clear mapping between CDPs and Agile practices or principles. CDP1 is brought to the teams by design, by the mere fact that teams are created to deliver a given product for a given customer. CDPs 3 to 8 are implicitly supported by the strong focus on group cooperation, which appears in how the Agile ceremonies are designed, and also by the existence of a specific scrum master role in Agile teams to deal with behaviors. CDPs 6 to 8 are also made easier to manage by having teams' work measured and all discussions being held in regards to data. Finally, knowing that we are very often talking about employees, CDP 2 is managed by giving some autonomy to team members to self-assign their work, as long as it aligns with purpose and goals, which is a means to align business constraints with the member's values or interests.
DSW: It is quite interesting that the mapping is indirect and leads to the question of whether Agile teams might function even better if the CDPs are made more explicit. This is a good opportunity to highlight two other aspects of Elinor Ostrom’s conceptual framework, in addition to the CDPs: Auxiliary design principles (ADPs) and the distinction between a functional design principle and its implementation. The CDPs are called “core” because they are needed for cooperation, which is a common denominator for all groups. Additional design principles are needed by some groups but not others to accomplish their particular objectives. For the groups that need them, the ADPs are as important as the CDPs. Are there aspects of Agile that are important for software development but perhaps not for other kinds of groups?
LA: I have led teams or consulted with clients in numerous contexts and industries, and I found the same basic ideas still hold. There are variations according to whether you are in a software, hardware, or service environment, whether you are in a start-up or in a well-established corporation, but not as much as to say there are more principles. So far, I have always managed to stick to the Agile Manifesto, the core ceremonies, and the key roles. The only trick is that the Agile Manifesto explicitly refers to software, but replacing it with product does the job.
DSW: Turning to the distinction between a functional design principle and its implementation, this is a one-to-many relationship. Just as there are many ways to skin a cat, there are many ways to implement a given design principle and the best way for any given group might be highly contextual. Cookie-cutter solutions are unlikely to work and every group needs to experiment to find the best way for itself. What you said earlier about “a battlefield for models and certifications” suggests that some forms of Agile might be erring in the direction of cookie-cutter implementations. True?
LA: This is true for the companies that have a business based on massively replicating the same work over and over, they rely on “off the shelf” approaches and certifications and stick to this. I cannot say it is not important to know the basics, neither to rely on certifications. I have certifications myself, but it does not compare to my fifteen years of experience in Agile in various companies and industrial sectors. I rather see it as a natural evolution of the consulting market. As Agile is now reaching the top level of firms, it takes the shape of an adaptation of the core principles to the constraints of executive leadership. While big firms like BCG are taking this share, others are taking the easier and more replicable part, and much more rely on models.
DSW: Thanks! As you know, ProSocial World draws upon a mindfulness-based technique called Acceptance and Commitment Training (ACT) in addition to Ostrom’s framework. This involves an exploration of inner thoughts and feelings that lead to counterproductive behaviors and a commitment to working around them to achieve the goals of the group. Has Agile converged upon these methods as well?
LA: One of the recognized added-values of Agile is that, when done well, it provides great results in terms of well-being and motivation. Agile does not explicitly mention anything about ACT, but if we look closer, how does agile work? Every two weeks, teams deliver product features, collect feedback and adjust their way of working based on retrospectives. This very much looks like a transposition of ACT to groups, with frequent short actions towards value (value for the customer), and retrospectives to reflect. There is such an obvious fit that I am convinced that ACT will progressively become part of the Agile body of knowledge.
DSW: Please say more about how Agile principles can be taken to the scale of a large organization. How does it go beyond two-pizza groups and tribes? Is there a point at which an Agile social organization must fit within another kind of social organization?
LA: David, this is what makes Agile so interesting!
When scaling from teams to Tribes, the main point is in coordinating teams that have some autonomy. This is usually not too difficult, since it is a lot about processes.
When scaling one level up, to organizations in the thousands or tens of thousands, we need to connect Agile ways of working with non-Agile processes, that cannot be easily changed because they are compliant with regulations or compliance constraints. And this is also where we face cultural issues between groups of professionals who have a very different view of the world while working for the same company. For example, it is pretty hard to talk to finance about empowered teams delivering work iteratively, knowing that these people have such a strong pressure to report accurate and compliant financial results to markets. But even for finance, an approach called “Beyond budgeting” – that very well fits with Agile - is taking off. At the executive team level, with tons of pressure on each member of the leadership team, a shared sense of purpose can become the alignment factor that makes it possible to make big changes in all corporate functions.
DSW: I’d like to share the opinions of an associate who works for a big tech firm that employs Agile and isn’t very happy with it. My associate likes the basic concept of Agile, as described by the manifesto, but thinks that it has been corrupted by the “Capital A” agile industry. He thinks that Agile made the most sense at the beginning of the software industry when products were being actively developed for clients. Now those products have already been developed to a high degree of sophistication, so it is hard to see what requires rapid iteration in the relationship with a client. And when Agile is used within a corporation, there is an insidious tendency to replace real performance outcomes with proxy metrics, resulting in pointless bean-counting. There isn’t a lot of room for innovation in the projects that are assigned to Agile teams, and some members of cross-functional teams just remain idle. And the amount of time spent in meetings, according to my associate, is “endless”.
You have hinted at some of these problems in your previous answers, but how would you respond to my associate, either by way of disagreement or advice about how to overcome these limitations of Agile?
LA: This does not surprise me, unfortunately. What your associate is describing is usually what happens when fair Agile practices hit the lack of flexibility of existing processes in organizations. It often happens with budgeting, compliance, or even legal, where the groups involved have a duty of preventing business issues to happen, and by doing so also prevent a lot of good things to happen. It is often believed that Agile is very permissive, but it is not: it is simply a new balance between empowerment and control. It replaces a priori top-down directives with empowerment and a posteriori checks but does not remove verification.
So the solution is to onboard these control functions in Agile too, and bring enough flexibility in their processes so that they can still achieve their protective role while taking part in the achievement of the company’s purpose.
The pressure on businesses will increase, due to more regulations, more transparency, better pressure on quality. The problem is that in the last years, we have seen that as the business complexity tends to increase, organizations react by creating more internal “complicatedness”, which indirectly creates more rules and coercion. This has been very well shown by studies led by Morieux and Tollmann. The need for better ways to deal with complexity will accelerate.
DSW: I’d like to end with the analogy of a cultural innovation such as Agile as like a biological species that originates in a certain time and place, spreads and diversifies to a degree based on its success, but then comes up against barriers, like the geographical distribution of a species, beyond which it is unknown. What are the barriers that are limiting the spread of Agile? Realistically, where do you think that Agile will be ten or twenty years from now? What can be done to catalyze the cultural evolution of Agile—or even better, the explicitly recognized active principles that make Agile function well as a highly cooperative and adaptable form of social organization?
LA: Technology and culture are evolving so fast now, that it is really difficult to answer. It is very tempting to think of the future of Agile by simply extrapolating what we know today, and replicate current Agile practices to ecosystems of corporations, to nations, or to new technologies. But this would certainly be too restrictive. I find it necessary to isolate the very core principles of Agile and restart from there. And to me, these core principles are simply “purpose with data”. Groups act together because they share a common purpose, and the ability to do this in a sustainable way relies on their ability to reflect on feedback from real data. Today’s Agile relies on this. It started in software this way, this is how SpaceX works today by testing launchers and collecting data to improve, and I believe organizations and society will evolve thanks to this ability. We will find new ways of working thanks to these mechanics.
And, to fully answer your question, the barriers I see will essentially be the availability of data. Which data will be available, and to whom. Those who will own the data will influence the evolution of society and organizations (well, even more in the future than today).
DSW: Thanks very much, Laurent, for this deep dive into Agile! I look forward to continuing to work with you in the future
LA: Thank you for having me, David. Let’s keep connecting our insights of research and on the field experience to help create more sustainable organizations.
