Chapter 1. What are the components or participants of a good behavioral science team?

Behavioral design requires a multidisciplinary approach, something our field has agreed on since its inception. Our teams must therefore also represent a variety of backgrounds and skillsets. In building or developing a “good” behavioral science team, its exact composition should balance the delicate interplay between the nature of the organization within which it operates, its goals, and the role or identity of the team within this broader business context. 

Most people don’t ever decide on a strategy or goal before just stuffing their backpacks with the first things that come to mind. This lesson is helpful in thinking about how to build a behavioral science function within your organization. There’s an impulse to just start stuffing our backpacks with tools. A data scientist! A theorist! A consultant! A designer! But it’s almost always better to first think about strategy. This may sound obvious, but it is surprising how often we might overlook it. When organizations reach out for guidance on how to set up a behavioral team, my first question is always some variation of “What do you hope to achieve?” Only a handful have ever had an immediate answer to this.
– Zarak Khan

1.1 Composition

Academic background, research, design, and communication skills should all be taken into consideration when considering the composition of a behavioral design team. Depending on the industry and type of organization you work in, the best behavioral design team might come from a variety of disciplinary backgrounds. 

For those of us in technology-centered or -related industries, this might involve being in a team with experts including engineers, designers, product managers etc. 

“ The best behavioral science teams are diverse, interdisciplinary and cross-functional. They mesh product managers and engineers with designers and anthropologists, business leaders and strategists. And what are they not? They are not comprised entirely of behavioral scientists, sitting in the corner chatting about heuristics and drinking their own kool-aid while the rest of the organization carries on with the important stuff.”
– Aline Holzwarth

For those of us in organizational and industry contexts leaning more heavily towards applied research, we might find ourselves in a team representing different verticals of behavioral science. 

“From my perspective, a good team consists of a diverse group of people with various backgrounds paired with a solid academic qualifications in behavioural science. Good applied behavioural science practice requires not only knowledge of relevant academic insights and research methods, but also the ability to evaluate, consolidate, structure and connect them to real-world challenges.”
–Torben Emmerling‍

Diversity is another crucial axis to map onto in behavioral design team composition. Beyond academic and disciplinary background diversity, the best behavioral design teams also pay attention to diversity across intersections of age, gender, race, ethnicity and professional experience.

‍“The dream team will consist of millennials and boomers. Diversity is non-negotiable for truly understanding behaviour in context. A cross-section of public and private industry expertise always enables deeper understanding of the mindsets of customers, employees, and suppliers. Multi-disciplinary roles should combine designers, psychologists, artists, data visualisation and UX/CX, experts trained in statistics, experimentation, and key theories. When assembling teams, I tend to always look for hunger and intellectual curiosity.” 
– Nuala Walsh‍

Diverse teams show higher competence at solving complex problems, and behavioral problems are nothing if not complex! Diverse teams also bring with them a diversity of lived experience, the bedrock of empathizing with and designing solutions for a variety of user populations. 

‍“The research shows that diverse teams perform better at solving complex problems and creating innovations. You need a team with a wide range of experiences across a wide spectrum of dimensions. You need people with formal academic training in the behavioral sciences, but also people who have experience getting things done in the real world. Ideally, you will also have people from different academic disciplines, as most “new ideas” are not de novo innovations, but rather about bringing together existing solutions from other fields. And crucially, you need people from different backgrounds, cultures, and with a diversity of lived experiences.”
– Josh Wright‍

Importantly, not all behavioral design work begins with the formation of a whole team. More often than not, organizations new to the field hire one behavioral scientist to cover all of their behavioral questions and design needs. Carlos Hoyos articulated that this is essentially a “unicorn behavioral scientist” who needs to know enough about a spectrum of methods, applications and skills to address and navigate a myriad of behavioral problems and solutions. 

‍“So, basically you need a unicorn as a behavioral scientist to actually excel in every job. And the difficult part is that many of these skills don’t match with each other (a methodological scientist who is also creative and also knows how to communicate well? Really?).”
– Carlos Hoyos‍

For those of you thinking about adding behavioral design expertise to your organization, or perhaps you have already been working with that one unicorn behavioral scientist, consider realizing the potential of behavioral know-how for your organization by expanding the team to fill at least four archetypes: the manager, the researcher, the creator, and the scientist, as recommended by Carlos Hoyos. 

‍“That’s why I think a powerful Behavioral Science team should be built with the following archetypes:

• The manager: an expert in business and team management, as in public relations. Their responsibility is to make sure the intervention is aiming at the right problem and to attain buy-in from stakeholders.
• The researcher: an expert in understanding human behavior through different research methodologies and discovering the right insights and problems to work on.
• The creator: an expert in problem-solving and intuitive thinking and the keeper of innovation and creative solutions.
• The scientist: an expert in experimentation methodologies and the keeper of the reliability of the information. 

This doesn’t mean you need one person for each archetype or that they only work on that specific job, but you do have to make sure that your team can cover these 4 archetypes. Maybe you can pull it off with 2 talented people or maybe you need whole teams for each role. The important thing is that we understand that solving behavioral problems is not possible with just one universal Behavioral Scientist role and we, as practitioners, should develop different skills and maybe specialize in some to later collaborate in synergy.”
– Carlos Hoyos ‍

Similar roles are combined in several successful behavioral design teams, creating a blend of disciplinary backgrounds and individual strengths that lead to impactful behavioral design work. ‍

Blended teams allow us to dedicate specialist knowledge to each stage of the behavioral design process. We believe each team should consist of a minimum of four people. Each represents one of four key disciplines. At Cowry, our Behavioural Researchers conduct implicit and explicit research to unearth valuable customer insights. Our Behavioural Architects use secondary research and models to identify barriers and create a series of strategic & innovative interventions. Our Behavioural Designers apply neuro design principles to refine and execute the interventions in the form of engaging conceptual designs. And our Experimental Designers implement the solutions ethically and responsibly, test for efficacy, and use the results to build a wider picture of the impact of our nudges. Collaborating across areas of expertise ensures that behavioral science is implemented across all stages of the creative process and overall leads to the construction of more creative, balanced, and effective solutions.
– Raphy March‍

Once the journey towards building a behavioral design function has begun, supporting the integration of this expertise is an ongoing process and requires its own time and resources. 

‍Second, we need people to have a shared understanding of how to make change happen.  Having a expert in a separate unit is a good start but this will hardly result in transformational outcomes:  change happens at the coal face of everyday decisions, often made in hurry.  This is why expertise needs to be codified in a way people see the value and feel equipped to apply it.
– Colin Strong‍

1.2 Skills

Regardless of background, members of the best behavioral design teams are competent in a variety of expertise-specific and transferable skills. Impactful behavioral design requires skills ranging from research, design, and communication, to problem-solving, creativity, and stakeholder management. Behavioral design teams are successful when they synergize each other’s backgrounds and skills in the context of their behavioral science field of application. 

“In addition to technical skills, a good team has highly developed communication and listening skills to properly define and understand a behavioral problem in its individual context, to address and explore it efficiently, and to articulate its findings effectively. It should also be able to successfully deliver within time and resource budgets (project management skills). Moreover, it includes design expertise capable of creating, trialing and analysing interventions in both physical and digital contexts (design skills).”
‍– Torben Emmerling‍

The best behavioral design teams are deeply skilled in the technicalities of research methods, specifically, intervention design and evaluation. Combined with stellar communication skills, behavioral design teams become powerhouses of insights that can be shared and resonate across entire organizations. 

‍“A good BE team is ideally comprised of people with deep technical skills, which are harder to teach, balanced with consulting skills which are easier to teach (presenting, managing stakeholders, driving forward programs of work and drawing tight links between research and business objectives). Technical skills refer to core training in judgment and decision making, experimentation, data analysis.”
– Etinosa Agbonlahor 

For behavioral design teams embedded in digital intervention and technology product design, technical skills might also involve those relating to the technology being used to deliver the intervention. It will be very beneficial for such behavioral design teams to collaborate with engineers and other technical experts to leverage technology for its full potential in intervention design.‍

An overlooked quality in excellent behavioral design is a collaboration with technical teams and deeply understanding any technology used in an intervention. As behavioral designers, we have an opportunity to leverage the available technology to the fullest in our interventions—but only if we understand how it works in some detail. I work in digital health, and can think of times when I didn’t use a technology feature that would have improved the intervention because I didn’t realize what it did, and when I included a feature in my design that required my technical partners to create extra code that probably wasn’t worth the effort in terms of the benefit it added to the product. In my current role, our behavioral science team works hand-in-hand with our AI scientists and platform engineers so that neither of these scenarios happen. We end up in a virtuous cycle where behavioral designers make the most of our technology, and our engineers and AI scientists work on new technical functionality to enhance the power of our interventions. Behavioral designers don’t have to know how to code, but they should know how to talk to the people who do. 
– Amy Bucher‍

Beyond the technical and creative behavioral science and design skills, communication skills are at the top of the priority list for the best behavioral design teams. ‍

“All insights are only valuable if delivered in an actionable way. Applied Behavioral Scientist help with framing messages, and they must apply that to their work.
- Connor Joyce‍

Behavioral design teams have their work cut out for them when it comes to developing and practicing impactful communication skills that cut through the noise of user needs and values,  behavioral and complex problem data, intervention design and development processes, evaluation assessments, stakeholder interests, and organizational or business value. What we might encapsulate in the category of storytelling skills, behavioral designers need to be proficient in communicating the process, impact, and value of behavioral design to a variety of audiences. ‍

Good storytelling. No matter how rigorous your quantitative analysis or randomized trial, people need a good story to buy into moving forward with behavioral design solutions. This is where a team that is equipped with data storytelling and visualization, the ability to capture and retell insights directly from users, and a strong capacity to build compelling narratives does the best work.
- Rachel Rosenberg‍

Communication often centers on navigating stakeholder interests, identities, and objectives. Behavioral design teams need to be tapped into all of these and invest the time to create and maintain clarity and cohesiveness in stakeholder positions in reference to their work to achieve the best results. ‍

Stakeholder management. As behavioral design teams show their work and tell good stories, often the final hurdle is influencing key stakeholders. Every team stands to benefit from a team lead that forms strong relationships with product and organizational leaders as one of their primary roles. And further, team members should embed themselves within teams, so as to build strong relationships with designers, researchers, engineers, and managers. These bonds are ultimately the glue between rigorous, hypothesis-driven solutions and implementing experiments that result in real (and iterative) behavior change.
- Rachel Rosenberg‍

Communication skills are, arguably, the difference between impactful and unsuccessful behavioral design work.  ‍

50% of our cortex is dedicated to visual processing. So, although how we describe products and services to customers is important, how we visually communicate that information is equally important. If we fail to guide the reader’s attention to the information we need them to read or the action we need them to take, we undermine all of the behavioural insights, strategy and science that has gone into the creation of an intervention.
– Raphy March‍

1.3 Team identity and role

The significance of stellar communication skills for behavioral design teams goes beyond explaining their process or sharing research findings or design insights. Within different organizational contexts, the behavioral design team will need to facilitate and build strong relationships with a variety of stakeholders, thus ensuring the team’s longevity and growth. ‍

In the private sector, ideas are adopted based on how well they are communicated. Although our work is grounded in science, many behavioural scientists give too much prominence to academic theory meaning that the overall value isn’t articulated to the client. This can lead to a lack of repeat business and reduce the likelihood of implementation. Behavioural scientists should be thinking like consultants if they want to build strong and long-term relationships.
- Raphy March‍

For both new and mature behavioral design teams, reflecting on and intentionally designing their role in the organizational context they work in will bring valuable insights into how to best leverage their processes, competencies, and ultimately, demonstrate their value in achieving organizational outcomes. ‍

Beyond the group's composition, where they form will play a significant role in how they create value. Behavioral science teams have successfully embedded themselves into larger product teams; meanwhile, others have become shared services and act as internal consultants. Once established, they must determine if they desire to work on current or future products and if they desire to work directly on products or instead utilize research to better position them. The benefit of being close to the product means immediate impact but at the cost of competing with other established groups to have one's recommendations followed. Alternatively, working on research that can help set direction can generate significant value at the risk of delayed impact.
- Connor Joyce 

As behavioral design teams and organizations seeking to add or already working with behavioral design teams, we will be set up for longevity and impact if we take into consideration the variety of behavioral science and design application areas, organizational contexts, and team compositions and base our navigation of each team’s role and identity on its unique position, value-add, and scope of work.    

While applied behavioral science has grown considerably, it is still uncommon in organizations and the work done by such teams varies quite a bit. As such, the expectations for a behavioral science function are likely to be very different between its stakeholders and collaborators. It’s all too common for these inconsistencies to result in too wide of a scope of work and inconsistent projects. It’s on behavioral scientists to remove that confusion by identifying the intersection of organizational need and their capabilities that can provide the most value and simplify it for easy understanding. It’s essential that behavioral scientists are able to circumvent [organizational] constraints and produce valuable work independently. Of course, the team should strive to remove organizational barriers and enhance their capabilities over time, but those working in new specialized functions don’t have the time to wait. They must provide demonstrable value in the short term to earn the trust and investment to create future opportunities.
– Erik Johnson ‍

1.4 Culture

A behavioral design team might have a formidable composition of disciplinary backgrounds and skills, but without an empowering culture, the team’s impact is unlikely to be sustainable or achieve buy-in from a diverse set of stakeholders. ‍

“An ideal team should, of course, enjoy working and finding creative solutions to behavioural problems together, taking into account as many different perspectives as possible. Good collaboration and good fun is so important.”
– Torben Emmerling‍

How to define team culture and its components is up to every individual team and their organizational context, but some foundational principles to consider include mindset, feedback mechanisms, trust-building, and values. ‍

Good teams usually have a culture of candid feedback, willingness to think outside the box when partnering with other disciplines (for example, how do we reconcile the qualitative HCD approach with the behavioral scientists preference for quantitative RCTS; what systems do we build to allow us pushback with stakeholders when we run into ethical grey-areas), and ideally are intrinsically motivated by the work they do.
- Etinosa Agbonlahor 

To fully realize the potential of a behavioral design team we might want to focus on creating culture add, instead of culture fit, and create and environment where team members can flourish in their individual areas of strength and learn from each other to achieve synergies of expertise and process. ‍

Teams that develop clear values together, as the team forms and grows, do better collective work. Often behavioral scientists share a discipline for rigor, a critical eye for the status quo, and a commitment to holding the situation and context, rather than the person, accountable for missteps. That said, the best teams love knowledge sharing, support humility and a growth mindset, and share a vision for improving lives. Teams that are curious about each other’s strengths and areas for growth are better suited to support one another. And, successful behavioral design teams opt for culture adds over culture fits and in doing so benefit from a wide range of experiences and backgrounds, rather than a narrow set of world views.
- Rachel Rosenberg‍

Centering organizational outcomes in the culture of behavioral design teams is also recommended, especially to ensure the longevity of these teams.   

‍Balancing customer-focused outcomes, commercial outcomes and buying down the risks that come with innovation.  A business leader once told me "profitability is sustainable." A BE team that hopes to exist for decades needs to be profitable, as an easy way to demonstrate its value to the org and future-proof itself.  Not everyone holds this view, but it's certainly my bias.
- Etinosa Agbonlahor

Ultimately, it is the behavioral design team’s leadership who will define how the composition, role, and culture of the team is designed and warrants a deeper reflection on the types of leaders and leadership practices required to bring the best out of the team. 

‍Building a successful behavioral science team is similar to building any successful team. It starts with leadership. To create a capability that delivers genuine insight and end-user advantage, vision and goal clarity are assumed.
- Nuala Walsh‍

Chapter 2. What makes for good user research?

User research is one of the foundations of behavioral design and it behooves us to dive into what makes user research good and how can we best leverage it in our work. 

‍Really good behavioral design teams and good user research are inextricably linked–you can’t have one without the other, two items are consistently overlooked in both: diversity and consistent process.
- Josh Wright‍

As an applied science community, we rely on standard scientific methodology and rigour to bring value to user research processes by integrating these methods with business or organizational strategy. ‍

The best user research does three things. Firstly, the researcher understands their own biases, resisting the tendency to latch onto preferred findings despite what the data suggests. Secondly, great research understands users’ biases and actively controls impression management and social desirability. Thirdly, great research integrates findings directly into strategy and gives the client the tools to self-administer change.
- Nuala Walsh‍

2.1 Timing

Something we might not take into consideration enough is the impact timing has on determining the success and impact of conducting user research. Behavioral problems are as diverse and complex as the organizational, budgetary, and business contexts within which they are addressed. As such, the timing of user research needs to be carefully evaluated to ensure it makes the most of the resources available for solving a particular behavioral problem. ‍

User research is most powerful at two phases of a behavioral design project: at the beginning when understanding the problem space, and later when prototyping the solution(s). In the beginning, user research can inform discovery, diagnosis, and ideation to get more ideas on the table and expand the number of directions to explore. Once those ideas have been tested — through an RCT or other quantitative means — and solutions generated, it’s time to bring back the humans with all their messy, disorganized opinions. These two phases are ripe for user research because they take advantage of what people can reliably tell you — what behaviors they’ve exhibited in the past, and not what they think they’ll do in the future. The how, but not the why. When user research is contained to the limits of human introspection, it can truly shine.
- Aline Holzwarth‍

User research can only be conducted, let alone conducted with and for maximum impact if there is enough and the right kind of stakeholder buy-in for user research processes and resource requirements. Depending on your organizational context, the stakeholders and decision-makers who need to approve resources for user research might not immediately see the value of conducting it at all, or following rigorous research methods to ensure the validity, reliability, and applicability of the insights it delivers. Often, user research is seen as something that can be conducted based on using members of your own team as research participants, a highly biased version of user research. ‍

“People believe their own experience as patients is more common than it is and don’t see the need to pressure-test their “me-search” with real research.”
- Amy Bucher‍

Still, it might be the only place that user research can even enter the picture if the field is new to the organization and the behavioral design team needs to then take on the role of advocating for user research and educating stakeholders and the internal organizational community on the value and methods of good user research. With time, and hopefully increasing stakeholder buy-in, more resources will become available to conduct good user research.‍

“It’s so fulfilling to share research results (or ideally have them observe sessions) with someone in this mindset and actually see the shift in their facial expression when they realize their assumptions were off-base.”
- Amy Bucher‍

2.2 Understanding the problem

Before we choose a user research method or even identify the right timing for user research, a deep dive into the behavioral problem at hand needs to be conducted. 

"‍Echoing the famous words of Chermayeff: "Design is directed toward human beings. To design is to solve human problems by identifying them and executing the best solution", I believe that any successful problem-solving process first requires a detailed understanding of the behavioural challenge at hand. In the same way that one would not accept treatment from a doctor without prior medical assessment, one should not rely on a behavioural design solution without preceding behavioural and contextual research. A proper definition, research and identification phase is simply essential to any applied behavioural science project."
- Torben Emmerling‍

In fast-paced organizational contexts where user research or research in general might not be of the same kind of operational and tactical nature as most processes within that context, carving out the necessary time for understanding the behavioral problem might be especially challenging. 

‍The biggest mistake a behavioral scientist can make is to rush the preliminary diagnostics stage and assume understanding of motivations. Too often, this stage is skipped in the rush to conduct a glamorous field experiment. Start with the end outcome in mind. Pre-register the intervention on an open-source platform as a simple control tool that negates the temptation to rewrite history.
- Nuala Walsh‍

Within the phase of understanding the behavioral problem at hand, it is also crucially important to pay attention to the context within which the problem occurs and might be solved. This goes beyond the organizational context within which the behavioral design team is working and stems from the information and research available to understand and solve the problem at hand. With limited resources and tight timelines, it can be tempting to extrapolate insights from existing research and user insights to address even a new behavioral problem or to use the insights from one user population for another in the interest of saving time. As behavioral design teams, it is our responsibility to shine light on the importance of bringing contextual information into understanding the problem and only then deciding whether existing insights or insights from another set of circumstances also applies to the problem at hand. ‍

“We can’t assume that past research on behavioral problems remains valid if there have been context changes that might affect those barriers. The pandemic has been an excellent illustration of this; it’s hard to think of a behavioral area, whether it’s mental well-being, work, leisure, relationships, finances, or physical health, not affected in some way by both the coronavirus and the accompanying societal effects. At the very least we now need to include virus-related barriers when designing for behaviors like health screenings or in-person activities.”
- Amy Bucher‍

Importantly, we must also remember that user research is not needed to solve every behavioral problem, especially if as a team, we have accumulated the know-how and insights from a variety of projects that can be used to answer the same types of questions across different behavioral problems. ‍

“Behavioral designers can evaluate when previously generated research insights might be “good enough” for a current behavioral problem, and where knowledge gaps need to be filled. For example, we might be able to apply insights about how people use prompts and reminders in a mobile app across several different problem spaces. But we’d want to deep dive on behavioral barriers for each set of behaviors those prompts and reminders target. The mechanics of a medication reminder might be similar for different health conditions, but the barriers to actually taking the medication are different depending on the side effect profile, whether it needs to be taken with food, or how it is administered. Over time, a behavioral design team might accumulate a body of insights that can serve multiple behavioral challenges along with the wisdom to know when new insights are needed.”
- Amy Bucher‍

2.3 Using complementary methods

Good user research makes the most of a range of research methods and triangulates data and insights across to deliver the most nuanced and impactful findings. In taking our time to understand the problem and choose the right method, we add value to the organizations we work with and in, and to the user populations our interventions and solutions ultimately serve. This means considering both qualitative and quantitative methods as equally important choices in answering behavioral questions and deciding depending on the whole project context, which one to use at any given time. ‍

“You absolutely must run an experiment!” they cry, over and over. Behavioral scientists are known for fighting the good fight when it comes to proper assessment of impact and inferring causation from an intervention. But in their efforts to advocate for sound methodologies in testing — pushing the RCT as their tool of choice — behavioral scientists may neglect the importance of qualitative user research. They may miss the fact that when paired together, user research and experimentation can complement each other and lead to better outcomes than either method on its own.
- Aline Holzwarth‍

Knowing which user research method to use when is a topic that deserves its own report, but to give a taste of the kinds of questions you might consider, think along the spectra of qualitative vs. quantitative methods, stated vs. revealed preferences, and existing vs. new data. ‍

"Good research is ideally organised along different complementary methods and various sources of information. Recognising the often observed difference between stated and revealed preferences, one should observe or even experience true behaviour in context whenever possible. I would also recommended to draw on existing behavioural data whenever possible. In situations where contextual observations are not possible, one can resort to proprietary user testing and structured interviews or surveys with a representative sample."
- Torben Emmerling‍

An often overlooked concept to include in user research and the behavioral design process is lived experience. Although some qualitative research methods tap into this, the most impactful design often includes lived experience as a part of the design process, not just as research insights. Including individuals with the lived experience of the behavioral problem you are solving for in your team will have the biggest impact, if this is not possible, consider regular check-ins with users who have the relevant lived experience as a part of your design process. ‍

"Good user research is also critical to a good process. Good user research includes looking at both quantitative and qualitative data. It involves observational research and user interviews, but make sure to focus on the what and the how, and little on the why (as humans are terrible at really knowing the why of our own actions and decisions). Perhaps the most overlooked is incorporating lived experience into your process. Ideally that is by having team members who have lived experience with the issue that are being faced (again diversity is important), but if that is not possible make sure to involve the user with the lived experience in your design process. We find lived experience to be undervalued and making sure it is part of the process will avoid embarrassing and costly mistakes later."
- Josh Wright‍

Chapter 3. What makes for a good behavioral intervention?

Evaluating the impact of behavioral interventions often aims to answer the questions of “which interventions work, in which contexts, and for who?”. Ultimately, the interventions we design are as good as their ability to meet the outcomes we set for them, which highlights the need to define the scope and realistic outcomes for intervention at the outset. Recently, a number of meta-analyses in the field have provided new evidence for the impact of behavioral interventions. 

‍For the past decade, figuring out ‘what works’ has been at the core of the applied behavioural science mission. And for good reason, especially given the sampling issues and replicability concerns. Although there is a long way to go, the ship appears to have weathered a storm of critiques and negative publicity around the legitimacy of its evidence base and methods of inquiry. One of the key points of defence is a number of recent large scale meta-analyses that appear to provide support for the effectiveness of behavioural-informed interventions. These studies are all contributing to a growing understanding of what interventions work, in which contexts, and for who. We now have good evidence that decision structure interventions (e.g. defaults, friction reduction) tend to work more effectively than interventions that provide additional information (social norms, progress visibility) or decision support (reminders, commitment devices).
- David Perrott‍

In the third and last chapter of this report, let’s dive into what makes for a good behavioral intervention!

3.1 Problem-centered

Good behavioral interventions are problem-centered. ‍

It’s almost too obvious to state: a good behavioral intervention grows out of the problem, not the solution. But too many behavioral interventions are hammers looking for nails, borne out of a company’s core product. A coaching service because the company has coaches. An app because the company sells apps. A cookbook because the company makes cookbooks. In order to design for behavior change, we need to fully understand the problem space, map out the user’s journey and identify opportunities to decrease friction and increase fuel to get to the desired behavior. And this might involve a coach, or an app, or a cookbook; we might end up there, but we can’t start there.
- Aline Holzwarth‍

3.2 Ethical

Behavioral design should be ethical. Combining the ethics of scientific research and the ethics of design practices might provide the most nuanced guidelines for behavioral design ethics. 

‍Behavioral scientists and designers must learn from the honesty experiment scandal and replication crisis. No matter how good your research discipline or moral intent, the data sources and integrity must be double-checked, robust and replicable. Large-scale researchers sensibly submit their methodology to an ethics committee. Why? Over-enthusiasm can generate backfire effects and create issues that date back to the Stanford Prison Experiment.
- Nuala Walsh‍

In addition to ensuring ethical behavioral intervention research and design processes, we might want to turn our focus onto more foundational ethical questions such as should the behavior at hand be changed at all, and even beyond this, are those in charge of changing it the right people to be in this role? 

‍With a growing understanding of what behavioral interventions work (and where), my sense is that it is time to shift much more attention towards the other big critique our field faces, ethics. Just because we can change behavior should we? Just because an intervention works, should we use it? Just because we can measure the impact of an intervention, should we be tracking behavior? There are no easy answers to these questions, but my sense is that it is time to address these ethical elephants in the room. Especially from the perspective of private sector practitioners.
- David Perrott‍

Our contributor, David Perrott, offers six questions to start your own inquiry into these deeper topics on behavioral intervention ethics:

1. How might we move beyond universal ethic standards, which tend to be constrained by the particular cultural norms and expectations from which they were derived?
2. How might we create a more inclusive conversation around the ethics of behavioral science, to ensure a rich and diverse range of perspectives on the topic?
3. How might integrate ethical consideration into the organizational behavioral design processes, without creating additional friction that slows down intervention deployment?
4. How do we frame the value and incentivize ethical consideration, to increase buy-in across the different levels of decision making within an organization?
5. What practical tools, frameworks, and other resources can we create to facilitate more ethical consideration and better decision-making within organizations?
6. What skills, mental models, and intuition pumps should practitioners adopt to improve their ethical thinking and decisions making within organizations?

Feel free to reach out to David, the editors of this report, and the wider community to get the discussion going!

3.3 Process-driven

Following a clear and evidence-based process of behavioral design is likely to lead to a better chance of achieving the desired impact and outcomes in the evaluation phase. A good place to start is the use of proven behavior change frameworks and theories. Including such frameworks in the design process gives another source of information to triangulate with user research and stakeholder input to reach an intervention design most likely to have the desired effect. 

‍Practically, it’s best to avoid bundling too many interventions as it’s hard to isolate effects.  Disciplined frameworks like COM-B, EAST, MINDSPACE, or NUDGE remain highly useful for practitioners.
- Nuala Walsh‍

Importantly, having an evaluation component to assess the impact of behavioral interventions is already a sign of “good” interventions. With no evaluation, there can also be no learning opportunities for how to improve the process or the design of behavioral interventions in the future. ‍

"From my experience, the biggest mistake is to not evaluate interventions at all. In the end, the effect of an intervention on a target group can only be successfully proven, once compared to a control group. The second mistake is to not control for possible spillovers. Potentially, unwanted side effects can overshadow the positive impact of an intervention and ultimately make it unattractive. The third commonly observed mistake is the sometimes unrealistic expectation of the results of behavioral interventions. In principle, any result of a robust field experiment, especially a null effect, represents an important finding. Moreover, even small effect sizes of a few percent, which are common in behavioural interventions, can lead to significant changes when applied to a large group of people."
- Torben Emmerling‍

Behavioral design might be practiced using different processes depending on the team and the context within which it occurs, but a few steps in the intervention design process are foundational to achieving the desired impact. These include, but are not limited to literature reviews, behavioral diagnoses, and hypothesis generation.  ‍

‍Most of the time, people have attempted to solve the same problem you’re looking at and written about it in academic journals. Not only that, but they've rigorously evaluated the results. This is why one of the starting points of problem-solving in behavioral science is a review of existing academic literature to inform hypotheses about how to solve a particular problem. Once we've leveraged existing knowledge, behavioral scientists conduct a behavioral diagnosis. This is a map of all the small details within the environment in question. Its goal is to reveal the psychologies pushing people to act or not act in certain situations. With these core inputs, we generate hypotheses about how to solve a problem based on actual behavior. These hypotheses can then be tested both through quantitative and qualitative research, and typically provide a strong starting point for the iteration.
- Kristen Berman‍

Another process element that might be helpful for various behavioral design teams and types of interventions is what our contributor Michael Hallsworth names “creative frugality”. Resources are usually finite in intervention design projects and to optimize what is available, it is often beneficial to create by combining the existing rather than investing in something completely new. 

‍Creative frugality. In other words, the intervention was created by recombining existing materials or capabilities in new ways. For example, can we identify how an existing communication can be repurposed, rather than paying for a completely new one? Can a capability built for moving resources between business units be reverse-engineered as a randomization mechanism? In the public sector, these kinds of innovations not only improve the cost-effectiveness of interventions - they may be necessary to make any change happen in the first place.
- Michael Hallsworth

Alongside creative frugality is an increased focus on how we measure. Behavioral problems can be solved in a myriad of ways but whether our interventions succeed at the solution can only be determined with the right types of measurement. ‍

Elegant measurement. Have we found an opportunity to measure behavior with the minimum of bias and burden - e.g. without relying on proxies or self-reports? Are there pragmatic yet smart experiment design choices we can make to avoid spillovers, attrition, or similar problems?
- Michael Hallsworth

The behavioral design process that leads to impactful interventions must be refined through time and practice, which is why having a portfolio approach to intervention design might be especially beneficial for behavioral design teams. The portfolio of interventions allows for better identification of what has worked, what has not worked, and where the quickest wins might be when approaching a new behavioral problem. 

Taking a portfolio approach is critical. At ideas42 we believe that behavioral design is ultimately a tool for innovation. When you are innovating you should fail some of the time. If you are not failing, you are either not measuring failure correctly, and/or you are not taking enough risks. A portfolio approach allows you to balance attacking low-hanging fruit with simpler and more tried and true approaches, with conquering bigger challenges using riskier and more complex designs.
- Josh Wright

3.4 Adds value 

Lastly, good behavioral interventions add value. Value can be defined in various ways, we will discuss a few here, but do share with us how you would define the value that a good behavioral intervention might add!‍

The best interventions aren’t just about size or scale, short-term or long-term effects. They inspire future research and lay the groundwork for others to follow. The best interventions don’t just solve a problem for personal interest’s sake. From the courtroom to the boardroom, they reduce bias and add social value to customers, voters, patients, defendants, or employees. The best interventions help companies avoid poor acquisitions or investments, and enhance overall effectiveness. Better data analytics combined with personalized techniques now make great interventions not just possible in 2022 but the delivery of commercial and prosocial outcomes also highly probable.
- Nuala Walsh‍

Behavioral interventions in a policy context can show the value they add by clearly connecting their impact to desired policy outcomes. 

‍The behavior or policy “matters”. What I mean here is that it was possible to explain why the intervention goal had some value or significance. For example, governments need to collect tax to exist. Antimicrobial resistance can upend modern medicine. Good interventions could convincingly show how the specific targeted behavior is linked to these big issues.
– Michael Hallsworth‍

Good behavioral interventions might also add value to the field of behavioral science and design. Some might argue that every behavioral intervention adds value to the field, by providing a valuable learning experience regardless of the outcomes achieved. ‍

Clear “value add” from behavioral science. Ideally, the intervention was constructed so it could credibly test a defined concept from behavioral science - and thereby develop the field itself. For example, how exactly do social norms play out in the real world? Where does the omission bias seem to be relevant and how exactly can it be operationalized?
- Michael Hallsworth‍

Arguably, the value of good behavioral interventions might be best measured by the value they add to the users for whom they are designed. After all, no other type of value can be achieved without serving the needs of the target users first and foremost. 

‍“Interventions that only increase usage and retention may help businesses grow, but only when connecting a feature to the behaviors that the users want to change can a team say the intervention is successful. For example, usage can measure a feature intended to help an individual not engage in work during a vacation in two different ways. The first looks purely at the usage and then infers if people are using it that it is working. The second looks to see if when people use it, they disengage by not sending emails or joining calls.”
- Connor Joyce‍

Conclusions

As the field of behavioral design matures, we have a unique opportunity to reflect on what is working well, what could be improved and, ultimately, where the impact and value of our interventions stem from. In this Behavioral Design 2022 report, we’ve collated contributions from leaders in the field on three important topics: what makes for a good behavioral design team, what makes for good user research, and what makes for a good behavioral intervention. As a community, we aim to use behavioral design for good and it is up to us to share, learn from, and improve the processes that lead to realizing this goal. 

Recommendations

Whether you are thinking about adding behavioral design to your organization, or want to optimize the expertise you already have, here are a few recommendations to keep in mind on team composition, the role of user research, and how to know whether the interventions being designed are impactful: 

1. Consider a cross-functional team to create the most impactful behavioral design, including expertise in research, design, experimentation, and communication. 
2. Consider providing skill-development support for your behavioral design team beyond their academic training to accelerate the team’s impact as a whole. 
3. Consider evaluating or intentionally designing the culture of your behavioral design team to create a supportive environment for the team to achieve their full potential. 
4. Consider your organizations’ assumptions about user research and create opportunities to unpack these to catalyze the value added from user research processes. 
5. Reflect on how you evaluate the impact of behavioral interventions and identify areas of opportunity for better measurement, more ethical practices, and an increased value add for target users.
6. Reflect on the value the behavioral interventions you and your team design adds to society, the field itself, or to the people, they are designed for. How might you increase the types of value added or the level of impact achieved?