ScienceDirect
Contents
Team members shuffle into the conference room. Some have headlights. Others shoot knowing looks at one another but
a look of fear on their face. Others have a decidedly defen- remain silent. Finally, a senior member hesitantly points out
sive posture. It soon becomes obvious that members’ seating that the extra steps involved in getting approval (introduced
choice coincides with their functional role in the company by the new VP) have slowed down his work. The VP snaps
(sales, accounting, or product development). The CEO and back defensively and the room falls silent again. Visibly
the VP of xyz corporation come in last and sit at the head of uncomfortable, the well-meaning CEO shifts gears and
the table. I have been told ahead of time that the company begins walking through a timeline of milestones from their
had just hired the VP and that his transition into the team has most recent project, noting which were completed on time
been a bumpy one. Many of the team members have been and which were missed. He asks each functional group one at
with the company for a very long time and were accustomed a time to provide their perspectives on why. The discussion
to reporting directly to the CEO. The VP feels he has inher- quickly turns heated as sales, accounting, and product
ited a team that is fiercely loyal to the charismatic CEO and development groups complain about one another. An hour
resistant to the introduction of an intermediary. later, team members shuffle out of the conference room
The CEO begins their team self-evaluation session by looking either confused, embarrassed, angry, or frustrated.
offering his take on the team’s coordination processes, asked for their inputs.
followed by the new VP. Next, he solicits team members’ I have seen similar scenarios unfold many times before in
inputs using general questions such as “How does everyone “After action reviews” conducted by military teams,
think it’s going?” “How can we improve?” “What are “debriefings” of medical teams and aircrews, and “post-
your concerns?” Some members look like deer caught in mortems” conducted by project teams in industry. In the
https://doi.org/10.1016/j.orgdyn.2017.11.002
0090-2616/© 2017 Elsevier Inc. All rights reserved.
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
2 K.A. Smith-Jentsch
weeks that followed the team self-evaluation session at xyz the freedom to talk about the things they saw as most
corporation, I was asked to speak with members one-on-one. important. The problem with this approach, however, is that
Each raised issues that would have been highly relevant to team members are often unsure about which topics are
discuss in the prior session. When I asked them why they had relevant to bring up. General questions like “How do you
not been brought up earlier, I got familiar answers: think we are we doing?” or “What could we do better?” tend
to yield equally general impressions that lack the detail to be
“We jumped from topic to topic so quickly. I kept losing
actionable. Moreover, an unstructured team discussion is
my train of thought.”
typically chaotic. Research has shown that without a
prescribed structure, members’ ability to retrieve relevant
“I’m fairly new and I wasn’t really sure if what I had to say
memories is disrupted by listening to one another’s
was relevant.”
disconnected inputs. This has been referred to as
collaborative interference.
“I didn’t want to offend the new VP.”
Reviewing team performance using a chronological struc-
ture while seemingly intuitive has disadvantages as well.
“I didn’t want to disappoint the CEO.”
Leaders who use this approach typically seek to identify
where in a chain of events teamwork broke down. A replay of
“I thought if I mentioned that problem it would make us in
events in the order they occurred is viewed as less likely to
product development look bad.”
miss important details. However, it also has some unin-
tended negative consequences. When a team discusses their
“I didn’t bring it up because, although it was an error, it
past performance chronologically, the outcome of a specific
didn’t end up delaying the milestones for this particular
project or event is made particularly salient. Knowledge of
project.”
performance outcomes biases memories and assessments of
I integrated and summarized team members’ inputs (with performance processes. This phenomenon is referred to as
their permission) in a report for the CEO and followed up hindsight bias. Thus, to the extent a team organizes their
with a working lunch. The CEO was surprised by the wealth of discussion around the chronology of a specific project or
insights and suggestions his team had expressed, yet fru- performance episode, members are likely to selectively
strated that they had not shared these during the team self- recall and discuss only those actions that directly contrib-
evaluation session. I suggested we try a second time, using a uted to the outcome. Moreover, it can lead teams to discuss
method of structuring and facilitating the discussion that has their faulty processes to the exclusion of their effective
been shown to significantly improve team performance. He processes. Prior research has shown that individuals develop
agreed. richer knowledge and more adaptive skills when both posi-
tive and negative examples are reviewed. In other words, it
LEVERS FOR HIGH IMPACT TEAM SELF- is just as important to know what to keep doing as it is to
know what to stop doing.
EVALUATION Finally, when a team structures its self-evaluation around
the timeline of a particular project or event there is a
Whether they are referred to as debriefings, after action tendency to narrowly discuss solutions and make plans that
reviews, post-mortems, or team building sessions, most would lead to a better outcome provided they face the same
teams periodically meet to discuss their processes and to or similar situation again in the future. To the extent that a
make course corrections. However, the manner in which team faces varied and unpredictable performance chal-
these sessions are conducted varies substantially both across lenges, this approach will be maladaptive. Specifically, if
and within organizations. Much research has been done to a particular process is deemed (and remembered as) effec-
understand how, when, and why some strategies result in a tive or ineffective solely on the basis of the idiosyncratic
greater impact than others. However, the results of this outcome of one project, there is a danger that a team may
research are often published in sources that are read pri- try to “fix something that is not broken” or miss an oppor-
marily by the communities that have funded the research. tunity to correct something that is likely to be problematic in
These are largely the military, NASA, the FAA, law enforce- the future.
ment, and the medical community. The objective of this Structuring a team’s discussion of their performance
article is to summarize best practices from this research and around categories of teamwork is a less common approach,
to describe a practical strategy that ties them together and but one my colleagues and I have found results in greater
has been shown to significantly improve team performance. performance improvement. Doing so helps team members to
develop more generalizable knowledge. The term teamwork
Structure mental models refers to the beliefs one holds about what
effective teamwork looks like. Research has shown that
The three most common approaches to structuring the con- individuals use their mental models to interpret novel situa-
tent of what is discussed during team self-evaluations are (1) tions and to guide their behavior. When mental models are
no real structure (or an inconsistent one), (2) a chronological highly situation-specific, individuals are less likely to recog-
discussion of a project or past event, or (3) a discussion of nize the relevance of their prior experiences in novel situa-
performance issues by type or category (e.g., communica- tions where they may be highly relevant. This has been
tion, workload sharing). Leaders who impose no formal or referred to as the inert knowledge problem. However, as
consistent structure on team discussions of their perfor- they accumulate experience, general rules are gleaned from
mance often report that their intention was to give members concrete instances making mental models of teamwork more
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
abstract. This allows individuals and teams to recognize opinions or whose opinions are considered most credible.
familiar patterns of behavior embedded in unfamiliar situa- Differences in teammates’ status and tenure on a team, as
tions. My colleagues and I found that when team self-evalua- well as their personalities, cultural norms, gender, and
tion sessions were organized around core teamwork relationships with one another all play a role in determining
principles members structured their mental models in a less whose inputs will be offered and how much consideration
situation-specific manner and this significantly improved those inputs will be given. Moreover, when geographically
their ability to adapt to new and challenging performance dispersed teams meet virtually to discuss their performance
situations. Doing so also led to a more balanced discussion of processes those co-located with a larger group of members
positive and negative processes that was less prone to hind- are more likely to contribute to the conversation than are
sight bias. those working with a smaller group (or alone) in a particular
In addition to structuring the content of what is discussed location. Finally, members may resist challenging what
during a team’s self-evaluation, there are different ways to appears to be the group consensus to avoid rocking the boat.
structure the order in which members offer their inputs. The This phenomenon, referred to as groupthink, is most likely to
three most common approaches are (1) individual turn-tak- happen when a team is highly cohesive and has a strong and
ing, (2) turn-taking by functional specialties within the team charismatic leader.
(e.g., sales, marketing, software development), or (3) Beyond motivational factors, team members may not
organic turn-taking. The first two approaches are typically participate in team self-evaluation simply because they do
used with the good intention of ensuring everyone’s per- not realize that they hold unique expertise. Transactive
spective is heard. However, both have unintended negative memory refers to the knowledge team members hold regard-
effects. Using a prescribed order for individual turn-taking ing the distribution of their unique expertise. Studies have
(e.g., simply going around the room) often leads to a dis- shown that teams with a stronger transactive memory more
jointed conversation as the issues brought up by consecutive efficiently utilize their human capital. For instance, my
speakers may be completely unrelated. This has been shown colleagues and I found that when air traffic controllers were
to reduce individuals’ ability to recall information. It also in greater agreement with respect to their relative strengths
puts members on the spot and can lead to defensiveness. and weaknesses they were more willing to accept feedback
Turn-taking among subgroups within a team has the added and assistance from their teammates. In this way, transac-
disadvantage of activating what are known as team fault- tive memory has the potential to make a team’s critique of
lines. Team Fault-lines are hypothetical dividing lines that their own performance more effective. The reverse is also
splinter teams into factions based on clusters of shared and true. When teams critique their own performance, this has
unshared characteristics (e.g., culture, age, education, the potential to strengthen transactive memory. However,
technical specialty). When such fault-lines are in the fore- this will not happen if the leader or a small group of senior
front of team members’ minds they are said to be “active.” teammates dominate the discussion.
Active fault-lines have been linked to greater conflict, lower In this regard, team members’ perceptions of their team
cohesion, and lesser information exchange. In the context of climate will determine whether they are willing to partici-
a team’s critique of their past performance, active team pate. Team climate refers to members’ perceptions of the
fault-lines have the potential to inhibit an open exchange of type of behavior that is expected and accepted in a team.
feedback among team members. Moreover, team members Formal team leaders play an important role in creating and
are more likely to react defensively to feedback provided to maintaining a team climate that is supportive of team self-
them from members on the other side of the fault-line. evaluation. Research has shown that teams are less likely to
Asking questions of the team as a whole, one category at engage in self-evaluation the more their formal leader per-
a time, puts the emphasis on the glue that holds a team forms this function. Thus, although it may be challenging for
together rather than the characteristics that separate them. a leader to resist sharing their own opinions prior to request-
This also allows consecutive speakers to build on one ing input from their teams, doing so can have a big pay-off.
another’s comments in a more coherent way. Additionally, the way in which a formal leader reacts when a
In sum, results from prior research suggest that teams are team member offers their input or admits his/her own
better able to learn and improve their performance when mistake can have a swift impact on the likelihood that
they systematically review their processes against cate- any other member of the team will do so in the future.
gories of teamwork and when an organic turn-taking
approach is used. Doing so decreases the negative effects
of collaborative interference on members’ ability to recall A STRATEGY FOR HIGH IMPACT TEAM
information, reduces the negative influence of team fault- SELF-EVALUATION
lines, and guides members in developing mental models of
teamwork that are transportable to new situations. The strategy I proposed to use at xyz corporation has been
tested on hundreds of teams of different types over the
Facilitation past 15 years (see Table 1 for a list of team types). In this
strategy, referred to in numerous prior publications as
Team members collectively possess more knowledge than Team Dimensional Training (TDT), team members gather,
each does individually. Therefore, the potential benefits of face-to-face or virtually, to review their past performance
team self-evaluation are optimized when team members’ with the expressed purpose of continuous process improve-
unique expertise and perspectives are fully utilized. Unfor- ment. These meetings typically last 45 min to 1 h. A facil-
tunately, those with the most relevant expertise on a given itator (internal or external to the team) serves the primary
topic are not always the ones who choose to voice their role of keeping the discussion organized, encouraging and
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
4 K.A. Smith-Jentsch
Table 1 Types of Teams That Have Used Team Dimensional roles and expertise. For instance, top performers from a
Training variety of disciplines are often brought together for critical
but short-term projects (e.g., healthcare teams). Such “star
Sports teams (e.g., soccer)
teams” collectively possess the knowledge to solve their own
Theater production teams
problems yet it is unlikely that a single team leader would
Software development and sales teams
have the breadth of knowledge to do this on his/her own.
Research and development teams
Third, the facilitator may be either internal (e.g., leader
MBA student teams
or team member) or external (e.g., instructors, outside
Law enforcement teams
evaluators, HR personnel) to the team. When teams are
Correctional facility teams
familiar with the process and have a fairly positive working
Aircrews
climate, it is best for an internal team member to serve as
Healthcare teams
facilitator since it helps the team to develop confidence in
Firefighting teams
their ability to be self-sufficient. This is particularly impor-
Navy tactical teams (submarine, surface ships)
tant for teams that do not always have ready access to
Nuclear power plant teams
external support (e.g., crisis intervention teams, space flight
NASA mission control teams
teams). However, I have found it useful for an objective
NASA astronaut teams
outsider to facilitate TDT sessions when a team has serious
pre-existing conflict or trust issues. Such was the case at xyz
corporation. For this reason, I facilitated their TDT session
Table 2 Defining Features of Team Dimensional Training myself.
1. A team meets (in person or virtually) with the expressed
purpose of improving the quality of their teamwork Setting the Stage
processes and learning from past experiences (45 min to
1 h). Before a TDT session begins, the facilitator sets the stage for
2. Team members (rather than the facilitator or leader) are success by strategically placing members and by setting
primarily responsible for critiquing their processes and expectations regarding the purpose and process of the ses-
solving problems noted. sion.
3. The discussion is organized around 4 teamwork categories.
4. A facilitator asks the team as a whole (organic turn-taking) Placement
for concrete examples (2—3 positive, 2—3 negative) of each In general, those least likely to participate (e.g., new mem-
teamwork category, encourages, reinforces, and summarizes bers, lower status members, shy members) should be placed
team member inputs. where they will naturally receive the most eye contact from
the facilitator (e.g., front, center). Additionally, it is best to
avoid allowing members to sit in clusters that reflect team
reinforcing team members’ observations and suggestions. fault-lines.
The discussion is organized around four teamwork cate- In the initial session I observed at xyz corporation, the
gories. Specifically, the team as a whole (organic turn-tak- physical placement of teammates around the table kept
ing) is asked to describe concrete instances (both positive fault-lines associated with functional area in the forefront
and negative) from their past performance that are relevant of everyone’s mind. It also allowed members of a functional
to one category at a time. The following sections provide specialty to whisper side-bar comments to one another that
greater detail on the key elements of this process (see were never shared with the rest of the team. This time, as
Table 2) using my experience at xyz corporation to illustrate team members shuffle into the room I ask them to please
each step. take a seat between teammates from a functional area other
than their own. They hesitantly accommodate my request. I
Facilitators also ask the CEO and VP not to sit side by side but to mix
themselves in within the rest of the team. This is intended to
The primary role of the facilitator in a TDT session is to ask convey the feeling that everyone’s inputs are equally desired
the right questions, in the right way, and to encourage, in this setting.
summarize, and reinforce team members’ inputs. In other
words, the facilitator does not need to “have all the Purpose
answers.” This has a number of practical benefits. First, I tell the team at the start that the purpose of the session is
the facilitator need not have observed the team’s perfor- to give them an opportunity to improve their teamwork
mance during the performance period being critiqued. This processes. I let them know that we will discuss past incidents
is particularly helpful when a leader inherits an intact team, as a mechanism for learning rather than assigning blame. In
or when a team is so large and/or physically distributed that other words, these incidents serve as a means to an end
no one person (including the leader) has first-hand knowl- rather than the end in and of itself. I tell the team that this is
edge of everything that transpires (e.g., a multi-national also an opportunity for them to learn about one another’s
team). Second, a facilitator does not need to have the unique expertise, strengths, areas for development, exter-
expertise to evaluate each members’ performance them- nal contacts, and work preferences. With this goal in mind, it
selves. In many cases, team leaders are “generalists” is important that everyone is willing to contribute openly and
whereas team members have highly specialized technical honestly.
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
6 K.A. Smith-Jentsch
Table 4 Sample Questions examples of instances when she had a suggestion that was
not shared with the team. The rest of the team was both
Information exchange
surprised and impressed at her ideas given no one else
(+) What sources of information has your team used in the
realized that she had dual majored in marketing.
past (or during a particular task) to make decisions?
( ) In retrospect, what sources of information have been
available (or will be in the future) that you have not used IMPACT
but could be helpful?
Communication delivery Weeks after the TDT session at xyz corporation, I met with
(+) What are some standard or agreed-upon terms your the CEO and VP once again. They told me that several of the
team regularly uses? What do you expect is meant by those suggestions offered in our session had already been put into
terms? practice. Additionally, the VP happily reported that team
( ) Who can describe a time when a team member used members had begun to work “with” him rather than
non-standard or ambiguous terminology? What would be a “against” him. Finally, the CEO had made several staffing
more appropriate or less ambiguous statement? changes. Having recognized through our session that his
Supporting behavior administrative assistant was performing duties well outside
(+) Who can describe an instance when someone on the of her job description, the CEO had changed her job title to
team noticed that another teammate was overloaded and office manager and gave her a sizable raise. In addition, after
provided helpful assistance to them? What signaled to you our session the CEO met individually with the accountant
that assistance was needed? who turned out to be quite talented at marketing. In this
( ) Looking back, can someone describe a time when they conversation, she told him she was more interested in
could have benefitted from assistance but did not receive marketing than in accounting and was considering searching
it? Who was in a position to help and how? for a marketing job outside of the company. The CEO imme-
Leadership/followership diately offered her a marketing position with xyz corporation
(+) Who can describe an instance when someone on the and avoided losing a valuable employee.
team stated (or requested) the team’s priorities? These anecdotal findings are consistent with empirical
( ) Looking back, who can describe an instance when results my colleagues and I have found in a series of quanti-
priorities were unclear and it would have been helpful if tative studies over the course of 15 years. When compared to
someone had clarified priorities? What was the priority at team self-evaluation sessions that were unstructured,
that time? chronologically-structured, or structured around functional
specialty, TDT sessions led to statistically significant
improvements in team attitudes, knowledge, and skills as
asked for an example of a time when information was not well as performance outcomes (see Table 5).
passed when it could have been helpful, a member of the
marketing staff noted that he felt the new VP was secretive. Table 5 Demonstrated Performance-related Benefits
Similar to the first meeting, the VP’s knee-jerk reaction was
to react defensively, this time declaring emphatically that he Motivational (self-report)
was “an open book.” Unlike the first meeting, however, we Collective efficacy –— team members reported greater
did not simply move on after an awkward silence. This time, I confidence in their collective ability to solve their own
asked the team member to follow up with a concrete exam- problems together.
ple in which he felt information was not openly shared. He Team climate –— team members described their climate as
stated that the CEO always left his door open when meeting being more open, participative, united, and forgiving.
with teammates and that everyone felt welcome to sponta- Knowledge (written tests, self-assessments)
neously join the discussion. However, the new VP had a Teamwork mental models –— team members’ mental
practice of closing his office door when speaking to individual models of teamwork were more generalizable.
team members. To him, this meant that individuals were not Role-specific teamwork knowledge –— team members
all receiving the same information. The VP had not realized reported learning more about how their role-specific
this was the impression he was giving, thanked the team- behavior impacts others on their team.
mate for his feedback and agreed to adopt the “open-door Behavior (ratings from outside observers)
policy” in the future. Not only was the miscommunication Teamwork skills –— team members demonstrated more
resolved, but the entire team witnessed the VP accept effective information exchange, communication delivery,
constructive criticism which led to a palatable lightening supporting behavior, and leadership/followership.
of the mood in the room. Use of transactive memory system –— a greater proportion
Discussing concrete examples rather than general impres- of team members contributed to the process of team self-
sions also allows team members to showcase their unique evaluation and planning.
expertise. For instance, when asked for an example of a time Results (objective data)
when opinions or recommendations were not shared but Team decision making –— teams made higher quality
would have been useful a member of the accounting staff decisions and fewer technical errors.
noted that she often had recommendations regarding mar- Training efficiency –— teams and individuals required less
keting given that she also had a marketing degree but that formal training in order to reach the same performance
she has kept her ideas to herself since her current position criteria.
was in accounting. I asked her to give us some concrete
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002
+ Models
ORGDYN-641; No. of Pages 8
8 K.A. Smith-Jentsch
SELECTED BIBLIOGRAPHY
For a meta-analysis of the impact of individual and team of climate and junior members’ willingness to speak up see
debriefings, see Tannenbaum and Cerasoli’s article entitled, Smith-Jentsch, Salas, and Brannick’s article entitled, “To
“Do team and individual debriefs enhance performance? A transfer or not to transfer?: Investigating the combined
meta-analysis,” published in Human Factors, (2013). For a effects of trainee characteristics, team leader support
quantitative study of the impact of Team Dimensional Train- and team climate,” published in the Journal of Applied
ing, see Smith-Jentsch, Cannon-Bowers, Tannenbaum, and Psychology, (2001). For a study on retrieving inert knowledge
Salas’s article entitled “Guided team self-correction: see, Gentner, Loewenstein, Thompson, and Forbus’s article
Impacts on team mental models, behavior, and effective- entitled, “Reviving Inert Knowledge: Analogical Abstraction
ness,” published in Small Group Research, (2008). For a Supports Relational Retrieval of Past Events,” published in
meta-analytic review of collaborative inhibition see Marion Cognitive Science, (2009). For a study of positive and nega-
and Thorley’s article entitled, “A Meta-Analytic Review of tive feedback and the impact of after action reviews, see
Collaborative Inhibition and Post-collaborative Memory: Ellis and Davidi’s article entitled, After-Event Reviews:
Testing the Predictions of the Retrieval Strategy Disruption Drawing Lessons From Successful and Failed Experience,”
Hypothesis,” published in Psychological Bulletin. For more published in the Journal of Applied Psychology, (2005). For
details on ameliorating the negative influence of team faul- more information on the impact of leadership on groupthink
tlines see Veltrop and Hermes, Postma and de Haan’s article see Schafer and Crichlow’s article entitled, “Antecedents of
entitled “A Tale of Two Factions: Why and When Factional groupthink: A quantitative study,” published in The Journal
Demographic Faultlines Hurt Board Performance,” published of Conflict Resolution, (1996). For a quantitative study
in Corporate Governance: An International Review, (2015). linking transactive memory to teammates’ willingness to
For a quantitative study of hindsight bias in decision making accept support from one another see Smith-Jentsch, Kraiger,
see Connolly and Bukszar’s article entitled, “Hindsight Bias: Cannon-Bowers, and Salas’s article entitled, “Do familiar
Self-Flattery or Cognitive Error?,” published in Journal of teammates request and accept more backup? Transactive
Behavioral Decision Making, (1990). For a quantitative study memory in air traffic control,” published in Human Factors,
linking leader behavior during a team briefing to perceptions (2009).
Please cite this article in press as: K.A. Smith-Jentsch, How to conduct a high impact team self-evaluation session, Organ Dyn (2017),
https://doi.org/10.1016/j.orgdyn.2017.11.002