1 of 30

KM 101

Managing the Lessons Learned Life Cycle

Dennis Pearce

January 2025

2 of 30

Organizational vs. Individual Learning

  • Organizations learn through their members�
  • They can learn independently of any specific individual but not independently of all individuals�
  • Organizational knowledge can reside in:
    • Databases and files
    • The minds of a collection of individual employees
    • The relationships between employees
    • Work processes

3 of 30

  • Persistent independent existence�
  • Transferable between individuals or subunits of the organization�
  • Applicable (“know-how” is an essential component)�
  • Situational awareness (“know-when”)�
  • Can be cognitive (lesson) or behavioral (lesson learned)
    • Recognized, captured, and stored for future use (cognitive)
    • Applied, resulting in behavior change (behavioral)

Characteristics of Lessons

4 of 30

A ‘good work practice’ or innovative approach that is captured and shared to promote repeat applications or an adverse work practice or experience that is captured and shared to avoid a recurrence.� - US Dept. of Energy 2004

Knowledge generated by reflecting on experience that has the potential to improve future actions. A lesson learned summarises knowledge at a point in time, while learning is an ongoing process.

- International Fund for Agriculture Development

A lesson learned is a knowledge or understanding gained by experience. The experience may be positive, as in a successful test or mission, or negative, as in a mishap or failure. Successes are also considered sources of lessons learned. A lesson must be significant in that it has real or assumed impact on operations; valid in that it is factually or technically correct; and applicable in that it identifies a specific design, process, or decision that reduces or eliminates the potential for failures and mishaps, or reinforces a positive result.

- SELLS 2000, currently used by the American, European, and Japanese space agencies

Definition of a Lesson Learned

5 of 30

Initiation

Recogni-tion

Validation

Scoping

Capture

Storage

Dissem-�ination

Institution-alization

Mainten-�ance

Lessons Learned Life Cycle

6 of 30

Initiation

Recogni-tion

Validation

Scoping

Capture

Storage

Dissem-�ination

Institution-alization

Mainten-�ance

Lessons Learned Life Cycle

Where we often tend to focus

7 of 30

Gates to Organizational Learning

Process for Identifying and Capturing

Process for Validating

Process for Storing and Disseminating

Process for Testing

Once an individual learns, how fast does the organization as a whole become aware of that knowledge?

How well can the organization determine the accuracy, value, and importance of the new knowledge?

How well is knowledge retained by the organization as people come and go?

How well can the organization identify if and when it has learned?

Initiation

Recogni-tion

Validation

Scoping

Capture

Storage

Dissem-�ination

Institution-alization

Mainten-�ance

8 of 30

Initiation

  • Must be an event with a beginning and end�
  • Lessons can be
    • Emergent (after the event)
    • Methodical (deliberately designed)
      • prototypes
      • test marketing
      • experiments�
  • Lessons can come from
    • Experience
    • Observation
    • Experimentation
    • Simulation

The most valuable lessons often come from the least experience

9 of 30

Recognition

  • Lessons cannot come into existence until they are recognized
  • Organizations can increase their chances of discovering (and thus learning) lessons by actively looking for them�
  • Sources
    • Anything that diverts the organization from “business as usual”
      • negative (accidents, financial difficulties, quality or PR problems)
      • positive (entering a new market, developing a new technology)�
    • Business as usual
      • more difficult to detect
      • special techniques may be required (AAR)�

10 of 30

Recognition

  • Sources (cont.)
    • Capturing and tracking development, manufacturing, or customer problems�
    • Management directed efforts
      • informal (“Go see what you can learn about this...”)
      • formal – responsibility of a specific part of the organization�
    • Consensus arising out of a Community of Practice�
    • Counterfactual (what-if) thinking
      • must be self-focused and upward-directed
      • example: FAA’s Aviation Safety Reporting System
        • incident reports from pilots, mechanics, flight crews, controllers
        • not used for accidents or criminal violations
        • “near misses” can be just as valuable as actual crashes

11 of 30

Efficient and effective transfer of knowledge within an organization requires:�

  • A knowledge producer
  • A knowledge consumer
  • A knowledge intermediary, who acts as:
    • facilitator
    • gatekeeper
    • interviewer
    • interpreter
    • system maintainer
    • buffer�

Capture

12 of 30

  • Can be difficult, because many lessons arise from events that happened only once�
  • Might not be able to be validated in the sense of confirming truth�
  • Can be validated in the sense of gaining organizational acceptance:�

Validation

Type of Event

Validation Process

Event occurs more than once to the same person or team

Consistent outcomes over time

Event occurs only once to one individual or team but multiple times throughout the organization

Use a Community of Practice to identify commonality, compare experiences, and come to consensus

Event occurs one time in the entire organization

Review by a team of experts

Event occurs one time, but no one in the organization has experience

Bring in external expertise to evaluate, or defer to the judgment of a “lessons learned reviewer” based on significance

13 of 30

Storage

  • Content – everything central to the lesson itself:
    • situation, driving event, cause, lesson learned, recommendation, application, evidence of effectiveness, benefits �
  • Identification – information required to separate one lesson from another
    • unique identifier, title, author, date, approver �
  • Categorization – the taxonomic structure, key words, and other labels that enable the user to find the lesson�

Note: Not everything about a lesson is known at the same time, so some of these are not just fields in a repository but also process steps

14 of 30

Scoping

Elements of an effective scoping process include:

  • Catching redundant lessons and eliminating them or preventing them from entering the system�
  • Extracting and distilling a lesson down from what might be a collection of interrelated knowledge �
  • Interpreting and evaluating lessons for their broader applicability, adoption, and adaptation�
  • Prioritizing lessons and assessing the best method(s) for their dissemination, then packaging them appropriately�

15 of 30

Scoping After Storage

“INBOUND TO ZZZ, CREW REQUESTED TO LAND AT ‘N COMPLEX’ AND SUBSEQUENTLY REQUESTED TO LAND ON TXWY L IN FRONT OF FBO. TWR RESPONDED CLRED TO LAND ‘L8J.’ COULDN’T FIND ‘J’ ON ARPT DIAGRAM AND ADVISED TWR, AND THEY SAID IT’S BY THE S COMPLEX! BY THIS TIME WE HAD CROSSED BOTH ACTIVE RWYS AT ABOUT 800 FT AND TWR RECLRED US TXWY P IN FRONT OF FBO. TWR WASN’T UPSET AND THERE WAS NO CONFLICT BUT IT COULD HAVE BEEN A PROB.”

16 of 30

Dissemination Methods (DoE)

17 of 30

Dissemination Methods (DoE)

18 of 30

Bundling Lessons: Knowledge intermediaries search databases, documents, etc. for patterns of lessons.

Two approaches:

  • Horizontal: Collecting groups of lessons into special packages:
    • Wildland Fire Lessons Learned Center’s yearly AAR “roll-ups”
    • FAA ASRS monthly newsletter Callback
  • Vertical: Tying lower-level lessons together to present a higher-level lesson or executive summary

Dissemination

19 of 30

Dissemination

Novices

Shared Work Practitioners

Shared Work

Producers

Increasing Context

Expertise-Seeking Novices

Potential audiences can be segmented based on their relationship to the lesson to be learned. There are three primary types of audience requiring various levels of context:

20 of 30

Institutionalization

The linchpin of the entire lessons-learned life cycle, but by far the most difficult to manage. Even the best organizations have problems in this area.

Common institutionalization fallacies:�

  • “If You Build It, They Will Come”
    • Belief that the mere existence of lessons in a repository or document will cause people to learn them
    • After deployment, system creators are often puzzled as to why people aren’t learning
    • May find at least one of the following:
      • system was not designed with sufficient understanding of user preferences
      • users do not have time to use the system
      • content is not perceived to be relevant or actionable
      • receipt of the lesson does not coincide with an activity where it can be put to use

21 of 30

Institutionalization

Common Fallacies (cont.)

  • “Why Can’t We All Just Get Along?”
    • Belief that knowledge intermediaries aren’t necessary
    • Belief that capturing and sharing lessons should be part of everyone’s job
    • Most people left to themselves do not make the time or have the inclination to stop and reflect on what they have learned unless prompted by an outside force (management direction, AAR, performance metric, etc.)
    • Many very smart people with valuable knowledge to share are not good writers or communicators

22 of 30

Institutionalization

Common Fallacies (cont.)

  • “We Tried That Five Years Ago”
    • The “Hot Stove/Green Room Effect”
    • Creates a bias in organizational learning against risky and novel alternatives
    • Influences the actual lesson learned:
      • surface-level lesson is that the new approach is worse than the old one
      • deeper uncaptured lesson may be that the new method is actually better but was not given sufficient time
      • AARs can push the organization into the “double-loop” learning that uncovers these deeper lessons

23 of 30

  • True learning at the individual level has to be voluntary
  • The best that can be done is to design systems and processes such that they increase the likelihood that lessons will be learned:
    • Repositories – lesson data made available
    • Stories – “Springboard stories” or learning histories
    • Case Studies – taking stories and integrating them into training
    • Phase Gates – using lessons as gates between phases of a process
      • awareness lessons: checklists, reminders
      • action lessons: recommended procedures or activities
    • Process or system changes – fundamental changes embedded into the organization (e.g., automobile safety)

Institutionalization

24 of 30

Maintenance

Two facets:

  • Maintenance of the content
    • To ensure it remains up-to-date and relevant
    • Content not just deleted or replaced, but also refined, modified, and added to�
  • Maintenance of the system
    • To modify the life cycle processes in an effort to facilitate increased learning
    • Metrics:
      • Activity
      • Volume
      • Employee opinion surveys, anecdotes, and testimonials
      • Evaluate content for business value or alignment with business goals
      • Learning curves – problems logged per development project, customer complaints per month, etc.

25 of 30

Level

Lesson Producer

Lesson Consumer

1

Not even aware that there is a need to capture lessons

Not even aware of a need to learn

2

Aware of the need to capture but lacks the time/resources/ability

Aware of need to learn but unaware of available lessons

3

There is a repository, but it is either out-of-date or supported by personal initiative – no formal, sustained process

Aware of the existence of lessons, but the volume of content or number of databases is too overwhelming

4

Capture process exists, but it is viewed as a chore or assignment rather than as a benefit. Capture takes place at the end of project rather than as events happen.

Can find lessons that might be useful, but they are too sketchy or hard to understand or interpret

5

Good, well-accepted process that captures lessons easily in an ongoing manner, but only for a particular audience

Lessons are definitely useful but there is not enough time to look at them

6

Well-written lessons, available in multiple forms in a central location, easily searched, easily accessible by everyone in the company. Processes in place for institutionalizing and embedding lessons. Organizational as well as individual learning.

Everything is at my fingertips, easily accessible and easily applicable. I may even be learning lessons without knowing it.

Lessons Learned Maturity Model

26 of 30

References

27 of 30

References (cont.)

28 of 30

References (cont.)

29 of 30

References (cont.)

30 of 30

References (cont.)