Rethinking Learning Measurement in the Flow of Work

IMAGE CREDIT:  
Image credit
iStock

Rethinking Learning Measurement in the Flow of Work

Subheading text
How adopting new ways of working can help deliver learning in the flow of work

    • Author
    • Author name
      Deloitte Consulting LLP
    • August 13, 2020
    ads }

    Post text

    Lead Author

    Julie Hiipakka

    Vice President, Learning Research Leader

    Bersin, Deloitte Consulting LLP

    Lee Ackerman

    Digital Transformation Leader

    Deloitte Canada

    Key Takeaways

    • Learning and development teams must adopt an agile mindset as they work to deliver value in the flow of work.
    • With organizations adopting new ways of working, it's important for learning and development (L&D) teams to embrace new approaches to support learning in the flow of work.
    • Data collection and the insights they generate are enhanced when learning teams talk to the people and teams getting the work done.
    • Leveraging a mix of sources for data collection can show better results when evaluating work and the different ways of working.
    • Expanding learning units beyond the individual or conventional team structures can improve understanding of how workers interact with each other as part of multiple learning networks.
    • As the value of experimentation in learning gains more recognition, L&D teams need to discard presumptions around what and how these activities can be measured.  

    Overview

    As the pace of change and demands for innovation in the world of work increase, the half-life of many skills keeps shrinking. In response, large organizations with digital aspirations are adopting new ways of working, encouraging leaders and workers to be iterative, transparent, collaborative, and feedback-driven. Further, these companies are becoming flatter as they empower teams to use their expertise to become nimbler and more innovative.    

    These changes to work have led organizations to recognize that traditional approaches to learning are insufficient. The pace of formal training is too slow; we can measure courses, hours, and completions, but none of those allow us to gauge impact. This gap limits the efficiency of interventions and prevents organizations from scaling learning efforts—teams are learning in the flow of work, but learning and development (L&D) groups are on the outside looking in. In this article, we follow the journey of a hypothetical L&D team to demonstrate how L&D professionals can rethink their approaches to what they measure, how they measure, and where they can look to enhance learning as they deliver value in the flow of work.

    Beginning with New Ways of Working

    Like the rest of the organization, L&D must embrace the values and practices that support its own agility. Accomplishing that means breaking free of old habits, mindsets, and perceptions. L&D professionals should start this process by rethinking how they work to get into the flow of work.

    KEY POINT

    To embrace the values and practices that support its own agility, L&D must break free of old habits, mindsets, and perceptions.

    We'll model what this might look through a hypothetical example. Consider “D. Bank,” a personal and commercial bank with digital aspirations. The company’s L&D group seeks to improve its capacity to innovate and increase its value to the company. The group recognizes the need to support learning in the flow of work and is eager to tackle the challenge through new ways of working. As a first step toward that goal, the group launches a small, cross-functional team composed of L&D professionals, an Agile coach, and two developers from IT—a “two-pizza” team." The members decide to call themselves the WOW team, as they are fascinated by ways of working (WOW). As part of a software-driven organization, the team knows that it must be iterative, incremental, transparent, collaborative, and feedback-driven. Because continuous learning and improvement applies to everyone at D. Bank, the team strives to be as innovative and adaptive as all the other teams at the organization.

    Taking a Multisource Approach to Data Collection

    L&D teams have historically relied on data from their own systems, limiting the potential to consider how data from the actual work might be leveraged for learning. The WOW team decides to look outside the systems of L&D and HR to see what it can find. The Agile coach and the developers share that D. Bank’s software teams use some standard collaboration and development tools to facilitate getting the work done. As the WOW team studies these tools and their applications, it realizes just how many learning activities there are to consider: messaging, team structures, pull requests, automated builds, unit testing, code check-ins, and more. For example, a pull request provides an opportunity for both giving and receiving feedback. These are two key competencies that should appear within a team and are critical learning activities.

    As the WOW team continues its work, the group realizes that connecting the data from existing learning management systems (LMSs) and learning experience platforms (LXPs) with the learning activities being discovered won't be easy—after doing some research, the team gets excited by the possibilities provided by tools that support the Experience API (xAPI) to capture learning activities and drive associated analytics. The flexibility of xAPI to recognize the broad range of learning activities in software development is critical to learning in the flow of work.

    KEY POINT

    The flexibility of xAPI to recognize the broad range of learning activities in software development is critical to learning in the flow of work.

    At this point, the WOW team is cooperating and feeling good about its progress. However, everyone recognizes that they still have some big questions to answer, including:

    • Which learning activities do we want to capture?
    • How do we gain insights from those learning activities?
      • How do we know when an intervention is needed?
      • How do we know when there’s been a positive learning outcome?  

    Determining the Right Unit of Learning Measurement

    Data from work can measure learning, but it has to be categorized in a way that makes it useful. L&D teams need to consider the unit of measurement in how performance is tracked. As the WOW team keeps thinking about learning, it has its “eureka” moment when it realizes that the basic unit of value delivery isn’t the individual—it’s the team! After digging deeper, the Agile coach explains that D. Bank has utilized different types of team structures, such as pods, squads, chapters, and guilds. The company's approach to agility and teams is slightly more complex than it appears: an individual is a member of each existing team type, each of which has a different purpose and expectation around the style and frequency of interaction.

    KEY POINT

    As teams continue to measure and analyze ways of working, they need to consider team maturity.

    After considering how team structure relates to learning, the WOW team recognizes that belonging to a given team is just the beginning. To gain more understanding, WOW team members must categorize and interpret the interactions and activities that occur within and across each team. They add social learning analytics, team competencies, and team development stages into the mix to accomplish this. Regarding the former, the WOW team considers the interactions between the members of the social network (teams and teams of teams are, after all, social networks), how members communicate and support one another in finding useful content, their disposition to learning, and the larger context of their interactions. The WOW team also includes Tuckman’s team development stages as a reminder that as a team continues to measure and analyze ways of working, it needs to consider team maturity. For example, a less mature team may not be as effective with pull requests, as fear of offending a new teammate might limit feedback.

    Questioning Assumptions about What Should, Can, and Will Be Measured

    The Agile coach also nudges the team to think about experimentation and the powerful role it plays in learning. For instance, the coach explains that the creation of minimum viable product (MVP) and A / B testing supports experiments focused on capturing customer feedback. If an A / B test helps to identify a feature that customers don’t want, for instance, that can be a valuable learning activity (and outcome). The developers on the team note that this mindset is important in shaping how D. Bank development teams do their day-to-day work. They explain that when creating software, developers need to analyze test-case execution and build failures to connect the events to discussion, learning, and future results. Observing teammates discuss a failing test or build is a critical learning activity—and one that should be celebrated. The conversation within the team continues, and excitement builds as they realize that they can analyze how these learning activities map to ongoing quality metrics and outcomes.

    This WOW team has made tremendous progress in understanding the path in front of it. However, at this point, all it has is a vision and collection of ideas. Moving forward, the team plans to create a backlog of its ideas and start putting together a roadmap for its development efforts. It also wants to start sprinting as it works toward an initial MVP to help validate its ideas and hypotheses. Iterative execution and ongoing learning will likely be key to the team's success.

    Reflection

    The D. Bank L&D team represents many learning organizations that have recognized the need to change. To embrace and enact this type of shift, learning organizations must start with how they work, focus on different things, and learn from the process to help their organizations evolve.

    To learn more about how to leverage technology and data in the flow of work, download our latest report in the Learning in the Flow of Work Solution Provider Landscape Series.

    Key Takeaways

    • Learning and development teams must adopt an agile mindset as they work to deliver value in the flow of work.
    • With organizations adopting new ways of working, it's important for learning and development (L&D) teams to embrace new approaches to support learning in the flow of work.
    • Data collection and the insights they generate are enhanced when learning teams talk to the people and teams getting the work done.
    • Leveraging a mix of sources for data collection can show better results when evaluating work and the different ways of working.
    • Expanding learning units beyond the individual or conventional team structures can improve understanding of how workers interact with each other as part of multiple learning networks.
    • As the value of experimentation in learning gains more recognition, L&D teams need to discard presumptions around what and how these activities can be measured.

    Community forecast feedback

    View the community's ratings after you leave your own below.

    Average year

    All readers

    --

    Average year

    Qr readers

    --

    Average vote

    All readers

    --

    Average vote

    Qr readers

    --

    Average vote

    All readers

    --

    Average vote

    Qr readers

    --

    Average vote

    All readers

    --
    --
    --

    Average vote

    Quantumrun readers

    --
    --
    --

    Average vote

    Company readers

    --
    --
    --